Ir al contenido

Foto

FXE Generator Tool (Accurate lip synching)


  • Por favor identifícate para responder
26 respuestas en este tema

#1
Hellfire_RWS

Hellfire_RWS
  • Members
  • 623 mensajes
This tool will use speech recognition to analyze an audio file and auto-generate a matching FXE lip-syncing data file for use by the NWN2 game engine.

Make sure you read the ReadMe.txt file that is included!
By 0100010

FXE Generator Tool


Editado por Hellfire_RWS, 16 julio 2010 - 01:51 .


#2
drechner

drechner
  • Members
  • 300 mensajes
Very cool! Thanks :)

#3
dunniteowl

dunniteowl
  • Members
  • 1.559 mensajes
This is more than 'Very Cool," drechner. It is pure awesomeness in a can. The lip synching the FXE generator provides is so damn good that you'll think that character is actually talking! It is THAT GOOD. Trust me, you use this to generate your VO parts and folks will bow down and worship you as a God Amongst Men when it comes to VO work.



I know you're working on BG:NWN2 and I know you'll be doing voice sets. This would make it 'a cut above' what I know you're already doing. I am not sure how much more work it would be, though from my Interview with 0100010 a while back (I'll find and post the link to the Citadel Community News Update it's in later) indicated that setting it up and getting it to generate the files is relatively Big Bang for Little Buck.



dunniteowl

#4
Hellfire_RWS

Hellfire_RWS
  • Members
  • 623 mensajes
Top post updated with an example video

DNO, please sticky this one and the one in the toolset thread... pleeeeease.

Editado por Hellfire_RWS, 16 julio 2010 - 02:19 .


#5
dunniteowl

dunniteowl
  • Members
  • 1.559 mensajes
Wish granted. Sticked!

#6
Morbane

Morbane
  • Members
  • 1.883 mensajes
This is quite a thing - including the post of a video right in the thread - new things make for new ways.



Are there syntax / pronunciation cues that override the actual displayed text. I mean without downloading the tool - it seems like the voice is really natural sounding - but syntax cues could enhance strangely spelled names and such.

#7
dunniteowl

dunniteowl
  • Members
  • 1.559 mensajes
Not completely certain what you mean Morbane, however I hope that this link will do something to sort of help answer your question: Citadel Community News Update: 12/29/2009 Interview: Do You Binary? A Pair of Interviews With 0100010.



The second interview with binary in that edition talks a lot about the FXE tool and does get sort of technical. Beyond that, go ahead and ask your questions here and I'll poke binary a bit (pun intended) to get him to elaborate, either here or in another interview.



regards,

dunniteowl

#8
Hellfire_RWS

Hellfire_RWS
  • Members
  • 623 mensajes
The FXE tools does not generate speech from text. it takes a wave file and converts it to face movements for NWN2 characters. 0100010 Was just using pregenerated voices for the test.

#9
Morbane

Morbane
  • Members
  • 1.883 mensajes

Hellfire_RWS wrote...

The FXE tools does not generate speech from text. it takes a wave file and converts it to face movements for NWN2 characters. 0100010 Was just using pregenerated voices for the test.


I understand thx.

#10
0100010

0100010
  • Members
  • 87 mensajes
It does not generate speech from text. However a text file which correctly matches the pre-recorded wav file will significantly help the tool produce better results.

#11
Eguintir Eligard

Eguintir Eligard
  • Members
  • 1.832 mensajes
I thought this was already done on the other forum? I was about to ask cause I forgot the authors name but I know it wasn't an RWS member.

#12
dunniteowl

dunniteowl
  • Members
  • 1.559 mensajes
The author's name is directly above your post Eg.



dno

#13
Eguintir Eligard

Eguintir Eligard
  • Members
  • 1.832 mensajes
I had a bit of a problem with this file. Is there a none issue that, if you have a long silence, it stops generating fxe animations?



I had a two sentence (but not too long) line with a pause. Both files either ran through too fast, or only read the first sentence then went silent (lip flapping wise) while the sound continued. I shortened the pause and it seemed to help. I wouldn't mind if this issue was fixed though.



I also have a question, I chose the c_dragon for the fxe and tried it on a blue dragon, it doesnt seem to work. Do dragons have lips on their models?

#14
0100010

0100010
  • Members
  • 87 mensajes
Did you read the readme file? Try avoiding wavs that are longer than 10 seconds. Having very cleanly recorded files is very important. If it cant recognize the text or the sound volume in the recording was too low it won't generate speech as well. I'd consider breaking up the two sentences into separate files and dialogue lines in your conversation.



I've never tested it on the c_dragon model. I don't know if the dragon models support lip movements.

#15
Eguintir Eligard

Eguintir Eligard
  • Members
  • 1.832 mensajes
Yes I read it, the line isnt too long. Its not near 10 seconds, its just the pause (for effect) that seems to stop the thing working, even when I type in the words for it.

#16
Vaalyah

Vaalyah
  • Members
  • 953 mensajes
Sorry, I am totally unaware of the subject. Maybe (better, almost for sure) a stupid question. Since you said the tool uses speech recognition... does this tool work only for English language?

#17
dunniteowl

dunniteowl
  • Members
  • 1.559 mensajes
Well it's based on the 'memes' of the sounds of words, so, it seems to me it should be able to function in any language, though it may require some experimentation and playing. I am not sure 0100010 is currently reading the forums, school's started back up and I know he's hitting the books lately.

There is an interview on this system at the NeverWinter Citadel Project and it's pretty complete in details. See my post above created 3 months ago for the link. It may help you to understand better what you might be able to do in Italian (or, possibly, in any other language.)

dno

#18
Vaalyah

Vaalyah
  • Members
  • 953 mensajes
Thank you very much. Thought it is still a bit early for me to use this kind of tools! I am a newby in modding!

#19
0100010

0100010
  • Members
  • 87 mensajes
i'm still reading the forums.. just was busy over the last month.


#20
Obadiah

Obadiah
  • Members
  • 5.710 mensajes
Sweet!

#21
0100010

0100010
  • Members
  • 87 mensajes
My hope is that sometime between now and spring I will have time to make up update to this, To allow for direct editing of the phoneme timing sequence values, and incorporate the facial animation expressions that are also in the FXE file.

#22
slowdive.fan

slowdive.fan
  • Members
  • 235 mensajes
This looks very interesting. I hope to use this for the IWD NWN2 conversion project. Thanks for putting this together for us. It'll be great to see Arundel's lips move with the original IWD voiceovers.

#23
IxionReborn

IxionReborn
  • Members
  • 84 mensajes
Awesome, now I just need some voice actors.

#24
johnbgardner

johnbgardner
  • Members
  • 185 mensajes
What program do you need to combine the .001 and .002 files?  Windows can't open them.

#25
0100010

0100010
  • Members
  • 87 mensajes
7-zip. google it.