Post by giftfish on Jan 5, 2017 19:43:14 GMT
Below is a port from the thread on the old forum, reformatted for readability. Highlights only; posts that don't add important information are excluded.
Sir | FaceFX Editor
Postby SirCxyrtyx » 23 Jun 2016, 17:32
Deager and giftfish have got FaceFx pretty much figured out, which means I can code up a tool that's much easier to use than the current one.
Thanks for suggesting I re-use the curve editor giftfish. I was a bit skeptical at first that it would work, since FaceFX and Matinee are completely separate things made by different companies, but as you can see in my wip screenshot below, it worked out perfectly.
Not ready for primetime yet, but it's working way better than I hoped. It's not particularly pretty, but it's workable.
Postby giftfish » 24 Jun 2016, 12:17
sircxyrtyx --
Technical question for you related to code in the existing toolset that's relevant to this new tool. Bear with me, as it's a little complicated; I'll try to be clear.
Package Editor and the current FaceFXAnimset editor each have an export/import functions related to FaceFX. These are different than the normal "dump BIN" functions.
Package Editor supposedly imports/exports "FXA" files. My understanding of the exact difference is between an FXA file and what is called a FaceFXAsset in UDK is blurry. UDK's own FaceFX help file documentation doesn't mention FaceFX Assets. The UDK wiki talks about them, in the context of importing the FXA "FaceFX Actor" file from Maya/Max (which contains the "reference pose and bone poses") and linking it to the FaceFX Asset associated with a skeletal mesh. The FaceFXAsset is the object class we find in main PCC files with a name like: BIOG_FaceFX_Assets.SFX_Drell_FaceFX. There are no FaceActorFile objects in the files, as far as I know.
The FaceFXAnimset Editor edits exports that are in reality FXE files. Here's how the UDK help file explains them:
External Anim Sets consist of FXE and FXM files. The FXE files store animation data which can be linked up to an actor file (*.FXA) in FaceFX Studio or at runtime in your game. In FaceFX Studio, this process is called “mounting” and “unmounting” animation sets. FXE files are usually accompanied by a matching FaceFX Mini Session file (*.FXM). FXM files store all of the data needed by FaceFX Studio but not your game. This includes audio data and phoneme information.
The FaceFXAnimset Editor has an option to "recreate and dump FXA" (from the FXE). Package Editor's option is called, "FaceFX > Export". I tested both, and, interestingly, the output does not match.
Output from the Package Editor feature only works when a FaceFXAsset or FaceFXAnimset export are selected and the output hex very closely matches that of the parent object. There are some differences at the beginning and end.
Output from the FaceFXAnimsetEditor very closely matches the selected FXE; it even includes string references, which I'm almost certain shouldn't be there.
Presumably, WV implemented this feature so that the FXA could be opened inside of UDK/FaceFX Studio/Maya/3DMax, if the modder wants. But, from what I can tell, one of them doesn't work properly. It's also possible that neither work properly. We should probably try to find out.
To recreate an example of what I described above, do the following:
--Open BioD_CitHub_HospitalP2.pcc in Package Editor and export obj 411 as BIN. Also use the FaceFX > Export feature with this object selected.
--Open BioD_CitHub_HospitalP2_LOC_INT.pcc in Package Editor and export obj 1302 as BIN.
--Open BioD_CitHub_HospitalP2_LOC_INT.pcc in Package Editor and select obj 1302, then "recreate FXA and dump".
Postby SirCxyrtyx » 24 Jun 2016, 16:59
Doing "FaceFX > Export" on a FaceFXAsset dumps the FaceFXAsset data, and doing it on a FaceFXAnimSet dumps the FaceFXAnimSet Data. Doing "recreate and dump FXA" on a FaceFXAnimSet is not doing any kind of processing of the data, it's just dumping the binary data for that export, the exact same as doing "FaceFX > Export" on that export in the Package Editor. If the FaceFXAnimSet is an FXE, then I suppose it's named incorrectly and ought to be called "dump FXE".
by giftfish » 24 Jun 2016, 17:34
Okay, that clears up part of my confusion (the FXA/FXE misnaming).
My guess is that neither should be called FXA, though, as I don't think either are an FXA. I'd bet that if you exported the FXA from the FaceFXAsset that it wouldn't open in Max. I can't verify that, but I strongly suspect it. I wonder if UDK would think it's an FXA or a FaceFXAsset.
Either way, aren't both functions basically redundant with dump BIN? Couldn't the BIN extension just be renamed FXA/FXE?
Postby SirCxyrtyx » 24 Jun 2016, 23:43
Pretty much. You'd have to trim four bytes from the beginning and end too, but that's trivial enough in a hex editor. I can remove them from Package Editor as they just clutter up the menu.
Postby Deager » 11 Aug 2016, 20:03
Question. First off actually, a statement. I hate animcutscenes. Now, to the question.
As I expected, the mouth movement and all that good stuff, when an animcutscene is involved, are embedded in that animcutscene. I'm looking through BioD_CitCas_530Goodbye.pcc and I see object 254 is Cit004_Goodbye_M_A.FXA_ROM_Miranda_M and in the Interpreter / Hex Editor I sure see stuff that looks a lot like the patterns I've seen for the FaceFX stuff. But our new FaceFX Editor doesn't recognize it. And I can tell something is different because when I grab a smallish group of bytes that sure look like "points" I don't find them in BioD_CitCas_530Goodbye_LOC_INT.pcc.
I know there's a way to do this and I refuse to give up on the new lines I've created for the goodbye scene. I don't even care if I have to do this all via HxD but does anyone know the slight difference between the LOC_INT and the base version of the file? Or has anybody already figured out how to animate mouths and gestures in an animcutscene?
Seriously, I've put in the time on these lines, I think I have it down, and I'm willing to do straight up hex to make it happen so even pattern recognition help would be awesome.
Thanks,
Deager
EDIT: I did that post fast last night before going to bed but mulled it over a bit in my sleep. Since the BioD_CitCas_530Goodbye.pcc must relate to not just the LOC_INT.pcc version but also other languages present in the file...I'm thinking it does not have the lip sync stuff in it. I do think it has all the gestures and movement as the LOC_INT.pcc has none of those for the characters. However, it still does have the m_ stuff so I still do not understand from where the BioD_CitCas_530Goodbye.pcc is getting the lip animation.
So, just throwing that out there as well. And even if I have to scrap, wow, probably 50 to 80 hours of work, so be it. It is was it is. /EDIT
Postby giftfish » 12 Aug 2016, 06:26
Ah, yes, welcome to my world
Fun, huh?
From my experience, this is how animcutscenes work:
1. All game assets used for an animcutscene are in the main file EXCEPT the dlg files themselves and the dialogue audio. If you look at the import list you should find the dlg object and a bunch of WwiseAudio imports. The latter are for the WwiseEvent (VO_Play) objects from the LOC_INT.
2. I suspect the bank must also be used, but in Kaidan's case, the bank isn't imported; it's definitely not in the main file. The package the bank belongs to is imported, though. Not sure if that matter or not. I don't understand all the theory behind imports yet.
3. Even if there is other stuff (matinee tracks) in the LOC_INT, it doesn't seem to be used. The dlg file is only used so much as dictated by the main sequence for the animcutscene. Things usually don't flow in order.
4. If the animcutscene you're looking at is anything like part 1 of Kaidan's romance scene (pre-dream), the FXA is defined by interp and not by line. So, if the first interp for the scene is 35s, then the FXA for that will be 35s in length. I suspect it won't have any string ID pointers which might be why the tool can't open it. Taking a quick look at one of Kaidan's... yeah, this seems right.
---
Btw, I thought your decided to wait until the new editor was finished and had some of the bugs worked out? I made the same call pretty quickly after finding out most of the changes I needed to do for the radio dialogue resulted in the editor crashing or didn't work due to the type of animation I needed to edit (or add from scratch).
Postby Deager » 12 Aug 2016, 06:44
Thanks for all the info. I'll pour over this tomorrow at home.
As for waiting...yes and no. The no is if I have a line I'm going to "zero out" or a line which just needs a short, quick edit. Anything involving a big shift of existing lines or new lines from other files I'm waiting on. But a word but here or there without much other stuff involved I'm already beginning work.
So in this test, I actually have Miranda set to say nothing as opposed to "Back to the fight?" to start off the goodbye scene. Subtitle is gone which is great, and I should be able to get rid of the audio as I have done this in other animcutscenes before, but yeah, the lip animation...either the new editor will do it (which you may have stated already but I'm so tired and busy I would have missed it) or I'll figure something out.
Anyway, again, thanks for the post and I'll read it tomorrow. Just wanted to get a quick word out.
Postby Deager » 12 Aug 2016, 08:06
Instead of a classic ninjaDeagerEDIT, I'll make a new post regarding the current stability of the editor.
Since I was getting FaceFX errors in the dump file when adding and removing points, as we call them, I knew that was a bad idea. But, I saw basically zero risk in editing existing points as it's not changing the size of the file or the format of anything...it's just changing existing hex values. Anytime I see a numerical value needing a change which can be done straight in hex, and if I see an me3explorer tool able to do the same without changing anything else, I feel safe using the tool.
Granted, I'm having some odd anomalies with new names getting created on the Names list. Not a huge deal and I don't see a pattern yet so I'm not posting anything on Git until I figure it out. Also, it's just adding garbage names so I don't really care since it's just a few extra bytes worth of data is all.
Postby Kinkojiro » 10 Dec 2016, 07:34
All I want for Christmas is a upgrade to FaceFX!
I am finally reaching the stage where we can create entirely new conversations using snippets of other convos. I have done 2 for Miranda Mod. This is great and only made possible by the FaceFX editor.
A problem is the actual facefx. The fact we can drag and drop facefx between files is incredible. However often I have edited the audio. For example I take the vanilla phrase: "The Reapers are the only thing we should be concerned about. Its not your fault."
I remove all the audio to just leave "its not your fault". Easy enough in audio, first 4secs deleted and push through wwise and sorted.
For faceFX I just drag and drop the old "The Reapers are the only thing we should be concerned about. Its not your fault." line into the edited file, rename the numbers and works perfectly. However actually editing the face fx down to just "its not your fault" is very tedious.
In face fx editor this means doing 2 things:
(1) For every group (so 20 or so) removing any key under 4seconds. This isnt actually that hard, as the delete button works and automatically jumps to the next one.
(2) For every key point (5-20 keys per group) I have to change the time by 4 seconds. This takes absolutely ages.
Ideally what I want to be able to do is either cut or delete a portion and everything moves by 4 seconds, or have the ability to manually tell it to move every point by a certain amount of time.
Ultimately it would be great to be able to copy, cut and paste a selection of key points so it is easy to build the face fx out of a couple of phrases that we join together. This maybe too much to ask, but even just the ability to move all the key points in a group by a set amount of time would still be a huge timesaver.
Postby giftfish » 10 Dec 2016, 07:52
@kinko -- Yeah, this is the same suggestion I made (along with many others) in June, when doing my first round of experimentation after release. The issue is still open here on Git. My thought was that we need the ability to drag the mouse on screen and select a group of datapoints, then all could be slid up/down the timeline and keep their locations relative to each other.
Without this the labor/time investment is still extremely high depending on want you want to do. This and the Audio Editor are the two main reasons I haven't yet headed full force into all the dialogue editing for TMv3.0. With redoing all of Huerta from scratch, creating two versions of that convo, the new ship dialogs, and a full length romance scene, we're talking a huge time investment. Months and months. Implementation of a group editing feature in FaceFX and the new Audio Editor would reduce this investment significantly.
END
Sir | FaceFX Editor
Postby SirCxyrtyx » 23 Jun 2016, 17:32
Deager and giftfish have got FaceFx pretty much figured out, which means I can code up a tool that's much easier to use than the current one.
Thanks for suggesting I re-use the curve editor giftfish. I was a bit skeptical at first that it would work, since FaceFX and Matinee are completely separate things made by different companies, but as you can see in my wip screenshot below, it worked out perfectly.
Not ready for primetime yet, but it's working way better than I hoped. It's not particularly pretty, but it's workable.
Postby giftfish » 24 Jun 2016, 12:17
sircxyrtyx --
Technical question for you related to code in the existing toolset that's relevant to this new tool. Bear with me, as it's a little complicated; I'll try to be clear.
Package Editor and the current FaceFXAnimset editor each have an export/import functions related to FaceFX. These are different than the normal "dump BIN" functions.
Package Editor supposedly imports/exports "FXA" files. My understanding of the exact difference is between an FXA file and what is called a FaceFXAsset in UDK is blurry. UDK's own FaceFX help file documentation doesn't mention FaceFX Assets. The UDK wiki talks about them, in the context of importing the FXA "FaceFX Actor" file from Maya/Max (which contains the "reference pose and bone poses") and linking it to the FaceFX Asset associated with a skeletal mesh. The FaceFXAsset is the object class we find in main PCC files with a name like: BIOG_FaceFX_Assets.SFX_Drell_FaceFX. There are no FaceActorFile objects in the files, as far as I know.
The FaceFXAnimset Editor edits exports that are in reality FXE files. Here's how the UDK help file explains them:
External Anim Sets consist of FXE and FXM files. The FXE files store animation data which can be linked up to an actor file (*.FXA) in FaceFX Studio or at runtime in your game. In FaceFX Studio, this process is called “mounting” and “unmounting” animation sets. FXE files are usually accompanied by a matching FaceFX Mini Session file (*.FXM). FXM files store all of the data needed by FaceFX Studio but not your game. This includes audio data and phoneme information.
The FaceFXAnimset Editor has an option to "recreate and dump FXA" (from the FXE). Package Editor's option is called, "FaceFX > Export". I tested both, and, interestingly, the output does not match.
Output from the Package Editor feature only works when a FaceFXAsset or FaceFXAnimset export are selected and the output hex very closely matches that of the parent object. There are some differences at the beginning and end.
Output from the FaceFXAnimsetEditor very closely matches the selected FXE; it even includes string references, which I'm almost certain shouldn't be there.
Presumably, WV implemented this feature so that the FXA could be opened inside of UDK/FaceFX Studio/Maya/3DMax, if the modder wants. But, from what I can tell, one of them doesn't work properly. It's also possible that neither work properly. We should probably try to find out.
To recreate an example of what I described above, do the following:
--Open BioD_CitHub_HospitalP2.pcc in Package Editor and export obj 411 as BIN. Also use the FaceFX > Export feature with this object selected.
--Open BioD_CitHub_HospitalP2_LOC_INT.pcc in Package Editor and export obj 1302 as BIN.
--Open BioD_CitHub_HospitalP2_LOC_INT.pcc in Package Editor and select obj 1302, then "recreate FXA and dump".
Postby SirCxyrtyx » 24 Jun 2016, 16:59
Doing "FaceFX > Export" on a FaceFXAsset dumps the FaceFXAsset data, and doing it on a FaceFXAnimSet dumps the FaceFXAnimSet Data. Doing "recreate and dump FXA" on a FaceFXAnimSet is not doing any kind of processing of the data, it's just dumping the binary data for that export, the exact same as doing "FaceFX > Export" on that export in the Package Editor. If the FaceFXAnimSet is an FXE, then I suppose it's named incorrectly and ought to be called "dump FXE".
by giftfish » 24 Jun 2016, 17:34
Okay, that clears up part of my confusion (the FXA/FXE misnaming).
My guess is that neither should be called FXA, though, as I don't think either are an FXA. I'd bet that if you exported the FXA from the FaceFXAsset that it wouldn't open in Max. I can't verify that, but I strongly suspect it. I wonder if UDK would think it's an FXA or a FaceFXAsset.
Either way, aren't both functions basically redundant with dump BIN? Couldn't the BIN extension just be renamed FXA/FXE?
Postby SirCxyrtyx » 24 Jun 2016, 23:43
Pretty much. You'd have to trim four bytes from the beginning and end too, but that's trivial enough in a hex editor. I can remove them from Package Editor as they just clutter up the menu.
Postby Deager » 11 Aug 2016, 20:03
Question. First off actually, a statement. I hate animcutscenes. Now, to the question.
As I expected, the mouth movement and all that good stuff, when an animcutscene is involved, are embedded in that animcutscene. I'm looking through BioD_CitCas_530Goodbye.pcc and I see object 254 is Cit004_Goodbye_M_A.FXA_ROM_Miranda_M and in the Interpreter / Hex Editor I sure see stuff that looks a lot like the patterns I've seen for the FaceFX stuff. But our new FaceFX Editor doesn't recognize it. And I can tell something is different because when I grab a smallish group of bytes that sure look like "points" I don't find them in BioD_CitCas_530Goodbye_LOC_INT.pcc.
I know there's a way to do this and I refuse to give up on the new lines I've created for the goodbye scene. I don't even care if I have to do this all via HxD but does anyone know the slight difference between the LOC_INT and the base version of the file? Or has anybody already figured out how to animate mouths and gestures in an animcutscene?
Seriously, I've put in the time on these lines, I think I have it down, and I'm willing to do straight up hex to make it happen so even pattern recognition help would be awesome.
Thanks,
Deager
EDIT: I did that post fast last night before going to bed but mulled it over a bit in my sleep. Since the BioD_CitCas_530Goodbye.pcc must relate to not just the LOC_INT.pcc version but also other languages present in the file...I'm thinking it does not have the lip sync stuff in it. I do think it has all the gestures and movement as the LOC_INT.pcc has none of those for the characters. However, it still does have the m_ stuff so I still do not understand from where the BioD_CitCas_530Goodbye.pcc is getting the lip animation.
So, just throwing that out there as well. And even if I have to scrap, wow, probably 50 to 80 hours of work, so be it. It is was it is. /EDIT
Postby giftfish » 12 Aug 2016, 06:26
Ah, yes, welcome to my world
Fun, huh?From my experience, this is how animcutscenes work:
1. All game assets used for an animcutscene are in the main file EXCEPT the dlg files themselves and the dialogue audio. If you look at the import list you should find the dlg object and a bunch of WwiseAudio imports. The latter are for the WwiseEvent (VO_Play) objects from the LOC_INT.
2. I suspect the bank must also be used, but in Kaidan's case, the bank isn't imported; it's definitely not in the main file. The package the bank belongs to is imported, though. Not sure if that matter or not. I don't understand all the theory behind imports yet.
3. Even if there is other stuff (matinee tracks) in the LOC_INT, it doesn't seem to be used. The dlg file is only used so much as dictated by the main sequence for the animcutscene. Things usually don't flow in order.
4. If the animcutscene you're looking at is anything like part 1 of Kaidan's romance scene (pre-dream), the FXA is defined by interp and not by line. So, if the first interp for the scene is 35s, then the FXA for that will be 35s in length. I suspect it won't have any string ID pointers which might be why the tool can't open it. Taking a quick look at one of Kaidan's... yeah, this seems right.
---
Btw, I thought your decided to wait until the new editor was finished and had some of the bugs worked out? I made the same call pretty quickly after finding out most of the changes I needed to do for the radio dialogue resulted in the editor crashing or didn't work due to the type of animation I needed to edit (or add from scratch).
Postby Deager » 12 Aug 2016, 06:44
giftfish wrote:
Btw, I thought your decided to wait until the new editor was finished and had some of the bugs worked out? I made the same call pretty quickly after finding out most of the changes I needed to do for the radio dialogue resulted in the editor crashing or didn't work due to the type of animation I needed to edit (or add from scratch).
Btw, I thought your decided to wait until the new editor was finished and had some of the bugs worked out? I made the same call pretty quickly after finding out most of the changes I needed to do for the radio dialogue resulted in the editor crashing or didn't work due to the type of animation I needed to edit (or add from scratch).
Thanks for all the info. I'll pour over this tomorrow at home.
As for waiting...yes and no. The no is if I have a line I'm going to "zero out" or a line which just needs a short, quick edit. Anything involving a big shift of existing lines or new lines from other files I'm waiting on. But a word but here or there without much other stuff involved I'm already beginning work.
So in this test, I actually have Miranda set to say nothing as opposed to "Back to the fight?" to start off the goodbye scene. Subtitle is gone which is great, and I should be able to get rid of the audio as I have done this in other animcutscenes before, but yeah, the lip animation...either the new editor will do it (which you may have stated already but I'm so tired and busy I would have missed it) or I'll figure something out.
Anyway, again, thanks for the post and I'll read it tomorrow. Just wanted to get a quick word out.
Postby Deager » 12 Aug 2016, 08:06
Instead of a classic ninjaDeagerEDIT, I'll make a new post regarding the current stability of the editor.
Since I was getting FaceFX errors in the dump file when adding and removing points, as we call them, I knew that was a bad idea. But, I saw basically zero risk in editing existing points as it's not changing the size of the file or the format of anything...it's just changing existing hex values. Anytime I see a numerical value needing a change which can be done straight in hex, and if I see an me3explorer tool able to do the same without changing anything else, I feel safe using the tool.
Granted, I'm having some odd anomalies with new names getting created on the Names list. Not a huge deal and I don't see a pattern yet so I'm not posting anything on Git until I figure it out. Also, it's just adding garbage names so I don't really care since it's just a few extra bytes worth of data is all.
Postby Kinkojiro » 10 Dec 2016, 07:34
All I want for Christmas is a upgrade to FaceFX!
I am finally reaching the stage where we can create entirely new conversations using snippets of other convos. I have done 2 for Miranda Mod. This is great and only made possible by the FaceFX editor.
A problem is the actual facefx. The fact we can drag and drop facefx between files is incredible. However often I have edited the audio. For example I take the vanilla phrase: "The Reapers are the only thing we should be concerned about. Its not your fault."
I remove all the audio to just leave "its not your fault". Easy enough in audio, first 4secs deleted and push through wwise and sorted.
For faceFX I just drag and drop the old "The Reapers are the only thing we should be concerned about. Its not your fault." line into the edited file, rename the numbers and works perfectly. However actually editing the face fx down to just "its not your fault" is very tedious.
In face fx editor this means doing 2 things:
(1) For every group (so 20 or so) removing any key under 4seconds. This isnt actually that hard, as the delete button works and automatically jumps to the next one.
(2) For every key point (5-20 keys per group) I have to change the time by 4 seconds. This takes absolutely ages.
Ideally what I want to be able to do is either cut or delete a portion and everything moves by 4 seconds, or have the ability to manually tell it to move every point by a certain amount of time.
Ultimately it would be great to be able to copy, cut and paste a selection of key points so it is easy to build the face fx out of a couple of phrases that we join together. This maybe too much to ask, but even just the ability to move all the key points in a group by a set amount of time would still be a huge timesaver.
Postby giftfish » 10 Dec 2016, 07:52
@kinko -- Yeah, this is the same suggestion I made (along with many others) in June, when doing my first round of experimentation after release. The issue is still open here on Git. My thought was that we need the ability to drag the mouse on screen and select a group of datapoints, then all could be slid up/down the timeline and keep their locations relative to each other.
Without this the labor/time investment is still extremely high depending on want you want to do. This and the Audio Editor are the two main reasons I haven't yet headed full force into all the dialogue editing for TMv3.0. With redoing all of Huerta from scratch, creating two versions of that convo, the new ship dialogs, and a full length romance scene, we're talking a huge time investment. Months and months. Implementation of a group editing feature in FaceFX and the new Audio Editor would reduce this investment significantly.
END
