Sound replacement possible?

[tracktwo]

Great! Glad to hear you've successfully reproduced my results.

 

As for the customization cue, I think I know the reason why they aren't playing. The voice package file for the soldiers has the customization sound nodes directly within it. They are copies of the sound nodes from one or more of the voice banks for that soldier voice. While Zhang/Annette/blue shirts have the customization cue defined in the voice package, the sound nodes themselves are actually references in the import table to sounds defined in one of the bank files. The strategy layer appears to load only the voice pack package and not any banks, so the game can't find the sounds. My intent once I get some sample content was to put copies for the customization cues directly in the package and hopefully this will allow them to play.

[Amineri]

Definitely true that the four sample voice packages I tested with were configured differently than the others in the Voices subfolder (the ones in the Voices subfolder have the cues embedded, as you say).

 

However, the only thing I had to specify for loading was the root voice package :

Package=Voice_BSMVoice1
Package=Voice_BSFVoice1
Package=Voice_ZhangVoice1
Package=Voice_AnnetteVoice1

and not any of the voice banks specifically.

 

The two options I can think of now are :

1) The sound cues not being embedded are causing the voice to not load properly

2) The XComContentManager.RequestContentArchetype native function only properly dynamically loads the voice packages it finds algorithmically, and not those that are manually specified in DefaultContent.ini

 

Regardless, I think we'll have to have sample new voicebank to test.

[tracktwo]

Yep, that will be the most conclusive. I did have success playing customization sounds for a new language that used the existing soldier archetypes as sounds, so if it is an algorithmic thing its smart enough to recognize the packages despite being associated with a different language id.

[Amineri]

The RequestContentArchetype call that loads the voice bank looks like :

    XComContentManager(class'Engine'.static.GetEngine().GetContentManager()).RequestContentArchetype(m_kVoiceRequest.eCategory, m_kVoiceRequest.iID, m_kVoiceRequest.iSubID, self, OnFullPawnContentLoaded, bAsync);

Note that it is being passed the Voice ID and subID, which are previously set using :

            kRequest.eCategory = 7;
            kRequest.iID = m_kAppearance.iVoice;
            kRequest.iSubID = m_kAppearance.iLanguage;
            m_kVoiceRequest = kRequest;

This is the code that appears to be dynamically loading the voice bank, but note that it doesn't necessarily have access to the config data. The native code for loading could be skipping the VoicePackageInfo configuration and using the algorithmic method to try and locate the voicebank package to load.

[tracktwo]

I've successfully created a voice bank with a custom language and had the customization cue play, so it looks like it isn't algorithmic, it just requires the sound data to be present directly in the same package as the XComCharacterVoice object. I managed to get a partial customized voice into the game last night, but it's not complete yet.

 

What's working: Creating an entirely new language with new sound content, selectable in the UI and with working customization sound cues. It does not require force loading all the voice bank packages, it just requires defining a new VoicePackageInfo line in the defaultcontent.ini file and making the small patches to introduce a new language enum value already mentioned before.

 

What's not working: The sound banks I created aren't quite right and fail to de-serialize when loading the tactical game. This actually crashes the game right now, but I have some ideas to test later tonight and hopefully will have it working soon. All of this work was done using the UDK and PatchUPK, I'll post the full details once it's completely working.

 

What I don't yet understand is how to correctly create the bank files. I can create most of the bank files in the UDK, but they will require some post-processing after cooking using PatchUPK to complete them. Ideally the entire thing could be done just through the UDK, but having a post-processing script isn't the worst thing in the world if we really need to go that route.

 

An XComCharacterVoiceBank consists of two main pieces: the "normal" class with a bunch of variables of type sound cue that will refer to all the sound cues defined in the bank, and then a special native variable at the end. You can see this by opening up XComGame.XComCharacterVoiceBank in the explorer and seeing a big list of cue variables followed by the native Map_Mirror variable.

 

Using the UDK to create bank packages, the "normal" part works fine: the cooked bank contains a default property list for all the cue variables, set to my sound cues, and looks identical in the explorer to a stock bank file. Perfect. But, if you look at the object data for a stock bank in the explorer, there is a chunk of data immediately following the default property list. This is associated with that native Map_Mirror variable. I know the structure of that data (it's fairly straightforward) but I don't know how to get that data into my custom cooked package in any way other than a post-processing PatchUPK step. Opening up my custom cooked bank, the object ends right at the end of the default property list for all the cue variables.

 

This makes sense; the UDK doesn't really know anything about whatever native code is used to deserialize an XComCharacterVoiceBank or how to serialize it. Ideally there would be some way to extend the UDK to teach it about that format and have it automatically injected. I don't know enough about the UDK to know how to do that or even if it's possible. The Dll Binding looked promising, but all the examples I can see only discuss using native functions, and are executed at game runtime. I need to serialize a native variable, and do it at package cook time. If anyone has any pointers/knowledge in this area I'd really appreciate it, otherwise I'll keep researching but probably stick with the PatchUPK post-processing step for now.

 

 

[wghost81]

tracktwo, to serialize native variable into package you need to change c++ serialization code. And c++ code is not accessible with free UDK version. So yes, if XComCharacterVoiceBank is a native class which has its own c++ serialization code, we probably won't be able to cook it purely with UDK.

 

BTW, the game can read and process uncooked image/sound packages. Have you tried to save package and then use it directly without cooking?

Edited October 9, 2014 by wghost81

[tracktwo]

Thanks, Wghost. That makes sense, and the patchupk step can be automated so it isn't too bad.

 

I haven't tried uncooked packages. ogg encoding happens as part of cooking, so the uncooked packages would contain wav data and be quite a bit bigger for the sound banks. The uncooked package would still be missing the native variable data that maps the cues in the voice bank to the appropriate variables in the XcomCharacterVoiceBank class, but I'll try some experiments and see what happens :)

[tracktwo]

I thought I'd post all the details and experiments that I have so far. I still haven't been able to get a custom bank to load, but I think I'm so close I can taste it :smile:

 

Here's a tutorial for creating a custom language voice pack with sound banks.

 

Step 1: Prep work

 

 

You need .WAV files in 16-bit PCM format to use as your content. The X-Com voices are 22050Hz and mono, but you can probably use higher sample rates and more channels, the files will just be bigger. You don't need a full set to start, just a few sounds is enough.

 

You also need to install the X-Com compatible UDK. Get it from here: http://www.nexusmods.com/xcom/mods/485

 

 

 

Step 2: Create stub version of the voice classes

 

 

Create the folders XComGame and XComGame\Classes under Development\Src in the UDK installation. Inside the Classes folder, create the files XComCharacterVoice.uc and XComCharacterVoiceBank.uc.

 

Open the XcomGame.upk in the CookedPCConsole folder of the XEW installation in UE Explorer and search for the XComCharacterVoiceBank class, and copy everthing into your .uc file of the same name. The function names aren't strictly needed, all you really need is the class declaration and all the var() declarations.

 

For XComCharacterVoice, the definition you want is slightly different from what you see in UE Explorer. Paste in this:

class XComCharacterVoice extends Object
    hidecategories(Object)
    native(Unit);

var() array<string> VoiceBankNames;
var() SoundCue CharacterCustomizationCue;

Now add the XComGame package to the config files for the UDK. In Config\DefaultEngine.ini add these lines in the sections as noted, after any packages already listed in the section:

[Engine.ScriptPackages]
+NativePackages=XComGame
 
[UnrealEd.EditorEngine]
+EditPackages=XComGame

Now compile the scripts, either from the command line or from the unreal frontend (Binaries\Unrealfrontend.exe in your UDK installation).

 

 

 

Step 3. Create the voice bank

 

 

Open up the unreal editor (there is an icon in the unreal front end for UnrealEd)

 

In the content browser, click "Import" and select one or more of your files. It's easiest to import all the sounds at once, so shift-click or ctrl-click the files you want and hit open.

 

In the dialog that pops up, choose a package name for your voice bank. I'll use MaleSoldier1_Robotic_Bank0, because this is the first (0th) bank of the first male soldier voice in my "robotic" language, which are truly awful sounding text-to-speech clips I created.

 

The name field should be populated with your sound file name. Whatever you used is fine for now, but Xcom uses a naming scheme like SM01Reload02 for the 2nd reload clip of the 1st male soldier. Its easiest to import if you use a consistent scheme and name your sound files appropriately.

 

Click the "auto-create cue" box under options.

 

Hit ok.

 

If you selected more than one file, it'll open up another dialog with your package already filled in. If it looks good, hit OK, or OK to all.

 

You'll now see a pane with a bunch of blue squares for your sounds and sound cues. You should also see your package name under "NewPackages" in the packages pane on the bottom left of the browser. Right click it and select "save". It'll prompt you for a place to save, I used the "Content" folder under the UDK installation.

 

Now click the "Actor Classes" tab at the top of the browser window. De-select all the checkboxes, and scroll down to the bottom of the list. You should see "XComCharacterVoiceBank" there, from the script you compiled earlier. Right click it and select "Create archetype". Enter your package name in the package box, and give the archetype a name. X-Com usually calls the banks by the same name as the package, so I used "MaleSoldier1_Robotic_Bank0" again.

 

Click back to the content browser tab and you should see a new red square labelled archetype. Double click it and a dialog will open that has a big table of names corresponding to the cue variables in XComCharacterVoiceBank.

 

For each of the sounds you imported, click on the corresponding cue in the browser (not the sound, the cue) to highlight it. Then click the little green left-facing arrow next to the event you want to associate with that cue. The field should populate with the object name of the cue you selected. Repeat this process for all your cues. You can leave the rest of the events blank.

 

Save your package again. You now have a sound bank.

 

 

 

Step 4: Create the Voice Package

 

 

Now we're going to go through a similar process to create the voice package. This is the package that references all the voice banks for your voice pack. In the stock X-Com, each voice has 15 banks. We'll only create one, because it isn't actually working yet anyway.

 

Click back to the Actor Classes tab and locate the XComCharacterVoice entry. Right click it and create a new archetype. This time, choose a different package name: this one is for the voice, not the voice bank. I'll match the standard XCom voice scheme and use MaleSoldier1_Robotic as the package name. And also to match X-Com, but be oddly inconsistent with the banks, I'll use Voice_MaleVoice1_Robotic as the archetype name. I don't know why they're prefixed with voice. (Fun fact: the DLC voices do it the other way around, using a Voice_ prefix on the package instead of the archetype)

 

Back to the content browser, hit "Import" again. Import one sound cue to use as the "customization cue". This is the sample sound that plays when you select the voice in the soldier customization UI in-game. You can actually have multiple sound cues to choose from randomly in-game, but adding random cues is a bit more advanced, so we'll just stick with one for now. Remember to select "auto-create cue" and use the same package as your voice archetype.

 

Now double click the archetype. Under Voice Bank Names, Click the green + icon to add a new entry. Add the string "XComCharacterVoiceBank MaleVoice1_Robotic_Bank0.MaleVoice1_Robotic_Bank0", without the quotes. That is, the name of the voice bank class, a space, then the name of your bank package, a dot, and the name of your archetype inside the voice package. Select your cue in the browser and then assign it to the customization cue field as you did before.

 

Save your new package, preferably in the same place you created the voice bank.

 

 

 

Step 5: Cook

 

 

 

Shut down the Unreal Editor. If you have other "Newpackages" lying around you can ignore them, as long as you remembered to save your voice bank and voice package.

 

Add your packages to the DefaultEngine.ini you edited in the script step, in the following section:

[Engine.PackagesToAlwaysCook]
+SeekFreePackage=MaleVoice1_Robotic_Bank0
+SeekFreePackage=MaleVoice1_Robotic

In the unreal front-end, click "cook" and wait for it to complete.

 

 

 

Step 6: Add the cooked packages to the game

 

 

You should now have .upk packages of the same name as your voice package and bank in the CookedPC sub-folder of the UDK, except suffixed with _SF. Copy them to the CookedPCConsole folder of XEW\XComGame. Do not copy the XComGame package from the UDK, it's only a stub. I also had some trouble where initially the game wasn't finding the cooked XComGame package, so the sound bank package had bits of XComCharacterVoiceBank defined within it. If you open the bank in UE Explorer, you should not see any "Classes" tab, and under "Content" you should see only your package name and another package with the same name and the _SF suffix. If not, try doing a clean recompile of the scripts and re-cooking in the front-end.

 

Edit the Config\DefaultContent.ini to register the voice with the game. You can do it as an entirely new language, or add it as a new soldier voice in an existing language. Adding a new language requires making other mods to the game, to expand the language enum list and adjust the number of languages to cycle in the customization UI. I'll describe using an existing language in this tutorial, we'll make a 7th male english voice. (Note: female voices work differently, unfortunately, they require .upk mods to add additional voices beyond the default 6)

 

In defaultcontent.ini, add the following lines:

VoicePackageInfo=(VoiceType=eCharVoice_MaleSoldier1,Language=eCharLanguage_English,IsMec=false,ArchetypeName="MaleVoice1_English.Voice_MaleVoice1_English")

VoicePackageInfo=(VoiceType=eCharVoice_MaleSoldier2,Language=eCharLanguage_English,IsMec=false,ArchetypeName="MaleVoice3_English.Voice_MaleVoice3_English")

VoicePackageInfo=(VoiceType=eCharVoice_MaleSoldier3,Language=eCharLanguage_English,IsMec=false,ArchetypeName="MaleVoice4_English.Voice_MaleVoice4_English")

VoicePackageInfo=(VoiceType=eCharVoice_MaleSoldier4,Language=eCharLanguage_English,IsMec=false,ArchetypeName="MaleVoice5_English.Voice_MaleVoice5_English")

VoicePackageInfo=(VoiceType=eCharVoice_MaleSoldier5,Language=eCharLanguage_English,IsMec=false,ArchetypeName="MaleVoice6_English.Voice_MaleVoice6_English")

VoicePackageInfo=(VoiceType=eCharVoice_MaleSoldier6,Language=eCharLanguage_English,IsMec=false,ArchetypeName="MaleVoice7_English.Voice_MaleVoice7_English")

VoicePackageInfo=(VoiceType=eCharVoice_MaleSoldier7,Language=eCharLanguage_English,IsMec=false,ArchetypeName="MaleVoice1_Robotic.Voice_MaleVoice1_Robotic")

Only overwriting some of the voices and leaving the rest as defaults was a little buggy, so I specify them all. Note that the 2nd voice actually uses MaleVoice3_English, because there is no MaleVoice2_English package. MaleVoice7 will use your new package, so name the package and archetype correctly.

 

 

 

Step 7: Test it!

 

 

Load the game and customize a soldier. You should be able to select a 7th voice in the english language for male soldiers, and it should play your sound cue. Going into the tactical game, however, won't play anything. This is because the bank isn't fully set up yet. It's missing important data structures that the native code uses to load your bank. I haven't yet figured out how to get these to work, so far all my attempts have crashed the game :sad:

 

 

 

 

If you try this and it works, please let me know. If you try it and the steps aren't as I described or a particular step doesn't work, also please let me know and I'll edit this post to update/clarify the step. Also, if there are simpler ways to do these things, let me know! I'm a UDK rookie.

 

 

[Amineri]

I tried doing a little research on Unreal Engine SoundCues, and discovered there is apparently an additional, specialized UE tool for handling these : the UE SoundCue Editor (references : http://udn.epicgames.com/Three/SoundCueEditorUserGuide.html and http://udn.epicgames.com/Three/SoundCueReference.html )

 

This appears to be a visually-driven graph-type editing system for configuring various sound effects for different cues to map to particular sounds. In particular there is a Random Node, with description :

 

 

The Random Node is used to randomly trigger a Sound Node Wave from within a group of possible Sound Node Waves. Weight controls the probability a Sound Node Wave will be triggered relative to other Sound Node Waves in the Actor. The check box RandomWithoutReplacement will exhaust the entire list of possible Sound Nodes before repetition. Inputs are added for each audio file by right clicking on the Random Node and selecting Add Input. Sound Node Waves may be connected directly to the Random Node but you can also add nodes between them for additional control.

 

So apparently the Random Node can assign non-uniform weight from a single cue to multiple possible sound files. If this tool was used to create the vanilla VoiceBanks, it may also have to be used in order to create new VoiceBanks.

 

The Sound Cue Editor can be accessed from within the UDK Editor application by selecting a Sound Cue, right-clicking, and selecting "Edit Using Sound Cue Editor".

 

New Sound Cues are created by selecting a package/folder, clicking new, and then selecting Factory / SoundCue.

 

-------------------------

 

Not sure if this is a factor in why the Voice Bank isn't working, but I thought I'd share in case it is

[wghost81]

Since current problems is too big and complex, I decided to split it to several small problems and resolve those one by one.

 

So. The first problem: adding a new language to soldier customization UI.

 

To add a new language we need to alter three files: XComStrategyGame.upk, XComStrategyGame.int, DefaultContent.ini.

 

Here is a small patch for XComStrategyGame.upk to extend the number of available choices:

 

 

UPK_FILE=XComStrategyGame.upk
OBJECT=XGCustomizeUI.AdvanceLanguage
[BEFORE_CODE]
//MaxChoice = 7 - 1
0F 00 <.MaxChoice> 93 2C 07 26 16
[AFTER_CODE]
//MaxChoice = 8 - 1
0F 00 <.MaxChoice> 93 2C 08 26 16

 

 

XComStrategyGame.int part is easy - just add a new m_arrLanguages entry to [uISoldierCustomize] section:

[UISoldierCustomize]
...
m_arrLanguages[7]="Test"

And for DefaultContent.ini one needs to add a new VoicePackageInfo entry to [XComGame.XComContentManager] section:

[XComGame.XComContentManager]
...
VoicePackageInfo=(VoiceType=eCharVoice_MaleSoldier1, Language=7, isMec=false, ArchetypeName="Voice_BSMVoice1.BSMVoice1")

In this example Male Blueshirt voice is added for the new language.

 

All this will allow to select a new language with customization UI and the language will work in game (except for the fact that it won't be played in customization UI, because Voice_BSMVoice1 package doesn't have actual sound embedded).

 

The problem of creating new voices also consists of two separate problems: UI samples and tactical game voices. But before I proceed with those, there are several notes on how things are organized.

 

Voice packages, voice banks, content management and the other mysteries I don't understand

 

Mystery number 1: there are no voice package info for XCOM soldiers, defined anywhere. Those 6 default voices are either coded in c++ or build dynamically, based on existing packages and some c++ defined rules. [Content.VoicePackages] section inside DefaultContent.ini doesn't correspond to those options and deleting it doesn't affect anything in game. But those default options can be re-defined by adding VoicePackageInfo to [XComGame.XComContentManager] section. For example, this will set first entry for Russian language to Blueshirt voice, but won't affect any other entries, nor number of entries available:

VoicePackageInfo=(VoiceType=eCharVoice_MaleSoldier1, Language=5, isMec=false, ArchetypeName="Voice_BSMVoice1.BSMVoice1")

For the sake of experiment I removed an entire Voice folder and got all the options, but no sounds, so it seems to be coded in c++.

 

Mystery number 2: voice packages and voice bank packages seem to be completely unrelated. <- resolved, I'm an idiot.

 

So. To make a voice sample play in customization UI, we need a voice package (similar to Voice_FemaleVoice1_SF.upk package and the others) with actual sound, cue and XComCharacterVoice objects inside. This solves the strategy part new voices problem.

 

Tactical part problem consists of two parts: guessing a proper voice bank file name(s) and making a proper voice bank package.

 

Tactical part problem is caused by native binary data, specific to xcom, which can't be serialized into package with UDK.

 

Voice bank package consists of several generic UE objects and one xcom specific object - XComCharacterVoiceBank. XComCharacterVoiceBank has configurable SoundCue entries, which can be handled with UDK, and native EventToPropertyMap variable, which maps event numbers to sound cues and gets serialized with c++ code.

 

EventToPropertyMap binary structure has 4 bytes of map size (number of entries) followed by actual entries. Each entry has 1 byte of event number (ECharacterSpeech enum, most probably) and 4 bytes of import table object reference. Those import table objects are references to XComCharacterVoiceBank class variables (SoundCue objects):

ObjectProperty'XComGame.XComCharacterVoiceBank.AlienMoving'
...
ObjectProperty'XComGame.XComCharacterVoiceBank.Wounded'

So, to properly add EventToPropertyMap into the package, we need to add those entries to the import table, construct EventToPropertyMap binary data and append it to XComCharacterVoiceBank object.

 

Bank packages are connected to voice packages through VoiceBankNames array. Each entry is a string with reference to bank package. Example:

XComCharacterVoiceBank FemaleVoice1_English_Bank0.FemaleVoice1_English_Bank0

Edited October 11, 2014 by wghost81