R&D XCOM UI/Flash modding

[tracktwo]

I've been doing some experimentation with creating custom flash UIs/elements in XCOM to make modding UI changes simpler. I'm planning on using this thread to share what I find out and hopefully some of the rest of you can find this useful too.

 

So far I've been able to create a custom flash movie to act as a UI and displayed it in both the tactical and the strategic game. Here's a screenshot of the strategic version, hooked up to the situation room and triggered by a "Campaign Summary" UI option. The flash object is simple and just drops a blue rectangle on the screen.

 

 

 

http://i.imgur.com/0M7B6M3.jpg

 

 

 

Current knowledge/Assumptions

 

Here's what I know or think I know so far:

 

1. The UDK uses ScaleForm to embed flash into the game.

2. The version of the UDK used by XCOM supports only Flash 8 and ActionScript 2.0.

3. These versions of Flash are quite old and are no longer supported by the CC version of Flash, it doesn't support anything back further than Flash 10.3 and AS 3.0. Flash CS6 supports these versions, but I don't have this version.

4. It looks like even emitting Flash 10.3 SWFs will work in the UDK, at least the simple ones do, I'm assuming that as long as you stay away from any feature that is only present in versions later than 8 it will be ok.

5. ActionScript 3.0, on the other hand, is radically different from 2.0 and I don't think ActionScript created by Flash will work at all in the UDK.

6. Open Source projects like mtasc will compile actionscript 2.0. It might be possible to create a kind of hybrid workflow between Flash CC to author the basic flash layout and then use something like FlashDevelop to handle the scripting part to add actionscript 2 classes to the project.

 

Getting Flash into XCOM

 

So far I have two ideas fow how to get custom flash into xcom. The first I've tested, and the second I haven't.

 

The first option is to import the custom .swf into the UDK and add it to a new blank map via Kismet, then load the map into xcom as a streaming map on top of whatever map you want to display the flash in. Kismet can communicate with both the flash movie and the xcom unrealscript, and the flash map can be loaded either programmatically or via the defaultmaps.ini file.

 

Pros: Fairly easy to do, and very self-contained. Custom actionscript code can be created and compiled in your own packages associated with the custom map, all you need to do is create the appropriate stubs for the xcomgame or xcomstrategygame classes you want to reference and create the appropriate sequence objects for Kismet. Little hex modding to get it to work, just enough to stream the map when you want it to show up and to unload the map when you're done.

 

Cons: Any raster images in the .swf will be converted to textures by the UDK import process, which means these textures need to get added to the main Textures.tfc file or the game will crash. AFAIK we don't have any tools for patching the Textures.tfc file. Some .swfs may not have any bitmapped images in them and then this won't matter.

 

The second option is to mimic what Xcom already does. Import the .swf into the udk as for the first option, except instead of adding it to a blank map, just cook the package to create a "UI" package like the vanilla ones (UICollection_Tactical, UICollection_Strategic, etc). These then need to be referenced by unrealscript to load and play. The existing ones use special uscript classes that are subclasses of the classes FxsScreen or FxsPanel.

 

Pros: Better/easier texture handling so bitmapped images in the .swf should work better. (Is this true, or do the textures still need to be in the .tfc?)

 

Cons: More invasive. New class definitions need to be created, probably in their own packages, and these need to be loaded from the vanilla unrealscript. This means injecting new names and imports into the xcomgame/xcomstrategygame packages. The Fxs* base classes also appear to be a little magic, it looks like they involve a lot of native code to do stuff that we can't really see or understand easily. E.g. the AS_* functions to invoke actionscript seem to have some magic associated with them for parameter marshaling, and I don't know if this will even work with custom compiled subclasses or if it'd require access to the native development tools.

 

 

Flash Authoring

 

This is tricky since the XCOM UDK is so old. I really hope XCOM2 supports a slightly newer UDK that at least supports AS3 (apparently this was added to the version of ScaleForm included in the Nov 2011 UDK, so they only just missed it). Anyone lucky enough to have a Flash CS6 license can probably author a compatible flash file fairly quickly, but the rest of us will probably need to hack together some sort of weird hybrid solution. I've been experimenting with FlashDevelop and trying to construct a swf that when compiled looks sort of like the ones extracted from the xcom packages, but so far I haven't had much luck. I have almost zero knowledge of flash and actionscript internals and how to structure a project, though, so I'm really stumbling around this aspect of it. I need to spend some more time learning flash details to make more progress here.

 

 

So that's the status so far. I think I'm really close to being able to generate arbitrary UIs layered on top of XCOM, but I'm just stumbling at the flash scripting part. I'd really like to hear from anyone with more flash experience about how to create a .swf that looks compatible with the UDK. If anyone out there knows a little bit of flash and can help me out, please let me know, I'd appreciate it!

Edited July 7, 2015 by tracktwo

[LiQuiD911]

try this: create the udk map with the new textures and cook it. Then patch the resulting upk replacing the references of the Texture tfc wirh someother name as in the eu map pathes. then rename the newly created tfc with that name(it's advised to use one from the upk name table).

[tracktwo]

Thanks, that's a great idea. I'll try that.

 

Update on getting more complex flash stuff to work, including scripting:

 

 

 

http://i.imgur.com/0W8YKWW.jpg

 

 

 

This text was updated via kismet setting a variable in the flash file:

 

 

 

http://i.imgur.com/TdvoeCq.jpg

 

 

 

So I think I've got this at least partially figured out now and things are looking quite promising so far.

 

And more news on the actual flash authoring front. I found a couple of alternative flash creation packages. The first is Vectorian Giotto, which is free. I had some trouble getting it to work yesterday but now that I have a working example I will try it again. When I was doing some research yesterday I came across a forum thread that said that it couldn't handle .png format images, which is definitely something that we need to make this work ok. I don't know if that's still the case or not.

 

This particular sample was created with the trial version of SWF Quicker, which is why the first screenshot has the watermark across the top. This isn't free, but it has a 1 month trial, looks to have a pretty good feature set, and supports lots of flash versions natively all the way back to and including flash 8 and AS 2.0. It's $85 though, which I would consider shelling out for if I knew it'd continue to be useful for XCOM2 modding too. All things considered, that's only the cost of 4 months of an individual Flash CC subscription from Adobe.

 

I'm going to go back to working with Vectorian Giotto for now and see if I can reproduce exactly this result, and also see what I can do about .png support and getting the textures working correctly with LiQuiD911's suggestion.

[Zyxpsilon]

Maybe you already know about this "Freeware Tool"... but JPEXS is what i mostly use (recommended by XMTS) to inspect and edit most SWF & GFX assets when developping my Re-CLR projects. Not sure if it could serve as an authoring device though as it essentially does various editing functions.

Have a look... https://www.free-decompiler.com/flash/features/

 

Another more professional open-source program seems to offer the essentials as well. I never could find the time to dig deeper into its complex features (after having found JPEXS above), although it looks pretty solid.

And -- it's free too... http://www.flashdevelop.org/

(( PS; Oooopppsss... I just realized you've been testing that one! ))

 

Maybe XMarksTheSpot could offer you some in-direct help too, btw.

 

Otherwise, the web is filled with various solutions -- Open-Dialect & so on.

Good luck!

Edited July 8, 2015 by Zyxpsilon

[tracktwo]

I've been using JPEXS to inspect flash files, but it had some warnings about using it for significant editing. I'm not sure if it's any better now or not with later versions but I had some corruption when I was using it to do some simple editing of the ActionScript code for the radar mod.

 

I managed to get Vectorian Giotto working with the same simple flash file as I had created with SWF Quicker, so it looks viable, at least for basic stuff. The lack of lossless support is a bit of a bummer though. However, the person who wrote the original tutorial on it I found also released a tool to post-process a SWF generated by Giotto and replace the jpgs it exports with the original pngs. The tutorial and link to the program are here.

 

Here's a new screen shot with the journal reading logic hooked up. No processing yet, it just reads the first entry from the journal and dumps it on the screen in raw form. The star is a png image that I created and embedded in the .swf. The UDK import process converted it to a texture, as expected. However, it didn't need to be in the textures.tfc file - it just loaded and everything worked!

 

 

 

http://i.imgur.com/eRI2Zvs.jpg

 

 

 

Edit: The typo "Startint Continent" isn't mine, it's in the vanilla journal writing code for the start of a campaign entry. :smile:

 

Edit #2: The link to the lossless post-processing utility for Giotto is broken. But I found it in a google code project here (untested, I have no idea what this .exe actually does. YMMV if you decide to download strange executables from random google code projects and execute them :) )

Edited July 8, 2015 by tracktwo

[tracktwo]

Tonight's update:

 

The xcom fonts work in text fields in flash. It's easiest to use HTML textfields and just embed a font tag with the face $TitleFont, $NormalFont, or $ProtoFont. To use a plain text field with a font set statically you'd need to refer to the font through an ImportAssets directive and I'm not entirely sure how to get there yet. Either way, this is good news for a consistent look to the text in custom UIs without needing to acquire and redistribute the fonts.

 

On the less-good front, although it's minor, I haven't had much success in getting the UI to resize correctly according to resolution changes. I can detect the current resolution and scale things appropriately -- that works fine. What doesn't work is responding to the user changing resolution while the UI is active. That's not a huge deal, as resolution changes should be rare and simply closing and reopening the UI should fix it if it's written to check the resolution and adjust to it at startup.

 

The details:

 

It looks like the UIs respond to resolution changes via a class Environment that's included in most of the vanilla UI SWFs. This is a singleton class that is responsible for recording callback functions for objects that are interested in receiving resolution or input change events. There is a function SetView() which takes as parameters the new resolution (among other things). This function does some computation on the new resolution and then iterates over all the registered listeners in the .SWF, invoking the callback for each so they can query the new resolution and adjust their UI elements.

 

So how does the Environment object in each of the active UI .swfs actually get called when the resolution changes? From what I can see, magic. In the UnrealScript there is a function UIOptionsPCScreen.UpdateViewport that is called when the resolution is changed. This calls UpdateViewportNative, which is a native function that actually does the work, and I can't easily see what happens in there. However, the log reports some interesting things:

 

[0005.02] DevGFxUI: UGFxInteraction::Send - CALLBACK_ViewportResized
[0005.06] DevGFxUI: Environment.SetView: 1280, 720, 0, 0, 0

The first log line looks like it's happening somewhere in the native code, the second is a trace call in the actionscript itself. My guess is that since XCom generally doesn't use Kismet to load the UIs, it uses UnrealScript classes that are subclasses of FxsPanel and friends, there is probably something in the native code associated with those classes. This would keep track of all active instances of FxsPanel and invoke the evironment callback for each movie, which in turn dispatches the event off to all the UI elements in that .swf that have registered themselves with it.

 

Eventually I'll try creating a UI through one of these subclasses instead of through Kismet and see if that works instead. Or rather, I'll probably have kismet invoke some unrealscript to instantiate a subclass of one of these GUI classes instead of using the Open GFx Movie kismet.

 

Lastly, you can enable ActionScript tracing into the normal xcom log by editing DefaultEngine.ini and commenting out the line +Suppress=DevGFxUI. That's how I was able to see those callbacks regarding the viewport change.

[Amineri]

So, you've inspired me to a little poking around of my own. :) All of the following is tracing the second path described by tracktwo -- a possible non-Kismet path.

-----------------

It appears that each UI class (for this example I'll use UIOTS from the strategy game) defines the GFXMovie package that it plays, as well as an associated ID. For UIOTS, this is in the defaultproperties value :

s_package=/ package/gfxOfficerTrainingSchool/OfficerTrainingSchool

s_screenId=gfxOfficerTrainingSchool

This corresponds to the path to the GFXMovie object gfxOfficerTrainingSchool.OfficerTrainingSchool located in UICollection_Strategy_SF.upk.

Loading of new UI screens appears to be handled by UIFxsMovie.LoadScreen :

myValue.Type = 3;

myValue.S = string(screen.GetScreenId());

myArray.AddItem(myValue);

myValue.S = string(screen.GetPackage());

myArray.AddItem(myValue);

myValue.Type = 4;

myValue.B = screen.m_bAnimateIntro;

myArray.AddItem(myValue);

Invoke(string(s_mainClip) $ ".LoadScreen", myArray);

The GetScreenId and GetPackage functions are native functions that return names (which are coerced to strings). I'm presuming that these return the name equivalents of the s_package and s_screenId class variables defined above, which are then coerced back to strings. They might be replacing the "package" portion of the string with the appropriate UICollection package.

The final Invoke line is passing the Screen ID and package info to the s_mainClip component of the default GFxMovie, which is defined in UIFxsMovie as :

s_mainClip=_level0.theInterfaceMgr

MovieInfo=SwfMovie'gfxInterfaceMgr.interfaceMgr'

That is, there is a "master" gfxInterfaceMgr actionscript handler which loads additional GFX movie files. UIFxsMovie extends GFxMoviePlayer, and the MovieInfo variable must be defined in that base UnrealEngine class.

The invocation of LoadScreen happens (in this example) in UIOTS.Init with the line :

manager.LoadScreen(self);

the manager parameter is of type UIFxsMovie. The interfaceMgr GFXMovie package is embedded directly within the XComGame.upk package. I extracted the gfxInterfaceMgr GFX component embedded within XComGame, and within the InterfaceMgr class is the method :

function LoadScreen(id, fileLocation, animateIntro)

For the typical case of no animated intro, the fileLocation parameter (e.g. / package/gfxOfficerTrainingSchool/OfficerTrainingSchool) is used :

_loc4_ = this.createEmptyMovieClip(id,_loc6_);

Bind.swf(_loc4_,fileLocation,{onComplete:DelegateWithParams.create(this,this.onScreenLoaded,id),containedScreenID:id});

The screenID field appears to primarily used to maintain uniqueness. That is, attempting to load a screen with the same screenID will result in the previous instance being destroyed.

if(!(this[String(id)] == undefined))

{

    var _loc5_ = "WARNING: InterfaceMgr already has \'" + id + "\' ... ";

This details how UIOTS maps at the lower levels to load the appropriate GFx movie file.

-------------------

Next, I'll describe how UIOTS itself is created and maintained. Within XComHQPresentationLayer is the protected class variable :

var protected UIOTS m_kOTS;

XComHQPresentationLayer also contains a specific state for the OTS, 'State_OTS'. Invoking XComHQPresentationLayer.UIOTS() pushes this state. This happens in XGBarrackUI.OnChooseMainMenuOption :

if(iNewView == 7)

{

PRES().UIOTS();

}

Within 'State_OTS', the following method instantiates a new UIOTS instance and initiates it :

simulated function Activate()

{

m_kOTS = Spawn(class'UIOTS', self);

m_kOTS.Init(XComPlayerController(Owner), GetHUD());

//return;        

}

That is, invoking PRES().UIOTS() pushes the 'State_OTS', which triggers the spawning/initiating of the UIOTS class. During initialization, the LoadScreen call handles the loading of the movie object.

Each UI<type> class (typically) has an associated XG<type>UI manager class, although in some cases multiple UI classes share a single manager class. Each UI class typically has a GetMgr() method which provides access to the manager class. For UIOTS, this is implemented via :

return XGOTSUI(XComHQPresentationLayer(controllerRef.m_Pres).GetMgr(class'XGOTSUI', (self), m_iView));

The GetMgr implementation in XComHQPresentationLayer has a lazy implementation. It first checks the dynamic array m_arrScreenMgrs for an existing instance of the manager. If it fails to find it, it initializes a new instance of the manager. For UIOTS, the associated manager is XGOTSUI. Note that the UI/Manager paradigm is only used in the strategy layer.

The main difference between the 2 classes is that a UI class inherits from UI_FxsScreen, while a manager class inherits from XGScreenMgr (which in turn inherits from XGStrategyActor). Thus XGScreenMgr has easier access to various strategy game systems.

Trying to follow the Firaxis paradigm with the existing modding capabilities presents a few challenges :

1) Adding a new UI<newname> class. This might be done via mutators, although the UI_FxsScreen base class would need to have a stub created, I think.

2) Adding a new XG<newname>UI manager class. As with the UI class, this could possibly be done via mutators, although more stubs might need to be created.

3) Adding a new protected variable and state to XComHQPresentationLayer. Doing this seems like a much more difficult task.

I'll be pondering ways that this might be worked around...

[tracktwo]

Ah, great, thanks for the investigations! My thinking for how to trigger these, which is complete speculation right now, I haven't attempted it, is to use a blank streaming map with kismet to instantiate a Singleton class that manages the ui object. Or, instead of kismet and the map you could use a mutator maybe to go off and access the Singleton.

 

Creation of stubs for the Fxs classes was on my to-do list for this weekend :)

[tracktwo]

From Amineri's links to the GFxMovie and the gfxInterfaceMgr manager, I think I've figured out how the resolution changes work. Again this is filtered through my limited understanding of ActionScript, so here goes:

 

The LoadScreen functions creates new movie clips to be associated with the screens and panels that represent all the UI elements in the game, eg the OTS or the main menu or whatever. This is the Bind.swf() stuff in LoadScreen. It looks like the UI is organized as the interface manager being the "main" swf, and all other elements added as child movies under this main movie. My first sample colored rectangle movie doesn't do this - it creates a whole new movie completely independent of the ui manager.

 

I think the way name resolution works in ActionScript this means that all of these swfs in the hierarchy will share some name scoping rules. References to Environment.instance() will all refer to the instance of the "root" movie. Actual resolution changes are handled by UIFxsMovie.SetResolutionAndSafeArea, which invokes .SetUIView on the interface manager, which in turn calls Environment.instance().setView(), which then triggers all the notifications for all children. Since my little rectangle movie creates its own movie separate from the interface manager, my references to Environment will be referring to the one I have in my movie, not the one in the master movie, which is why nothing works :smile:

 

Next step: move my little movie out of of Kismet and into a UI_FxsScreen sub-class and try to get it to work.

Edited July 10, 2015 by tracktwo

[tracktwo]

I've successfully loaded my sample movie through a UI_FxsScreen subclass, and it looks like this is correctly registering itself as a sub-movie in the main interface manager. I haven't tested the screen resolution change callback yet, but it looks like it is using the root Environment class instead of the one I injected.

 

Here's how I did it:

 

I created two unrealscript classes, a UI class (UICampaignSummary), and a manager class (XGCampaignSummaryMgr).

 

The UI class extends UI_FxsScreen and sets the s_package and s_screenId member variables to the desired settings. For example, I used gfxCampaignSummary as the screenId and "/ package/gfxCampaignSummary/CampaignSummary" as the package. It also defines some functions that are needed for initialization and shutdown. I only implemented Init() and OnInit(), this was enough for now, but I'll need to add more functionality later. So it looks like this:

 

 

 

class UICampaignSummary extends UI_FxsScreen
    hidecategories(Navigation)
    implements(IScreenMgrInterface);
   
simulated function Init(XComPlayerController _controllerRef, UIFxsMovie _manager)
{
    BaseInit(_controllerRef, _manager);
    _manager.LoadScreen(self);
    Show();
}

simulated function OnInit()
{
    super.OnInit();
}

defaultproperties
{
    s_package="/ package/gfxCampaignSummary/CampaignSummary"
    s_screenId="gfxCampaignSummary"
}

 

 

 

the Mgr class extends Actor and contains a variable of type UICampaignSummary. I created a function Create() that spawns the UI object and initializes it. This manager class can act as the corresponding Manager class as Amineri described above, but for now I am mostly using it as a stand-in for the presentation layer. Normally instances of these UI element classes are members of a presentation layer class, e.g. XComHQPresentationLayer. Adding new member variables to the class is a fair bit of work and hex modding, so I didn't bother with it. One thing the PresentationLayer does have is a reference to the UIInterfaceMgr object that is basically the master UI flash object. This is accessible from the GetHUD() function in the presentation layer, so I just get this by accessing the presentation layer through the game controller. My basic manager class looks like this:

 

 

 

class XGCampaignSummaryMgr extends Actor;

var protected UICampaignSummary _ui;

simulated function UIInterfaceMgr GetHUD()
{
    return XComHQPresentationLayer(XComHeadquartersController(XComHeadquartersGame(class'Engine'.static.GetCurrentWorldInfo().Game).PlayerController).m_Pres).GetHUD();
}

function Create()
{
    _ui = Spawn(class'UICampaignSummary',self);
    _ui.Init(XComPlayerController(Owner), GetHUD());
}

defaultproperties
{
    _ui=none
}

 

 

 

Creating these two classes required creating a whole bunch of stub classes in the XComGame and XComStrategyGame packages in the UDK just so everything would compile.

 

The next step is actually getting the game to load the UI. Again I'm trying to minimize hex modding, and especially minimizing the need to inject new names and import entries into XcomGame or XcomStrategyGame. So the solution I went with is to keep using the streaming map model and kismet, but now kismet doesn't create the movie, it just calls into UnrealScript to invoke functions on the manager object. So, in my streaming level I deleted all the kismet nodes for dealing with the UI, and instead dropped an actor of the XGCampaignSummaryMgr class into the map. When the map is loaded, this will spawn an instance of this actor (which is invisible, so it doesn't matter where it is). Then in Kismet, I need to call the Create() function on that actor. So I created a kismet object variable linked to that actor I dropped, and created a new sequence action "SeqAct_CreateCampaignSummaryScreen", tying the actor variable into the input to this node. The actual sequence action looks like this:

 

 

 

class SeqAct_CreateCampaignSummaryScreen extends SequenceAction
    hidecategories(Object)
    forcescriptorder(true);
   
var() XGCampaignSummaryMgr CSActor;

event Activated()
{
    CSActor.Create();
    OutputLinks[0].bHasImpulse = true;
}

defaultproperties
{
    bCallHandler=false
    VariableLinks(0)=(ExpectedType=Class'Engine.SeqVar_Object',LinkDesc="Actor", PropertyName=CSActor)
    ObjName="Create UICampaignSummary"
}

 

 

 

Then it was just a matter of importing the .swf into the UDK int the gfxCampaignSummary package with the name CampaignSummary, cooking everything, copying the resulting map and other packages into the CookedPCConsole folder, and loading the gfxCampaignSummary package in defaultcontent.ini. Started up the game and my pre-existing "Campaign Summary" menu option that streams the level worked: it streamed the map, which created the "manager" actor, and then kismet invoked the sequence action that invoked Create, which spawned and initialized my movie.

 

Lots of the grunt work of creating all these stub classes and sequence actions is now done, so adding new ones should be more straightforward.

Edited July 11, 2015 by tracktwo