R&D XCOM Map Alterations

[Amineri]

I think that everyone agrees that one of the biggest things in XCOM is the repetition of the maps.

 

According to various sources (sorry I don't have links), Jake Solomon VERY much wanted to have randomized maps in the XCOM:EU release but was told that it just wasn't a practical thing (I think I recall seeing somewhere that it was a choice between randomized maps and destructible maps?).

 

So, apparently in the upcoming XCOM:EW expansion there aren't going to be a million new maps. In XCOM:EU there are between 59 to 68 unique maps (some are slight variations of each other), and the Slingshot DLC adds 3 additional maps that are only available on specific Council Missions. (source http://www.ufopaedia.org/index.php?title=Maps_(EU2012))

 

Apparently the new expansion will contain 47 new maps (source http://www.ufopaedia.org/index.php?title=XCOM:_Enemy_Within_DLC_(EU2012)), which doesn't quite double the number of maps, only ~70% more diversity.

 

I decided to try and dig in a bit to try and understand more about the maps after seeing an earlier post that the maps contain fair bit of the UI actionscript. It got me thinking about what else might be in the map files.

 

For my initial testing I started out with the "URB_Bar.upk" map. The base install size is 19.6 MB, which is fairly hefty considering that the compressed XComTacticalGame.upk is only ~7MB.

 

Unsurprisingly the URB_Bar.upk map is compressed. The usual UE Decompressor (from Gildor) decompresses the unreal package without problem. The decompressed package file is 35.0 MB.

 

I was able to open and the decompressed URB_Bar.upk file in the umodel Unreal model viewer program. The only additional file I seemed to need was the lighting.tfc file.

 

The URB_Bar.upk file contains copies of all of the art assets used in the map. Everything from buildings to fire hydrants, video game machines to delivery vans. It appears that the art assets for each map are packaged up and included within each map upk, which explains why these files are so large.

 

This makes the maps very modular (I think that in theory even if you only had 1 map that map would play normally). It also makes it so that new assets could be included in new maps without modifying the rest of the game (the Slingshot DLC in particular has some unique art assets in its maps).

 

The URB_Bar.upk also contained copies of many of the weapon FX, such as plasma bolts hitting something, as well as some of the psionic special effects. It also contained a copy of all of the perk/ability icons, as well as other UI icons. XMarksTheSpot had reported earlier that the file contains actionscript used to create most of the tactical game UI.

 

Each map doesn't appear to contain ALL of the various map art assets used, however. The URB_Bar.upk for example did not contain any trees -- there aren't any trees in this map. I suspect that there is a common library of art assets that are available to the map creators at Firaxis. Each map developer pulls the art assets in for each map as it is created. After completion the art assets are all packaged together into the map upk.

 

This means that some assets will be duplicated across several maps. For instance a particular car may be present in multiple maps -- the car model and animation assets for it exploding would be duplicated in each mapfile.

 

It is possible to open the decompressed map file in UE Explorer, but there didn't appear to be any decompile-able code for UE Explorer to work with.

 

Finally, it is possible to use the Unreal Engine extractor to break apart the decompressed mapfile. I did this with the URB_Bar.upk and now have a very large folder full of stuff to try and parse through.

 

---------------------

 

The first thing I'm going to play with is trying to track down how the art assets are placed on the map in order to try and modify their placement.

 

One issue is going to be distribution of such maps. Since they are highly modular and contain a lot of Firaxis art assets I'd be leery of distributing a full map upk.

 

Am open to thoughts on any of this...

[Amineri]

One early interesting find is the GameData_Toughness folder. This was created by the UE Extractor.

 

For URB_Bar.upk it lists the following materials. Looking though the files with HxD they appear to have the toughness as defined:

 

Wood (Destructible Actor 0x16) : toughness 0x28 = 40

Trucks (Destructible Actor 0x15) : toughness 0x1F4 = 500

Metal (Destructible Actor 0x0E) : tougness 0x5A = 90

Small Car (Destructible Actor 0x12) : toughness 0x1F4 = 500

Plastic (Destructible Actor 0x0F) : toughness 0x32 = 50

Masonry (Destructible Actor 0x0C) : toughness 0x50 = 80

Glass (Destructible Actor 0x09) : toughness 0x0A = 10

Foliage (Destructible Actor 0x08) : toughness 0x14 = 20

Deco (Destructible Actor 0x06) : toughness 0x0A = 10

Cloth (Destructible Actor 0x04) : toughness 0x14 = 20

 

Presumable there are more destructable actor types (like trees and UFO walls), but these are the only actors in the URB_Bar.upk

 

Presumably these could be modded (although I'm not yet sure how). If nothing else they provide a guide in how to set/mod the environmental damage stats of various weapons (in particular cars and trucks have toughness 500 which explains why a rocket with 500 env damage destroys them, but that it takes 2 grenades each with 250 environmental damage to make them explode).

[Amineri]

The primary "class" (or maybe it's more correct to call it an object) that provides the high level information is labeled "TheWorld".

 

After extraction it contained a "PersistentLevel.Level" file that is 320 KB. It also had a folder "PersistentLevel" that contains a wealth of difference parts.

 

18 spawnpoint files of the format:

XComSpawnPoint_##.XComSpawnPoint

 

A couple hundred XComLevelActor files

 

7 XComInteractiveLevelActor files (presumably the doors in URB_Bar.upk)

 

Several hundred XComDestructibleActor files

 

18 XComFloorVolume files

 

43 XComFracLevelActor files

 

5 AmbientSoundSimple files

 

Other file types include:

VisGroupActor

StaticMeshActor

SpotLight

PreFabInstance

PointLight

PointInSpace

GeneratedMeshAreaLight

Emitter

DecalActor

 

--------------

 

Needless to say these maps look to be pretty complex constructions, which would explain why map randomization was rejected and why the new expansion looks to have a fairly limited number of maps (if the 47 number is true).

 

Hopefully someone more familiar with how Unreal Engine maps are created can chime in here.

[XMarksTheSpot]

This is all pretty interesting :smile: Kinda wasteful approach Firaxis took there with packaging all that redundant stuff into the level files :ermm:

 

I suspect what you stumbled upon in your last post are references to Kismet nodes, the Unreal Engine's visual scripting system which is separate from UnrealScript (and may, in fact, replace UnrealScript in future versions of the engine, from what I've heard).

I don't know if there are any Kismet editors around that work on packaged game files, from what I can gather there is limited functionality for that in a modding tool for Mass Effect 3, ME3Explorer. You could try checking out their forums for some additional pointers, e.g. this thread.

[Amineri]

I suspect that part of the reason for packaging up the maps was for business reasons -- to allow maps to be sold as DLC. Since each map contains all of the assets needed to run that map, it makes it easier for the devs to sell maps a la carte.

 

Also, the publicly given reason for the non-randomized maps was that to add the destructable terrain. I'm wondering if including the destructible actors in the map file is required in order to be able to transform it into another destructible actor.

 

Given that the spawn points appear to be defined in the UnrealScript unit (96 per tile) it makes sense that all of the destructable actor locations would be specified in the same way.

 

---------------

 

The big trick would be to try and figure out where the actors (both static and destructible) are instantiated and each instance's location defined.

 

When I was looking through the URB_Bar.upk, the cargo van's weren't defined as a single unit but as the cab, base, side and roof. I think this is what allows the player to move units up in to the area exposed if the sides/back are destroyed. However, there are two cargo vans in the Bar map, but only one prototype mesh appears to be defined in the upk. So, somewhere the two copies of the van parts must be getting created, and with separate locations.

 

If that can be figured out then at least each map could be re-arranged, although there would be definite limitations based on which art assets were included with each of the already-published maps. I also wouldn't be surprised if some of the maps had a few assets still stored that didn't make it into the displayed map.

[bokauk]

I had a look into creating a map editor as part of ToolBoks a few months ago and had a little bit of success moving some entities around, including spawn points.

As you have also discovered, only existing assets already in the map can be used and attempting to add assets to the map would be extremely difficult. This kind of killed my interest in a map editor, although I'm sure different spawn locations would be good for some mods.

I think you might have to move buildings around on a tile-by-tile bases. It's easier to move entities such as cars, but from what I remember, even they are made up of several parts which you have to manually edit.

I too picked URB_Bar for my experiments :smile:

[Amineri]

Finally decided to re-visit this topic.

 

I decompressed a different map, LSmallScout_River.upk. Definitely only some of the map information is showing up in UE Explorer, as it's not really designed for viewing UE levels. Even though we call them maps, in Unreal terminology they are levels.

 

Even decompressed, the XCOM maps/levels are 'cooked'. Being an Unreal neophyte, I did some research and discovered that cooking an unreal package optimizes it for a particular hardware architecture (e.g. PC vs PS3 vs XBOX 360 vs Mac). Consoles can only run cooked unreal files, while PCs can run a mixture of both cooked and uncooked. Cooking for the PC does however improve performance, so all of the XCOM released files for PC are cooked. The big hint is in the folder name PCCookedConsole. ;)

 

Now, by explicit design the Unreal level editor cannot edit cooked level files. Apparently it would make cheating on and Unreal-based multiplayer 3D shooter pretty easy, so that's just something that has to be lived with.

 

In theory a cooked level can be opened in the UE SDK, but cannot be saved. I haven't tested this theory, though.

 

----------------------------------

 

On another note, all of the possible maps available are configured in the DefaultMaps.ini file. This is not stored in the xcomgame.exe Resource Cache, but appears to be read from the file. However, users with any DLC will have a merged version of the file in the My Games folder. My merged file includes the three Slingshot DLC maps.

 

One thing of particular interest to me is that the Council special mission maps that include dynamic aliens (those aliens that drop in), have that dynamic alien defined in the DefaultMap.ini config file.

 

For example:

Trainyard Bomb Disposal mission:

Maps=(	MapFamily=URB_Trainyard,			Name=URB_Trainyard,					DisplayName="URB_Trainyard_Bomb (Mission Variant Bomb Defusal Urban Trainyard)",			MissionType=eMission_Special,		CouncilType=eFCMType_Bomb,		bInRotation=false, 	TimeOfDay=eMissionTime_Day, 	ERegion=eMissionRegion_None,		StreamingMaps[0]=(MapName=URB_Trainyard_Stream, 	Loc=(X=0,Y=0,Z=0), Rot=(Yaw=0)),  StreamingMaps[1]=(MapName=CIN_DropshipIntros, 	Loc=(X=0,Y=0,Z=0), Rot=(Yaw=0)),		DynamicAliens[0]=eChar_ThinMan)

The defined dynamic alien is a Thin Man. This is true for all of the vanilla maps.

 

The only exception I noted is in the Confounding Light DLC map, which has:

 

Maps=( MapFamily=URB_TrainStation, Name=DLC1_2_CnfndLight, DisplayName="DLC1_2_CnfndLight", MissionType=eMission_Special, bInRotation=false, TimeOfDay=eMissionTime_Night, ERegion=eMissionRegion_Asia, StreamingMaps[0]=(MapName=DLC1_2_CnfndLight_Stream, Loc=(X=0,Y=0,Z=0), Rot=(Yaw=0)), StreamingMaps[1]=(MapName=CIN_DLC1_DropshipIntros, Loc=(X=0,Y=0,Z=0), Rot=(Yaw=0)), DynamicAliens[0]=eChar_Muton)

 

The defined dynamic alien is a muton, which is consistent with how the map plays.

 

Now to the really interesting part!

 

This dynamic alien appears to actually be used in the function XComMapManager.AddMapDynamicContent, which is one of the few functions which isn't a native function :

static simulated function AddMapDynamicContent(string MapDisplayName, out TPawnContent Content)
{
    local XComMapMetaData MapData;
    local int I;

    GetMapInfoFromDisplayName(MapDisplayName, MapData);
    // End:0x18C
    if(MapData.DynamicAliens[0] != 0)
    {
        I = 0;
        J0x50:
        // End:0x18C [Loop If]
        if(I < 2)
        {
            // End:0x17E
            if(MapData.DynamicAliens[I] != 0)
            {
                Content.arrAlienPawns.AddItem(class'XGGameData'.static.MapCharacterToPawn(MapData.DynamicAliens[I]));
                Content.arrAlienPawns.AddItem(class'XGGameData'.static.MapCharacterToPawn(class'XGBattleDesc'.static.GetSecondaryAlienForContentLoading(MapData.DynamicAliens[I])));
            }
            ++ I;
            // [Loop Continue]
            goto J0x50;
        }
    }
    //return;    
}

Unfortunately this function only appears to be used to load the pawn's mesh/animation content (failing to do this will cause the game to crash, as I discovered with the "more aliens" mod).

 

The alternative appears to be to structure the Council mission maps more. Instead of randomizing the Council missions, they could be set up on a "easiest" to "hardest" range. Easier maps would appear nearer the beginning of the game and harder maps nearer the end of the game. This would allow for configuring harder aliens directly in the DefaultMaps.ini file.

 

For example, there are five bomb mission maps (Big Cemetary, Highway Bridge, Slaughterhouse, Train Station, and Trainyard). These could be broken into an early/mid map, 2 mid/late maps, and 2 late maps. The early/mid map could use Thin Men (but would only be available from April to maybe May), the mid/late maps could use Mutons (and be available May through August), while the late maps would drop in Muton Elites.

 

I don't know if it's possible to somehow redefine the same map multiple times, but with different dynamic aliens. Will have to look into it.

[bokauk]

Nice find, this could be interesting, definitely worth a further look :smile:

Not sure if you already know, but you can duplicate a map just by copying the map entry in:
\My Games\XCOM - Enemy Unknown\XComGame\Config\XComMaps.ini

Give it a different name: "Name=URB_Trainyard2"

Then make a copy of the map's UPK file and rename it to the new name.

You should then be able to just change the dynamic alien in XComMaps.ini for the map you just duplicated :smile:

Not sure if this is what you mean or if I'm just stating the obvious. I've been out the loop for a while now :blush:

[Amineri]

Nice find, this could be interesting, definitely worth a further look :smile:

 

Not sure if you already know, but you can duplicate a map just by copying the map entry in:

\My Games\XCOM - Enemy Unknown\XComGame\Config\XComMaps.ini

 

Give it a different name: "Name=URB_Trainyard2"

 

Then make a copy of the map's UPK file and rename it to the new name.

 

You should then be able to just change the dynamic alien in XComMaps.ini for the map you just duplicated :smile:

 

Not sure if this is what you mean or if I'm just stating the obvious. I've been out the loop for a while now :blush:

 

This is exactly what I was thinking of (just hadn't actually tested it yet).

 

This would allow for duplications of the council mission maps but with different dynamic aliens.

 

The only further patch-up required would be in the Council mission map selection function XGFundingCouncil.BuildMission. This function defines the map name (e.g. kMission.strMapName = "URB_StreetOverpass_Assault (Mission Variant Assault on Overpass)";) based on EFCMission. A little extra logic to append something to the mapname based on desired dynamic alien should work perfectly.

 

In fact it might be possible to make multiple Maps= entries that reference the same map upk file but have different display names, which would definitely be more disc-space friendly. Will report more after I do some testing.

[Amineri]

Potentially some progress in altering the alien spawn point locations.

 

Following the same general steps that Bertilsson used to alter the XCOM units' spawn points (except the 59th byte part of step 3):

 

1. Decompress.exe URB_Demolition.upk
2. Extract.exe URB_Demolition.upk
3. Open any of ..\URB_Demolition\TheWorld\PersistentLevel\XComSpawnPoint_11.XComSpawnPoint to XComSpawnPoint_16.XComSpawnPoint which contains location of a spawnpoint at the 59:th byte, in each file, exactly as predicted.
4. Copy the entire hex-contents
5. Open the decompressed URB_Demolition in HxD
6. Find the copied hex-contents and modify to liking.

 

The alien spawn points appear to be changable as well.

 

I'm pretty sure that the alien spawn points are in the objects starting with XComAlienPod_#. There are child classes for this, so on some maps these can be appended with an extra descriptor (e.g. XComAlienPod_Abduction_#). These appear to correspond to the classes in XComGame.upk.

 

The LSmallScout_River.upk map has 5 XComAlienPod_# objects (0-4).

The URB_Bar.upk map has 5 XComAlienPod_Abduction_# objects (0-4), which makes sense since this is an abduction map.

In contrast the AlienBase01.upk map has 13 XComAlienPod_# objects (0, 7, 8, 13-15, 17, 30-35).

 

I haven't yet drilled down to determine where the Location is in each file (and it may be in a different hex location in the _Abduction cases). Also, not all of the XComAlienPod objects are the same size, which is potentially problematic.

 

Aliens spawnpoints are computed using the XComAlienPod.GetSpawnPoint function :

function XComSpawnPoint_Alien GetSpawnPoint(int iAlien, out Vector vLoc_Out, optional bool bUseDefault)
{
    local XComSpawnPoint_Alien kSpawnPoint;

    bUseDefault = false;
    if(!bUseDefault && CustomStartLocations.Length != 0)
    {
        if(iAlien < CustomStartLocations.Length)
        {
            vLoc_Out = CustomStartLocations[iAlien].Location;
            vLoc_Out = XComTacticalGRI(WorldInfo.GRI).GetClosestValidLocation(vLoc_Out, none,, false);
            kSpawnPoint = Spawn(class'XComSpawnPoint_Alien',,, vLoc_Out, rotator(Location - vLoc_Out),, true);
            return kSpawnPoint;
        }
    }
    else
    {
        if(iAlien < NumAliens)
        {
            vLoc_Out = GetDistributedLocationAround(Location, iAlien, Rotation, true);
            kSpawnPoint = Spawn(class'XComSpawnPoint_Alien',,, vLoc_Out, rotator(Location - vLoc_Out),, true);
            return kSpawnPoint;
        }
    }

The first branch appears to be for special CustomStartLocations (probably for scripted missions like the DLC and Temple Ship missions).

 

Most aliens on most missions are likely distributed using the second branch. The GetDistributedLocationAround function places each alien around XComAlienPod.Location (aliens are limited to being in a 3x3 tile area centered on Location).

 

So technically there aren't alien spawn points in each map (not like XCOM spawn points). XCOM unit spawn points are 1 per unit. Alien spawns are defined as 1 XComAlienPod object per pod, and the actual spawn points are computed on-the-fly.

 

Location should be a built-in vector-type element that should be in the XComAlienPod object in the map file.