Jump to content

Save game file format?

Elodran

By Elodran
in Enemy Unknown

 Share

Recommended Posts

FogGene Apprentice

FogGene

Posted
Posted (edited)

I've taken a look at the Package Loading part of Content Streaming, but it doesn't seem like the structures match. The whole save structure might be a match to the package format, though. I think you suggested this before. in that case perhaps the collection of Index Tables altogether is an export table?

 

I'm at a loss here, to be honest. This below is an example of an Index Table collection header; as in a single header for all tables. So far none of the data there has helped me figure out a way to get start and end offsets for the collection, which is what I'd need to reliably skip the content (that or the actual format).

 

 

 

Offset(h) 00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F

002077F0                                   22 00 00 00 55             "...U
00207800  52 42 5F 54 72 61 69 6E 79 61 72 64 20 2D 20 30  RB_Trainyard - 0
00207810  31 2F 31 37 2F 31 34 20 32 32 3A 31 34 3A 35 32  1/17/14 22:14:52
00207820  00 42 00 00 00 55 52 42 5F 54 72 61 69 6E 79 61  .B...URB_Trainya
00207830  72 64 5F 42 6F 6D 62 20 28 4D 69 73 73 69 6F 6E  rd_Bomb (Mission
00207840  20 56 61 72 69 61 6E 74 20 42 6F 6D 62 20 44 65   Variant Bomb De
00207850  66 75 73 61 6C 20 55 72 62 61 6E 20 54 72 61 69  fusal Urban Trai
00207860  6E 79 61 72 64 29 00 AE 16 00 00 00 00 00 20 F0  nyard).®...... ð
00207870  C4 00 10 28 C5 00 00 80 BF 00 10 04 45 00 10 1C  Ä..(Å..€¿...E...
00207880  45 00 40 40 44 01 2A 00 00 00 36 00 00 00 0C 00  E.@@D.*...6.....
00207890  00 00 50 6A 00 00 FF FF FF FF FF FF FF FF FF FF  ..Pj..ÿÿÿÿÿÿÿÿÿÿ
002078A0  FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF  ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ
002078B0  FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF  ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ
002078C0  FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF  ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ
002078D0  FF FF FF FF FF FF 00 00 00 00 00 00 00 00 00 00  ÿÿÿÿÿÿ..........
002078E0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
002078F0  00 00 00 00 00 00 FF FF FF FF FF FF FF FF 05 00  ......ÿÿÿÿÿÿÿÿ..
00207900  00 00 4E 6F 6E 65 00 00 00 00 00 FF FF FF FF FF  ..None.....ÿÿÿÿÿ
00207910  FF FF FF 4D 00 00 00 4C 00 00 00 00 00 00 00 00  ÿÿÿM...L........
00207920  00 00 00 00 00 C0 43 00 00 00 00 FF FF FF FF FF  .....ÀC....ÿÿÿÿÿ
00207930  FF FF FF 05 00 00 00 4E 6F 6E 65 00 00 00 00 00  ÿÿÿ....None.....
00207940  FF FF FF FF FF FF FF FF AF 16 00 00              ÿÿÿÿÿÿÿÿ¯...

 

 

 

Edit: code in hexbase; added missing byte.

Edited by FogGene
Link to comment
Share on other sites
wghost81 Mentor

wghost81

Posted
Posted (edited)

UDN info does not contain format description, it describes the logic behind package loading. All the info I was able to gather on XCOM upk format is summarized here: http://www.nexusmods.com/xcom/download/1000003820. From the information you posted, saves have similar, but not exact, structure. What I mean is that saves contain a bunch of concatenated serialized data. Mostly properties, which is understandable. It also should contain name list (and it in fact does). It probably has a very different header (ad it does), but may have a map package embedded, as it will be the easiest way to save map state. Not the whole map, probably TheWorld.PersistentLevel objects, as they are persistent by their name :smile: and should be saved. I also see WorldInfo object on your screens, which is a part of all map packages too.

 

The unknown data in packages I was talking about is located between NumCompressedChunks and NameOffset. It looks similar to the example you've posted before.

Edited by wghost81
Link to comment
Share on other sites
FogGene Apprentice

FogGene

Posted
Posted

It looks like it will take me a while to figure out that list.

 

On the positive side, the GUI and I are reaching an understanding, so now I can display (and edit) object properties individually. Now if I only I knew how to display all properties together...

 

 

 

http://i.imgur.com/QikwJc2.png

 

 

Link to comment
Share on other sites
wghost81 Mentor

wghost81

Posted
Posted

I see PersistentLevel objects. :smile: Its the map, alright. :smile: And this is how that pods looks inside the "inactive" map (i.e. in map package):

 

0x00002664 (9828): XComAlienPod'TheWorld.PersistentLevel.XComAlienPod_0'
	TypeRef: 0xFFFFFF63 -> XComAlienPod
	ParentClassRef: 0x00000000 -> 
	OwnerRef: 0x0000091F -> PersistentLevel
	NameIdx: 0x00000ABD (Index) 0x00000001 (Numeric) -> XComAlienPod_0
	ArchetypeRef: 0x00000000 -> 
	ObjectFlagsH: 0x00000000
	ObjectFlagsL: 0x02070001
		0x00000001: Transactional
		0x00010000: LoadForClient
		0x00020000: LoadForServer
		0x00040000: LoadForEdit
		0x02000000: HasStack
	SerialSize: 0x00000142 (322)
	SerialOffset: 0x015C95F4
	ExportFlags: 0x00000000
	NetObjectCount: 0
	GUID: 00000000000000000000000000000000
	Unknown1: 0x00000000
UObject:
	PrevObjRef = 0xFFFFFF63 -> XComAlienPod
Can't deserialize stack: skipping!
UDefaultPropertiesList:
UDefaultProperty:
	NameIdx: 0x00000813 (Index) 0x00000000 (Numeric) -> PodIndex
	TypeIdx: 0x000005D2 (Index) 0x00000000 (Numeric) -> IntProperty
	PropertySize: 0x00000004
	ArrayIdx: 0x00000000
	Integer: 0x00000000 = 0
UDefaultProperty:
	NameIdx: 0x00000612 (Index) 0x00000000 (Numeric) -> LightEnvironment
	TypeIdx: 0x00000767 (Index) 0x00000000 (Numeric) -> ObjectProperty
	PropertySize: 0x00000004
	ArrayIdx: 0x00000000
	Object: 0x000001FF = DynamicLightEnvironmentComponent_313
UDefaultProperty:
	NameIdx: 0x0000075D (Index) 0x00000000 (Numeric) -> NumAliens_Min
	TypeIdx: 0x000005D2 (Index) 0x00000000 (Numeric) -> IntProperty
	PropertySize: 0x00000004
	ArrayIdx: 0x00000000
	Integer: 0x00000001 = 1
UDefaultProperty:
	NameIdx: 0x0000075C (Index) 0x00000000 (Numeric) -> NumAliens_Max
	TypeIdx: 0x000005D2 (Index) 0x00000000 (Numeric) -> IntProperty
	PropertySize: 0x00000004
	ArrayIdx: 0x00000000
	Integer: 0x00000003 = 3
UDefaultProperty:
	NameIdx: 0x00000814 (Index) 0x00000000 (Numeric) -> PodMesh
	TypeIdx: 0x00000767 (Index) 0x00000000 (Numeric) -> ObjectProperty
	PropertySize: 0x00000004
	ArrayIdx: 0x00000000
	Object: 0x00001D5D = StaticMeshComponent_1
UDefaultProperty:
	NameIdx: 0x000001FC (Index) 0x00000000 (Numeric) -> CenterpieceMesh
	TypeIdx: 0x00000767 (Index) 0x00000000 (Numeric) -> ObjectProperty
	PropertySize: 0x00000004
	ArrayIdx: 0x00000000
	Object: 0x00001D5E = StaticMeshComponent_2
UDefaultProperty:
	NameIdx: 0x00000630 (Index) 0x00000000 (Numeric) -> Location
	TypeIdx: 0x0000096C (Index) 0x00000000 (Numeric) -> StructProperty
	PropertySize: 0x0000000C
	ArrayIdx: 0x00000000
	InnerNameIdx: 0x00000A60 (Index) 0x00000000 (Numeric) -> Vector
	Vector (X, Y, Z) = (0xC5804314, 0x4306E696, 0x43C22751) = (-4104.38, 134.901, 388.307)
UDefaultProperty:
	NameIdx: 0x00000894 (Index) 0x00000000 (Numeric) -> Rotation
	TypeIdx: 0x0000096C (Index) 0x00000000 (Numeric) -> StructProperty
	PropertySize: 0x0000000C
	ArrayIdx: 0x00000000
	InnerNameIdx: 0x00000899 (Index) 0x00000000 (Numeric) -> Rotator
	Rotator (Pitch, Yaw, Roll) = (0x00000000, 0xFFFFC000, 0x00000000) = (0, 4294950912, 0)
UDefaultProperty:
	NameIdx: 0x000009E4 (Index) 0x00000000 (Numeric) -> Tag
	TypeIdx: 0x00000740 (Index) 0x00000000 (Numeric) -> NameProperty
	PropertySize: 0x00000008
	ArrayIdx: 0x00000000
	Name: 0x00000ABD (Index) 0x00000000 (Numeric) = XComAlienPod
UDefaultProperty:
	NameIdx: 0x00000750 (Index) 0x00000000 (Numeric) -> None
Stream relative position (debug info): 0x00000142 (322)

 

That group of unknown entries is actually a stack and I don't know anything about it, except it has fixed 22 bytes size in upk files and present if HasStack flag is set. Here are UPKUtils sources, properties are handled in the beginning of UObject.cpp.

 

Every object on your screen-shot is a world object instance, saved as default properties list. Quite the same as in regular packages, but contains all the CheckpointRecord variables.

Link to comment
Share on other sites
FogGene Apprentice

FogGene

Posted
Posted (edited)

Maybe I'm wrong, but I don't think we're talking about the same thing.

 

I've checked your code, and while it is very nice to have it as a reference, properties are not the problem. My problem was, and still is, the index tables. I have not found a tool yet that can take on those, or a known format that comes even close to defining their structures. Then again, I haven't had much time lately to spend on this.

 

They also cannot be skipped easily. Block sizes are variable and undefined; at first I thought the offset for each individual chunk might be calculated as a function of its index position in the list, based on each table's index distribution. But no luck so far. Signature search won't work consistently if the last table is not empty, not that I want to do it that way anyhow.

 

My properties screenshot, the persistent level objects, was just to show current progress in property code (outdated now anyways, but irrelevant). I posted an example screen of hex data for one index table that looked like this:

 

 

 

http://i.imgur.com/8r3boA3.png

 

 

 

That's just a very small sample, though. If you're curious about this, I have a couple of files with a whole index table collection. One with 21 tables and the other with 30; each table can have from 0 to more than 5k entries, depending on the save; each entry is accompanied by its respective compressed(?) data block.

 

https://skydrive.live.com/redir?resid=DF1F7772939FE5B5!451&authkey=!AJMWDFt74uzq5DM&ithint=folder%2c

Edited by FogGene
Link to comment
Share on other sites
dubiousintent Grand Master

dubiousintent

Posted
Posted

Have you tried Amineri's UPK-Modder tool, which is able to parse and update the UPK table indexes? Her source is available there as well (but it's in Java).

 

The discussion in the 'R&D - New Modding Tools' thread is the basis of the 'UPK File Format' article, which covers that structure's Table Indexing. I think that is what wghost81 is referencing, as she contributed much of the information. Hopefully that structure may provide some insight into your problem.

 

-Dubious-

Link to comment
Share on other sites
wghost81 Mentor

wghost81

Posted
Posted

FogGene, serialized data contain not only properties, but the whole objects. TheWorld.PersistentLevel is an object of yet unidentified format which contains what you call an "index table". Here's an example:

http://i.imgur.com/W0mfu3a.png

You can not tread save file as a simple list of default properties. There have to be import/export tables, dubiousintent mentioned, with a full info on all objects.

Link to comment
Share on other sites
FogGene Apprentice

FogGene

Posted
Posted (edited)

OK. I think I see what you guys are saying. What I'm calling an Index Table would be then a list of unidentified objects.

 

If so, what I'd have to do is find each object (by index or some other means) in the UPK's Exports Table and get, at least, its size. Luckily for me, from what I read in the R&D - New Modding Tools thread, it seems you guys have already figured that stuff out. Theoretically then, I'd only have to use your code to read the object's size for each of my indexed entries and skip forward. Or I may be understanding this wrong yet again (perks of not being a programmer and not knowing the UE, I imagine).

 

In any case, the first thing to figure out would be what those indexes are to each object in the save (entry in namelist? entry in objectlist? object index?, else?). Then use that info to do the above.

 

That sounds like fun. And likely also beyond my limited capabilities. This was a nice learning exercise, anyhow :smile:

 

 

Edit: Thank you to all who contributed in this thread for your help and your patience.

Edited by FogGene
Link to comment
Share on other sites
  • 10 months later...
kelnor277 Newbie

kelnor277

Posted
Posted

I'm assuming this isn't a Necro since it's a thread that's referenced in the SaveGame file format wiki.

 

I've created a program that extracts Roster data from an In Base game save (Does not work on a save generated while in a mission yet. I think know why it doesn't work but I'm not ready to tackle that problem yet.)

Eventually the program will be a way to view your soldiers and sort, filter, etc.

 

I've only tested it on one save at this point, and it's a Long War save.

 

Still I figure viewing the code might help people decode the save files for their own purposes.

 

Some extra resources and tips and tricks.

 

I used Eliot's UE Exporer to get the definitions of the in game structures that were saved to the save file.

It really helps if you because now you know what the object looks like once instantiated in the game.

It seems that any object with a Checkpointrecord struct will serialize everything in the struct when saved.

 

The 010 Editor really helped me reverse engineer the save file. Especially the C based templates feature.

 

Also this: http://me3explorer.freeforums.org/xcom-eu-ew-save-structure-t978-10.html

Some guy managed to decode a lot of properties structures.

 

My source: https://github.com/Kelnor277/Xcom2012SoldierViewer

Link to comment
Share on other sites
 Share
Go to topic listing

  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...