wghost81 Posted December 8, 2013 Share Posted December 8, 2013 (edited) Since wiki is still down, I'll start here. Don't like wiki formatting, anyway. :smile:I want to document UPK file format, so it will be easier for anyone to implement their own tools. Here's what I know so far.UPK formatUPK file consists of:package header name list object list (aka export list) import list actual objects dataTechnically, lists are part of the header, as header size counts lists in. Header structure int32 Signatureint16 Versionint16 LicenseVersionint32 HeaderSizeint32 FolderNameLengthnull-terminated string FolderNameint32 PackageFlagsint32 NameCountint32 NameOffsetint32 ExportCountint32 ExportOffsetint32 ImportCountint32 ImportOffsetint32 DependsOffsetint32 Unknown1 (seems to be equal to HeaderSize)int32 Unknown2int32 Unknown3int32 Unknown4int32x4 GUIDint32 GenerationsCountint32x3xGenerationsCount generationsint32 EngineVersionint32 CookerVersionint32 CompressionFlagsvariable size unknown data chunkgenerations are int32x3 objects with following structure:int32 ExportCountint32 NameCountint32 NetObjectCountDependsOffset seems to point to some object inside import table (haven't checked yet). Beginning with NameOffset there is a list of NameCount entries. Each entry is variable-size and has following format:int32 NameLengthnull-terminated string NameStringint32 Unknown1int32 Unknown2Beginning with ExportOffset there is a list of ExportCount entries. Each entry is variable-size and has following format:int32 ObjTypeint32 ParentClassRefint32 OwnerRefint32 NameListIdxint32 Field5int32 Field6int32 PropertyFlagsint32 Field8int32 ObjectFileSizeint32 DataOffsetint32 Field11int32 NumAdditionalFieldsint32 Field13int32 Field14int32 Field15int32 Field16int32 Field17int32xNumAdditionalFields UnknownFieldsBeginning with ImportOffset there is a list of ImportCount entries. Each entry is fixed-size and has following format:int32 PackageIDint32 Unknown1int32 ObjTypeint32 Unknown2int32 OwnerRefint32 NameListIdxint32 Unknown3OwnerRef is positive value for Export objects and negative value for Import objects. Object data chunks format depends on object type. For example, scripted function has 0x30 bytes of header, last 8 bytes are memory size and file size respectively. Then go fileSize bytes of actual code, ending with 0x53 and then some unknown data. Hints, tips and notes on UPK format and object data formats are welcomed. :smile: Edited December 8, 2013 by wghost81 Link to comment Share on other sites More sharing options...
dubiousintent Posted December 8, 2013 Share Posted December 8, 2013 (edited) Thanks. I'll port this over to the Wiki when it gets back up. (Feel free to remind me if I should lose track.) -Dubious- Edited December 8, 2013 by dubiousintent Link to comment Share on other sites More sharing options...
EliotVU Posted December 9, 2013 Share Posted December 9, 2013 (edited) Guess you haven't discovored my article on the package's format yet? http://eliotvu.com/page/unreal-package-file-formathttp://eliotvu.com/news/view/34/deserializing-unreal-packages-data-structureshttp://eliotvu.com/news/view/35/deserializing-unreal-packages-tables Edited December 9, 2013 by EliotVU Link to comment Share on other sites More sharing options...
wghost81 Posted December 9, 2013 Author Share Posted December 9, 2013 I did found first article, but not the last two. Guess, that's because they were under news, not package format category. :smile: And googling didn't helped. Thanks for the info! Link to comment Share on other sites More sharing options...
dubiousintent Posted December 10, 2013 Share Posted December 10, 2013 Posted as wiki article 'UPK File Format - XCOM:EU 2012', including links to Eliot's articles. Link to comment Share on other sites More sharing options...
wghost81 Posted December 10, 2013 Author Share Posted December 10, 2013 Thanks! Having centralized source of information is good, as I spent several days trying to google something on the subject. Link to comment Share on other sites More sharing options...
wghost81 Posted December 10, 2013 Author Share Posted December 10, 2013 One word on lists indexes. I don't like the "subtract one" thing, as it is confusing and doesn't seems "native". Name list is an array of data with zero starting index. Object lists are array of data with starting index equal to 1, as zero-reference indicates null-object. When unpacked to memory, object lists are prepended with null-object with zero starting index and then other objects from upk file are appended to the list. This way you don't need to subtract/add anything when reading/editing, which makes operations more transparent. Link to comment Share on other sites More sharing options...
dubiousintent Posted December 10, 2013 Share Posted December 10, 2013 Updated the article with 'List indexing'. -Dubious- Link to comment Share on other sites More sharing options...
Amineri Posted December 10, 2013 Share Posted December 10, 2013 One word on lists indexes. I don't like the "subtract one" thing, as it is confusing and doesn't seems "native". Name list is an array of data with zero starting index. Object lists are array of data with starting index equal to 1, as zero-reference indicates null-object. When unpacked to memory, object lists are prepended with null-object with zero starting index and then other objects from upk file are appended to the list. This way you don't need to subtract/add anything when reading/editing, which makes operations more transparent. XMTS and I did the same thing in the Java code we are (still!) working on. It's a lot easier to add that initial null object to the list and be done with it. Link to comment Share on other sites More sharing options...
dubiousintent Posted December 10, 2013 Share Posted December 10, 2013 Hmm. Does that mean the initial null object is not automatically created by the existing decompiler tools, and is a recommendation to be adopted manually? If so, I should rewrite that section in the article. -Dubious- Link to comment Share on other sites More sharing options...
Recommended Posts