Jump to content

UPK Utils


wghost81

Recommended Posts

Here are source and test files for my UPKUtils project:
https://github.com/wghost/UPKUtils

UPKUtils is a set of simple console applications for analyzing and patching uncompressed UPK files. Main "library" consists of two files: UPKUtils.h and UPKUtils.cpp. Other files are using this library to perform actual operations on UPKs:

  • ExtractNameLists — extract all header info (package info and name/object lists) and echo it into console. Use redirection to put it into some text file.
  • FindObjectEntry — find object entry by object name and extract its data into binary file (optional).
  • MoveExpandFunction — move and/or expand function by its name.
  • PatchUPK — patch UPK file using modfile.
  • RepairFunction — discontinued, waiting for Amineri to finish her work. :smile:

Helper library to parse modfiles: ModParser.h, ModParser.cpp.

 

UPKUtils is C++11 Code::Blocks project. I use latest MinGW to build it.

 

UPKUtils compiled and ready to use with some examples on nexus:

http://www.nexusmods.com/xcom/mods/448

 

I will explain more about modfile format in following post.

Link to comment
Share on other sites

  • Replies 235
  • Created
  • Last Reply

Top Posters In This Topic

Modfile format

 

Modfile is a simple human-readable txt file which uses format similar to ToolBoks custom mod. PatchUPK is fully compatible (at least, I hope :smile: ) with ToolBoks custom mods: it can read and implement all the changes,

 

Modfile consists of keys (NAME=VALUE), sections ([sECTION_NAME]) and comments. Comments are begin with { and end with }. You may place comments anywhere, they also can be multiline. Right now PatchUPK is case-sensitive, so be careful! I'm not an expert on text parsing, so I made it as I could, sorry. :sad:

 

List of supported keys and their allowed values:

  • MOD_NAME, AUTHOR and DESCRIPTION are self-explanatory. :smile: Value is simple text. Can be multiline.
  • UPK_FILE - name of UPK file to patch.
  • OFFSET - dec or hex (0x) value.
  • FUNCTION - full function name to patch.
  • REL_OFFSET - relative offset, dec or hex (0x) value. If absolute offset value is set through OFFSET or FUNCTION key, REL_OFFSET is added to that value, so you can patch parts of function code without a need to specify it's full offset.
  • FUNCTION_FILE - file (with path) to read data from. Can be used with OFFSET or FUNCTION or by itself, if file name has Full.Function.Name.Function format.
  • NAMELIST_NAME - change namelist name. Value is of OLDNAME:NEWNAME.
  • EXPAND_FUNCTION - move and expand function by its name. Value is Full.Function.Name:NEWSIZE.
List of supported sections:
  • [MODDED_HEX] - section which contains modded space-separated hex data (as in ToolBoks custom mod).
  • [bEFORE_HEX] and [AFTER_HEX] - hex data for search-and-replace style patching,
  • [/bEFORE_HEX] and [/AFTER_HEX] are permitted, by not used anyway, since parser uses different method to find data blocks.
Some examples:

 

Move and expand:

 

UPK_FILE=XComStrategyGame.upk
EXPAND_FUNCTION=XGStrategyAI.GetAltWeapon:300

Find and replace:

 

UPK_FILE=XComGame.upk

[BEFORE_HEX]
CC 04 00 00 18 03 00 00 0F 00 FF B2 00 00 25 07 C9 04 96 00 FF B2 00 00 2C 04 16 07 51 00 9A 38 3A 35 7A B2 00 00 7C B2 00 00 00 00 1A 00 FF B2 00 00 01 A6 B2 00 00 38 3A 24 10 16 04 0B 06 BB 04 07 BB 04 9A 38 3A 35 7A B2 00 00 7C B2 00 00 00 00 1A 00 FF B2 00 00 01 A6 B2 00 00 38 3A 24 00 16 0F 35 7A B2 00 00 7C B2 00 00 00 01 1A 00 FF B2 00 00 01 A6 B2 00 00 24 10 07 BF 03 81 2D 01 88 B2 00 00 16 0F 00 FE B2 00 00 25 07 BC 03 96 00 FE B2 00 00 36 35 42 00 00 00 43 00 00 00 00 00 01 95 B2 00 00 16 07 DA 01 9B 38 3A 35 3D 00 00 00 3F 00 00 00 00 00 10 00 FE B2 00 00 35 42 00 00 00 43 00 00 00 00 00 01 95 B2 00 00 38 3A 24 00 16 55 35 7B B2 00 00 7C B2 00 00 00 00 1A 00 FF B2 00 00 01 A6 B2 00 00 7D 00 1B 54 0F 00 00 00 00 00 00 35 3D 00 00 00 3F 00 00 00 00 00 10 00 FE B2 00 00 35 42 00 00 00 43 00 00 00 00 00 01 95 B2 00 00 35 3A 00 00 00 3F 00 00 00 00 00 10 00 FE B2 00 00 35 42 00 00 00 43 00 00 00 00 00 01 95 B2 00 00 16 16 07 C4 02 9B 38 3A 35 3C 00 00 00 3F 00 00 00 00 00 10 00 FE B2 00 00 35 42 00 00 00 43 00 00 00 00 00 01 95 B2 00 00 38 3A 24 00 16 55 35 7B B2 00 00 7C B2 00 00 00 00 1A 00 FF B2 00 00 01 A6 B2 00 00 7D 00 1B 54 0F 00 00 00 00 00 00 35 3C 00 00 00 3F 00 00 00 00 00 10 00 FE B2 00 00 35 42 00 00 00 43 00 00 00 00 00 01 95 B2 00 00 35 39 00 00 00 3F 00 00 00 00 00 10 00 FE B2 00 00 35 42 00 00 00 43 00 00 00 00 00 01 95 B2 00 00 16 16 07 AE 03 9B 38 3A 35 3B 00 00 00 3F 00 00 00 00 00 10 00 FE B2 00 00 35 42 00 00 00 43 00 00 00 00 00 01 95 B2 00 00 38 3A 24 00 16 55 35 7B B2 00 00 7C B2 00 00 00 00 1A 00 FF B2 00 00 01 A6 B2 00 00 7D 00 1B 54 0F 00 00 00 00 00 00 35 3B 00 00 00 3F 00 00 00 00 00 10 00 FE B2 00 00 35 42 00 00 00 43 00 00 00 00 00 01 95 B2 00 00 35 38 00 00 00 3F 00 00 00 00 00 10 00 FE B2 00 00 35 42 00 00 00 43 00 00 00 00 00 01 95 B2 00 00 16 16 A5 00 FE B2 00 00 16 06 C5 00 06 B8 04 55 35 7B B2 00 00 7C B2 00 00 00 00 1A 00 FF B2 00 00 01 A6 B2 00 00 2A 00 1B 54 0F 00 00 00 00 00 00 38 3D 35 6A B2 00 00 71 B2 00 00 00 00 01 82 B2 00 00 4A 16 16 55 35 7B B2 00 00 7C B2 00 00 00 00 1A 00 FF B2 00 00 01 A6 B2 00 00 2A 00 1B 54 0F 00 00 00 00 00 00 38 3D 35 69 B2 00 00 71 B2 00 00 00 00 01 82 B2 00 00 4A 16 16 55 35 7B B2 00 00 7C B2 00 00 00 00 1A 00 FF B2 00 00 01 A6 B2 00 00 2A 00 1B 54 0F 00 00 00 00 00 00 38 3D 35 66 B2 00 00 71 B2 00 00 00 00 01 82 B2 00 00 4A 16 16 06 C9 04 A3 00 FF B2 00 00 16 06 0B 00 04
[/BEFORE_HEX]

[AFTER_HEX]
A8 04 00 00 18 03 00 00 0F 00 FF B2 00 00 25 07 89 04 96 00 FF B2 00 00 2C 04 16 07 51 00 9A 38 3A 35 7A B2 00 00 7C B2 00 00 00 00 1A 00 FF B2 00 00 01 A6 B2 00 00 38 3A 24 10 16 04 0B 06 7B 04 07 7B 04 9A 38 3A 35 7A B2 00 00 7C B2 00 00 00 00 1A 00 FF B2 00 00 01 A6 B2 00 00 38 3A 24 00 16 0F 35 7A B2 00 00 7C B2 00 00 00 01 1A 00 FF B2 00 00 01 A6 B2 00 00 24 10 07 7F 03 81 2D 01 88 B2 00 00 16 0F 00 FE B2 00 00 25 07 7C 03 96 00 FE B2 00 00 36 35 42 00 00 00 43 00 00 00 00 00 01 95 B2 00 00 16 0F 01 98 B2 00 00 38 3A 35 3D 00 00 00 3F 00 00 00 00 00 10 00 FE B2 00 00 35 42 00 00 00 43 00 00 00 00 00 01 95 B2 00 00 07 84 01 97 01 98 B2 00 00 25 16 55 35 7B B2 00 00 7C B2 00 00 00 00 1A 00 FF B2 00 00 01 A6 B2 00 00 7D 00 1B 54 0F 00 00 00 00 00 00 38 3D 01 98 B2 00 00 25 16 16 0F 01 98 B2 00 00 38 3A 35 3C 00 00 00 3F 00 00 00 00 00 10 00 FE B2 00 00 35 42 00 00 00 43 00 00 00 00 00 01 95 B2 00 00 07 79 02 97 01 98 B2 00 00 25 16 0F 00 57 B3 00 00 25 07 79 02 98 00 57 B3 00 00 35 39 00 00 00 3F 00 00 00 00 00 10 00 FE B2 00 00 35 42 00 00 00 43 00 00 00 00 00 01 95 B2 00 00 16 55 35 7B B2 00 00 7C B2 00 00 00 00 1A 00 FF B2 00 00 01 A6 B2 00 00 7D 00 1B 54 0F 00 00 00 00 00 00 38 3D 01 98 B2 00 00 24 00 16 16 A5 00 57 B3 00 00 16 06 E3 01 0F 01 98 B2 00 00 38 3A 35 3B 00 00 00 3F 00 00 00 00 00 10 00 FE B2 00 00 35 42 00 00 00 43 00 00 00 00 00 01 95 B2 00 00 07 6E 03 97 01 98 B2 00 00 25 16 0F 00 57 B3 00 00 25 07 6E 03 98 00 57 B3 00 00 35 38 00 00 00 3F 00 00 00 00 00 10 00 FE B2 00 00 35 42 00 00 00 43 00 00 00 00 00 01 95 B2 00 00 16 55 35 7B B2 00 00 7C B2 00 00 00 00 1A 00 FF B2 00 00 01 A6 B2 00 00 7D 00 1B 54 0F 00 00 00 00 00 00 38 3D 01 98 B2 00 00 24 00 16 16 A5 00 57 B3 00 00 16 06 D8 02 A5 00 FE B2 00 00 16 06 C5 00 06 78 04 55 35 7B B2 00 00 7C B2 00 00 00 00 1A 00 FF B2 00 00 01 A6 B2 00 00 2A 00 1B 54 0F 00 00 00 00 00 00 38 3D 35 6A B2 00 00 71 B2 00 00 00 00 01 82 B2 00 00 4A 16 16 55 35 7B B2 00 00 7C B2 00 00 00 00 1A 00 FF B2 00 00 01 A6 B2 00 00 2A 00 1B 54 0F 00 00 00 00 00 00 38 3D 35 69 B2 00 00 71 B2 00 00 00 00 01 82 B2 00 00 4A 16 16 55 35 7B B2 00 00 7C B2 00 00 00 00 1A 00 FF B2 00 00 01 A6 B2 00 00 2A 00 1B 54 0F 00 00 00 00 00 00 38 3D 35 66 B2 00 00 71 B2 00 00 00 00 01 82 B2 00 00 4A 16 16 06 89 04 A3 00 FF B2 00 00 16 06 0B 00 04 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B
[/AFTER_HEX]

Rename function:

 

UPK_FILE=XComStrategyGame.upk
NAMELIST_NAME=GetAltWeapon:#GetPodProgs

Patch part of the function by relative offset:

 

UPK_FILE=XComGame.upk
FUNCTION=XGBattleDesc.InitAlienLoadoutInfos     
REL_OFFSET=0x28
[AFTER_HEX]
A8 04 00 00 18 03 00 00 0F 00 FF B2 00 00 25 07 89 04 96 00 FF B2 00 00 2C 04 16 07 51 00 9A 38 3A 35 7A B2 00 00 7C B2 00 00 00 00 1A 00 FF B2 00 00 01 A6 B2 00 00 38 3A 24 10 16 04 0B 06 7B 04 07 7B 04 9A 38 3A 35 7A B2 00 00 7C B2 00 00 00 00 1A 00 FF B2 00 00 01 A6 B2 00 00 38 3A 24 00 16 0F 35 7A B2 00 00 7C B2 00 00 00 01 1A 00 FF B2 00 00 01 A6 B2 00 00 24 10 07 7F 03 81 2D 01 88 B2 00 00 16 0F 00 FE B2 00 00 25 07 7C 03 96 00 FE B2 00 00 36 35 42 00 00 00 43 00 00 00 00 00 01 95 B2 00 00 16 0F 01 98 B2 00 00 38 3A 35 3D 00 00 00 3F 00 00 00 00 00 10 00 FE B2 00 00 35 42 00 00 00 43 00 00 00 00 00 01 95 B2 00 00 07 84 01 97 01 98 B2 00 00 25 16 55 35 7B B2 00 00 7C B2 00 00 00 00 1A 00 FF B2 00 00 01 A6 B2 00 00 7D 00 1B 54 0F 00 00 00 00 00 00 38 3D 01 98 B2 00 00 25 16 16 0F 01 98 B2 00 00 38 3A 35 3C 00 00 00 3F 00 00 00 00 00 10 00 FE B2 00 00 35 42 00 00 00 43 00 00 00 00 00 01 95 B2 00 00 07 79 02 97 01 98 B2 00 00 25 16 0F 00 57 B3 00 00 25 07 79 02 98 00 57 B3 00 00 35 39 00 00 00 3F 00 00 00 00 00 10 00 FE B2 00 00 35 42 00 00 00 43 00 00 00 00 00 01 95 B2 00 00 16 55 35 7B B2 00 00 7C B2 00 00 00 00 1A 00 FF B2 00 00 01 A6 B2 00 00 7D 00 1B 54 0F 00 00 00 00 00 00 38 3D 01 98 B2 00 00 24 00 16 16 A5 00 57 B3 00 00 16 06 E3 01 0F 01 98 B2 00 00 38 3A 35 3B 00 00 00 3F 00 00 00 00 00 10 00 FE B2 00 00 35 42 00 00 00 43 00 00 00 00 00 01 95 B2 00 00 07 6E 03 97 01 98 B2 00 00 25 16 0F 00 57 B3 00 00 25 07 6E 03 98 00 57 B3 00 00 35 38 00 00 00 3F 00 00 00 00 00 10 00 FE B2 00 00 35 42 00 00 00 43 00 00 00 00 00 01 95 B2 00 00 16 55 35 7B B2 00 00 7C B2 00 00 00 00 1A 00 FF B2 00 00 01 A6 B2 00 00 7D 00 1B 54 0F 00 00 00 00 00 00 38 3D 01 98 B2 00 00 24 00 16 16 A5 00 57 B3 00 00 16 06 D8 02 A5 00 FE B2 00 00 16 06 C5 00 06 78 04 55 35 7B B2 00 00 7C B2 00 00 00 00 1A 00 FF B2 00 00 01 A6 B2 00 00 2A 00 1B 54 0F 00 00 00 00 00 00 38 3D 35 6A B2 00 00 71 B2 00 00 00 00 01 82 B2 00 00 4A 16 16 55 35 7B B2 00 00 7C B2 00 00 00 00 1A 00 FF B2 00 00 01 A6 B2 00 00 2A 00 1B 54 0F 00 00 00 00 00 00 38 3D 35 69 B2 00 00 71 B2 00 00 00 00 01 82 B2 00 00 4A 16 16 55 35 7B B2 00 00 7C B2 00 00 00 00 1A 00 FF B2 00 00 01 A6 B2 00 00 2A 00 1B 54 0F 00 00 00 00 00 00 38 3D 35 66 B2 00 00 71 B2 00 00 00 00 01 82 B2 00 00 4A 16 16 06 89 04 A3 00 FF B2 00 00 16 06 0B 00 04 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 53
[/AFTER_HEX]
There is a fully functional mod example in nexus mod file archive (Larger Pods), it produces exact same version as the one released for patch 1. Should be fully compatible with unpatched EW.

 

PatchUPK does not support unicode (not in text, nor in paths)! As I said, I'm no expert on parsing.

 

Command line format

PatchUPK modfile.txt [PATH_TO_UPKS]

 

PATH_TO_UPKS is optional, if omitted upks are searched in current working directory.

Edited by wghost81
Link to comment
Share on other sites

It does not need any installation. It's a set of console programs with command line arguments. If you run a program without any, you will receive "help" line like this:

ExtractNameLists
Usage: ExtractNameLists UnpackedResourceFile.upk
So, in this case you need to type ExtractNameLists Path\to\XComGame.upk, for example, and see a wall of text on the screen. :smile: You may redirect program output with >

ExtractNameLists Path\to\XComGame.upk > list.txt
Other programs work the same way. PatchUPK is fully functional upk patcher and can be used to install/distribute mods. Although, right now I'm working on GUI wrapper for PatchUPK to make it more user-friendly.
Link to comment
Share on other sites

dubiousintent, thanks!

 

You may put it anywhere, just give the correct path to upk file in the command line.

 

I usually copy upk files from game folder to different working directory and then start analysing/patching. Utils are in the same dir too. In this case you don't need to bother with paths and may use upk copies directly, without fear of breaking something during experiments.

 

BTW, I'm a Linux user, actually, so I do like console. :smile: I use FAR manager under windows, it has "old-school" look and feel, but is powerful and easy to use with console apps.

Edited by wghost81
Link to comment
Share on other sites

Oh: then we need to clarify if you are referring to the 'XCOM Console' or the 'Linux Console' / 'Windows command prompt'? Originally I thought you meant these were XCOM Console additions, but now I'm thinking you mean they are standalone command line tools. (Confused minds want to know! :blush: )

 

-Dubious-

Link to comment
Share on other sites

Oops... found a stupid mistake in the code which could lead to PatchUPK crash. Will update programs soon, but right now you should avoid any spaces before/after key and section name! Use KEY=VALUE and [sECTION_NAME] with immediate end of line to avoid any troubles!

 

Sorry for this. :sad:

 

UPD: updated both github sources and nexus files binaries to v.0.9.1

Edited by wghost81
Link to comment
Share on other sites

  • Recently Browsing   0 members

    • No registered users viewing this page.

×
×
  • Create New...