UPK Utils

[wghost81]

Added new tool to main UPKUtils project called DecompressLZO. It can decompress all upk files: compressed (with header) and fully compressed (without header). Works with LZO compression only.

 

For whatever reason github decided that all the files inside project were modified. Linux/Windows incompatibility or something else related to git. Anyway tracking source changes is problematic now. And now I can understand why some people dislike github. :sad:

[Amphibious]

wghost why does the

 

"XComLZO Packer/Unpacker
Incorrect max block size!"

 

error occur when i try to unpack some upks. ones that hold refrences to textures inside tfc files to be exact.

[wghost81]

XComLZO is intended for raw data packing/unpacking. Using it for script packages was a temporal solution. Use new DecompressLZO utility (which is a part of UPK Utils now) which can handle all packages.

[Amphibious]

XComLZO is intended for raw data packing/unpacking. Using it for script packages was a temporal solution. Use new DecompressLZO utility (which is a part of UPK Utils now) which can handle all packages.

wilco

[wghost81]

Fixed a bug with adding new member variables to classes (see Nexus UPKUtils page).

 

I've tested adding new export variables extensively: local, member and ini-configurable variables are all working perfectly. I wasn't able to add a new function, though: UEExplorer decompiled it correctly, game ran well up to the point I tried to call that function and game crashed. I don't know why, but most probably I've messed up some object parameters. Anyway, adding a properly working function definitely needs more testing.

[abaxter]

I have had significant difficulties in using PatchUPK.

 

It is probably my own fault, however:

 

I am using 64bit Linux with a compiled version of UPKUtils. I was only able to patch the UPKs when the UPK was in the folder with the binaries. It would not follow paths at all.

[wghost81]

Old habits die hard. :smile: Open PatchUPK.cpp find

        if (upkPath[upkPath.length() - 1] != '\\')
            upkPath +=  "\\";

and comment it out

        /*if (upkPath[upkPath.length() - 1] != '\\')
            upkPath +=  "\\";*/

Recompile. Enjoy.

Usage example:

./PatchUPK CameraZoom-EU-EW.txt SomeFolder/uncompressed/

[wghost81]

Latest update: http://www.nexusmods.com/xcom/mods/448

 

REPLACE_HEX and REPLACE_CODE keys finally implemented and allow to replace ALL before hex/code with after hex/code.

 

New INPL specifier allows to resize objects in-place. UNINSTALL and UPDATE_REL switches allow to control Patcher behavior.

 

PatcherGUI can now batch-apply several mods with one mouse-click. Have fun! :smile:

Edited October 2, 2014 by wghost81

[Anderkent]

Hi all,

 

Is there a summary of all patchergui syntax somewhere? I've been scanning the thread backwards, trying to find out how the pseudocode works, but it's quite an effort :tongue: The first posts only talk about direct hex manipulations.

 

 

For context, I'm trying to do a toy mod - set steam locations to be constant, rather than randomized. I found the function in upkviewer, managed to decode the hex by referencing the unhood byte reader (https://code.google.com/p/unhood/source/browse/trunk/UnHood.Engine/BytecodeReader.cs), but not sure what the proper way to specify the transformations in the mod file are.

 

(also, since I totally ignored the memory size thing so far, I expect it really won't work :tongue:)

 

The annotated bytecode can be seen in this gist: https://gist.github.com/JacekLach/604f370127fe6232f887. As expected, it applies but the game crashes :tongue:

 

ETA:

ugh, found out why it crashes. Didn't notice that native calls have to be ended with 0x16. That means I'm over-budget on the function length though, so I guess I'll have to edit the wrapping if to fix jump offsets. Ugh.

 

ETA2:

So with the current code:

MOD_NAME=Static Steam Vents - LW
AUTHOR=Jacek `Anderkent` Lach
DESCRIPTION=All your steam vents will spawn in the 6th column. Yay!

UPK_FILE=XComStrategyGame.upk

OBJECT=XGBase.GenerateTiles:AUTO

[BEFORE_CODE]
07 // jump unless 
E0 03 // jump offset
96 00 F4 28 00 00 00 F6 28 00 00 16 // I < numSteamVents
0F 00 F5 28 00 00 A7 36 00 F7 28 00 00 16 // iTile = Rand(arrDeepRockTiles.Length)
0F // let
  35 // struct
    19 05 00 00 // field: iType
    1C 05 00 00 // struct: XGBase.TTerainTile
    00 01 // [end of struct ?]
	// struct object:
    10 // dynamic array access
      // array index: [
      10 // dynamic array access
        // index [
        00 // local variable
	      F5 28 00 00 // GenerateTiles.iTile
  	    // end of index ]
	    // array:
	    00 // local variable
	      F7 28 00 00 // arrDeepRockTiles
	    // end of array
      // end of index
      // array:
      01 // instance var
        C9 28 00 00 // m_arrTiles
  26 // = 1
55 // Array.add(
  01 C8 28 00 00 14 00 // m.arrSteamTiles
  10 // arrIndex
    00 F5 28 00 00 // iTile
    00 F7 28 00 00 // arrDeepRockTiles
  16 // Array.add)
  
[AFTER_CODE]
07 // jump unless 
D9 03 // jump offset
96 00 F4 28 00 00 00 F6 28 00 00 16 // I < numSteamVents
0F 00 F5 28 00 00 A7 36 00 F7 28 00 00 16 // iTile = Rand(arrDeepRockTiles.Length)
0F // let
  35 // struct
    19 05 00 00 // field: iType
    1C 05 00 00 // struct: XGBase.TTerainTile
    00 01 // [end of struct ?]
	// struct object:
    10 // dynamic array access
      // index:
      92 // +
        90 // *
          2C 07 // 7
		  00 F4 28 00 00 // local: I
		  16 // end *
        2C 0C // 12 = 5 + 7
		16 // end +
      // array:
      01 // instance var
        C9 28 00 00 // m_arrTiles
  26

55 // Array.add(
  01 C8 28 00 00 14 00 // m.arrSteamTiles
  92 // +
    90 // *
      2C 07 // 7
      00 F4 28 00 00 // local: I
      16 // end *
    2C 0C // 12 = 5 + 7
    16 // end +
  16 // Array.add)

the patch applies and seems to decompile fine in UE, but game still crashes. Inspecting function bytecode, I think the virtual/file sizes are off? Isn't 'AUTO' supposed to handle that for me?

 

(the values at offsets 0x28 and 0x2c are 'CD 04 00 00' = 205, and '7D 03 00 00' = 125 respectively; the buffer view seems to be 0x3BB long, and the token view ends with 0x4C4. am I doing something wrong?)

Edited October 31, 2014 by Anderkent

[wghost81]

Doc files are here: https://github.com/wghost/UPKUtils/tree/master/doc PatchUPK_Readme.txt has all the commands explained.

 

BEFORE/AFTER_HEX won't adjust memory (virtual) size for you, only serial (file) size. UEE doesn't pay much attention to this value, but the engine does and the game crashes if script virtual size is incorrect.

 

Virtual size value in a 4 byte long integer at offset 0x28:

REL_OFFSET=0x28
UNSIGNED=12345 // actual virtual size here

To determine function virtual size, switch to tokens view in UEE and navigate to last (EOS) token. You'll see two numbers before it: (XXX/YYY). First number is a virtual offset of this token and last is a serial offset. Since EOS token is always 1 byte long, virtual size of the whole script is equal to XXX+1.

 

If token view ends before EOS, this means virtual size is to small. Set it to some bigger value (twice as original, for example) and check if EOS is visible.

 

If you'll switch to using pseudo-codes, corresponding Patcher commands will auto-adjust virtual size along with serial size.

 

PS As I recall, there already are two mods which do the similar things:

http://www.nexusmods.com/xcom/mods/490

http://www.nexusmods.com/xcom/mods/494

Edited November 1, 2014 by wghost81