Rewriting Scripts

[twinj]

I have learnt what you must do when you rewrite scripts completely. It can be difficult to stay within the parameters I am bout to share but certainly doable.

 

Firstly I will have to show you the headers in a script or function. Note that my knowledge is incomplete and that this is a sort of workaround to enable better changes. I got this reply from Eliot (the man who made UE Explorer) while writing this.

 

The 16 last bytes are two 32bit integers, the former one represents the bytecode size in memory(when loaded) and the latter the bytes written in the file.

 

You only have to modify the former integer when editing the bytecodes.

Every token that references an object counts an additional 4 bytes.

 

All the other data is not relevant, TextPos and Line just means the character position and line the function was declared before it was compiled.

0x00000000 : CharToCorpse                    :UFunction                        => CharToCorpse(72)
0x00000000 : ExportSize                      :Int32                            => 230
0x00000000 : NetIndex                        :UIntProperty                     => iChar(71)
0x00000004 : NameIndex                       :String                           => None
0x0000000C : NextField                       :UScriptStruct                    => XGUnitVisibilityInformation(69)
Super
ScriptText
0x00000018 : Children                        :UIntProperty                     => iChar(71)
CppText
0x00000020 : Line                            :UInt32                           => 1712
0x00000024 : TextPos                         :UInt32                           => 41209
0x00000028 : ByteScriptSize                  :Int32                            => 179
0x0000002C : DataScriptSize                  :Int32                            => 167
0x000000D7 : NativeToken                     :UInt16                           => 0
0x000000D7 : OperPrecedence                  :Byte                             => 0
0x000000DA : FunctionFlags                   :FunctionFlags                    => Defined, Static, Public
0x000000DE : FriendlyNameIndex               :UNameTableItem                   => CharToCorpse

 

The issue is size. We currently cannot write code past the script size without breaking the rest of the upk file. We also could not do much with code other than do small to medium changes like swapping jumps or constants. Now we can pretty much NULL everything as long as the virtual size stays the same or you can successfully update the virtual size.

 

Ill use a simple example. The code below is split into 4 sections:

1. Header
   
2. Script bytes
   
3. End of script token,
   
4. Footer
   

 

simulated function OnInit()
{
   super.OnInit();
}

 

CB 02 00 00   47 55 00 00   00 00 00 00   CB 02 00 00
7A 02 00 00   00 00 00 00   00 00 00 00   00 00 00 00
5A 00 00 00   E3 0B 00 00   0D 00 00 00   09 00 00 00

1C 7A 02 00   00 16 04 0B 

53 00 00 00 

02 01 02 00   6B 56 00 00   00 00 00 00

 

A function header seems to inherit from Class Struct. The header is 48 bytes long. This will become important for understanding jump offsets.

 

The first section holds information about the script and is split into INDEX's and DWORDs. Don't worry if you do not know what there are yet. You will find out!

 

Header:

The first 16 bytes:

INDEX -> Object reference to export table: [self] 4 bytes in file 8 in memory.

INDEX -> Object reference to export table: [Children] Usually "None." A91F0000 in XComStrategyGame.4 bytes in file 8 in memory.

DWORD -> ???? 4 bytes in file

INDEX -> Object reference to export table: [First Paramater] These are read until [self] is reached and include [Locals] and [ReturnValues] 4 bytes in file 8 in memory.

 

The next row.

DWORD -> ???? 4 bytes in file

DWORD -> ???? 4 bytes in file

DWORD -> ???? 4 bytes in file

DWORD -> ???? 4 bytes in file

 

The last row.

DWORD -> Line Number

DWORD -> TextPosition

DWORD -> Loaded Size (Virtual size)

DWORD -> Script Size ) (Script Size on disk) will point to the 0x53 end of script token in a file.

 

We aren't really concerned with the above except for the number of paramaters or objects in between the [self] and [FirstParam] and the two sizes.

 

From using the header information and applying it to the sample code we see that there is no return value, parameters or locals indexed as the 2 values are the same (0xCB020000) .

 

The size values in the header are: 0x0d000000 and 0x09000000. Please not that these values are actually read in pairs backwards by the (PC) computer!! 0x0000000d and 0x00000009

13 and 9! The way bytes are read can be confusing at first and not all machines read and store bytes in teh same way. This is know as Little Endian and Big Endian for reasons I may explain later.

 

Virtual Size = 13

Script Size = 9

 

You will always note that the VS will always be a multiple of 4 higher than the script size. In this case it is only 4. It may be 0, 4, 8, 12 and so on!

 

This means that one of the lines in the script must have 1 read which points to an Index reference. Lets look!

 

The code 1C 7A 02 00 00 16 04 0B .

 

1C is the token for a final function. 7A 02 00 00 is an object reference otherwise known as an INDEX. It is 4 bytes. 16 is the token for ending an expression. 04 return and 0b void.

 

A very simple function! That index read when loaded is actually expanded into 8 bytes! 9 + 4 = 13.

 

So when editing scripts you must keep the VS and FS the same. You can do this by creating index reads that do nothing or repeat themseleves and by writing in OB voids int he code to keep the FS correct.

 

Theoretically you should be able to add or take extra Index reads if needed by updating the VS repective to the reads. I have not yet got this to work. When counting the amount of INDEX's be sure to include the parameters, locals and return values as 1 read each!

 

This theory is also related on how to count jump offsets.

Edited December 23, 2012 by twinj

[twinj]

Reserved...!

[bokauk]

Theoretically you should be able to add or take extra Index reads if needed by updating the VS repective to the reads.

This is amazing, great work finding this out, twinj!

 

I think this is a much needed breakthrough and the biggest leap forward we've had since being able to modify UPKs. It really does open up so much more potential for modding :)

 

What's almost more amazing is that this was almost burried on page 2 and no one had replied!

 

Kudos + 1; :thumbsup:

 

Thank you :)

[twinj]

Theoretically you should be able to add or take extra Index reads if needed by updating the VS repective to the reads.

This is amazing, great work finding this out, twinj!

 

I think this is a much needed breakthrough and the biggest leap forward we've had since being able to modify UPKs. It really does open up so much more potential for modding :)

 

What's almost more amazing is that this was almost burried on page 2 and no one had replied!

 

Kudos + 1; :thumbsup:

 

Thank you :)

 

Thanks. You can in fact edit the VS in the script header to correspond to the amount of index reads. This means there is not needed code workarounds.

 

UE explorer has a statements list which shows the index reads and their positions. This can help with count a lot. To the point of not needing to count in some situations, just using the values. Same with jumps!

[johnnylump]

I'm trying to parse all this, but I'm dense. Is it possible to add lines of code to functions, if we can calculate the correct length and get that in the function header? Perhaps a step-by-step walkthrough would be helpful.

[osito2dancer]

I've always had a doubt about this: Can you "hook" DLL function calls to UPKs? If so, you could generate code, place a dll call into your script (and fill with no-ops to erase the originally-replaced function) and play with it.

 

O2D

[twinj]

I'm trying to parse all this, but I'm dense. Is it possible to add lines of code to functions, if we can calculate the correct length and get that in the function header? Perhaps a step-by-step walkthrough would be helpful.

 

There really isnt a step by step way to do this. You have to just understand how the script is expanded in size when loaded so you can adjust the jumps and lengths as needed.

 

Currently there is no way to write over the file length of the script. If you get clever and find a better more efficient way to rerwrite some code which saves room then you have space. Have to get creative.

 

I am working on a decompiler where I will add the ability to rewrite sizes.. if I can rewrite them it does not also mean that they will actually load as well.

 

 

I've always had a doubt about this: Can you "hook" DLL function calls to UPKs? If so, you could generate code, place a dll call into your script (and fill with no-ops to erase the originally-replaced function) and play with it.

 

O2D

 

Not that I am aware of.

Edited January 21, 2013 by twinj

[johnnylump]

I'm trying to parse all this, but I'm dense. Is it possible to add lines of code to functions, if we can calculate the correct length and get that in the function header? Perhaps a step-by-step walkthrough would be helpful.

 

There really isnt a step by step way to do this. You have to just understand how the script is expanded in size when loaded so you can adjust the jumps and lengths as needed.

 

Currently there is no way to write over the file length of the script. If you get clever and find a better more efficient way to rerwrite some code which saves room then you have space. Have to get creative.

 

I am working on a decompiler where I will add the ability to rewrite sizes.. if I can rewrite them it does not also mean that they will actually load as well.

 

 

Good luck.

 

I gather we can't make bigger functions because each function's location in the upk is referenced in a bunch of other places, so all functions have to stay at the same starting point?

 

But smaller ones are OK because we can pad out the function with null statements. Is my understanding correct?

[bokauk]

I gather we can't make bigger functions because each function's location in the upk is referenced in a bunch of other places, so all functions have to stay at the same starting point?

 

But smaller ones are OK because we can pad out the function with null statements. Is my understanding correct?

That's my understanding of things currently too. Being able to "move" the starting location of a function+header would be very helpful!

 

twinj, do you know if this is/isn't possible yet? Are we able to update the places that reference the function+header position? :)

[johnnylump]

I gather we can't make bigger functions because each function's location in the upk is referenced in a bunch of other places, so all functions have to stay at the same starting point?

 

But smaller ones are OK because we can pad out the function with null statements. Is my understanding correct?

That's my understanding of things currently too. Being able to "move" the starting location of a function+header would be very helpful!

 

twinj, do you know if this is/isn't possible yet? Are we able to update the places that reference the function+header position? :)

 

Follow-up:

 

Can we add parameters to existing functions? One workaround to make longer scripts is to find some unused function or debugging function, rewrite it, and call it from the main function we're trying to edit. If that's workable, next step would be catalog these functions.