Expanding function size in UPK

[Drakous79]

Christmas soon so it's present time :)

[wghost81]

A copy of old function is appended to the end of upk file? What would happen if there are both old and expanded function in the upk and offset and size is set to the old one?

No, an old function code stays in place, so other objects offset won't change. Yes, there are two functions in upk after move/expand operation. And yes, you can "switch" between them by simply changing the offset and data size in objectlist table.

 

Bertilsson, it's too complicated. At least for me. :smile:

[Bertilsson]

Bertilsson, it's too complicated. At least for me. :smile:

Nah, it only seems too complicated right now.

When you have spent a few sleepless nights obsessing about it, it will come to you! Promise! ;)

 

But seriously, from what I can tell you have already implemented almost all of it in the upktools? You even seem to have an option to expand function size already in place.

 

So isn't it then "only" a "simple" matter of identifying that dataToInsert.length > sizeOfTargetFunction and in that case to trigger size expansion and insert at new location instead?

 

Sorry if I'm ignorant to the issue and missed something obvious...

[wghost81]

Well, after a "few sleepless nights" I found an easy way to make uninstaller and implement Bertilsson's idea. :smile: Stay tuned! :wink:

[Bertilsson]

 

A copy of old function is appended to the end of upk file? What would happen if there are both old and expanded function in the upk and offset and size is set to the old one?

No, an old function code stays in place, so other objects offset won't change. Yes, there are two functions in upk after move/expand operation. And yes, you can "switch" between them by simply changing the offset and data size in objectlist table.

Not sure if this is the same idea as you have already come up with... But after giving this a little thought...

 

Wouldn't it be possible to always create a new function at end of file for any change to an idenfied function, and to also append a "custom header" directly before or after the "new" function data.

 

That way you would have the original version completely intact and an easy to find reference that identifies this function as a modded function and a handle to the previous version.

 

Something like bytecode for wghost81 + previous storage location would be enough?

[wghost81]

Bertilsson, yes, adding a custom header to expanded function is a good idea, thanks!

[Amineri]

I must confess to being a bit envious that I didn't think of this excellent and brilliant solution to easily increase function size.

 

In terms of adding a custom header to the expanded function in order to better track/uninstall such larger functions ...

 

One way would be to add a string constant near the beginning (or perhaps at the end would be more appropriate?). A basic string constant with no assignment will remain valid runnable code but won't execute anything / change any state. Defining one is as simple as 1F <bytes> 00 .

 

The issue with placing this at the beginning of the function is that functions with optional parameters must have particular commands executed first or the code will crash.

 

At the very least a 0x0B token for each optional parameter will make the optional parameter be the default value if not specified by the caller.

 

The more complex definition is :

49 <2 byte relative jump> <value> 15 

Note that this is the only use I've seen using the 0x15 token to terminate a "parameter" list instead of the more usual 0x16. This hex (the above line or a 0x0B) must be the first thing present in a function, and have to be in a particular order based on the parameter list ordering.

 

So, my suggestion would be to place this "expanded function info string" at the end of the function just prior to the 0x53 end-of-script token.

 

The breakdown would then be:

- 48 byte function header

- regular code up to (but not including) 0x53 EOS token

- expanded function info string footer (e.g. "wghost81" + <oldfunction info>)

- 0x53 EOS token

- final 8 bytes that follow 0x53

 

Since you're making the function bigger anyhow there is plenty of room to fit in such a footer. It is valid code that won't change any state.

[Bertilsson]

But is it really necessary?

 

It does complicate things (not much but still more than nothing) as the function header must be adjusted with respect to virtual size in addition to not being able to treat the entire function as one big chunk of data.

 

Edit: Also a small observation I have done is that it seems perfectly fine to chuck in any data directly after the 53 token... I normally use that location as a buffer when cutting and pasting bytes in hxd... In the end i replace the surplus bytes with 0B... But on occasion when I have forgotten that step the game has been running perfectly fine without complaint even though random leftover bytes have been there.

Edited December 10, 2013 by Bertilsson

[Amineri]

You're probably correct that having extra bytes after the 0x53 EOS token is okay (I just don't have much experience with it) -- assuming that the file size and memory size are correctly adjusted in the function header.

 

If it goes after the EOS then memory and file sizes in the header wouldn't have to be directly adjuted (just the object size in the Export table). The trick would be preventing modders from removing the meta-data un-install information :)

[wghost81]

We don't need to add extra bytes into function object. We may add ANY bytes after object_offset + object_size (entire object, not function). As Bertilsson suggested, it may be some ID (like PatchUPK md5hash: 7aa056c9605f7b31725d4bc47cd24dd9) plus old object offset and size. This way no one could ever see or remove it withoud direct hex editing. Program will read object data, when look for ID, and if ID is found, extract original object offset/size.

 

And may I remind you, that not all objects are functions. :wink: Someday we might want to expand some other object, when we'll find a way to do so. :smile:

 

BTW, I haven't found/understood info on object types in objectlist. I want to check object type and auto-expand it only if it is a script code.

 

UPD: so stupid of me... objectType is actually an ImportObject reference, which holds type string.

Edited December 11, 2013 by wghost81