How to build class/object references

[Amineri]

I've been mucking about with the hex code enough that some of it is starting to make sense.

 

Even the constructions that look something like : m_kSoldier.m_kSoldier.kClass.eType

 

They actually follow some pretty straightforward rules which I thought I'd share.

 

There are two types of "dot" constructions, class/object and struct.

 

Class/object uses the 0x19 context token

Struct uses the 0x35 struct token

 

Here I'm going to focus on the 0x19 context token

 

Suppose I wanted to build something that looked like:

 

<object1>.<object2>

 

The breakdown looks like so:

19 -- context token

## ... ## -- reference to <object1>

## ## -- two bytes (little endian order) representing the virtual size of <object2>

## ## ## ## -- the return value of <object2)

00 -- this is always zero ... not sure if it can fill any other role

## ... ## -- reference to <object2>

 

Each object can be either a class variable or a class function.

A class variable has the structure : 01 ## ## ## ##

A class function usually has the structure : 1B ## ## ## ## 00 00 00 00 <parameters> 16

Sometimes a function may be a "final" function, then it has the structure 1C ## ## ## ## <parameters> 16

 

For a class variable, the return value is the same as the object reference itself.

For a class function, the return value is ... the return value. If the value has no return value it is set to 00 00 00 00

 

Example1:

m_iFoo : 01 44 32 00 00

m_iBar : 01 87 21 00 00

 

m_iFoo.m_iBar

19 -- context token

01 44 32 00 00 -- reference to class variable m_iFoo

09 00 -- virtual size of m_iBar (5 file + 4 additional virtual bytes)

87 21 00 00 -- return value of m_iBar (the base reference without 01 class var token)

00 -- just a zero

01 87 21 00 -- reference to class variable m_iBar

Example2:

m_iFoo : 01 44 32 00 00

Bar( ) : 1B 99 25 00 00 00 00 00 00 16

Bar ReturnValue : 3E 77 00 00

iSnafu : 00 D4 AA 00 00

m_iFoo.Bar(iSnafu)

19 -- context token

01 44 32 00 00 -- class variable m_iFoo

13 00 -- size of Bar(iSnafu)

3E 77 00 00 -- return value Bar

00 -- just a zero

1B 99 25 00 00 00 00 00 00 00 D4 AA 00 00 16

---------------------

larger constructs are daisy-chained together in a similar manner.

<object1>.<object2>.<object3> has the form:

19 -- context token

19 -- context token

## ... ## -- reference to <object1>

## ## -- size of <object2>

## ## ## ## -- return value of <object2>

00

## ... ## -- reference to <object2>

## ## -- size of <object3>

## ## ## ## -- return value of <object3>

00

## ... ## -- reference to <object3>

All of the context tokens go at the beginning of the construction. There will be 1 0x19 token for each "dot" in the construction (if all members are CLASS objects -- structs follow different rules.

The return values and reference can be looked up in various places within UE explorer without having to dig through the hex viewer.

Notes:

1) If the virtual size is incorrect UE Explorer will decompile the object correctly but the program will crash upon execution. I often use the token view in UE Explorer to help find and correct the virtual sizes in these contexts

2) If the Return Value is incorrect the program may crash or simply not return the correct value depending on where in the construction the incorrect value is located.

-----------------------

Hopefully this helps some people in attempting to write up new hex code :)

[dubiousintent]

Started a Wiki article for this on 'How to build class.object references - XCOM:EU 2012'.

 

-Dubious-

[Amineri]

Here's an example of some code I'm working on now. The original line's breakdown is:

kLockerItem.bClassLocked = !STORAGE().IsClassEquippable(kSoldier.GetClass(), byte(kItem.iItem));
	14 -- Boolean LET
	2D -- boolean token
	35 A7 2C 00 00 AB 2C 00 00 00 01 -- struct bClassLocked
	00 FD 2C 00 00 -- local variable kLockerItem
	81 -- NOT
		19 -- class context token
		1B 23 27 00 00 00 00 00 00 16 -- STORAGE
		47 00 -- size of next context (IsClassEquippable(...))
		76 40 00 00 -- return value next context
		00 
		1B 33 14 00 00 00 00 00 00 -- IsClassEquippable(
			19 -- context token
			00 FF 2C 00 00 -- local variable kSoldier
			0A 00 -- size of next context (GetClass())
			91 45 00 00 -- return value next context (GetClass())
			00 
			1B B5 0E 00 00 00 00 00 00 16 -- GetClass()
			38 3D -- int-to-byte conversion
			35 C5 02 00 00 C8 02 00 00 00 00 48 -- struct iItem
			00 2D 00 00 -- local variable kItem
		16 -- execute IsClassEquippable
	16 -- execute NOT

My goal is to build a new line:

kLockerItem.bClassLocked = !STORAGE().IsClassEquippable(kSoldier.m_kSoldier.kClass.eWeaponType, byte(kItem.iItem));

From elsewhere in the code I already have the construction:

m_kSoldier.kClass.eWeaponType 
35 2F FF FF FF 7D FA FF FF 00 00 35 B4 F9 FF FF 74 FA FF FF 00 01 01 EC 44 00 00 

The thing to note here is that structures are built in the opposite order to objects.

 

So

35 -- struct token
2F FF FF FF 7D FA FF FF 00 00 -- eWeaponType reference
35 -- struct token
B4 F9 FF FF 74 FA FF FF 00 01 -- kClass reference
01 -- class variable token
EC 44 00 00 -- m_kSoldier reference

The first step is to build kSoldier.m_kSoldier.kClass.eWeaponType. The order is 35 <eWeaponType> 35 <kClass> 19 <kSoldier> <m_kSoldier>

 

The full breakdown is:

kSoldier.m_kSoldier.kClass.eWeaponType

35 -- struct token
2F FF FF FF 7D FA FF FF 00 00 -- eWeaponType
35 -- struct token
B4 F9 FF FF 74 FA FF FF 00 01 -- kClass
19 -- context token
00 FF 2C 00 00 -- local variable kSoldier
09 00 -- size of next context (m_kSoldier)
EC 44 00 00 -- return value next context (m_kSoldier)
00 
01 EC 44 00 00 -- class var m_kSoldier

With that in hand I substitute it for the original kSoldier.GetClass() parameter in IsClassEquippable()

 

This yields:

kLockerItem.bClassLocked = !STORAGE().IsClassEquippable(kSoldier.m_kSoldier.kClass.eWeaponType, byte(kItem.iItem));

14 -- Boolean LET
2D -- boolean token
35 A7 2C 00 00 AB 2C 00 00 00 01 -- struct bClassLocked
00 FD 2C 00 00 -- local variable kLockerItem
81 -- NOT
	19 -- class context token
	1B 23 27 00 00 00 00 00 00 16 -- STORAGE
	47 00 -- size of next context (IsClassEquippable(...))
	76 40 00 00 -- return value next context
	00 
	1B 33 14 00 00 00 00 00 00 -- IsClassEquippable(
		35 -- struct token
		2F FF FF FF 7D FA FF FF 00 00 -- eWeaponType
		35 -- struct token
		B4 F9 FF FF 74 FA FF FF 00 01 -- kClass
		19 -- context token
		00 FF 2C 00 00 -- local variable kSoldier
		09 00 -- size of next context (m_kSoldier)
		EC 44 00 00 -- return value next context (m_kSoldier)
		00 
		01 EC 44 00 00 -- class var m_kSoldier
		38 3D -- int-to-byte conversion
		35 C5 02 00 00 C8 02 00 00 00 00 48 -- struct iItem
		00 2D 00 00 -- local variable kItem
	16 -- execute IsClassEquippable
16 -- execute NOT

However, this hex code will not run like this. Changing the parameter to IsClassEquippable changed the overall virtual size of the IsClassEquippable context that follows STORAGE(). Specifically the 47 00 size has to be updated to reflect the larger parameter size.

 

So how to figure out the new virtual size? It can be done by hand, but it's much easier to let UE Explorer do the heavy lifting here. In a similar manner to how I put in placeholder value for jumps, the incorrect value will still decompile correctly. This allows me to use the Token view to figure out the new virtual size of the context.

 

The first step is to verify that the new line decompiles properly, which it does:

kLockerItem.bClassLocked = !STORAGE().IsClassEquippable(kSoldier.m_kSoldier.kClass.eWeaponType, byte(kItem.iItem));

The next step is to correct the context virtual size. To do this enter the Token view and find the line of code:

(0x1D8) LetBool(162) -> BoolVariable(29) -> StructMember(28) -> LocalVariable(9) -> NativeFunction(132) -> Context(130) -> VirtualFunction(10) -> EndFunctionParms(1) -> VirtualFunction(108) -> StructMember(68) -> StructMember(49) -> Context(30) -> LocalVariable(9) -> InstanceVariable(9) -> IntToByte(30) -> StructMember(28) -> OutVariable(9) -> EndFunctionParms(1) -> EndFunctionParms(1)
	kLockerItem.bClassLocked = !STORAGE().IsClassEquippable(kSoldier.m_kSoldier.kClass.eWeaponType, byte(kItem.iItem))

The IsClassEquippable context is the VirtualFunction(108) reference. 108 is in decimal, so convert to hex = 0x6C

 

This means that the 47 00 size in the context construction needs to be replaced with 6C 00.

 

With this change the new construction decompiles and runs correctly.

[wghost81]

I just had most interesting experience with virtual sizes.

 

Turns out, I forgot to change virtual size in one complex construction for Random Pods mod. And it worked with wrong sizes. I changed sizes to correct values determined with UE - it still works.

 

Code:

iTestPod = World().FindClosestValidLocation(World().GetPositionFromTileCoordinates(iPod[ 9] + Rand(iPod[7]),iPod[10] + Rand(iPod[8]), 0),false, false, false);

Bad sizes:

0F 00 6F BA 00 00 19 1B A1 7F 00 00 00 00 00 00 16 28 00 AC D2 00 00 00 1B 29 33 00 00 00 00 00 00 19 1B A1 7F 00 00 00 00 00 00 16 25 00 BC 0C 00 00 00 1B DD 3A 00 00 00 00 00 00 92 1A 2C 09 00 70 BA 00 00 A7 1A 2C 07 00 70 BA 00 00 16 16 92 1A 2C 0A 00 70 BA 00 00 A7 1A 2C 08 00 70 BA 00 00 16 16 1D 00 00 00 00 16 28 28 27 16

Good sizes:

0F 00 6F BA 00 00 19 1B A1 7F 00 00 00 00 00 00 16 6A 00 AC D2 00 00 00 1B 29 33 00 00 00 00 00 00 19 1B A1 7F 00 00 00 00 00 00 16 47 00 BC 0C 00 00 00 1B DD 3A 00 00 00 00 00 00 92 1A 2C 09 00 70 BA 00 00 A7 1A 2C 07 00 70 BA 00 00 16 16 92 1A 2C 0A 00 70 BA 00 00 A7 1A 2C 08 00 70 BA 00 00 16 16 1D 00 00 00 00 16 28 28 27 16

Code works, no crashes and I see no difference in game. That is very strange...

[Amineri]

There are definitely some mysteries to how/when the Unreal Engine crashes and when it simply tosses out a warning.

 

What worries me is that such behavior may be different on different systems, which could lead to bugs that only occur for some people that aren't reproducible on my system. I've still had quite a bit of strangeness surrounding using "borrowed" local variables -- sometimes it works, sometimes it causes CTD, and sometimes the variable value doesn't persist. No rhyme or reason that I've been able to comprehend.

[wghost81]

I have a word for the last one: garbage collection.

 

Unreal Engine has some strange garbage collection algorithm. I couldn't find very much info on the subject, but what I found convinced me not to use borrowed variables anymore.

 

I double-checked virtual sizes, fixed all the bad ones. Script still works, like engine doesn't care. Well, better be on the safe side and don't forget to calculate correct values anyway. :smile:

[Amineri]

To toss some more evidence onto the pile of "really try and avoid borrowing local variables" ...

 

I just found and squashed a CTD bug in the Long War EW beta which only popped up when Muton Berserkers used their BullRush ability. The strange thing about this was that neither JL nor I had made any mods to Bullrush -- at all.

 

It turns out that the source of the CTD was in XGTacticalGameCore.CalcOverallDamage, where I had been using two borrowed local variables (iPerk and iAbility). The function CalcOverallDamage is always called from XGAbility_Targeted.CalcDamage, but CalcDamage can be called from several different locations :

• XGAbility_Bullrush.ApplyEffect -- XGAbility_Bullrush is a child of XGAbility_Targeted
• Three different places within XGAbility_Targeted.RollForHit -- once for scatter shots, once for psionic abilities, and once for regular attacks

The game ran normally when CalcDamage was called from within XGAbility_Targeted.RollForHit, but would consistently crash when it was invoked from XGAbility_Bullrush.ApplyEffect. The source of the crash was the borrowed local variables. Once I removed those, the CTDs disappeared.

 

The two variables were borrowed from the function XGTacticalGameCore.CharacterIsPsionic :

simulated function bool CharacterIsPsionic(const out TCharacter kCharacter)
{
    local int iAbility, iPerk;

This function isn't ever called from either within XGAbility_Targeted or XGAbility_Bullrush. However these borrowed local variables worked when called via one call-chain, but not another.