UPK Utils

[wghost81]

Works fine for me. Answered in "UPK Patcher XSHAPE failed to work": http://forums.nexusmods.com/index.php?/topic/1834480-upk-patcher-xshape-failed-to-work/&do=findComment&comment=15846275

[projectmercy]

Is there a way to modify chunks of code within a function in such a way as the jump offsets after your change are updated for you?

 

I was attempting to edit XGStrategySoldier.LevelUp(). It has a single call on class select to PickAClass(), I wanted to instead put the logic on class selection inside this function, as it contains context on that soldier's capabilities. If I just try to use "BEFORE_CODE" and "AFTER_CODE" selecting the PickAClass() line, the offsets that exist after my change are not updated.

 

Test mod:

 

 

UPK_FILE=XComStrategyGame.upk
OBJECT=XGStrategySoldier.LevelUp:AUTO
[bEFORE_CODE]
// eClass = BARRACKS().PickAClass();
0F 00 <.eClass> 19 1B <BARRACKS> 16 0A 00 <XGFacility_Barracks.PickAClass.ReturnValue> 00 1B <PickAClass> 16

[AFTER_CODE]
// if(m_kSoldier.iRank == 7) {
07 [@ChevCheckEND]
9A
35 <XComGame.XGTacticalGameCoreNativeBase.TSoldier.iRank> <XComGame.XGTacticalGameCoreNativeBase.TSoldier> 00 00 01 <@m_kSoldier>
2C 07
16
// Achieve(29);
1B <Achieve> 24 1D 16
// }
[#ChevCheckEND]

 

 

 

 

The beginning of that function in UE Explorer without changes looks like:

 

 

function LevelUp(optional XComGame.XGTacticalGameCoreData.ESoldierClass eClass, optional out string statsString)
{
// End:0x75
if(m_kSoldier.iRank == 0)
{
// End:0x62
if(eClass == 0)
{
eClass = BARRACKS().PickAClass();
}
SetSoldierClass(eClass);
}
m_kSoldier.iRank += 1;
// End:0xD8
if(m_kSoldier.iRank == 3)
{
BARRACKS().GenerateNewNickname(self);
}
// End:0x107
if(m_kSoldier.iRank == 7)
{
Achieve(29);
}

 

 

 

After the change it looks like:

 

 

function LevelUp(optional XComGame.XGTacticalGameCoreData.ESoldierClass eClass, optional out string statsString)
{
// End:0x75
if(m_kSoldier.iRank == 0)
{
// End:0x62
if(eClass == 0)
{
// End:0x2F
if(m_kSoldier.iRank == 7)
goto J0x2F;
Achieve(29);
}
SetSoldierClass(eClass);
}
m_kSoldier.iRank += 1;
// End:0xD8
if(m_kSoldier.iRank == 3)
{
BARRACKS().GenerateNewNickname(self);
}
// End:0x107
if(m_kSoldier.iRank == 7)
{
Achieve(29);
}

 

[wghost81]

projectmercy, no there is no functionality for automatic offset updates.

 

If function isn't too big, you can use before-after code command to insert new code and then use find/modded hex commands to find and replace "bad" offsets. Old and new offset values can be calculated with UE Explorer.

 

But there is a bigger issue. If you're inserting code in the middle of some function, jump offsets will be incorrect, as patcher doesn't know starting memory offset and always starts with zero. labels were originally designed to use with REPLACEMENT_CODE, which replaces the whole script.

 

There are several approaches to deal with it. If you're inserting some simple logic, best way would be to manually recalculate offsets with UE Explorer. And if you're modifying a large part of the code, you can rewrite the whole function in pseudo-code.

 

I am planning to add "convert to pseudo-code" functionality to UPK Utils collection. But it will require partial implementation of decompiler, which isn't that easy. So this will take some time. :smile:

[projectmercy]

Np, thank you for your reply. I don't mind doing it, I just figured if there was a better way I'd do that.

 

I opted for just recoding the whole function in psuedo-code. It's not that bad once you get used to it, just takes a bit of time.

 

I was more thinking about mod collisions in the future. If my mod recodes the whole function, then you get into race conditions on stacking mods, but I guess when you're working on a game that wasn't designed to be modded, you have to make do!

[projectmercy]

OK,I have another question, as regards exported classes and their functions.

 

I was picking about trying to figure out if there was a way I could add things to the launch.log file for logging purposes from XComStrategyGame. I noticed that in XComGame there was XComOnlineEventMgr.DevOnlineMsg which took a string and dumped it to the log file. Also in there is XComOnlineEventMgr.UnlockAchievement.

 

I can access UnlockAchievement from an XComStrategyGame function, via:

 

 

 

// XComOnlineEventMgr(GameEngine(class'Engine'.static.GetEngine()).OnlineEventManager).UnlockAchievement(9);
19 2E <XComGame.XComOnlineEventMgr>
// GameEngine(class'Engine'.static.GetEngine()).OnlineEventManager
19 2E <Engine.GameEngine>
// class'Engine'.static.GetEngine()
12 20 <Engine.Engine> 0A 00 <Engine.Engine.GetEngine.ReturnValue> 00 1C <Engine.Engine.GetEngine> 16
09 00 <Engine.GameEngine.OnlineEventManager> 00 01 <Engine.GameEngine.OnlineEventManager>
0C 00 <NullRef> 00 1B <UnlockAchievement> 2C 09 16

 

 

 

But if I change it to something else (I haven't tried everything, but a few easy ones) , here's the log example:

 

 

 

// XComOnlineEventMgr(GameEngine(class'Engine'.static.GetEngine()).OnlineEventManager).DevOnlineMsg("ILIKEPIE");
19 2E <XComGame.XComOnlineEventMgr>
// GameEngine(class'Engine'.static.GetEngine()).OnlineEventManager
19 2E <Engine.GameEngine>
// class'Engine'.static.GetEngine()
12 20 <Engine.Engine> 0A 00 <Engine.Engine.GetEngine.ReturnValue> 00 1C <Engine.Engine.GetEngine> 16
09 00 <Engine.GameEngine.OnlineEventManager> 00 01 <Engine.GameEngine.OnlineEventManager>
13 00 <NullRef> 00 1B <DevOnlineMsg> 1F <%t "ILIKEPIE"> 16

 

 

 

 

 

I basically get an error in UPKUtils where it can't find the entry.

 

 

 

Uninstall script saved to D:\mods\XCOM\SoldierBootcamp\mod_test.txt.uninstall21.txt
Bad name: DevOnlineMsg
Bad token: <DevOnlineMsg>
Invalid/empty data!
Execution stopped at #5 command named [REPLACEMENT_CODE].

 

 

 

Suggestions? And/or Does anyone know a dead function in XComGame that accepts a string and is exported to XComStrategyGame? :wink:

Edited June 29, 2014 by projectmercy

[wghost81]

XComOnlineEventMgr is a class, defined in XComGame package. You can't directly access XComGame classes from inside XComStrategyGame package: you need to import every object you need first. ImportTable inside XComStrategyGame.upk has many XComOnlineEventMgr related entries. For example:

0xFFFFFC70 (-912): Function'XComGame.XComOnlineEventMgr.AreAchievementsEnabled'

Your access to XComOnlineEventMgr from inside XComStrategyGame.upk is limited to those entries. And since DevOnlineMsg isn't amongst those entries, you can't access it from inside XComStrategyGame.upk.

 

Use ExtractNameLists program, which is a part of UPKUtils collection, to extract package info and import/export tables.

 

In theory using latest PatchUPK release you can add new import and export entries. But this functionality is still unfinished and not safe for distributing mods, as it breaks uninstall files.

[projectmercy]

Thanks, that was all I needed. I just had to add it to the Names table.

 

UnlockAchievement also doesn't have anything in the import table, just the names table. I imagine this is something to do with the fact that it doesn't contain any return types, and that the class itself is in the import table.

 

 

Adding it to the names table:

 

 

[ADD_NAME_ENTRY]
<%u 13>
<%t "DevOnlineMsg">
<%u 0x00000000>
<%u 0x00070010>

 

 

 

 

Values now in launch.log

 

 

[0345.83] DevOnline: ILIKEPIE
[0345.83] DevOnline: ILIKEPIE
[0345.83] DevOnline: ILIKEPIE

 

 

 

If an uninstall is performed on a mod that includes these items, does it roll back everything but the Tables changes?

Edited June 29, 2014 by projectmercy

[wghost81]

That's an interesting discovery about UnlockAchievement and DevOnlineMsg. And great discovery about logging! Will help a lot with debugging some complex mods.

 

Adding new entries is incompatible with uninstall feature, irreversible (by PatchUPK means) and breaks all previously created uninstall files. My suggestion will be to use this for development purposes only and won't add any new objects to final release.

 

There are a number of problems, related to adding new objects. Before pseudo code was introduced, the biggest problem was object reference: if there are two different mods and both add new objects, different install order will result in different object references. Uninstall raises a bigger problem: if you delete an entry, created by the mod, what will happen with other entries, created with some other mods? Removing an entry from the middle of the table will result in object references shift and will break installed mods, which use those references. So I decided that new entries should never be removed with uninstall scripts.

 

Another problem is caused by badly designed uninstall feature. I've made an error when I decided to use absolute offsets and replacement hex for uninstall files. Install files are "smart" and make full use of "object-oriented" patching and uninstall files are "dumb". They work perfectly in situations where upk header is unchanged though and I've never imagined that adding new objects will be a common practice at that point. :smile: Anyway, uninstall feature has to be reworked from scratch to correct those issues and make it compatible with mods adding new objects.

[Amineri]

I was more thinking about mod collisions in the future. If my mod recodes the whole function, then you get into race conditions on stacking mods, but I guess when you're working on a game that wasn't designed to be modded, you have to make do!

 

Yes, mod collisions due to having to mod the same function/object are seemingly unavoidable. And resizing a function pretty much requires rewriting the whole thing, since (as you point out) all of the later jump offsets have to be changed.

 

When coding in Long War I try and avoid rewriting/resizing very large functions for just the reason (beyond that it's just a bunch of work), and instead try and mod smaller portions of the function. In some cases I have up to 5 different upk_mod modfiles that all modify different portions of the same function.

 

In part this is to maintain as much compatibility with other mods as I can, and in part this allows better modularization of the different major functional components within Long War.

 

-------------

 

Also, very nice find on figuring out how to write data to the Launch.log.

 

 

UnlockAchievement also doesn't have anything in the import table, just the names table. I imagine this is something to do with the fact that it doesn't contain any return types, and that the class itself is in the import table.

 

I suspect that this is because the function is accessed via the 1B Virtual Reference token which only utilizes the namelist reference. Presumably it's bound to the correct object at run-time. Final functions (which are rarer) or super functions are referenced via the 1C token which utilizes the specific Export/Import reference entry.

 

For Long War I'll probably come up with some other solution to access DevOnlineMsg from XComStrategyGame.

 

Doesn't adding name references suffer the same multi-mod install issues as adding import/export entries? If two different mods both add to the namelist, then the references to each would be different depending on install order. This would require mods to always use the string-based name reference to be converted to the numeric name reference at mod-install time, wouldn't it?

[projectmercy]

Doesn't adding name references suffer the same multi-mod install issues as adding import/export entries? If two different mods both add to the namelist, then the references to each would be different depending on install order. This would require mods to always use the string-based name reference to be converted to the numeric name reference at mod-install time, wouldn't it?

 

 

If the tool you're using to update the tables just pre-scanned for that entry, couldn't you just abort the insert and assume it was covered if the contents were the same? Especially for the name table. For the import/export table, I guess it would depend on if the signatures were equivalent.

Edited June 29, 2014 by projectmercy