UPK Utils

[projectmercy]

I have an idea which will help preserve existing mods compatibility. What if for string tokens not the lone symbol '>', but the combination of two subsequent symbols '">' (with any possible whitespaces in between) will be considered an end of token? This combination seems rare enough.

 

I think that would work well, combined with a note in the documentation about the limitation. If anyone needed to start a string with a > they could just put a space at the head.

[wghost81]

I think that would work well, combined with a note in the documentation about the limitation. If anyone needed to start a string with a > they could just put a space at the head.

No space will be needed, as I will check for <%t" (with any possible whitespaces in between) and set it as a string start. Then I will search for "> to find an end. So only "> combination (with or without spaces in between) won't be allowed inside.

 

Or I can disallow whitespaces in between "> and search for "> without whitespaces to determine a string end. In this case constructions like

<%t "Text "Text" > Text">

will be allowed, but this one won't work

<%t "Text "Text" > Text" >

[projectmercy]

I've been using <%t " right now. I could switch to without a space, but I find it makes it harder to read. I noticed in most of your mods though, you weren't using the space. I guess old habits die hard for me :)

 

I'm sure you could put in more complex checks on 'if <%t existed right before the ">, then assume it's not an end token' or something, but it starts to get down to the question of 'is it really worth your time.' I guess I'm thinking no. It looks like there's going to be a caveat no matter what you do. I vote you do whatever seems best/easiest and just document it. Consistency of design is golden.

[wghost81]

That's because first version of pseudo-code parser was space-sensitive. Now it's not.

 

I'll stick to "> with any possible whitespaces in between. It means, that "> and " > won't be allowed inside string token.

[wghost81]

I've uploaded modified patcher sources to github. I don't have access to windows right now, so you'll have to build it yoursef. :smile:

 

Some instructions can be found here: http://forums.nexusmods.com/index.php?/topic/1829365-modding-xcom-under-linux/

 

For windows I recommend using TDM-GCC with cmake and skip wxWidgets, as you need PatchUPK only. You will also need git to clone project repository. Build instructions are similar for both linux and windows.

[projectmercy]

Thanks, I'll check it out. Last time I looked there wasn't a makefile for the main project. Is that still true?

[wghost81]

projectmercy, there was a makefile (project file) for CodeBlocks IDE (cbp). I've added cmake project file (CMakeLists.txt) to "build" directory for easy cross-platform compilation. Check linux modding topic I've linked in previous post for build instructions.

[Amineri]

So I've been pondering the issue of how to add additional objects to the upks in a way that would be consistent and applicable across multiple mods. I thought I'd post up some general thoughts here. They are fully formed (or even fully researched yet), but seem like they might be leading in the right direction.

 

The general problems with inserting new objects come about because of :

1) Install-order issues / issues with installing multiple mods

2) Uninstall issues when multiple mods are installed

 

The basic issue is that the numeric reference to a newly created name/import/export list entry will depend upon it's install order. If the mod assumes certain that it's newly created list entries will have particular value, then if another mod that also creates entries is installed first the second mod will fail. Also, if the uninstallation of a mod actually removes the list entry, that will cause any subsequent list entries to change their position within the list, causing subsequently installed mods to fail.

 

Above discussion has shown that adding namelist entries is relatively safe. In this cases, an uninstaller should leave the namelist entries alone in order to preserve the ordering for any subsequent entries made by later mods.

 

The PatchUPK code for this was :

[ADD_NAME_ENTRY]
<%u 13>
<%t "DevOnlineMsg">
<%u 0x00000000>
<%u 0x00070010>

ALIAS=DebugLog:19 2E <XComGame.XComOnlineEventMgr> 19 2E <Engine.GameEngine> 12 20 <Engine.Engine> 0A 00 <Engine.Engine.GetEngine.ReturnValue> 00 1C <Engine.Engine.GetEngine> 16 09 00 <Engine.GameEngine.OnlineEventManager> 00 01 <Engine.GameEngine.OnlineEventManager> FF 00 <NullRef> 00 1B <DevOnlineMsg>

which added DevOnlineMsg to the namelist and then relied upon Engine.GameEngine.OnlineEventManager already being an importlist entry in XComStrategyGame and the run-time virtual function binding in the Unreal Engine.

 

As long as the uninstaller doesn't alter the namelist entry, uninstallation should be fine even if subsequent mods have added to the namelist -- having an extra namelist entry won't hurt anything.

 

 

Another key aspect to safe installation in the presence of other mods is using name-type refences to newly created list items instead of numeric references. In the vanilla XComStrategyGame.upk, there are 13.853 namelist entries. The last one is 13852 "ZoomRotationLimit". The UDK compiler appears to create the namelist in alphabetic order, although this doesn't appear to be a runtime requirement.

 

However, using name-type references (such as in the alias definition above) does create an implicit install order requirement -- the new entry has to be installed prior to attempting to install anything that utilizes it. This is less of an issue for PatchUPK since the format allows for making changes to multiple objects within a single file, but more an issue with UPKmodder, in which each object change is a different modfile (which is actually more useful for project management when handling ~1000 changes).

 

Regardless, when dealing with multiple mods, as long as each references its newly created entry by name in the modfile (converting to number only when applied to the hex), and as long as the uninstall process doesn't remove the list entries, leaving them as "deadweight", multiple mods should be able to be handled.

 

-----------

 

In terms of new code, the thing I'm going to experiment with is seeing if it's possible to create a new upk using UDK, and having that upk contain unrealscript objects, and then create new importlist objects in the existing XComGame/XComStrategyGame upks that would allow access to the newly created unrealscript.

 

This would have a couple of advantages, in my opinion :

1) Ability to use UDK compiler to build the new unrealscript instead of having to convert it to hex by hand

2) Limit the number/type of list entries being adding to the existing upks to only name/import list changes instead of having to add/create export list entries

 

The first part is nice, but the second part would allow the "hiding" extra variables within the newly created package file, limiting the extent of the changes necessary to the existing package files.

 

There are two issues that I think have to be resolved :

1) Is it possible to create a new upk using the September 2011 build version 845 and access unrealscript?

2) Does leaving "dead" importlist entries that no longer point to a valid upk cause the game to crash?

 

The first issue is partially resolved, in that I've been able to build a upk that contained 2DTexture objects and load them during the game. I haven't done so specifically for unrealscript, but it seems do-able.

 

The second issue relates to uninstallation. When uninstalling it would be necessary to leave existing importlist entries alone in order to preserve compatibility with subsequently installed mods, but it would be desired to clean up the new unrealscript package file and any config-file references to it. If the importlist is parsed and packages to it resolved when the package is loaded, this might cause crashing. At the least it might generate warnings within the Launch.log file.

 

---------

 

Anyhow, just some thoughts/ramblings on how we might be able to add new stuff. Open to thoughts/suggestions/critiques on the concept.

[wghost81]

Amineri, I haven't ever worked with UDK and the biggest question I have is how it will work with existing XComGame/XComStartegyGame classes? How do we import those into our new package? By simply declaring those as an import objects? Will UDK require original classes to build new package? Or having import references will be enough?

 

I agree with your thoughts on installation and uninstallation. Being able to create our own packages and use script compiler is huge. So this sure worth investigating. Unfortunately, I don't have windows now and I'm still struggling with 64/32-bit wine problems on my OS, so I don't know if I can help you here. :sad:

 

Testing incorrect import entries should be easy enough, though. We can try to add some crazy entry, combined of existing namelist entries and see how engine handles it.

[wghost81]

I experimented with incorrect import entries and it worked.

 

I've added completely weird package to XComStrategyGame.upk (EW) called wghost81 and game launched normally with no log warnings. Seems, import entries only get resolved if they're actually used.