OSX Modding tools

[psyflame]

Hi all, I'm interested in developing some compatibility tools so that people can actually use mods on the OSX (hopefully Linux as well) versions of XCOM.

 

As I understand it right now, there exists a tool to patch UPK and executable files given some delta file that is supplied by the modder. Can I get a source for this so I can try to build it on Unix?

 

Alternatively, does anyone have any advice on where to start on modding and mod compatibility tools for OSX? I'm not familiar with the modding situation of XCOM, but I really want to play Warspace Extension :tongue:

 

Update 8/12: I have finals this week so bear with me. I have the Steam version for Mac and I found the relevant strings inside the binary, DGC.ini forced load seems to work fine (can edit item prices outside of BalanceMods). When I can figure out an easily testable UPK change I will do that too.

 

If anybody has the non-Steam Mac versions, such as App Store or disc, if you want to send me your XCom Enemy Unknown.app (or just the binary inside the MacOS folder) then I am gonna whip up a tool to at least do the forced DGC external load patch today. After all, who needs to study? :psyduck:

 

The hash table is in a separate file in the OSX version, under Binaries/Win32/Hashes.sha (so it seems)

 

Edit: My patching script is working but I can only say that it works on my computer. v 0.000001 is attached if anyone wants to try it who has the Steam Mac version. It currently disables UPK hash checking for xcomgame, xcomstrategygame, xcomshell upk files and enables external loading of DGC.ini.

 

Since Unreal Package Extractor does not exist for OSX, *authoring* mods on a Mac platform seems significantly further away than *installing* them.

Edited August 12, 2013 by psyflame

[TheOldOne822]

few problems I see

1 XCOM ToolBoks requires .NET Framework 4 so would need to be rewritten to work on mac

2 we have not figured out how to disable the hash checks on mac so UPK changes would just crash it

3 the disc, mac store, and steam mac versions have the DGC.ini HEX settings for reading external and in different places and different formats for the stored ones

4 the 3 mac executable are different from each other and the windows one so executable hacks would not be easily ported between windows/mac or even mac store//mac steam

[dubiousintent]

The tool I used with the EXEPatcher script was 'bsdiff', which is open source, BSD Type 2 licensed. As I understand it, you can create derivatives from their source but must give credit and include a copy of their license with yours. The Wiki 'Modding Tools' article has the link and also one to the license. There are *nix and Python sources available through the linked site.

 

As written it only works with EXE files. However, I should think it could be modified to work with a UPK file as well.

 

-Dubious-

[toastedotter]

On a semi-related note, is there a way to mod the .ini files for XCom? I've been fiddling with the DGC.ini i could get my hands on, but I can't seem to find the elusive .ini that actually does the work. Mostly because I can't find the xcomgame.exe file I seem to need to find.

[TheOldOne822]

Amineri informs me that disabling the hash checks works on the mac versions and I have tested the forced DGC.ini load for myself and can tell you that does work but have not tested the UPK change.

So most of the mods should work on mac it's just a matter of finding a good way to patch the 3 different mac versions (Disc/Steam/Mac Store) and apply the upk/DGC changes on mac. The few that do need to change the exe (only long war and only for the Arc Thrower nerf AFAIK) will not work but most would.

[Bertilsson]

Is there anything windows specific regarding the compression of the upk-files?

If not, is there any reason why not the mac upk-files could be decompressed in windows and modded in windows using hxd and ue explorer? And is there any indication that the upk-files themselves are any different between the different mac versions?

 

Edit: I assume that if decompress and UE Explorer can be used it would not be much more work involved in converting to mac than what may be required for windows if Enemy within turns out to be not so minor... Still a lot of work though.

Edited August 10, 2013 by Bertilsson

[Amineri]

The other big thing that is useful for development (and for fun for some) is enabling the developer console. Currently this is done via an exe edit to the PC version (the edit itself is 'black box' to me, and was discovered by someone using a C++ IDE with the windows executable).

 

Gildor's decompress executable appears to only be currently available for PC, although it's a tiny exe at only 52kB. His UE Viewer is available for both PC and Linux (although the PC version has 139k downloads vs only 3k for Linux, so it's easy to see why he's focussed on PC releases). There's no UI at all, so ideally it wouldn't be much work to port the decompressor over to OSX. You'd need to contact Gildor to see about doing that, though. As a bonus any work done for that would also be applicable to any other Unreal Engine-based game that's been ported to OSX

[psyflame]

The tool I used with the EXEPatcher script was 'bsdiff', which is open source, BSD Type 2 licensed. As I understand it, you can create derivatives from their source but must give credit and include a copy of their license with yours. The Wiki 'Modding Tools' article has the link and also one to the license. There are *nix and Python sources available through the linked site.

 

As written it only works with EXE files. However, I should think it could be modified to work with a UPK file as well.

 

-Dubious-

Why does EXEPatcher generate diffs? At least, it appears to, based on the screenshot (can't run it as I'm on OSX.) Is there some reason why I can't just supply diffs premade from the various executable types?

[dubiousintent]

@psyflame: EXEPatcher does NOT generate diffs. It only applies the diff files to the executable. I just noticed yesterday that EXEPatcher had the screenshot of the EXEDiff script, and replaced it. Sorry if it caused confusion. EXEDiff is used to create the 'difference' patch files.

 

I have zero experience with OSX. The 'bsdiff' package for Windows is a commandline tool, and makes no registry entries. So assuming you can run a 'DOS type' commandline tool in OSX, it should be able to create and patch with the diffs. If OSX doesn't provide a Windows shell, but is Unix based instead, the source code for both the 'bsdiff' programs was originally written for *nix back in the 90s, and ported to Windows. If necessary, you should be able to roll your own from the source and distribute it, as long as you keep to the terms of the BSD license. The great thing about difference files is they don't really care about anything other than being able to determine the offset locations from the beginning of the file. Everything else is just a set of so many bytes to overwrite. (At least in concept.) Might need to watch out for 'little-endian' versus 'big-endian' bytes, though.

 

And yes, once you create the diffs for the various Mac versions of XCOM, they should be distributable and useable with the appropriate Mac versions of the games.

 

What offsets we know of for Mac versions are listed on the Wiki here. Updates to that information would be appreciated. You can just pass it along to me and I'll take care of posting it.

 

Good luck.

 

 

-Dubious-

Edited August 12, 2013 by dubiousintent

[TheOldOne822]

when I edited my Steam version on my old mac I found that it has the UTF32 Little-Endian format of the Mac App Store version but was at a different offset. I planed to update the wiki but was unable to edit the page (bit defender browser plugin issues)