XCMM - XCom Mod Manager

[Drakous79]

Parsing size information would be better placed in the install.xml that is coming as part of the interop with FMods CBD.

Agreed :)

[Drakous79]

Tested XCMM 1.07 Closed BETA:

 

XCMM started with "Incorrect settings: Please set up the application correctly." message, clicked OK, set name and paths, clicked Save and got error "Incorrect settings: The system cannot verify your unpacked resources.", clicked OK and saw this in Home Console:

SAVED [config\config.xml]

Closed XCMM and started it again and got "Incorrect settings: Please set up the application correctly." message again. All was set so I clicked Save and got "Incorrect settings: The system cannot verify your unpacked resources." error again. Clicked OK and wanted to check some log, but there's was file I could check. Well, I have other ways... Created C:\Program Files (x86)\Steam\SteamApps\common\XCom-Enemy-Unknown\XComGame\CookedPCConsole\ and copied there Core.upk and Core.upk.uncompressed_size, went to Settings / Edit Config, clicked Save and got message "Unpack Core.upk: Would you like to unpack the core upk file here to verify the current path?", clicked Yes and got "Game file handler: There was an IO error when modifying the game resources." message with "Gildor's Unreal Extractor: Extracting the resources has completed.".

DECOMPRESSED [Core.upk]
SAVED [config\config.xml]
SORTING FILE [Core.upk]
EXTRACTED RESOURCES [Core.upk]
Extraction process finished.

Went back to Settings / Edit Config, clicked Save and got message "Settings saved: The settings have been updated.". Closed XCMM, removed C:\Program Files (x86)\Steam, started XCMM, went to Settings / Edit Config, clicked Save and got message "Settings saved: The settings have been updated.". Once is Core.upk decompressed and unpacked in unpacked folder, everything is okay.

 

So I deleted XCMM and extracted it from the archive again to start with fresh installation, returned C:\Program Files (x86)\Steam back and started XCMM. Got standard "Incorrect settings: Please set up the application correctly." message, clicked OK, set name and paths, clicked Save, got message "Unpack Core.upk: Would you like to unpack the core upk file here to verify the current path?", clicked Yes and got "Game file handler: There was an IO error when modifying the game resources." message with "Gildor's Unreal Extractor: Extracting the resources has completed.".

DECOMPRESSED [Core.upk]
SAVED [config\config.xml]
SORTING FILE [Core.upk]
EXTRACTED RESOURCES [Core.upk]
Extraction process finished.

Loading package: C:\Program Files (x86)\Steam\SteamApps\common\XCom-Enemy-Unknown\XComGame\CookedPCConsole\Core.upk Ver: 845/59 Engine: 8916 [FullComp] Names: 828 Exports: 1552 Imports: 18 Game: 8000
C:\Program Files (x86)\Steam\SteamApps\common\XCom-Enemy-Unknown\XComGame\CookedPCConsole\Core.upk: uncompressed size 223449

My installation path is S:\Steam\SteamApps\common\XCom-Enemy-Unknown. It is not in default location, because I left less space for system partition and installed XCOM EU to SSD.

 

Make tab was ready with predefined name - yay! Provided 2 files by drag and drop and clicked Make, got "Make mod: Confirm to continue." and then "Finished mod: Mod creation has finished.".

ORIGINAL FILES [spoilerName.Function]
EDITED FILES [Drakous79.SpoilerName.Function]
MOD GENERATE ACTION
UPK FILENAME [XComStrategyGame.upk]
PROCESSING ORIGINAL [spoilerName.Function] FROM [XComStrategyGame.upk]
SEARCH HASH [ED CB 30 A1 39 34 C8 D5 35 1D 9E C5 6C 6F BE 0B 96 25 36 F4 ]
RESOURCE CHECKSUM [23434]
COMPARING ORIGINAL [spoilerName.Function] WITH EDITED [Drakous79.SpoilerName.Function]

Went to Install tab and clicked refresh but the mod was not there. Checked folders and there is only Customizable Special Soldiers.xmod.config, no export.xml, no copied original/edited files. Tried one more time with input files name match, nothing.

 

One thing to note - my description was around 1700 characters and could not scroll down to see the end of it. Can you add a scrollbar to description text field please?

 

 

Here my testing of XCMM ended and I moved to the portable version.

 

I used file created by XCMM 0.6 BETA. Decompressing went fine but the portable did not remove uncompressed_size.

****************************************************************************************
UPK FOUND - IS DECOMPRESSED [XComStrategyGame.upk]
INSTALLING RESOURCE CHANGES
****************************************************************************************
SEARCHING [XComStrategyGame.upk] FOR RESOURCE [spoilerName.Function]
WITH CHECKSUM [23434]
WITH HASH [ED CB 30 A1 39 34 C8 D5 35 1D 9E C5 6C 6F BE 0B 96 25 36 F4]
SEARCHING....
FOUND - CHECKSUM & HASH MATCH
WRITING HEX EDITS SpoilerName.Function]
SAVED [mods\Spoiler name\Spoiler name.xmod.install.log.xml]
****************************************************************************************
CREATING A SESSION BACKUP
XSHAPE ACTION
****************************************************************************************
STARTING AT OFFSET [26420592]
ENDING AT OFFSET [26557951]
BUFFER SIZE [137.359 KB]
SEARCHING FOR ENTRY [xcomstrategygame.upk] AS BYTES [78 63 6F 6D 73 74 72 61 74 65 67 79 67 61 6D 65 2E 75 70 6B ] IN [XComGame.exe]
ENTRY FOUND AT OFFSET [26428581]
EXE HASH [8B 69 F3 46 84 52 F6 B8 42 D3 78 AA 24 53 A2 34 B4 7D 9E 87 ]
UPK HASH [8B 69 F3 46 84 52 F6 B8 42 D3 78 AA 24 53 A2 34 B4 7D 9E 87 ]
PATCHING NOT REQUIRED.

I confirmed with UE Explorer that the change was not applied.

 

Ah, I see, what's missing in XCMM 0.6 BETA export.xml - what to edit. Also there was <Changes/> in 0.6 BETA.

<Changes>
 <HexEdit>
   <Offset>70</Offset>
   <Data>CF</Data>
 </HexEdit>
</Changes>

Corrected it by adding HexEdit, Offset and Data entries and the mod installed successfully - works in the game, after I delete uncompressed_size. Only got "XPatch: Error [31] Error pathcing Ini file.". Spelling of patching. My export.xml has this in it:

<Ini></Ini>

 

Here's good point to mention, I tested on vanilla installation... Next I clicked Restore Original and got "Restore Game: This will replace..." warning and "Restore Game: The files have been restored." and "Installer: XCMM Installer has finished re-patching the game." after clicking Yes. But XComStrategyGame.upk ended with uncompressed size and XComStrategyGame.upk.uncompressed_size was not restored. Seems like backup is not working properly.

 

Do you see? I can still break it :) I would love to have working Portable at disposal, because I have very cool mod for promotion 8)

[twinj]

This is getting crazy... lol When I test it all works. Arrg

 

Found the culprit for the cooked path unpacked path setting... I kept skipping over the actual default values thinking they were all updated and the getters got only new values.. the cooked path still gets the default. Where I thought I rebuilt the cooked path based on the new exe path. I did not. This is why it kept going back to C:. It could also explain all of the other troubles... Why you have no edits. Extraction could not happen at cooked path for other files so no comparison from a vanilla game and could also explain why you were able to do it with older version but you pointed to already unpacked paths.

 

There is always someone who doesn't install on default locations. I use to be one of those someone's. :thumbsup: I also do not like installing on default coz of spaces. It is much easier dealing with anything on paths without spaces.

 

Added scroll bars.

 

Will edit this reply with ongoing changes.

 

Oh and did you see my new XShape. It works really well! Will be adding the improvements to XCMM and XCMPI when I can manipulate ini files properly. It doesnt need to be 'set up' anymore. It just works. The only setting it needs now is XCom install. I might even be able to fix that if windows tracks that!

 

https://dl.dropbox.com/u/54222881/XCMM-v.107d-closed.zip

 

I didnt get to the rest ran out of time will come back to those. But testing should be a lot easier with the path being correct. Up to you of course. I do think that the rest of the problems might be fixed because of that one thing though.

Edited December 15, 2012 by twinj

[Drakous79]

Hehe crazy :) That's why I keep going. Great work fixing the path! Also scrollbar is great :)

 

Tested XCMM 1.07d:

 

Started XCMM. Got standard "Incorrect settings: Please set up the application correctly." message, clicked OK, set name and paths, clicked Save, got message "Unpack Core.upk: Would you like to unpack the core upk file here to verify the current path?", clicked Yes and got "Game file handler: There was an IO error when modifying the game resources." message with "Gildor's Unreal Extractor: Extracting the resources has completed.". OKed both.

Loading package: S:\Steam\SteamApps\common\XCom-Enemy-Unknown\XComGame\CookedPCConsole\Core.upk Ver: 845/59 Engine: 8916 [FullComp] Names: 828 Exports: 1552 Imports: 18 Game: 8000
S:\Steam\SteamApps\common\XCom-Enemy-Unknown\XComGame\CookedPCConsole\Core.upk: uncompressed size 223449

Why am I getting Input Output error?

 

Went to Make tab, filled in all informations, drag and dropped files (one with Drakous79. prefix), clicked Make and got "Make Mod: Confirm to continue." and "Finished Mod: Mod creatin has finished.", but no files were created except SpoilerName.xmod.config.xml.

ORIGINAL FILES [spoilerName.Function]
EDITED FILES [Drakous79.SpoilerName.Function]
MOD GENERATE ACTION
UPK FILENAME [XComStrategyGame.upk]
PROCESSING ORIGINAL [spoilerName.Function] FROM [XComStrategyGame.upk]
SEARCH HASH [ED CB 30 A1 39 34 C8 D5 35 1D 9E C5 6C 6F BE 0B 96 25 36 F4 ]
RESOURCE CHECKSUM [23434]
COMPARING ORIGINAL [spoilerName.Function] WITH EDITED [Drakous79.SpoilerName.Function]

 

XCMPI 1.07d:

 

Works fine, mod worked in the game, uncompressed_size was correctly removed and Restore Original restored it back along with compressed upk. Only error I got was "XPatch: Error [31] Error pathcing Ini file.". Spelling of patching. I deleted DGC.ini from install folder as the mod doesn't need it and in export.xml is <Ini></Ini>. Is there something like IF INI is empty, do nothing?

 

Btw how the portable continues after OKing decompression :thumbsup:

 

Vanilla installation/uninstallation worked great. But what about already modded game uninstallation? Do you have any plans for the feature?

 

Haven't tried new XShape yet. Will check it. I had no problems with XShape in the past, even made a batch to copy altered files and then XShape to ease the process for me.

[Drakous79]

You can leave XShape path empty and let the user browse for it upon the first launch.

 

There is entry in registry (knowing thanks to BlackAlpha) HKCU\SOFTWARE\Valve\Steam,SteamPath to which can be added the rest \steamapps\common\XCom-Enemy-Unknown.

 

Only downside is my registry entry has forward slash in it: s:/steam

Edited December 15, 2012 by Drakous79

[twinj]

You can leave XShape path empty and let the user browse for it upon the first launch.

 

There is entry in registry (knowing thanks to BlackAlpha) HKCU\SOFTWARE\Valve\Steam,SteamPath to which can be added the rest \steamapps\common\XCom-Enemy-Unknown.

 

Only downside is my registry entry has forward slash in it: s:/steam

 

Forward slash isnt a problem for java paths.

 

Was this in old version? (You can leave XShape path empty and let the user browse for it upon the first launch.)

 

New version of XShape doesnt have the batch features but will only shape files that are decompressed. The default config is for all files to be checked in the exe list. 19 files. It checks if they are decompressed to see whether to test the hash or not. If we need to repack it needs to be done after xshaping anyway so I saw this as the correct behaviour. To me an installer checks whether a file is uncompressed to install and opens the file if need be. Anyway this XShape will pin point exact location of any resource in the exe using the PE headers. So it is much faster than original version. It also uses a special search algorithm once it pinpoints the location rather than doing a brute force search like the original.

 

These features will help in future for ini editing and updating the header information in case of size changes. Once all patching is done I also want to recalculate the exe checksum or windows will think there is a virus.. as it does on my machine when i xshape. Looking for the correct algorithm to do this.

 

Hehe crazy That's why I keep going. Great work fixing the path! Also scrollbar is great

 

Coz of that I found another scroll bar error with mod files on the install pane... fixed. And thanks! :biggrin:

 

 

Works fine, mod worked in the game, uncompressed_size was correctly removed and Restore Original restored it back along with compressed upk. Only error I got was "XPatch: Error [31] Error pathcing Ini file.". Spelling of patching. I deleted DGC.ini from install folder as the mod doesn't need it and in export.xml is <Ini></Ini>. Is there something like IF INI is empty, do nothing?

 

There should be but I'll triple check it!

 

 

Started XCMM. Got standard "Incorrect settings: Please set up the application correctly." message, clicked OK, set name and paths, clicked Save, got message "Unpack Core.upk: Would you like to unpack the core upk file here to verify the current path?", clicked Yes and got "Game file handler: There was an IO error when modifying the game resources." message with "Gildor's Unreal Extractor: Extracting the resources has completed.". OKed both.

Loading package: S:\Steam\SteamApps\common\XCom-Enemy-Unknown\XComGame\CookedPCConsole\Core.upk Ver: 845/59 Engine: 8916 [FullComp] Names: 828 Exports: 1552 Imports: 18 Game: 8000

S:\Steam\SteamApps\common\XCom-Enemy-Unknown\XComGame\CookedPCConsole\Core.upk: uncompressed size 223449

 

Why am I getting Input Output error?

 

Went to Make tab, filled in all informations, drag and dropped files (one with Drakous79. prefix), clicked Make and got "Make Mod: Confirm to continue." and "Finished Mod: Mod creatin has finished.", but no files were created except SpoilerName.xmod.config.xml.

ORIGINAL FILES [spoilerName.Function]

EDITED FILES [Drakous79.SpoilerName.Function]

MOD GENERATE ACTION

UPK FILENAME [XComStrategyGame.upk]

PROCESSING ORIGINAL [spoilerName.Function] FROM [XComStrategyGame.upk]

SEARCH HASH [ED CB 30 A1 39 34 C8 D5 35 1D 9E C5 6C 6F BE 0B 96 25 36 F4 ]

RESOURCE CHECKSUM [23434]

COMPARING ORIGINAL [spoilerName.Function] WITH EDITED [Drakous79.SpoilerName.Function]

 

Has to be another cross path drive issue... has to be... my files are created. Could be a problem with the System dir call i am making to get program home... Will look into it more.

 

And thanks again dude! Wish I could give you more Kudo's!

 

Vanilla installation/uninstallation worked great. But what about already modded game uninstallation? Do you have any plans for the feature?

 

I want to but its proving a lot of work just for the simple stuff like the above!

[Drakous79]

Was this in old version? (You can leave XShape path empty and let the user browse for it upon the first launch.)

It wasn't. I just thought it could work. Once the user sets the path, there shouldn't be any problems with it. But I guess the best behavior is to provide the path already filled (from registry or something) and let the user check, if is it correct.

 

New version of XShape doesnt have the batch features but will only shape files that are decompressed.

Just added copy to XShape.bat, nothing big. I have workplace outside the game folder.

 

Will give new XShape a go.

 

And thanks again dude! Wish I could give you more Kudo's!

Thanks. You know it would make me fat! :)

 

I want to but its proving a lot of work just for the simple stuff like the above!

I can imagine :( time, compatibility issues. And inis making it a lot more difficult.

 

Maybe you could split the functionality. If a mod performs only UPK surgery and does not edit ini and other text files, allow reverse process for changed values. In case of uninstallation, values are changed back to the original and you can leave uncompressed UPK in CookedPCCConsole, because only rehashing the exe is needed. Such approach would work for vanilla and modded game and would not disrupt already installed mods.

 

It would require adding new button "Uninstall mod". In case of problems, the user can hit Restore Original, or Verify Game Cache with Steam. And in the end, new patch will overwrite it all.

 

With inis it is different. How to track, if the ini was modified by someone else, in case of returning backuped files. FMod's app?

[twinj]

Was this in old version? (You can leave XShape path empty and let the user browse for it upon the first launch.)

It wasn't. I just thought it could work. Once the user sets the path, there shouldn't be any problems with it. But I guess the best behavior is to provide the path already filled (from registry or something) and let the user check, if is it correct.

 

New version of XShape doesnt have the batch features but will only shape files that are decompressed.

Just added copy to XShape.bat, nothing big. I have workplace outside the game folder.

 

Will give new XShape a go.

 

And thanks again dude! Wish I could give you more Kudo's!

Thanks. You know it would make me fat! :)

 

I want to but its proving a lot of work just for the simple stuff like the above!

I can imagine :( time, compatibility issues. And inis making it a lot more difficult.

 

Maybe you could split the functionality. If a mod performs only UPK surgery and does not edit ini and other text files, allow reverse process for changed values. In case of uninstallation, values are changed back to the original and you can leave uncompressed UPK in CookedPCCConsole, because only rehashing the exe is needed. Such approach would work for vanilla and modded game and would not disrupt already installed mods.

 

It would require adding new button "Uninstall mod". In case of problems, the user can hit Restore Original, or Verify Game Cache with Steam. And in the end, new patch will overwrite it all.

 

With inis it is different. How to track, if the ini was modified by someone else, in case of returning backuped files. FMod's app?

 

Yeah FMods app is the way forward if we ever want to see this extra features. We have spoken about interop but I told him I want to get XCMM in top shape first before I update the code to be able to work CMB and CMB use XCMM or XMPI. If we can get better upk support then the idea is to be able to use his files and my files with a supervening install xml which work on both XCMM and CMB.

 

Issue is I really want to make sure that any exe editing is done properly so as to avoid issues with virus checking and crashing. Or at least reduce risks. What we are doing now is fraught with problems. I am working on the checksum updating util for the exe as part of the new XShape work.. Imagine if it worked when steam was online! lol..

 

I do think just simply reversing what was done in an install would be really handy and is what I should do. I promised FMod so after that after christmas! Its funny.. they were looking for developers for XCMM when it was being made in C#. I couldnt get it working and there was no reply to any messages so I just started from fresh.. I think I'm slightly mad! ;)

[Drakous79]

It works with Steam online and preventing phone home. Do you mean you want to update XComGame.exe so the game does not try to reupdate it?

 

C# may be more user friendly, if it just needs .NET installed and updated by Windows. Java requires good installation and many people can't ge it to work.

 

I promised FMod so after that after christmas!

Wonderful, that would be christmas present then :)

 

Have you found, why is "XPatch: Error [31] Error pathcing Ini file." error happening with empty <Ini></Ini>?

[twinj]

It works with Steam online and preventing phone home. Do you mean you want to update XComGame.exe so the game does not try to reupdate it?

 

C# may be more user friendly, if it just needs .NET installed and updated by Windows. Java requires good installation and many people can't ge it to work.

 

I promised FMod so after that after christmas!

Wonderful, that would be christmas present then :)

 

Have you found, why is "XPatch: Error [31] Error pathcing Ini file." error happening with empty <Ini></Ini>?

 

I havent but I suspect its because the line <Ini></Ini> needs to be completely removed.

 

I dont get how people can not get Java to work... It is an installable product just as is .NET. So many websites use java too.. what do they do then? Being a java developer if there is a problem with SE it's because I also have lower versions installed along with SDk's and EE with even different subversions and I can get them all to work. If not I tweak the windows path settings. Like you would have to do with any other program that needs to be called by command line.