XCMM - XCom Mod Manager

twinj · December 3, 2012

XCMM - Xcom Mod manager Test version.

GitHub repository - https://github.com/twinj/XCMM Git Hub removed download support...

LATEST VERSION 1.08a BETA - https://dl.dropbox.com/u/54222881/XCMM-v108aBETA.zip Drop box.

Portable Installer with demo mod.http://forums.nexusmods.com/index.php?app=core&module=attach&section=attach&attach_id=28237

This version has caused me a headache. It has a new uninstall feature which has caused me to write a lot of new code and to refactor even more.... I have not added improved XShape yet. I have not added FMods interop yet. Console is broken this will now only get fixed once all gui components work properly.

Uninstall features and direction
Can make a mod 'active' and 'inactive by installing and uninstalling.
Can NOT currently overwrite same resource with new mods. Resource example: CreateTSoldier.Function
Can write several patches to a upk.
Writing in a game state and install state feature so can track the current state of each resource and change the search as per the install.
Install tracking is half active where an Xmod saves its install log. The data here can be used as the new Resource so as to input new search when updating.
Not as yet sure how to tackle when a resource offset is in conflict with two mods. But there will be a solution.
Gamestate will allow modders to patch in any small mods and the create a mod based on the gamestate. In progress.

I would highly appreciate any testers and new testers as this is becoming quite large. I WILL need to reqwrite much of the GUI event handling as it is proving too sensitive to changes even thought the logic is the basically the same.

PI features: PI is not yet updated this is a test version only.

Install upk changes from embedded install.xmod.export.xml
Install ini changes from embedded install.ini ( Currently only supports XComGameCore.ini or DefaultGameCore.ini )
XShape XComGame.exe automatically.
Ability to restore game to state before install. (This is done by files copies NB: The installer only works for one mod install >> uninstall. It currently does not support multiple mod installs. If you want to modify further uninstall mod. Edit install to incorporate new mods then reinstall.
Ability to patch in small HTML to act as a readme for the mod.
Console output for debugging.

XMOD XML EXAMPLE

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<XComMod>
   <Name>ModEst1999</Name>
   <Author>Drakous79 / DaemonJax / twinj</Author>
   <Description>Changes the start year to 1999. The same year XCom classic was based on. Also changes the chance that a new rookie soldier is a woman. Because women were tougher in the 90's. Changes the chance from 33% to 50%.</Description>
   <Hash>C56E4E3655983D6DEE0522C04DEE6519A3BDF5EB</Hash>
   <ChangedFiles>
       <ResFile>
           <ResourceName>CreateTSoldier.Function</ResourceName>
           <UpkFilename>XComGame.upk</UpkFilename>
           <ByteSum>104516</ByteSum>
           <SearchHashLength>2023</SearchHashLength>
           <SearchHash>092450C0A758BADAE0552BBF8F0170CB821A7FE0</SearchHash>
           <Changes>
               <HexEdit>
                   <Offset>374</Offset>
                   <Data>02</Data>
               </HexEdit>
           </Changes>
       </ResFile>
       <ResFile>
           <ResourceName>IsFirstDay.Function</ResourceName>
           <UpkFilename>XComStrategyGame.upk</UpkFilename>
           <ByteSum>3110</ByteSum>
           <SearchHashLength>110</SearchHashLength>
           <SearchHash>A31541C2068D03EA1C20BD41C7ACEF102722ACB3</SearchHash>
           <Changes>
               <HexEdit>
                   <Offset>70</Offset>
                   <Data>CF</Data>
               </HexEdit>
           </Changes>
       </ResFile>
       <ResFile>
           <ResourceName>CreateMission.Function</ResourceName>
           <UpkFilename>XComStrategyGame.upk</UpkFilename>
           <ByteSum>40791</ByteSum>
           <SearchHashLength>1250</SearchHashLength>
           <SearchHash>0B8662E7723C262054DA2BB30B420AF1A6446311</SearchHash>
           <Changes>
               <HexEdit>
                   <Offset>101</Offset>
                   <Data>CF</Data>
               </HexEdit>
           </Changes>
       </ResFile>
       <ResFile>
           <ResourceName>InitNewGame.Function</ResourceName>
           <UpkFilename>XComStrategyGame.upk</UpkFilename>
           <ByteSum>4378</ByteSum>
           <SearchHashLength>143</SearchHashLength>
           <SearchHash>FC4EC23447B6D34BB1D36F3ED1C122EA02F92C90</SearchHash>
           <Changes>
               <HexEdit>
                   <Offset>101</Offset>
                   <Data>CF</Data>
               </HexEdit>
           </Changes>
       </ResFile>
       <ResFile>
           <ResourceName>InitNewGame.Function</ResourceName>
           <UpkFilename>XComStrategyGame.upk</UpkFilename>
           <ByteSum>9361</ByteSum>
           <SearchHashLength>301</SearchHashLength>
           <SearchHash>F37A2B9DD67AFEE415F682EC802D645037919D6C</SearchHash>
           <Changes>
               <HexEdit>
                   <Offset>101</Offset>
                   <Data>CF</Data>
               </HexEdit>
           </Changes>
       </ResFile>
   </ChangedFiles>
</XComMod>

OLD VERSIONS

1.06

There are bugs with installer where sometimes it will not handle decompressed files or restore them correctly. Replacing the files with original ones can do the trick. This is rare though... I cannot as yet pinpoint the cause.

This is a bug fix version with improvements to XShape and upk file extraction. Can now extract more than one file at a time. This is kicked off only when making mods. Can add better extraction policy if wanted.

Most of the refactoring came about for the the Portable Installer. I am now working on shrinking its size so it can be guaranteed to fit on attachment for most mods.

XShape is bugged in 1.05.

I know these are coming quickly but I want to get to a point where I can fork development to work on uninstallation/proper patching/ dynamic mod patching and portability. I want makers who use XCMM to be able to create static mods packages which contains all the ini changes, all upk changes and anything else into a self contained small and portable installable package without the need for users to download the larger XCMM.

I also ran into some painful bugs during this update session so didn't get much done. But I did add a java version of modPatcher. While probably not as efficient as RcData handling from Win32, as it seems dreadylein modPatcher uses, it does however allow you to mod a file with existing upk resource changes. I will be improving upon this as at the moment it does not check for file size changes or incompatibilty. So to edit a file you must keep the values with their current space.. that is if there is a '2' it cannot become a '21'. This is because I do not check the file padding when data changes.

I will be working towards GUI editing of Ini settings which are then exportable in a similar format as the Upk mods.

An important feature is one where upk modding can be done on the fly rather than statically.

I fixed other minor flaws in settings and with buttons.

V 1.04 - BETA - https://github.com/downloads/twinj/XCMM/XCMM-V1.04-BETA.zip

Small fixes to settings and config saver. Fixes error where not extracting resources but says it has.
When dropping or adding original files. Make sure they are form the unpacked path you gave. You will get a hash error if you do not.
If you have an edited resource which shares a name with another. Rename it if you want to save it in the mods save location or one of the files will be overwritten. This is not an issue for original files they are saved in relative paths. Looking for a better way to handle the files when making.
You must manually make sure that each edited file you drop into the maker is in the same order as the files you dropped in the edited.
If you delete any mods you can refresh the screen.
XShape outputs only console text on the Home panel. There are no windows yet.
The system tracks all edited upks within a session. You can run XShape manullay at the end to patch the exe.
When restoring any uncompressed files. XShape will test each upks hash. This is accessed by using the "Restore Original' button.
If for some reason a Maker session fails and you have to restart XCMM. The session is remembered. Fix any invalid settings.
The Gildor tool downloads open a windows to his website so you:
2. know where they came from so I do not get credit.
3. Could possibly offer him a donation.
4. Join his Facebook group.
[*]I have made many changes to the Gui version and have not tested the console versions for some time. They are likely broken. For now I do not see them as necessary to fix.

V 1.03 - BETA removed from github

This version is much more robust than the previous versions.

The library is quite extensive now. Will be able to add ini support, and on the fly mod patching.

Can now restore edited Upk's. This only works if files backed up or uncompressed by XCMM.
Core.upk must now be uncompressed to verify an unpacked path. XCMM should however do this for you.
Config file now saves.
Downloading tools now works correctly. Will add better support for external tools after we have basic functionality.
Making a mod from multiple upk files with multiple resource changes now works.
Installing a mod from multiple upk files with multiple resource changes now works.
XShaping the above enables game play.
and more...

Edited December 19, 2012 by twinj

twinj · December 11, 2012

Looking for user input for direction of XCMM. Anything goes.

Drakous79 · December 12, 2012

Hello :)

Tested XCMM 1.06 BETA and had few issues:

Tools are broken. Clicked yes to download, web browser opened at Gildor's page. Next I got error "Incorrect setting. The settings have been updated.". Only decompress.zip was downloaded and Tools folder is empty. Downloaded from main XCMM dialog. Archives are broken, nothing unzipped. Had to provide tools manually.
```
DOWNLOADED [http://www.gildor.org/down/32/umodel/decompress.zip]
DOWNLOADED [http://www.gildor.org/down/32/umodel/extract.zip]
```
Every start I got "Please set up the application correctly", while I already saved XCOM Config Settings.
After pressing Save in XCOM Config Settings dialog, I get error "The system cannot verify your unpacked resources.". Still checking Core.upk at C:\ and not in my path?
Once again I was lazy and want to have original and edited file in the same directory, So added _Drakous79 to edited file name before function extension. Got error "The number of files must be the same. Please correct mistake.". When they had the same name, all went fine.
When creating a mod with Make, I got this error: "Could not save mod files. Try again.". But seems like all files were saved (export and config xml, original and edited files).

Tried installing a mod to vanilla game. Clicked Yes for decompression and had to click Install again.At the end I got asked if to update changes to game executable with XShape and got "Error [30] Patching was not required." after clicking Yes. Mod failed to install (Name is a function I'm working on).

INSTALLING RESOURCE CHANGES
****************************************************************************************
SEARCHING [XComStrategyGame.upk] FOR RESOURCE [Name.Function]
WITH CHECKSUM [23434]
WITH HASH [ED CB 30 A1 39 34 C8 D5 35 1D 9E C5 6C 6F BE 0B 96 25 36 F4]
SEARCHING....
FOUND - CHECKSUM & HASH MATCH
WRITING TO RESOURCE [Name.Function]

Also XCMM did not remove XComStrategyGame.upk.uncompressed_size. Ran the game with XCMM still up and got Steam error "Failed to run game (unknown error)". After exiting XCMM, all was okay.

Personal wishes for XCMM:

Please fix XCMM checking Core.upk at wrong path and broken tools before moving on. I just wanted to do simple mode, one true to false and spent an hour with troubleshooting. I really look forward, when I run new version of XCMM and everything runs smoothly without errors.
Dynamic mods - so users can change values of the mod.
Would be great to have Make tab empty of files and text with my Author name predefined from XCOM Config Settings on the first start.
XCMM to continue installing a mod, after I click Yes to decompress if UPK files is compressed.
More people to help with testing and ideas.

Tested Portable 1.06 BETA:

Failed at decompression, paths were set correctly.

// notify.log
ERROR: Write failed

// Console output
D [XComStrategyGame.upk]
Error [18] Cooked Upk file still compressed.
DECOMPRESSED [XComStrategyGame.upk]

Personal wishes for Portable:

Portable window resizes after start to with and height defined in html.

I know I post all errors and stuff, but it's what I'm supposed to do :) Keep up the good work! Just make it error free now, so it's user friendly and we can use it for mod distribution.

Edited December 12, 2012 by Drakous79

twinj · December 13, 2012

Tools are broken. Clicked yes to download, web browser opened at Gildor's page. Next I got error "Incorrect setting. The settings have been updated.". Only decompress.zip was downloaded and Tools folder is empty. Downloaded from main XCMM dialog. Archives are broken, nothing unzipped. Had to provide tools manually.
DOWNLOADED [http://www.gildor.org/down/32/umodel/decompress.zip]
DOWNLOADED [http://www.gildor.org/down/32/umodel/extract.zip]

Gildor has updated his link. It is now version 33. I'll fix it. I'd rather just distribute the tools with the package. But he wants people to visit his website. So I took the auto download path with a window opening. Gildor also warns against this as he updates his links. They are easy to change and the tools can be setup manualy if need be. I still think an automated download is easier.

Every start I got "Please set up the application correctly", while I already saved XCOM Config Settings.

Cannot reproduce this error. Probably fixed as a result of fixing the above.

After pressing Save in XCOM Config Settings dialog, I get error "The system cannot verify your unpacked resources.". Still checking Core.upk at C:\ and not in my path?

I was sure this was fixed. Must have overlooked something. On my tests however cannot reproduce. Maybe leave till next version see what happens.

Once again I was lazy and want to have original and edited file in the same directory, So added _Drakous79 to edited file name before function extension. Got error "The number of files must be the same. Please correct mistake.". When they had the same name, all went fine.

I think I added some support for this? Dont know if it was in last patch. But you cannot have filenames that end incorrectly. Eg: Name.Function and XCOM.Name.Function will verify a match. Xcom.Name.Function.Drak and Name.Function will not. For filename end to test correctly it must be delimited by dots. Am moving towards ability to drop original files from anywhere but they must fit the naming convention. Generally the idea is once you have made the mod once you will have original files with the correct convention and can remake using those.

When creating a mod with Make, I got this error: "Could not save mod files. Try again.". But seems like all files were saved (export and config xml, original and edited files).

This is a stupid bug even though I am testing if an ini file does not exist in the mod it is still trying to save it... weird. Ok found error. It is letting me create a valid path with a null. Seems to be fixed. (notice I am updating and reporting at the same time. lol)

Would be great to have Make tab empty of files and text with my Author name predefined from XCOM Config Settings on the first start.

Removed the history xml from future distributions. It is still generated from users make attempts. Name now shows up as it should on first use.

XCMM to continue installing a mod, after I click Yes to decompress if UPK files is compressed.

Somewhat difficult due to the way threads are handled in swing and how I implemented the background tasks but I was able to tweak it in the end. Hope it hasn't broken Pinstaller. Need to refactor the GUI code so changes in one can affect the other when code is changed. Currently the code is a copy.

More people to help with testing and ideas.

I know right. However XCMM is still new and is probably not very helpful for those who dont really get how to work it when there is bugs. I know how to work around them and half the time dont find them coz of this. Tis why youve been an absolute asset. Need some simple docs or wiki on git hub. For me it saves me heaps of time testing mods. I can make a mod test it restore, try again really easy. Speeds up the process so much. Even the history for failed mod creation saves heaps of time. Testing XCMM is also getting easier... Just rename history mod. Create >> Install >> restore >> Xshape >> create!

Have been talking to FMod about some ideas and interaction with his tool for ini and int editing. He also would like to move toward dynamic mods. I need to refactor some code so we can achieve interoperability. Will be doing this before next release. This is why I am doing a call for more testing on this version so I can get it to a real working point.

Tried installing a mod to vanilla game. At the end I got asked if to update changes to game executable with XShape and got "Error [30] Patching was not required." after clicking Yes. Mod failed to install (Name is a function I'm working on).

Requires further testing. Seems to be a new problem with XShape exiting prematurely.

Portable window resizes after start to with and height defined in html.

I'll see if I can fit it in.

Also XCMM did not remove XComStrategyGame.upk.uncompressed_size. Ran the game with XCMM still up and got Steam error "Failed to run game (unknown error)". After exiting XCMM, all was okay.

I removed the file locks. Should fix not tested. They were annoying me too. Did not remove file? Requires further testing. Could not reproduce.

twinj · December 13, 2012

Found why XShape not patching... there was a break statement where it should not have been. I thought I tested installs.. Remember how I was experimenting with replacing full resources. It was that feature which broke it. ? But I was SURE that it wasnt in the last release? Anyway its fixed. Just need to test install and pinstall.

bokauk · December 13, 2012

Gildor has updated his link. It is now version 33. I'll fix it. I'd rather just distribute the tools with the package. But he wants people to visit his website. So I took the auto download path with a window opening. Gildor also warns against this as he updates his links. They are easy to change and the tools can be setup manualy if need be. I still think an automated download is easier.

I asked Gildor if I could distribute decompress.exe with ToolBoks which he was fine with and gave me permission. You should send him a quick PM on his forum too. Of course, I've credited him and linked to his site in the ReadMe :)

As you mentioned, he specifically asks not to link directly to his files, so getting permission would surely be best for everyone and also more convenient :)

twinj · December 13, 2012

Thanks sent a message! And already got a reply! Apparently the third to ask this week.! Edited December 13, 2012 by twinj

twinj · December 13, 2012

Anyone feel like testing the fixes? I am keeping this some what closed until release for now. I cant access my downloads on GitHub anyway.. lol

I want the next release to include the changes I talked to FMod about and I am working on upating XShape to not need a pin pointed offset or to go through whole file just to find the resources. My beta of these is working very well. It pin pointe the start of a resource! This has multiple uses.

XCMM 107c

Portable Installer

All the tests I performed. They both seem ok.

Drakous79 · December 13, 2012

Thank you for fixes and improvements. I'll test right after ordering some christmas presents :)

Asked Gildor about decompress today as well hehe.

Also I got the naming convention now. I just did it differently and couldn't understand what's happening.

Nice work updating and reporting at the same time. I'm also writing the report when testing.

Edited December 13, 2012 by Drakous79

twinj · December 13, 2012

Personal wishes for Portable:
Portable window resizes after start to with and height defined in html.

Oh I didn't add this. I only chucked in a frame repack call. This may adjust the window size but may be limited in scope. It was the easy option for now.

When I added it it was more for links and a break down of the mod. Parsing size information would be better placed in the install.xml that is coming as part of the interop with FMods CBD.

File locks do not seem to be closing.. making use of Java new AutoClosable feature...

Edited December 13, 2012 by twinj

XCOM

XCMM - XCom Mod Manager

Recommended Posts

twinj

Link to comment

Share on other sites

twinj

Link to comment

Share on other sites

Drakous79

Link to comment

Share on other sites

twinj

Link to comment

Share on other sites

twinj

Link to comment

Share on other sites

bokauk

Link to comment

Share on other sites

twinj

Link to comment

Share on other sites

twinj

Link to comment

Share on other sites

Drakous79

Link to comment

Share on other sites

twinj

Link to comment

Share on other sites

Recently Browsing 0 members

Browse

Activity

My Activity Streams

Explore