Finding the first step?

[Daemonjax]

Oh man, I'm so sorry. I figured this out last week with a different game and didn't think to include it in my post. Maybe if i came back more often, I could be a little more help... Also, I'm embarrassed I didn't think to try the decompressed file's hash.

great GD work

 

Naw, it's cool.. it was only like 8 hours of my life or whatever. ;)

[Gazzzz]

I haven't seen it mentioned here but this is what I discovered accidentially.

 

http://www.hentschke-keramik.de/transfer/xcomgame.jpg

 

Same message for xcomstrategygame.

(I had different "tinkering" backups in different sub-folders)

 

If the game automatically uses equally named (uncompressed) files, the whole argument of how to properly cook them might be moot.

[Daemonjax]

I haven't seen it mentioned here but this is what I discovered accidentially.

 

<image>

 

Same message for xcomstrategygame.

(I had different "tinkering" backups in different sub-folders)

 

If the game automatically uses equally named (uncompressed) files, the whole argument of how to properly cook them might be moot.

 

 

So, does it let you bypass any of the steps we've come up with to sidestep the game's compression/hashing?

Edited October 20, 2012 by Daemonjax

[Lethargus]

People may be getting a little ahead of themselves. We can unpack/decompile/repack and get the game to use a repacked/uncompressed upk. All really great stuff. But there's still the problem of recompiling any edited scripts. UE Explorer lets us export the decompiled scripts one at a time, but if you take a look at the extracted scripts themselves they're a series of binary files. We can't just paste in a text script and expect it to work.

 

This means we can edit simple things in the scripts like changing constants, or, if you happen to know the byte code for Unreal script change == to !=, < to >, things like that. But to still create entirely new effects/items/abilities I think we still need to find a way to re-cook the extracted files or at least recompile a script file.

 

Experimenting with extracting scripts using UE Explorer and then trying to rebuild them with the UDK has proved somewhat promising, but there's still a long way to go. There's a lot of setup required and so far I'm only able to recompile very very simple scripts because of all the inter-dependencies. When I try to recompile anything complex I run into syntax errors which means UE Explorer may be too old for the version of the UDK I'm using. Some editing might be able to solve that, since the meat of the content is there, but it would take a LONG time since youhave to export and then edit each class individually.

[Daemonjax]

I'm not even thinking about anything like that.

 

I think we should just be happy with hexediting upks for now.

 

Maybe we can even make a tool to make the hexediting faster.

 

Hoping for anything more and we're likely to be disappointed, but I'm more than willing to be pleasantly surprised. ;)

[Lethargus]

What would be the point of editing the upk files then? Replace textures and models I guess... or the very simple bytecode changes I mentioned above... anything beyond that really will require recompiling. Replacing the simple scripts I can get to recompile is not a problem. So to replace existing scripts the process would be:

 

1) Decompress the game upk file with the script to be replaced.

2) UPKUnpack the decompressed upk file

3) UE Explorer the decompressed upk file and export to your UDK build directory ALL its scripts

4) Now do the same for every upk file it depends on (probably XComGame and others)

5) Now edit ALL these .uc files to actually compile with your version of the UDK (This is the hard part imo, and I'm having limited success)

6) At this point you more or less have the game's build environment. Edit the scripts how you want, or add new scripts

7) Compile scripts with UDK to the .u files

8) UPKUnpack the compiled .u files

9) Copy and replace the edited script files over the unpacked files from step 2. If you added scripts you'll need to edit the index files to include them

10) UPKRepack the directory created in step 2, now with the edited/new scripts

11) Delete/rename any compressed size files from the game directory you no longer need

12) Resource edit the .exe to have the new SHA hash values for the edited upk files

 

If anyone can get to step 6 for the core upk files they could zip them up and share with community, perhaps. I don't think I have the time to go through ALL of them. And that's assuming you can match up the table definition locations correctly, because right now UE Explorer will include needed table definitions in all the exported classes that use them, which results in compile time errors until you figure out which class should truly be the owner. Small things like that aren't hard, they would just take a whole lot of time. More than I have free.

 

EDIT: You need to keep all the internal references the same too, which is why it's looking like you have to extract ALL the scripts and resources from a .upk file and basically rebuild the entire thing. Replacing individual edited scripts may or may not work depending on the script and the changes.

Edited October 20, 2012 by Lethargus

[Daemonjax]

If anyone can get to step 6 for the core upk files they could zip them up and share with community, perhaps.

 

Potential copyright problems with that. We should just share the changes itself, and not distribute game assets.

[MaliciousRean]

there shouldnt be any copyright issues unless you try to sell stuff.

the worst thing that can happen is that you'll be asked to take it down. nobody will go after simple modders with lawsuits.

Edited October 20, 2012 by MaliciousRean

[dose206]

there shouldnt be any copyright issues unless you try to sell stuff.

the worst thing that can happen is that you'll be asked to take it down. nobody will go after simple modders with lawsuits.

 

Have you looked at copyright law lately? It is my (admittedly non-expert) understanding that generally ANY redistribution of copyrighted material is forboden, whether for monetary gain or not. Anyhow, if it can be (moderately) easily worked around, there's no reason to step on toes.

Edited October 20, 2012 by dose206

[Lethargus]

Well, bad news anyway. I found some short cuts that let me completely extract all the classes in XComGame and XComStrategyGame, but they won't compile. They depend on what appears to be customized Engine and Core components (not surprising) so the simple solution was to overwrite the default UDK ones with decompiled versions from XCom. This is where I ran into a problem... The free UDK refuses to build them because it complains about DLLBind classes not being allowed to be native. Not really surprising.

 

I managed to remove the dependent scripts and try to compile just the majority of XComGame, but the UDK compiler seems to go into an infinite loop for me with no errors or warnings. I'll let it run for a while and see if it's really stuck, or just trying to handle the large number of files. Either way, it's not looking good for drastic script changes. I believe it's still possible, just looking to be beyond my capabilities since I don't know the upk format as well as I would like.