FogGene

OK. I think I see what you guys are saying. What I'm calling an Index Table would be then a list of unidentified objects. If so, what I'd have to do is find each object (by index or some other means) in the UPK's Exports Table and get, at least, its size. Luckily for me, from what I read in the R&D - New Modding Tools thread, it seems you guys have already figured that stuff out. Theoretically then, I'd only have to use your code to read the object's size for each of my indexed entries and skip forward. Or I may be understanding this wrong yet again (perks of not being a programmer and not knowing the UE, I imagine). In any case, the first thing to figure out would be what those indexes are to each object in the save (entry in namelist? entry in objectlist? object index?, else?). Then use that info to do the above. That sounds like fun. And likely also beyond my limited capabilities. This was a nice learning exercise, anyhow :smile: Edit: Thank you to all who contributed in this thread for your help and your patience.

Maybe I'm wrong, but I don't think we're talking about the same thing. I've checked your code, and while it is very nice to have it as a reference, properties are not the problem. My problem was, and still is, the index tables. I have not found a tool yet that can take on those, or a known format that comes even close to defining their structures. Then again, I haven't had much time lately to spend on this. They also cannot be skipped easily. Block sizes are variable and undefined; at first I thought the offset for each individual chunk might be calculated as a function of its index position in the list, based on each table's index distribution. But no luck so far. Signature search won't work consistently if the last table is not empty, not that I want to do it that way anyhow. My properties screenshot, the persistent level objects, was just to show current progress in property code (outdated now anyways, but irrelevant). I posted an example screen of hex data for one index table that looked like this: That's just a very small sample, though. If you're curious about this, I have a couple of files with a whole index table collection. One with 21 tables and the other with 30; each table can have from 0 to more than 5k entries, depending on the save; each entry is accompanied by its respective compressed(?) data block. https://skydrive.live.com/redir?resid=DF1F7772939FE5B5!451&authkey=!AJMWDFt74uzq5DM&ithint=folder%2c

It looks like it will take me a while to figure out that list. On the positive side, the GUI and I are reaching an understanding, so now I can display (and edit) object properties individually. Now if I only I knew how to display all properties together...

I've taken a look at the Package Loading part of Content Streaming, but it doesn't seem like the structures match. The whole save structure might be a match to the package format, though. I think you suggested this before. in that case perhaps the collection of Index Tables altogether is an export table? I'm at a loss here, to be honest. This below is an example of an Index Table collection header; as in a single header for all tables. So far none of the data there has helped me figure out a way to get start and end offsets for the collection, which is what I'd need to reliably skip the content (that or the actual format). Edit: code in hexbase; added missing byte.

Well, fumbling around with those saves was very instructive. Progress is slow yet, but I'll post what I've got so far. This is the save structure I'm using: This save structure I have seems for the most part correct. Except for a couple of wrong assumptions I had made regarding signatures, the saves load fine. However, because of this I have hit a major roadblock while reading the saves: - The problem is at the index tables. These are a group of int32 tables with their values ordered following an arithmetic progression. After each table there come unreadable structs of data of undetermined format (some are easy to guess, others nigh impossible). Each of these undefined structs seems to have an int32 index itself, the first 4 bytes. At least it was so for those that I could guess their structure. There are as many structs as indexes in the table, and as you probably have guessed the indexes match. Although remember they follow a specific arithmetic progression. Below there is an example of one of this tables (one entry). The problem is I can't for the life of me figure out these structs. I can skip this stuff by searching for signatures, but it would be a very unreliable way of parsing a save. Also, I think we wanna know what's in there. My guess is that something is reading the tables and the structs, and that something likely has the key to their format. ----------------------------------------- As a sort of better news, I can now read each object's properties and store them away in structs. I think I have figured out pretty much the properties format of all the objects I could find. I'm having a fight with the GUI in trying to figure out how to display them on the PropertyGrid, once I do they'll be editable as well. Screenshot below (sorry for not displaying the data, like I said me and GUI....). Edit: This is your save36 btw, Drakous ------------------------------------------ wghost81, thanks for those links. I'll check them out asap.

Very good info guys, much appreciated. Sorry I haven't responded earlier, I had a couple of busy days and could only do quick forum reads. I'm going to try to reply to a few interesting things, but before that a plea if I may: Is there a chance I could borrow some XCOM: Enemy Within save games to test the explorer? Last patched version, to keep things simple for now. Preferable Tactical View map savegames, but I'll take any Strategy View savegame too, since sooner or later I'll have to get around those anyway. I just finished the GUI for the full save read (it's not final at all, but check the last post here for some pics if curious) and I'd like to see if it works (or more likely where it fails) when reading other saves. --------------------------------------------- @ wghost81, I had checked the upk structure you posted in that thread, but I haven't had the chance to play with UPK Utils yet. Definitely in my to-do list. I hope it'll help me to figure out what everything is once the explorer reads data across saves consistently. ------------------------------------------ @Drakous79, Oh man... those CRCs are the epic find of the year. Remind me to thank you again whenever I get to the repacking part of the project. The rest of the structure seems to confirm my own findings. ----------------------------------------------------- @dubiousintent Thanks for the link. To be honest I'm pretty green in python, not really being a programmer myself. But it could be handy to have that code around. ------------------------------------------------------- This will help to figure out what is what in the save file. The structs look like on of the many property list entries, but I couldn't find one that exactly matched what you posted. I'm gonna try to focus on the property reader now, see if things clear up a little when we can see those properties. Too bad about the native functions, now that would've sped things up :tongue:

Hi, Sorry to resurrect this after almost a month. Hopefully you're still reading it. In my spare time I've been working on an XCOM savegame explorer, so to speak. You can follow the research at this thread in the me3explorer forums. You'll have to read the whole thread to get a feel on progress, I haven't updated the first post. The second page has a bit more up to date info (most of the stuff in the first page I was wrong about), but I still have a long way to go. In a few words, like Amineri and wghost81 have pointed out, the file is a bunch of chunks/blocks compressed in lzo1x format. Each chunk has a header with the signature you found (0x C1 83 2A 9E) and a list of decompressed and compressed sizes. Each block inside the each chunk should also have it's own header with compressed and decompressed size but, at least in the save I'm decoding, each chunk contains exactly one block so the sizes for chunk and block are all the same. We wrote a very simple quickbms script to decompress the blocks. Once decompressed you can see what appears to be a bunch of lists (names, properties, etc...). I haven't been able to make much sense of them yet, although most are quite readable. At the end of the file there is another compressed save file. I'm assuming that since my save is in a map (tactical view), that mysterious file is the strategy view. For now I am focusing on reading till the end of the file and make sure I can consistently do it for all saves. Then I'll start trying to figure things out. I have some C# code uploaded to sourceforge svn, with a GUI that outputs a few lists in readable format. It is private for now, but if you're a developer and are interested we could talk. Finally, I know ToolBoks has save editing functionality, so likely bohauk knows more about this than I do. I should check with him too. Cheers.

Awesome. Thanks very much for the link and the program.

Hi, I was wondering if UPKUtils is the famous UPKModder everybody keeps talking about. I tried searching for it, but I'm afraid I must not be very good at searching.

I'm not the mod author, that's Amineri. But, judging from what she has told us, I doubt it. I'd recommend you to wait until she shares an updated version of the mod. Sorry that I couldn't be of any more help. Now if you excuse me, I'm off to play the game. I haven't played EW yet and I'm eager to see the new stuff :smile:

That's strange. I have a few questions for you, if you don't mind. 1. What game version are you trying to install the mod onto, Enemy Unknown or Enemy Within? 2. If the answer above is Enemy Unknown, have you downgraded to patch 4? 3. What ToolBoks version number are you using? Unfortunately I can't test or even load the mod for a EU install, because I have Enemy Within myself and so I have to use a different ToolBoks. I can tell you that for the EW version of ToolBoks it loads and installs without issue, except for the warnings about the new comment operators. Other than that, the only thing I added to the mod is the link to this thread in the description (you can try to delete that if you want). Everything else is exactly as I used to play it, and it installed and worked fine, I assure you. ----------------------------------------------------------------- @Amineri, Thanks for the reply. It's more than enough to know that you're still around here and even have plans to update the mod eventually. Cheers

Hey, I have the ToolBoks custom mod I used for EU and I'd have no problem sharing it with you, Pukix. Thing is, I've been away for quite a while and I'm a bit reluctant to share anything that could break your game. I don't think these changes have been updated to work with Enemy Within, and maybe not even with any recent patches. Does anybody know what are the current instructions for players that don't have EW? Have any recent patches screwed old EU mods, such as this one? --------------------- Edit: Heh, I just saw a thread regarding this that might be an interesting read, to whoever it may concern: http://forums.nexusmods.com/index.php?/topic/1235585-downgrade-enemy-unknown-to-patch-4-356266-r-d/ Pukix, it seems you can downgrade to Patch 4 (see link above) and use old mods. Not sure if this is needed for this mod, though. I'd suggest to ask around before doing anything. In the meantime, I'm leaving here the custom mod I used to play with (see attachment). It should have all the relevant changes Amineri shared in this thread. Still, use at your own risk (and it probably wouldn't hurt if you double checked the hex code in your files, in case there are discrepancies like the ones I posted some time ago in this same thread).

Hey Amineri, I just put it together and did some initial testing. I tried it in a previous save with three countries lost. I had a skeleton key thanks to your code I previously installed and I had already completed the story-mission base assault. This is what I've got so far: - I was able to launch a satellite over each country, which triggered an alien base assault once I got coverage. - I launched one assault for testing purposes and was able to regain the country. Yay! - Upon finding the abduction pods inside the base, Dr. Vahlen dialogue about knowing what happened to the abductees was replayed (I think this was played the first time I assaulted the base, during the story). No other dialogues were played. - On mission completed I retrieved a Hyperwave Beacon, which was added to my storage. - Somehow my localization strings are not updated. I tried editing both XComStrategyGame.int files (one in XComGame folder, other in My Documents\blah...) to no avail. - Not relevant, but something I found interesting: Inside the base there were heavy and elite versions of the enemies, in concordance with the enemies I'm finding now during my regular missions. I have included johnnylump fixes, except for the change to XGSatelliteSitRoomUI.UpdateContinent, which I kept vanilla (I couldn't find these change in your posts though, maybe I missed that?). Edit: And I forgot to say, thank you so so much for sharing this. I should get that virtual goat now :D Also, for science, some discrepancies between your original hex and mine. XGCountry.SetSatelliteCoverage // Prevent satellite coverage over occupied countries from granting bonuses - Body Amineri 07 1B 00 F2 2D 01 32 24 00 00 2D 00 78 24 00 00 16 04 0B 14 2D 01 32 24 00 00 2D 00 78 24 00 00 19 1B 23 06 00 00 00 00 00 00 35 5D 1E 00 00 64 1E 00 00 00 00 01 34 24 00 00 16 30 00 00 00 00 00 00 1B F1 23 00 00 00 00 00 00 35 63 1E 00 00 64 1E 00 00 00 00 01 34 24 00 00 2D 00 78 24 00 00 16 07 D5 00 2D 00 78 24 00 00 1B B3 00 00 00 00 00 00 00 12 20 6D FE FF FF 09 00 D5 FB FF FF 00 02 D5 FB FF FF 4A 16 1B 64 03 00 00 00 00 00 00 16 06 F5 00 1B 64 03 00 00 00 00 00 00 16 40 01 2D 24 00 00 25 36 01 2D 24 00 00 16 04 0B 53 Fog.Gene 07 1B 00 F2 2D 01 32 24 00 00 2D 00 78 24 00 00 16 04 0B 14 2D 01 32 24 00 00 2D 00 78 24 00 00 19 1B 23 06 00 00 00 00 00 00 35 5D 1E 00 00 64 1E 00 00 00 00 01 34 24 00 00 16 30 00 00 00 00 00 00 1B F1 23 00 00 00 00 00 00 35 63 1E 00 00 64 1E 00 00 00 00 01 34 24 00 00 2D 00 78 24 00 00 16 07 D5 00 2D 00 78 24 00 00 1B B3 00 00 00 00 00 00 00 12 20 6D FE FF FF 09 00 D5 FB FF FF 00 02 D5 FB FF FF 4A 16 1B 64 03 00 00 00 00 00 00 16 06 F5 00 1B 20 27 00 00 00 00 00 00 16 40 01 2D 24 00 00 25 36 01 2D 24 00 00 16 04 0B 53 XGStrategyAI.CreateAlienBase // Body Amineri 0F 00 05 43 00 00 1C A3 FD FF FF 20 75 37 00 00 4A 4A 4A 4A 4A 4A 4A 16 0F 19 00 05 43 00 00 09 00 31 37 00 00 00 01 31 37 00 00 1C A3 FD FF FF 20 7A FE FF FF 4A 4A 4A 4A 4A 4A 4A 16 05 6C FC FF FF 00 A7 2C 05 16 0A C5 00 25 0F 19 00 05 43 00 00 09 00 36 37 00 00 00 01 36 37 00 00 25 0F 19 00 05 43 00 00 09 00 39 37 00 00 00 01 39 37 00 00 1C B9 FD FF FF 1E BE 9F 9A 3E 1E 5E BA C9 3E 16 06 36 02 0A 20 01 26 0F 19 00 05 43 00 00 09 00 36 37 00 00 00 01 36 37 00 00 2C 04 0F 19 00 05 43 00 00 09 00 39 37 00 00 00 01 39 37 00 00 1C B9 FD FF FF 1E BE 9F 1A 3F 1E 10 58 F9 3E 16 06 36 02 0A 7C 01 2C 02 0F 19 00 05 43 00 00 09 00 36 37 00 00 00 01 36 37 00 00 2C 02 0F 19 00 05 43 00 00 09 00 39 37 00 00 00 01 39 37 00 00 1C B9 FD FF FF 1E 2F DD E4 3E 1E 29 5C 0F 3E 16 06 36 02 0A D8 01 2C 03 0F 19 00 05 43 00 00 09 00 36 37 00 00 00 01 36 37 00 00 2C 03 0F 19 00 05 43 00 00 09 00 39 37 00 00 00 01 39 37 00 00 1C B9 FD FF FF 1E 58 39 54 3F 1E 96 43 0B 3E 16 06 36 02 0A 33 02 2C 04 0F 19 00 05 43 00 00 09 00 36 37 00 00 00 01 36 37 00 00 26 0F 19 00 05 43 00 00 09 00 39 37 00 00 00 01 39 37 00 00 1C B9 FD FF FF 1E 2B 87 96 3E 1E C5 20 F0 3E 16 06 36 02 0A FF FF 0F 19 19 00 05 43 00 00 09 00 31 37 00 00 00 01 31 37 00 00 09 00 C1 F9 FF FF 00 01 C1 F9 FF FF 1B C4 07 00 00 00 00 00 00 16 0F 10 2C AC 19 00 05 43 00 00 09 00 32 37 00 00 00 01 32 37 00 00 2C 32 0F 10 2C B3 19 00 05 43 00 00 09 00 32 37 00 00 00 01 32 37 00 00 2C 03 0F 10 2C AB 19 00 05 43 00 00 09 00 32 37 00 00 00 01 32 37 00 00 38 44 AB 10 19 1B 4C 0E 00 00 00 00 00 00 16 0A 00 43 41 00 00 00 1B F6 0E 00 00 00 00 00 00 16 12 20 6D FE FF FF 09 00 CA FF FF FF 00 02 CA FF FF FF 38 3F 10 2C B3 19 00 05 43 00 00 09 00 32 37 00 00 00 01 32 37 00 00 16 0F 10 2C B2 19 00 05 43 00 00 09 00 32 37 00 00 00 01 32 37 00 00 2C 04 0F 10 2C B0 19 00 05 43 00 00 09 00 32 37 00 00 00 01 32 37 00 00 2C 0F 0F 10 2C AF 19 00 05 43 00 00 09 00 32 37 00 00 00 01 32 37 00 00 2C 0A 0F 10 2C AE 19 00 05 43 00 00 09 00 32 37 00 00 00 01 32 37 00 00 26 0F 10 2C B4 19 00 05 43 00 00 09 00 32 37 00 00 00 01 32 37 00 00 26 19 1B 69 0E 00 00 00 00 00 00 16 14 00 AC 2F 00 00 00 1B A9 00 00 00 00 00 00 00 00 05 43 00 00 4A 16 04 0B 53 Fog.Gene 0F 00 05 43 00 00 1C A3 FD FF FF 20 75 37 00 00 4A 4A 4A 4A 4A 4A 4A 16 0F 19 00 05 43 00 00 09 00 31 37 00 00 00 01 31 37 00 00 1C A3 FD FF FF 20 7A FE FF FF 4A 4A 4A 4A 4A 4A 4A 16 05 6C FC FF FF 00 A7 2C 05 16 0A C5 00 25 0F 19 00 05 43 00 00 09 00 36 37 00 00 00 01 36 37 00 00 25 0F 19 00 05 43 00 00 09 00 39 37 00 00 00 01 39 37 00 00 1C B9 FD FF FF 1E BE 9F 9A 3E 1E 5E BA C9 3E 16 06 36 02 0A 20 01 26 0F 19 00 05 43 00 00 09 00 36 37 00 00 00 01 36 37 00 00 2C 04 0F 19 00 05 43 00 00 09 00 39 37 00 00 00 01 39 37 00 00 1C B9 FD FF FF 1E BE 9F 1A 3F 1E 10 58 F9 3E 16 06 36 02 0A 7C 01 2C 02 0F 19 00 05 43 00 00 09 00 36 37 00 00 00 01 36 37 00 00 2C 02 0F 19 00 05 43 00 00 09 00 39 37 00 00 00 01 39 37 00 00 1C B9 FD FF FF 1E 2F DD E4 3E 1E 29 5C 0F 3E 16 06 36 02 0A D8 01 2C 03 0F 19 00 05 43 00 00 09 00 36 37 00 00 00 01 36 37 00 00 2C 03 0F 19 00 05 43 00 00 09 00 39 37 00 00 00 01 39 37 00 00 1C B9 FD FF FF 1E 58 39 54 3F 1E 96 43 0B 3E 16 06 36 02 0A 33 02 2C 04 0F 19 00 05 43 00 00 09 00 36 37 00 00 00 01 36 37 00 00 26 0F 19 00 05 43 00 00 09 00 39 37 00 00 00 01 39 37 00 00 1C B9 FD FF FF 1E 2B 87 96 3E 1E C5 20 F0 3E 16 06 36 02 0A FF FF 0F 19 19 00 05 43 00 00 09 00 31 37 00 00 00 01 31 37 00 00 09 00 C1 F9 FF FF 00 01 C1 F9 FF FF 1B C4 07 00 00 00 00 00 00 16 0F 10 2C AC 19 00 05 43 00 00 09 00 32 37 00 00 00 01 32 37 00 00 2C 4B 0F 10 2C B3 19 00 05 43 00 00 09 00 32 37 00 00 00 01 32 37 00 00 2C 03 0F 10 2C AB 19 00 05 43 00 00 09 00 32 37 00 00 00 01 32 37 00 00 38 44 AB 10 19 1B 4C 0E 00 00 00 00 00 00 16 0A 00 43 41 00 00 00 1B F6 0E 00 00 00 00 00 00 16 12 20 6D FE FF FF 09 00 CA FF FF FF 00 02 CA FF FF FF 38 3F 10 2C B3 19 00 05 43 00 00 09 00 32 37 00 00 00 01 32 37 00 00 16 0F 10 2C B2 19 00 05 43 00 00 09 00 32 37 00 00 00 01 32 37 00 00 2C 04 0F 10 2C B0 19 00 05 43 00 00 09 00 32 37 00 00 00 01 32 37 00 00 2C 0F 0F 10 2C AF 19 00 05 43 00 00 09 00 32 37 00 00 00 01 32 37 00 00 2C 0A 0F 10 2C AE 19 00 05 43 00 00 09 00 32 37 00 00 00 01 32 37 00 00 26 0F 10 2C B4 19 00 05 43 00 00 09 00 32 37 00 00 00 01 32 37 00 00 26 19 1B 69 0E 00 00 00 00 00 00 16 14 00 AC 2F 00 00 00 1B A9 00 00 00 00 00 00 00 00 05 43 00 00 4A 16 04 0B 53 Shamed to say I don't have any questions regarding your changes, I just apply them with blind faith. At some point I should learn more about UE3 coding, but I'm not there yet. For now, just a tester :smile:

Hey Amineri, Just wanted to chime in one last time and say that following your instructions worked wonderfully. Alien turns are a bit longer now, but not unreasonably so (I actually like it in fact, builds up more tension). In return, Aliens do walk around the map instead of teleporting around, patrols I spot with battle scanners no longer warp away and aliens converge into the action areas in a realistic way. Also, these changes seem to have the effect of making alien patrols stumble into the XCOM team more often on their own. Not sure if you intended this last thing, but it is awesome. All in all, very pleased with these changes. Thanks a lot for sharing.

Wow that's a lot of hex changes. I'll put together a ToolBoks mod and test it ASAP. @Krazyguy75, yes I pretty much missed all that. Thanks. @johnnylump, thanks for the code updates, much appreciated.