Daemonjax

No. Modpatcher patches the exe with ini settings, because the game doesn't actually look at the DefaultGameCore.ini file on your hard drive. I don't see how you can confuse that with what we're talking about, especially since it appeared you had such a good understanding so far. :D Unless you mean how modpatcher is essentially a program that reads and modifies file(s)... Then, yes, they're essentially the same. ;) Anyways, I guess we better start this thing rolling. I don't think any more contributors will show up at this point. Let's hold off on the GUI part, as that'll take longer than anything else. Are you familiar with using version control software like SVN?

The RNG seems like it might be a bit on the streaky side, but over time it will average out for you (maybe a few hundred shots).

So simple. Why didn't I think of that? @echo off start /b /wait modpatcher DefaultGameCore.mod start /b /wait java -jar XSHAPE.jar 26444500 pause

Wait, there's a packer? I've been hex editing the the upk directly. As far as I know the packer works (source: dreadylein). In all honestly, I don't use it since I find it to be an unnecessary step. It's easier to explain the process as if I had used it, so I just explained it that way. :D But, if you're used to editing the upk directly, then you're not missing out on anything by doing it that way.

^^ You messed up somewhere. XSHAPE should have reported that 1 hash file was updated. The only explanation is that one or more of the files you thought you modified actually weren't. Even if you just switched to using the uncompressed upk file, the hash would still need to be updated. I would check if xcomgame.upk.uncompressed_size was actually renamed. XSHAPE will skip files that have a valid <filename>.upk.uncompressed_size. Also, download and use the new version of XSHAPE (v.11b), and then you could edit the batch to use the -v switch like so: @echo off java -jar XSHAPE.jar -v 26444500 pause And then copy and paste the results here. Might give us a clue. No matter what, you'll have to use XSHAPE. Otherwise, the game will crash. Personally, I don't actually use the repacking method at all.. but it's easier to explain as if I had, so I sometimes explain it that way. This time I didn't. I don't think that's the problem, however.

That explains it, then. If I delete the backup, it kinda defeats the purpose of the backup, however. :D No need to delete/move anything... just running XSHAPE again after each modpatcher use does the trick... How about changing modpatcher so it pulls the hash values from the current file?

Yeah, this has been the second out of two failed mods I tried. Both were modifying constants, and should have worked... but didn't. These constant-rich files are starting to taste like a honeypot. ;) Where the values used are actually defined might be in: \unpacked\XComGame\XGTacticalGameCoreNativeBase\ GetLowCoverBonus.Function GetHighCoverBonus.Function Or, it could be a massive display bug... maybe the changes do work, but they only way to know for sure would be to set them to like 200 and test via observations.

So far I've been able to: 1) Reduce the delay on Overwatch/Hunkering down from 1.75 seconds to 0.25 seconds 2) Allow the changing of item build times besides just satellites 3) Fix the ratio of male/female soldiers to 1:1 I thought I did another that worked, but maybe not. ;) About to put a new version up. There was a display problem with how I generated the human readable hashes if you've been using the -verbose switch. I padded the 0 to the wrong end, so if you wanted to search the exe by hex characters for the new hash, you'd not be able to find it. For most people it doesn't matter, it was just a display bug. The program actually functioned fine.

XCOM SHA Patcher for Executable (XSHAPE) v0.1b xcomgame.upk xcomstrategygame.upk Found 28433408 bytes in Binaries\Win32\XComGame.exe Starting at byte number: 26444500 xcomgame.upk entry found at: 26444932 Found Hash.: A6 C0 E1 3F 53 95 E5 49 55 F6 2A 25 A5 1F 29 8E 4B 1E 70 B2 Actual Hash: 8E 28 79 7B 44 D2 37 FB D9 B2 ED 4E C5 CF F3 7D 24 7F 68 F1 xcomstrategygame.upk entry found at: 26444965 Found Hash.: 78 B7 59 BB 9B 98 31 B4 28 DD 80 6F CF F2 71 E4 10 40 8C E2 Actual Hash: 24 53 5F 9E 9D 21 95 14 79 9B F0 CA 6F B8 89 87 2E 16 53 B3 xcomgame.upk SHA hash updated. xcomstrategygame.upk SHA hash updated. 2 SHA hash(es) updated. I have to run XSHAPE after each time I run modpatcher. Steam or modpatcher reverts the SHA hash values. Which is fine, I guess, but I just don't understand why. :D If I don't, game crashes. If it's modpatcher doing something you didn't intend, then the above output should help (XSHAPE with the -v switch) :D

UPDATE: It appears the game doesn't use these constant after all. But, maybe someone else will figure it out! So, yeah, assuming these constants are actually used in the game, here's how to mod low/high cover bonuses step by step: 1) uncompress XComGame.upk because this file has the tactical game elements 2) unpack it 3) load up the uncompressed XComGame.upk file with UE Explorer 4) navigate to XGTacticalGameCoreNativeBase 5) notice the constants: const LOW_COVER_BONUS = 20; const HIGH_COVER_BONUS = 40; We're going to add 20 to each. 6) Open your favorite hexeditor and open up both HIGH_COVER_BONUS.const and LOW_COVER_BONUS.const in the \unpacked\XComGame\XGTacticalGameCoreNativeBase folder LOW: Offset(h) 00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F 00000000 55 10 00 00 32 55 00 00 00 00 00 00 55 10 00 00 U...2U......U... 00000010 03 00 00 00 32 30 00 ....20. HIGH Offset(h) 00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F 00000000 56 10 00 00 32 55 00 00 00 00 00 00 56 10 00 00 V...2U......V... 00000010 03 00 00 00 34 30 00 ....40. You should immediately notice what you're going to want to change, so go ahead and do that. 7) save the two files, close UE Explorer, and then repack the folder into XComGame.upk 8 ) you can re-open that file in UE Explorer to verify your changes have the desired effect const LOW_COVER_BONUS = 30; const HIGH_COVER_BONUS = 60; 9) Go back to your \XComGame\CookedPCConsole folder, and backup your old XComGame.upk, and rename the XComGame.upk.uncompressed_size file to something else to get the game to load the uncompressed version of it. 10) Copy the new modified and repacked XComGame.upk to this folder 11) Run XSHAPE (otherwise your game will crash). It should say that 1 hash was updated. That's it, you're done. :D You'll have to test if that actually has any effect in the game. I don't know. Not all constants are actually used, unfortunately. Let me know how it goes :D UPDATE: It appears to not use these constants ;) At least, the displayed numbers ingame don't change. I'll play with it and see if I notice a difference anyways.

Hey, so I had to run XSHAPE again after running modpatcher. Are you making changes to the SHA hashes? I made the modification to my files to allow changing of the build times for all items (without upk modification, those ini settings don't do anything). Ran XSHAPE, then started the game up. 1 hash updated. Game ran fine... But I soon realized the default build times were unbalanced (7 days for a medkit, heh). So, I fired up notepad++, made some ini changes, ran modpatcher, then started the game... the game crashed. So, I ran XSHAPE and it said 2 hashes were updated. Which was weird, because, the game was just running with my uncompressed upk files before I ran modpatcher. Everything's fine now, but I was just curious as to why running modpatcher changed the SHA hashes stored in the exe. Using v.12. Just wondering. :D

Yeah, Unreal is non very mod-friendly imo... unless they specifically release tools for the game. But, I'll take what I can do as far as I can.

Yes, but it may have failed due to the bytes not being what the mod expected... like if the upk was changed by Firaxis and those bytes are different now. Of course, the user can bypass that (I wouldn't), but that's up to the user. The other thing we could do is in additional to storing the byte's offset within the file, we could also store the hash value of the bytes around the area to be modified, and directly search through the upk to find a match entirely by hash value. The reason I say we should use hash values is so that we avoid entirely the problem of distributing any part of the game's assets. Both ways have advantages disadvantages, and neither is particularly difficult to do. Perhaps using both methods simultaneously would be best, but then it might be slightly more complicated for the user to decide if they want to go through with the changes in certain circumstances. But, whatever. Choices are good, imo. Hmm... perhaps it could work like this: Player A (the modmaker) is creating his mod. He modifies a function from the unpacked files of a upk. Now, to create the mod, he needs the: 1) original unpacked file (filename.upk.original) 2) modified unpacked file (filename.upk.modified) 3) filename of the upk it was in Not too complicated. We can make it easier for him by having a configuration file that simply points to a static folder where all his unpacked files are, so he won't even need to worry about #1. Then he just uses this program, creates a mod package, and shares the file to other players. We can even have it automatically zip it up. And NONE of the game's assets would be contained in the file he shares. Nothing at all. I dunno, just thinking aloud, really :D

I think dose206 got it right. We should have no trouble with multiple mods writing to the same upk, but we can also prevent any potential problems from occurring with some simple checks. The user can bypass them and go full steam ahead if they choose, but I'd like something in there that checks if you try to load two mods that write to the same byte(s) in a upk. The general idea of things: 1. Load a .xcom.mod file 2. IF upk is compressed: Backup original Create uncompressed copy of it Rename <filename>.upk.uncompressded_size 3. IF bytes in .xcom.mod file don't match up with expected values (with user option to bypass check with a big warning sign) Patch in changes to .upk bytecode Update SHA hash in exe ELSE IF upk was compressed, then undo all changes So, the .xcom.mod file should not only contain the changes to be made, but ALSO it should contain the original values of the bytes it expected to be at those locations. For each byte to be changed we can have at least 4 fields: FILENAME, OFFSET, EXPECTED_VALUE, MOD_VALUE Sound good so far? Any other ideas? For the backup system... I think it would be best to only keep one copy... just the original compressed file, and after that, just uncompress and reapply whatever mods to it... rather than keeping X copies of slightly modified upk files. Kinda like a differential backup system. This idea somewhat changes the above, but whatever. What do you guys think?

Potential copyright problems with that. We should just share the changes itself, and not distribute game assets.

Good info... When you change the length of a function, when does it fail? When the game launches? Is it a catastrophic failure, or is it something more subtle? If it's on repacking, I can (maybe) easily get around that. Also, where did you get that information regarding special bytecodes for constants? It would be useful to be able to compute the bytecode for any constant. EDIT: So, anyone know why some bytecodes are missing from the file I scraped? Off the top of me head, 0x06 and 0x07 aren't in there (actually 0x06 is there, but it's SetCollision in the file. But that's not correct afaik). EDIT2: There may be a problem with how I formatted the output... 0x60 came up twice. Looking into it. EDIT3: Yeah, I padded the zeros incorrectly. ;) Fixed. Now FindPathTo 0x06 comes up twice. heh... so does 0x00. I double checked the ntl file, and that's how it is in there.

Scraped straight from UE Explorer's Native Tables (NativesTableList_UDK-2011-08.NTL): ! 81 == F2 != F3 && 82 ^^ 83 || 84 *= 85 *= C6 /= 86 += 87 -= 88 ++ 89 -- 8A ++ 8B -- 8C ~ 8D - 8F * 90 / 91 % FD + 92 - 93 << 94 >> 95 >>> C4 < 96 > 97 <= 98 >= 99 == 9A != 9B & 9C ^ 9D | 9E *= 9F /= A0 += A1 -= A2 ++ A3 -- A4 ++ A5 -- A6 Rand A7 Min F9 Max FA Clamp FB - A9 ** AA * AB / AC % AD + AE - AF < B0 > B1 <= B2 >= B3 == B4 ~= D2 != B5 *= B6 /= B7 += B8 -= B9 Abs BA Sin BB Cos BC Tan BD Atan BE Exp BF Loge C0 Sqrt C1 Square C2 FRand C3 FMin F4 FMax F5 FClamp F6 Lerp F7 Round C7 - D3 * D4 * D5 * 28 / D6 + D7 - D8 << 13 >> 14 == D9 != DA Dot DB Cross DC *= DD *= 29 /= DE += DF -= E0 VSize E1 VSizeSq E4 Normal E2 VRand FC MirrorVectorByNormal 2C ProjectOnTo DC IsZero DD == 8E != CB * 1F * 20 / 21 *= 22 /= 23 + 3C - 3D += 3E -= 3F GetAxes E5 GetUnAxes E6 RotRand 40 $ 70 @ A8 < 73 > 74 <= 78 >= 79 == 7A != 7B ~= 7C $= 42 @= 43 -= 44 Len 7D InStr 7E Mid 7F Left 80 Right EA Caps EB Locs EE Chr EC Asc ED Repl C9 == 72 != 77 ClassIsChildOf 02 IsA C5 == FE != FF + 0E - 0F LogInternal E7 WarnInternal E8 GotoState 71 IsInState 19 GetStateName 1C Enable 75 Disable 76 SaveConfig 18 Sleep 00 FinishAnim 05 SetCollision 06 SetCollisionSize 1B Move 0A SetLocation 0B SetRotation 2B MoveSmooth 81 AutonomousPhysics 83 SetBase 2A SetOwner 10 SetPhysics 82 Trace 15 FastTrace 24 Destroy 17 SetTimer 18 MakeNoise 00 PlayerCanSeeMe 14 GetURLMap 23 AllActors 30 DynamicActors 39 ChildActors 31 BasedActors 32 TouchingActors 33 TraceActors 35 VisibleActors 37 VisibleCollidingActors 38 CollidingActors 41 LineOfSightTo 02 CanSee 15 CanSeeByPoints 19 PickTarget 13 MoveTo F4 MoveToward F6 FinishRotation FC FindPathTo 06 FindPathToward 05 FindRandomDest 0D PointReachable 09 ActorReachable 08 PickWallAdjust 0E WaitForLanding 0F UpdateURL 22 FindStairRotation 0C IsSeatControllerReplicationViewer E7 Here's a direct link: http://dl.dropbox.com/u/32777109/Games/XCOM%202012/output.txt ^ then you can just hit F3 in your browser and search that way. Hope that helps :D UPDATE: Fixed zero paddings.

Yeah, I'll see if I can get around to it. Right now I'm working on a program to parse the upk bytecode table to make things a bit easier for everyone to learn how to mod this stuff. It's pretty much done, but I need to do some formatting. It's much easier to know what to hexedit when you have the information in front of you in some human readable form.

Well, modpatcher could easily be integrated into any GUI, since it's a command-line tool that has nothing to do with anything else that needs to be done. :D Same goes for Gildor's decompressor. I just wouldn't distribute them with the program (unless dreadylein/Gildor wanted to, of course). In fact, each and every requirement could be a seperate program: 1) Patching the UPK bytecode (moderately time-consuming) 2) Decompressing UPKs(already done -- Gildor's decompressor) 3) Filehandling of the little <filename>.uncompressed_size files (easy) 4) SHA hash patching of the main executable file (already done -- XHSAPE) Then we just need another program (a GUI, or yet another command line tool) to tie them all together. The other possible features are really all just different aspects of the thing, mostly polish (especially the GUI). A nice GUI isn't easy.

Personally I would just mod classic to be harder, rather than make impossible easier.

Right now XHSAPE does SHA patching of the main executable, nothing else. This program would have to do what XSHAPE already does, but it also has to do way more. We can do the project using more than one programming language if need be. The separate parts of the program could communicate with each other using network packets over loopback on some port :D Doing that isn't as hard as it sounds, and might even be preferable... at least for the GUI and Backend.

I'm not even thinking about anything like that. I think we should just be happy with hexediting upks for now. Maybe we can even make a tool to make the hexediting faster. Hoping for anything more and we're likely to be disappointed, but I'm more than willing to be pleasantly surprised. ;)

Looks right.... with 0x00 for padding afterwards. Thanks! I came to the same conclusion while hand editing the ntl file for human readability. Almost done, btw. EDIT: I should probably just stop and write a java program to do it.

Awesome, you've got the job. :D Also, thanks for the information guys! I can write a parser in java for those NTL files pretty quick! UPDATE: I'll have to tweak the file it by hand first, since the format isn't entirely consistent.

Right now we can change those and load the new values into the game. It just takes more work than it should, and its difficult for others to replicate the results without some knowledge of hexediting/programming. I'm working out the details right now for a program that would make it easy to distribute mods such as these. It'll probably take a week, optimistically, so maybe two weeks realistically. ;) A month, worst case scenario.