Jump to content

Photo

Rewriting Hidden Potential - Tutorial

mod tutorial

  • Please log in to reply
9 replies to this topic

#1
Zybertryx

Zybertryx

    Regular

  • Members
  • PipPip
  • 73 posts

Greets,

 

As I've finally learned how to make meaningful changes to game mechanics and functions beyond single byte replacements I'm going to fulfill a promise I made and lay out the process using a real function and a real change in a step-by-step manner. That said, I am a complete novice so there may be ways of doing things much faster or more efficiently that I've yet to make a habit of. It is not glamorous work.

 

Should note that I have absolutely no prior coding experience at all (and this will probably become quite clear as we move forward). Ironically, this is what the pros here have said qualifies me for writing this tutorial! :huh:

 

Six weeks ago I was using Toolboks and Modpatcher. Four weeks ago I was changing integers in the .upks with the help of UE Explorer and HxD. Two weeks ago I was asking questions like "How can I tell where the integer "1" is; I don't see any "2C 01" here?" And today I'm going to completely rewrite the Second Wave option "Hidden Potential".

 

First things first, and the hardest part can't be easily taught; knowing where a particular thing you want to change is handled in Vanilla. I suck at this part, so I fill my time by browsing through function after function in UE Explorer getting familiar with various functions and locations and often stumbling into "Ah Ha!" moments when I intuit that a particular function is handling a particular mechanic I'd like to see changed. That's how this Hidden Potential thing started, I just stumbled across it, stroked my chin and thought, "Yep, I'm doing it".

 

Looking at XGStrategyGame >>> XGStrategySoldier.LevelUpStats in UE Explorer I noticed it seemed to handle the "Hidden Potential" Second Wave option. (Yes, you reading now, ye interested and clueless parties, get UE Explorer, open up a decompressed "XGStrategyGame.upk", browse to the "XGStrategySoldier" function, expand it and click on the "LevelUpStats" section, have looks).

 

I don't use (or like) Hidden Potential because my mod is extremely anti-HP increases (excepting via equipment), very tight on Aim buffs, and very much dependent on the DGC values to make the classes distinct when leveling, but if I could tailor Hidden Potential for my mod then I would certainly use it, oh yes indeed - more variation in toons is always better - so long as it's sane.

 

So, with the suitable and necessary level of insatiable will in place, lets get to it.

 

---- Continues Below ----


Edited by Zybertryx, 18 July 2013 - 12:02 PM.


#2
Zybertryx

Zybertryx

    Regular

  • Members
  • PipPip
  • 73 posts

Part 1: Familiarising Yourself with the Vanilla Function (or at least the section that is to be modified or replaced)

 

The first thing is to have a good look at the function currently, have a think about what you'd like to change and how it could be changed, and look for anything can be safely deleted or optimised (via rewrite) to make room (byte wise) for any larger replacement code (should your changes require it). Unfortunately, we are constrained by byte totals. Total byte size of the function must remain unchanged. Thankfully, we have a few tricks in our arsenal to aid with this. I'll explain these methods when we come to writing out our replacement Hex in a later update here.

 

I wondered how best to go through this step-by-step without resorting to You Tube videos, embedded screenshots full of MSpaint scrawl or sleep inducing line-by-line cut/paste commentaries (which due to my poor technical grasp I'd be ill suited to do anyway). I may end up using all of the above at different points in the tutorial but for now I'm going to do it like this:

 

XGStrategySoldier.LevelUpStats (Vanilla) - Beginning of function and first class specific level up entries only, for brevity. There are three more near identically formatted entries for the other three classes. My commentary is relevant for all four class sections - Comments Inside.

 

Spoiler

 

--------- Continues Below ---------


Edited by Zybertryx, 19 July 2013 - 02:02 PM.


#3
Zybertryx

Zybertryx

    Regular

  • Members
  • PipPip
  • 73 posts

Part 2a: Fleshing out a Conceptual Replacement or So What Do You Actually Want It To Do?

 

For me, Vanilla Hidden Potential basically breaks the careful and thorough rebalance of the rest of my mod. My DGC entries for non Hidden Potential campaigns are as follows.

 

Spoiler

 

As you can see, compared to Vanilla this is all been ruthlessly squeezed. I also radically lowered the starting will for Rookies so that they were prone to panicking when their squadmates were killed or they themselves were seriously wounded (Game over, man! Game over!!). The +10 Will buff at "Sergeant" and later at "Colonel" have a clear effect on modifying their behaviour compared to the lower ranks.

 

Anyway, needless to say, Hidden Potential with all its ridiculous mobility, hit point and aim bonuses was until now not really an option for my mod.

 

Going back to Hidden Potential in the Vanilla function I linked previously, we can isolate the Vanilla mechanic (for Assaults, as example) like this:

 

Spoiler

 

Yup, that's it (I trust you can kinda "Read it" now too).

 

So to make it more conducive for my rebalance, I'd like it to look like this:

 

Spoiler

 

I think I might have to add Rank specific Will entries to mirror (though more randomly) my non Hidden Potential balance settings.

 

Another sequential conditional like

 

Spoiler

 

Well, that's all well and good Z, and cheers, it's starting to make sense, but how do we actually go about turning the Vanilla into the custom?

 

I'm glad you asked. We use the ludicrously awesome Token view in UE Explorer 1.3 beta or the mind numbingly awkward Buffer view of an earlier version (and then break it all down manually. . . gah).

 

To access these views, find the function in UE Explorer and right click it from the list, select "Object", "Token" or "Buffer (which is HEX)".

 

All three views will be vital for when we really get stuck into it although UE Explorer 1.3's token view pretty much contains all the data of all three. "Object" view is the default view (the decompiled code like these spoilers).

 

Token view is the crucial view for viewing call locations and jumps and Buffer view contains the raw Hex (with some great dynamic mouse over tool tips as well).

 

More on this in the next section.

 

--------- Continues Below ---------


Edited by Zybertryx, 19 July 2013 - 02:02 PM.


#4
Zybertryx

Zybertryx

    Regular

  • Members
  • PipPip
  • 73 posts

And notepad!

 

We must have notepad.

 

--------- Continues Below ---------



#5
Zybertryx

Zybertryx

    Regular

  • Members
  • PipPip
  • 73 posts

A wondrous creature flew to my window and whispered unto me a great insight.

 

This:

 

                    statOffense = 1 + Rand(1);

                    if(Roll(40)) <<<<< 40% Chance of an additional . . . 
                    {
                        statOffense = 1;  <<<<<< +1 Aim, and if they are . . .                   

                    }

 

Actually says that there's a 40% chance that there will be only +1 Aim gained (overuling the previous statOffense = 1 + Rand(1);) section. That is to say, that like this there won't be an additional +1 gained if the roll is successful (which is my intention).

 

To achieve that the code would need to swap the "=" for a "+=".

 

The next section of this tutorial will get straight into the grit; Hex, syntax and subtle nuances like this.

 

--------- Continues Below ---------


Edited by Zybertryx, 19 July 2013 - 11:38 AM.


#6
Zybertryx

Zybertryx

    Regular

  • Members
  • PipPip
  • 73 posts

Part 2b: Peeking Under The Hood or WTF Is All This HEX About?!

 

Following on from Part 2, and specifically addressing how it is we can go about turning our concept into a reality, it is necessary to firstly understand that each line in the decompiled code corresponds exactly to particular lines of hex and that it is the jump tokens alone that account for the correct formatting (the 'tabs', 'nests', 'bracks' and so on). Jump tokens will be handled in a subsequent post.

 

Using UE Explorer 1.2.3 beta's Token View by right-clicking the function "LevelUpStats" and selecting it from the list (Under the parent XGStrategySolder function) results in a treasure trove of information. Those of you using a previous version should seriously consider upgrading as although I actually got this mod happening using 1.2.2 It was somewhat tortuous.

 

UE 1.2.3's token View reveals the entire function (including the decompiled lines, the token/jumps and the HEX) like this:

 

Spoiler

 

You need to really look at this yourself in some detail (I highly recommend cut/pasting it to a fresh .txt) but if it feels like a bit of a bludgeon at the moment, let's start by breaking a couple of small examples down. I'll take a line at random:

 

Spoiler

 

Let's take a closer look at the call/phrase/line (the decompile friendly part) and compare it with the Hex. I'm going to colour code it so that we can more easily get savvy to the syntax.

 

Spoiler

 

The "statOffense" part is the five byte "00 29 45 00 00" section (the "local variable")

 

"0F" is the "=" and must precede the call (so to write "blahblah =" Sane-side you need to write "=blahblah" Hex-side).

 

The "2C" designates that the next byte is to be an Integer (in hexadecimal). So "2C 03" is "3".

 

You'll find that the composition of other local variables (and their contexts) are similar:

 

Spoiler

 

Actually this last one is a great example of something I had trouble with initially.

 

Notice how here the Integer "1" is not defined by the "2C 01" convention but is written as a standalone "26". For arcane reasons (probably due to the compiler, as this way requires one less byte) a "26" like this (when it corresponds to an Integer "1" in the decompiled code) is how you write the Integer "1". The Integer "0" can also be written in one byte by using "25".

 

(Integer) "26" = "1"

 

(Integer) "25" = "0"

 

For modders, values rendered like this are a pain and, if you have the bytes to spare when writing your own mods, you should always preference the "2C ##" convention, as to change single byte integers to values larger than "1" requires an extra byte (to effect the 2C ## convention). For the purposes of my rewrite here I will use the "2C 01" or "2C 00" convention for "1" and "0" (and all other sub 255 integers) for the ease of later moddability should a new kind of balance tickle my fancy; for future proofing. I'll be able to get away with this across the board as I'll be deleting a massive chunk of hex and likely not at all filling it all back up with active code.

 

Indecently, and I'm sure its the same for those of you who are soaking this up, the first thing I was doing Hex-side was sifting through the .Upks looking for juicy "2C ##"s I could edit pretty easily (after first identifying the relevant locations using UE Explorers Object View; Don't go in blind!)

 

After a while I began to view the .Upks as 'extended inis' - and you can get quite a bit done just by revaluing the integers like this.

 

I'm going to continue on assuming the reader is acquainted with some of these fundamentals. I'll mention others as they come up. The next section will deal with working out what to delete from the original function which opened this Tutorial following on with how, using the breakdown via Token (or Manual Buffer) View, we can begin to piece together a new hex chunk which will decompile in accordance with the conceptual outline described in Part 2a.

 

--------- Continues Below ---------


Edited by Zybertryx, 20 July 2013 - 02:57 PM.


#7
Zybertryx

Zybertryx

    Regular

  • Members
  • PipPip
  • 73 posts

Part 3a: Let's Organise Ourselves.

 

The next step is working out what we're going to remove to make room for our changes and what we need to keep to not break anything.

 

It's probably a good idea to go to UE Explorer's "Buffer View" at this point. Select the tab "Edit" from the Buffer View menu and then select the option "Dump Bytes." This copies the Hex for this entire function to your clipboard, where you can easily paste it to a clean .txt (notepad).

 

Switching back to UE Explorer's Token View, at the position 0x08F we can see the first switch to the Class case:

 

Spoiler

 

The highlighted Aqua (08F) is the position of this line. Paying attention to positions is not all that important at this stage but it will become much more important after we have spliced in our replacement code.

 

Everything above this line we are going to keep unchanged. So highlight and copy the relevant hex for this line and then do a search from the top of your "Dumped Bytes" in your .txt. When you've found it,  separate it (and everything preceding it) from everything immediately following it.

 

You should have divided the entire function hex block into two pieces now; a smallish piece which ends with "(continuing to) EC 44 00 00" and a much larger piece which starts with "0A F7 01 24 (continuing on)".

 

Now, that we have 'protected' the upper part of the function, lets isolate the lower part of the function we also wish to preserve.

 

Back into token view, we want to scroll right down to the entry at position 0x598 because that's where all the superfluous Vanilla stuff and the nasty Vanilla Hidden Potential stuff ends.

 

Spoiler

 

Note the exclamation mark in "If(!bRand)" is a "NOT" flag/token. I'll explain this in a little more detail later, but for now it will help to know that this exclamation mark is changing the meaning of this 'phrase' to "If(NOTHiddenPotential)then. . "

 

The stuff below that point is the all the stuff that actually is used when Hidden Potential is not enabled (it refers to the DGC.ini). We want to keep "If(!bRand)" itself and everything after it.

 

So, copy the relevant bytes for "If(!bRand)" - 07 6A 08 81 2D 00 23 45 00 00 16

 

Do a "Find" in your txt for it (it is a unique instance in this function - there's only one) and then, once found, separate the beginning of it and everything after it from the rest of the dumped Hex.

 

You should now have separated the "dumped bytes" into three distinct parts.

 

1) A shortish top part which contains the beginning of the function and ends with the first Class call - 05 30 FF FF FF 00 35 30 FF FF FF 7D FA FF FF 00 00 35 B4 F9 FF FF 74 FA FF FF 00 00 01 EC 44 00 00

 

2) The largest chunk which starts with 0A F7 01 24 04 05 73 FB FF FF (continuing until . . .)

 

3) Another largish chunk which starts with 07 6A 08 81 2D 00 23 45 00 00 16 (continuing to the end)

 

The largest chunk is the part of the function concerned with the unreferenced Vanilla DGC values as well as Vanilla's actually-referenced "Hidden Potential" mechanics (which we intend to completely customise).

 

Don't delete it from your .txt though! It's important to hold onto as when using "word wrap" in Notepad it can help us to ensure that our replacement code will be exactly the same size.

 

It might be best to name and save your separated "dumped bytes" .txt now. I'd also name the chunks; "Header" for the smallish first chunk (though it's more than just that), "Remove" for the large middle chunk, and "Keep" for the good sized lower chunk (or whatever's good for you).

 

Next, we'll nail down a working road map so we'll be able to work out the structure our new code should take. Following on from the road map of the next section, we'll lay out how we want our new mechanic to work, hex-line for hex-line, and I'll stress the importance of sticking to Vanilla's conventions as we do so (especially if we barely understand them).

 

--------- Continues Below ---------


Edited by Zybertryx, 19 July 2013 - 04:20 PM.


#8
Zybertryx

Zybertryx

    Regular

  • Members
  • PipPip
  • 73 posts

Part 3b: Road Mapping.

 

Now that we've separated the original hex into three parts (two we are going to keep and one part we are going to replace), we need to start getting an idea of how our replacement code should look, or how it should 'fit'.

 

For me, based on my DGC.ini rank settings, my plans for Hidden Potential are a little more complex than vanilla. That's fine, though one problem with increased complexity in refactoring is that it compounds that the amount of stitching required after the splice.

 

For now lets march ahead. Based on our Token view of the original function we can see the breakdown of the different calls and we can use these to create a new sequence more tailored to our particular needs.

 

Before we get too carried away though, lets take a good look at how Vanilla tied it all together as this will save us a world of pain later on if we build it into our code now. This is perhaps the most important part of the whole process, understanding how we can make our code work, even if we don't understand much about the particulars.

 

Spoiler

 

These three entries offer us a bit of a road map on how to begin our custom code. Note: If you recall, the first line, - switch(m_kSoldier.kClass.eType) - should be the final line of hex in the "Header" section of our saved .txt. which we're keeping.

 

Check out the sequence here, It goes:

 

Spoiler

 

And. . . if we scroll down in our Token views to the end of the seven Vanilla rank cases (case 1 to case 7, which we don't want and won't keep) we see this sequence:

 

Spoiler

 

Putting these two sequences together gives us:

 

Spoiler

 

Finally we need to look at how Vanilla 'closes' each specific Class entry, so we scroll down under the last of the level boosts for the first class to position 14F - (1F4/170) - in our Token view, and we see:

 

Spoiler

 

Cleaning this up conceptually we see:

 

Spoiler

 

And if we scroll down further we will see the beginning of the if(bRand) section of this new class preceded as before with a "default:" entry like this:

 

Spoiler

 

And if we put all of this together we see:

 

Spoiler

 

Continues until we have four cases (one for each class) with each ending in an 06 ## ## "goto" unconditional jump.

 

 

Now that we've have a clear 'road map' on how the vanilla code tied the "Remove" part (which we are about to completely replace) into the "Header" and "Keep" parts (which we've separated in our .txt.), we can at last begin construction of our new code in confidence.

 

 

--------- Continues Below ---------


Edited by Zybertryx, 19 July 2013 - 04:51 PM.


#9
Zybertryx

Zybertryx

    Regular

  • Members
  • PipPip
  • 73 posts

Part 4a: Constructing The Replacement Code

 

At this point we can start constructing the replacement code.

 

Using Token View we can easily isolate and grab the various parts we need, and using some inference, Buffer View, and by generally paying attention to things, we can use these pieces to construct new 'phrases' we'd like too.

 

I've decided I'm going to order the classes as Sniper, Heavy, Support and Assault as each of these corresponds to the case call "1", "2", "3" and "4" respectively.

 

So, based on the road map of the previous section, lets start constructing our new code for the Sniper class.

 

If you recall, the necessary sequence for a class specific Hidden Potential code was:

 

Spoiler

 

Working from that model, I've grabbed a bunch of entries from the token view in accordance with the above sequence:

 

Spoiler

 

---------------------------------------

 

It's important to note that with crude grabs like this, some of the information taken will be problematic for us when we splice it in. This is par for the course when modifying a large section like this though. In particular the jump references for Vanilla are specific to the precise locations of everything else in the Vanilla function. Our reworking/rewriting of things is going to throw all that way off. That's okay and it can (and must) be fixed up later (though that part is not fun).

 

If you've had a bit of look at Token View in your own time you've probably noticed that all "if" statements begin with an "07" and then two more bytes "F4 01" (an example). It's the 07 # # that constructs the "if". Technically these are "jump-if-not" Tokens. That is to say that, if the following conditional is false then jump away (to the position defined by the subsequent pair of bytes). These bytes are "Little Endian" and you'll need to take that into account when working with them (which we won't be dong in this section but as they're so important and really need understanding I thought I'd write a little primer for them here). What "little Endian" means (at least to me, in practicality) is that you write them in reverse order. So "F4 01" points to the position "1F4".

 

Similarly, "07 3D 04" would point to the position "43D".

 

This is the same for unconditional jumps too, which are all "06 ## ##" in style.

 

So. "07 9C 02" = Jump-to-29C-if-following-is-not-so.

 

And, "06 53 01" = Jump-to-153-unconditionally.

---------------------------------------

 

You can change the 'meaning' of a conditional (turn them into "Is Not" statements, rather than "Is" statements) by adding an "81" immediately following the jump token and prior to actual variable. The "81" token requires its own 'executor' - a "16" - which should be placed at the end of the completed 'hexphrase' (even if for other reasons a "16" is already present).

 

Example:

 

Spoiler

 

That was just an example and we won't be molesting our if(bRand) statements at all but we will be modifying a lot of our 'grabs' to suit our intended functionality and values. On that note, let's continue.

 

I grabbed (and pasted into a fresh.txt):

 

Spoiler

 

But some of that should be changed right now. Firstly, the "case 4" entry should be changed to "case 1" (from Assault, to Sniper). This is easily accomplished, a simple thing.

 

Spoiler

 

The other part from this initial grab I'd like to change is the "statOffense = 1 + Rand(5)" line. That's far too generous, even for Snipers, for my taste, so I'm going to change it to "statOffense = 1 + Rand(2)"

 

Like so:

 

Spoiler

 

Next up I'm going to grab one of those cool "X% chance of getting statboost X" lines and paste it in my .txt right after the just modified "statOffense = 1 + Rand(2)" line and before the "goto J0x598" jump. Then I'm going to grab another "StatOffense" and marry it to a "+=" to give significance to the roll. I'll clean up all the Token View clutter too so that's its nice and legible (and ready for an eventual stringing together of just the Hex components).

 

Spoiler

 

Now, we do have limits to what we can add for each entry byte-wise, but as I've run through this mod previously I know you can get away with three additional (and simple) sequential entries per class without any trouble. I'm going to go and have a think about what I really want Hidden Potential to do for my classes long term, and map out each class on my .txt., sequentially adding the buffs and values I'm after and in an order which will match the conceptual road map of Part 3. I'll post the results and discuss any lines I had to create that haven't been covered already, in the following installment.

 

--------- Continues Below ---------


Edited by Zybertryx, 20 July 2013 - 02:54 PM.


#10
Zybertryx

Zybertryx

    Regular

  • Members
  • PipPip
  • 73 posts

Part 4b: The Completed Code

 

So, after a bit of deliberation, consternation and procrastination, I've finally nailed down exactly how I want Hidden Potential to work for each of the four classes. Although this is tailored explicitly for my own balance I'm hoping that this can act as a template that anyone can modify to suit their own tastes. There's also much more content than the original entries contained and that was a slight concern due to the byte limits which constrain us. After careful investigation it appears it will all fit. It's really very fortunate that this function happened to contain so much dormant garbage.

 

First up, the Sniper class. It largely mirrors my non-Hidden Potential DGC settings (offering only a modest amount of randomisation against that measure) but it will suit me fine.

 

By using a combination of Token and Buffer views on the existing Vanilla function I've been able to piece together exactly the kind of 'phrases' I need to produce the intended results.

 

The first part of my working .txt now looks like this:

 

Spoiler

 

This continues on for the other three classes each with their own flavour and values.

 

For example, I've given the Heavy an additional unique entry, to ensure they are immediately saddled with a -1 Mobility hit. This was done to add a little flavour to the Rocket Launcher.

 

Spoiler

 

A robust list of various Hex tokens and their functionality can be found on the Nexus Wiki, here:

 

http://wiki.tesnexus...es_XCOM_Modding

 

Note, that there are sometimes different ways to write the same thing but that some functions or calls prefer one way to the other. If in doubt have a look for the token in the other sub-functions of the parent function in UE Explorer. If you find it used a certain way there, it's probably best to use it that way yourself. Also note that many of these require their own "16" executor. The mouse over tool tips of UE Explorer's Buffer View can really help to determine what kind of token is appropriate and which require their own executors.

 

With the complete line-by-line breakdown of my custom hex finished, it's time to prepare it all for a find/replace splice into the Vanilla function. Before we do that though, we need to ensure that our new code is the same byte size as the section we are replacing. If you're over that limit, then its hopeless; the original byte size of the function cannot be exceeded. But if you're under that limit, and we are, then we can 'puff up' our code to the necessary size with use of Null Operators; dormant "filler" code that will ensure our rewrite contains the same number of bytes as the code it replaces, but which will be totally benign and ignored by the engine.

 

More on this in the next section.

 

--------- Continues Below ---------


Edited by Zybertryx, 20 July 2013 - 02:53 PM.






Also tagged with one or more of these keywords: mod tutorial

Page loaded in: 1.044 seconds