Debugging Hex Code

[SirDrD]

What are some debugging techniques available with hex editing? If you resize a function, get the game to load, but it CTDS when the new code runs, what are some techniques to troubleshoot?

 

I know there is debug dump you can gets somewhere. How do you get that and does it tell you much?

Does the developers console help?

 

 

 

[XMarksTheSpot]

I don't really know about proper debugging techniques, but double-checking your modifications is usually a good start when dealing with CTDs. For that purpose its helpful to check whether your modified function decompiles correctly in UE Explorer. If it doesn't a likely candidate for the error is mismatched jump offsets from control structures, i.e. if-then-else or while blocks. Both UE Explorer and UPK Modder can help with figuring out the proper offsets.

 

Other than that, try adding your code modifications one line at a time to find any crashy statements in particular.

 

Sorry that this is all rather general, but this is basically what I had to go through to get the few UnrealScript modifications working that I attempted :smile: Not sure what would require further in-depth debugging, maybe give a few more details?

[SirDrD]

That is what I ended up doing. The error that ended up causing the CTD was I had a Boolean += operator when I wanted a float. So the code all looked right in UE explorer.

 

To figure it out I eliminated most of the lines to narrow down what was the problem. But that still caused CTD's until I realized I needed to re-adjust the jump offsets. I fixed those, which led me to eventually figure out the problem.

 

I was hoping there was some crash log available when the game crashes that would at least tell you the Hex offset that caused the error or something like. But I can work around that, like I guess the rest of you all did.

[wghost81]

Launch.log can sometimes help. Look at the last few lines for something like "script call stack".

[Amineri]

Here's the techniques I generally use when debugging more complex modifications that result in CTDs while executing :

 

1) Narrow it down to a single function.

1a) Apply and Revert various combinations of modfile as needed to narrow down the scope of the CTD as best possible

1b) When changes are "linked", use early return statements to skip executing a particular functions

 

 

2) Once narrowed down to a function, skip executing particular pieces of code to narrow down the CTD to a particular line

2a) Return statements can be added to skip execution of code past a certain point in the function, e.g. :

<line1>
return
<line2>

will prevent line2 from executing, so if the above doesn't CTD it means that line2 (or later) is the culprite

 

2b) Conditional jumps (e.g. 0x07 token statements) can be converted to unconditional (i.e. 0x06) to skip all code within a conditional. If a CTD stops happening after such a change, then the CTD is within the body of the conditional block

 

 

3) Once narrowed down to a particular line, figure out the problem with the line

3a) Make sure both absolute and relative jump offsets are correct (new v0.90+ UPKmodder checks for validity of absolute jump offsets, but not relative)

 

3b) Some composited lines can be broken apart to test pieces. For example :

<statement1> = <statement2>

can be broken into :

<statement1>
<statement2>

by replacing the 0x0F Assignement token with a 0x0B null-op. Functionally the two statements wouldn't do anything, but their code is still executed. Having broken them down into separate lines, the techniques from section 2 can be used to determine which portion of code is causing the CTD.

[SirDrD]

Thanks Amineri. These are good tips. It is certainly a lot easier That commenting out large sections of code, and keeping on adjusting jump references.

[VileTouch]

Open Skyrim.ini (My Documents/My Games/Skyrim) and add the following to it:

[Papyrus]
fPostLoadUpdateTimeMS=500.0
bEnableLogging=1
bEnableTrace=1
bLoadDebugInformation=1

Afterward, in that same folder you will see a new folder called Logs. Inside that: Script. Inside Script you will (eventually) find 4 logs. Double click and they should open with notepad or any text editor. Papyrus.0.log is the most recent. At the end of it you will see the last process to happen before the crash.

Edited December 18, 2014 by VileTouch

[dubiousintent]

Added as "Debugging Hex Code" section to the wiki article "Hex editing UPK files".

 

@VileTouch: You can PM a moderator to move your post to the Skyrim forum.

 

-Dubious-

Edited December 18, 2014 by dubiousintent