Corrupt Saves - StrCount > 0xFFFF - CTD on load

[silvericed]

Of course the simplest solution is to simply make strcount at 32 bit field but that takes very specialized knowledge.

It's possible, but as I said on another forum, it's not 'simply make strcount at 32 bit field'. Save file format use uint16 index in several structures in a save file to reference a string that located in global string array. And according to the wiki, there are lot of places to change in the binary.

 

Also it's best to hope that it doesn't really crash when string amount grows during runtime. What if runtime also use 16-bit to reference strings? Or maybe it doesn't crash, but counter overflow simply replaces older strings with new ones?

 

Though in long term it is indeed simpler than having to cut down mods, rename properties etc

Edited March 27, 2016 by silvericed

[bcsp]

Ok, simple was the wrong word to use. How about "most appropriate solution". Mod authors or mod users should not have to worry about strcount.

 

32 bits would provide for almost 4.3 million strings compared to 65k.

 

Do you know enough about assembly to have a chance of fixing the issue? I did some way back in college but this is beyond me. I've been trying to contact Boris about this since he seems to be the most experienced guy around this subject matter but I've had no luck so far.

[DavidJCobb]

As someone experienced with assembly?

 

You'd have to find every single part of the game that stores string IDs, and then patch all of it. If this requires changes to a class structure, every piece of code that works with that class will need changing as well. So right off the bat, the work needed could spread across most of the executable.

 

You'd also need to patch the code to save and load savegames, to use a UInt16 for pre-patch saves and a UInt32 for post-patch saves.

 

Honestly, without being able to refer to the source code, it'll probably be impossible to do this reliably. I don't even think we have a full list of what exactly uses the string table.

[bcsp]

:sigh:

I was afraid someone would say that.

Looks like we will have to rely on ridiculous workarounds then.

[silvericed]

@bcsp, I'm programmer, but honesty I was never into cracking, rarely looking into assembly to understand the logic, if the code isn't available.

@DavidJCobb, while looking at StringCache, BSFixedString it doesn't look like it spread across VM - this can be save file format (and the code that reads/writes it) issue only.

[DavidJCobb]

I originally heard about this bug by following a link to a discussion on LL; there, someone noted that exceeding the string limit at run-time results in Papyrus errors. I'm inclined to think the limit applies at run-time.

 

No idea if I can link to the source site here. I'm gonna guess "no."

[bcsp]

@silvericed

Can you explain what StringCache and BSFixedString is? Is there somewhere I can read up on it?

 

@DavidJCobb

I can not confirm that.

Papyrus log would have been one of the first places I checked when having this problem. The thing that is so maddening about this problem is that you can play for hours thinking everything is fine. It's only when you return you find out you have any problem.

 

If the game engines pointer overflowed and started overwriting strings mid game surely it would be pretty obvious? CTD or some other serious weirdness going on. I won't be able to try this myself until mid week sometime but if some one could try loading my strcount save breaker mod onto an already string heavy save it should surely trigger papyrus errors if it were true.

 

Since the crash fixes mod has posted a link to LL I'm going to assume it's ok. + there's this thread on Reddit: https://www.reddit.com/r/skyrimmods/comments/40sgt2/loverslab_is_no_longer_blacklisted_on_nexus/

 

This is the most active thread on LL regarding the strcount issue: http://www.loverslab.com/topic/56588-corrupted-saves/page-12

 

In fact I will update the OP with the threads we have found on the STEP & Bethesda forum about the string table problem. Having people googling for them is counterproductive.

Edited March 28, 2016 by bcsp

[DavidJCobb]

I've uploaded my Papyrus-related TES5Edit scripts here. Remember: these modify Papyrus-related data attached to forms in your mod; they don't modify your Papyrus scripts. The idea is to use Notepad++ to batch-edit your scripts, and then use this to batch-edit your forms' property data to match the edited scripts.

 

Papyrus log would have been one of the first places I checked when having this problem. The thing that is so maddening about this problem is that you can play for hours thinking everything is fine. It's only when you return you find out you have any problem.

Centrophy refers to Papyrus errors on page 7 of the LL thread.

[bcsp]

Thanks for the scripts :)

 

Interesting. I missed Centrophys post. I don't ever remember seeing bloated log files. I will test it out myself, but it will be Wed/Thurs probably before I can.

[bcsp]

Just did a super quick test on my laptop there. Load order isn't as heavy as my main rig but still has a save with 50k strings. Starting inside the sleeping giant inn I ran to whiterun at speedmult 300 (pressed for time here) and did a lap around the town for good measure and then saved at the main gate again.

 

Test1: No extra mods.

Papyrus log size: 194KB. 1,617 lines

 

Test2: added the properties version of the save breaker.

Papyrus log size: 195KB. 1,624 lines

 

Test3: add both the properties version and variables version of the save breaker.

Papyrus log size: 195KB. 1,646 lines

 

Saves at the end for test2 and test3 were obviously screwed.

Majority of the logs were spammed by a sorta well known to be spammy mod. And that save and MO profile aren't exactly clean anyway.

I'll will try to get clearer results later with a cleaner set up but there doesn't seem to be any massive bloating logs yet. Could the fact that the variables & properties in the save breaking mods are not initialized influence the results I wonder?

Edited March 28, 2016 by bcsp