Corrupt Saves - StrCount > 0xFFFF - CTD on load

[bcsp]

Thanks EoS.

Updated link in OP.

[zelazko]

Thanks EoS.

Updated link in OP.

Mods like Follower Trap Safety or Duel Combat Realism have negative string count? Does it mean those mods actually reduce string count number?

[bcsp]

Unlikely.

It's more likely that the mod consumes very few strings and the negative number is due to the natural fluctuation of string count during play.

Need EoS to clarify on that though.

[Gawad]

Follower Trap Safety didn't change my StrCount at all after installation. It's a set of three scripts that replace existing ones, so I suppose it could have, but there was no change (noted as 0 on EoS's spreadsheet).

 

bcsp, wouldn't it be possible that existing scripts be replaced by ones that have fewer properties, resulting in a lowering of StrCount during garbage collection?

 

Anyway, it's going to be tricky to get good numbers on the effect of mods, but there's no reason it can't be done with a little patience. Of course, establishing a testing methodology wouldn't hurt. And were probably going to have to consider a truncated mean to get rid of distorting outliers given the problem's nature. There's a function in Excel to do that (https://support.microsoft.com/en-us/kb/108280) but I don't know about Google docs.

Edited April 10, 2016 by Gawad

[bcsp]

Sorry didn't realize it replaced existing scripts. Yes, that is possible.

[EpitomyofShyness]

 

Mods like Follower Trap Safety or Duel Combat Realism have negative string count? Does it mean those mods actually reduce string count number?

 

 

 

Unlikely.

It's more likely that the mod consumes very few strings and the negative number is due to the natural fluctuation of string count during play.

Need EoS to clarify on that though.

 

 

Follower Trap Safety didn't change my StrCount at all after installation. It's a set of three scripts that replace existing ones, so I suppose it could have, but there was no change (noted as 0 on EoS's spreadsheet).

 

I've just been compiling the data I find, but I agree that it's highly unlikely any mod can actively reduce string count. With that in mind I could go through and replace all of the "negative" numbers people have presented with placeholder zeroes. I'm not familiar with excel, so if someone wants to use the data I collected in drive into a different spreadsheet which narrows down the outlier data I'll happily link to it and/or incorporate that info into my file somehow.

[Dienes]

I just followed a link to this thread from the bethesda SKSE thread and haven't yet read everything but this is very interesting and not something I had heard of before. I am hardly an expert at the skyrim.exe internals but I think I know more than the average mod user or author so I'm going to throw in my $0.02 about my understanding of what's going on. Maybe someone will be able to use it towards a real solution.

 

First string count is almost certainly stored as a 32 bit int (either signed or unsiged) inside the exe. I haven't verified this but I do know that a) skyrim is a 32 bit program so doing otherwise would give them more work, b) a number of other variables are 8 or 16 bits in data or save files but can the exe is just fine if you force them over (such as papyrus arrays being able to go over 256 entries thanks to skse or leveled lists being limited to 256 entries in an esp). c) If it were a 16 bit int overflowing it during gameplay would make new strings not work which should be pretty obvious and overflowing the int would put the extra 1 into the next memory address which would also probably be pretty bad and obvious. So that probably doesn't need to be worried about.

 

Second modifying the save file itself is a bad idea. It would render any save made with a patched version of skyrim invalid if the patch were removed but also make them unreadable in any 3rd party tools that didn't account for it. It would be far better to write normally to save if its less than 0xFFFF and otherwise write 0xFFFF and then write the real string count to the skse cosave or similar. That way if you removed the fix any save that would normally work would still work for both skyrim and any 3rd party tools.

 

Third my approach to actually solve this would be to locate the function that reads the string count from the save and then trampoline it to a custom loading function that handles reading from the cosave. This is the tricky part but should be doable with an skse dll plugin. A dll plugins is free to patch skyrim.exe's code at runtime (the example plugin does this) and is loaded before the game tries to load a save. It would also need to write to the cosave when saving but that should be no harder than the loading part. I sadly don't have the experience to write this quickly and don't have the time to do it slowly but I'd be happy to lend what knowledge I have of both skyrim and dll plugins to someone who wanted to work on it. Even without serious debugging tools something like CheatEngine should be able to help locate the loading and saving functions.

[DavidJCobb]

First string count is almost certainly stored as a 32 bit int (either signed or unsiged) inside the exe. I haven't verified this but I do know that a) skyrim is a 32 bit program so doing otherwise would give them more work, b) a number of other variables are 8 or 16 bits in data or save files but can the exe is just fine if you force them over (such as papyrus arrays being able to go over 256 entries thanks to skse or leveled lists being limited to 256 entries in an esp).

Both of these points are wrong. Using a UInt16 doesn't create more work for the devs, because while registers are 32-bit, there are assembly instructions and register aliases specifically meant to load and store smaller units of data. MOVZX, for example, can load smaller units of data into a register; and AL is (UInt8)EAX.

 

This also means that we can't be sure that the data is always treated as a UInt32 even when it's stored in a 32-bit register.

 

Second modifying the save file itself is a bad idea. It would render any save made with a patched version of skyrim invalid if the patch were removed but also make them unreadable in any 3rd party tools that didn't account for it. It would be far better to write normally to save if its less than 0xFFFF and otherwise write 0xFFFF and then write the real string count to the skse cosave or similar. That way if you removed the fix any save that would normally work would still work for both skyrim and any 3rd party tools.

A patch could increment the version number in a save file, and third-party tools can be updated to watch out for that. (Also, what third-party tools? The only one I know is TESVESSE, which is unmaintained and broken.)

 

Third my approach to actually solve this would be to locate the function that reads the string count from the save and then trampoline it to a custom loading function that handles reading from the cosave. This is the tricky part but should be doable with an skse dll plugin. A dll plugins is free to patch skyrim.exe's code at runtime (the example plugin does this) and is loaded before the game tries to load a save. It would also need to write to the cosave when saving but that should be no harder than the loading part.

Respectfully, this isn't a solution. You're just explaining how patches work and then saying we should make a patch that does something that is logistically difficult, without explaining how. :\

 

Even without serious debugging tools something like CheatEngine should be able to help locate the loading and saving functions.

no Edited April 10, 2016 by DavidJCobb

[DavidJCobb]

Also, the SKSE team pretty much already found the loading and saving functions -- or enough that we could find the rest. The problem is that we can't be sure how the data is handled at run-time, especially with the conflicting information that keeps getting posted in the various discussions being had on this topic.

[BowmoreLover]

Very interesting topic. Thanks bcsp!

 

Sadly, I'm having the same problem right now.
After becoming a thane of all of the city, I began Dawnguard quest. Immediately after the rescue the Serana, this problem has occurred >_<
I'm using DynDOLOD 1.47, FNIS 6.2, several custom-voiced-followers, and a number of immersion mods (Checko's mods, Isoku's mods, Hunterborn, RealShelter, WetAndCold, etc...)

Trends of strings in my save is as follows (# is save count):

#12 - 39,111
#15 - 44,706 DynDOLOD installed
#31 - 45,055
#100 - 46,286
#210 - 50,303
#300 - 52,443
#400 - 59,663
#501 - 61,572
#600 - 63,793
#705 - 65,435
#706 - 65,507
#707 - 65,434
#708 - 65,473
#710 - 65,496 Rescure the Serana
#711 - 65,515
#712 - Can't load!!