[LE] Papyrus: Any ideas for a technique to repeatedly change saved property values without compiling new code?

[opusGlass]

Ok so first off, this is about a framework that has already been deployed. It's in use by multiple mods so it's too late to redesign it. Answers saying I "should have done it differently" will not be useful.

 

I have a framework that allows MCMs and spell menus to be easily generated for creature mods, using only xEdit scripts.

 

The part of the framework that I'm worried about is the Splicer -- this is a script attached to an Activator that has two paired array parameters, a list of ActorBases and a list of integers (the encounter levels for those actors).

 

A mod can have a potentially unlimited number of Splicers, and they all share the same identical script. The differences between Splicers are determined by the initial values of their properties, as set in the Editor Window/xEdit.

 

This means that when a Splicer needs to be changed during an update, any existing saves will continue using the outdated version of that splicer, since the properties will never be set to their new "initial" value.

 

Since the framework is entirely in xEdit, I can't write a custom papyrus script for each update. I need some general update procedure that can be designed ahead of time without knowing the details of the update, that can be used repeatedly for every update.

 

Some ideas I've found:

 

-Replacing Splicers entirely any time they change, with a "disabler" script to attach to the old splicer that puts it out of use.

-Adding a new update script to the Splicer with every update that copies the initial values of its own properties into the Splicer script's properties. I actually could compile these dynamically in xEdit, since they will have identical code other than their name. It would also only work on SKSE systems, which is recommended but not required to use the framework.

 

Both of these techniques would potentially create a lot of "clutter" in the plugin and in the save file (and in the scripts folder for the last technique). Every update would require new script instances to be added and stored in the player's save file for every changed Splicer, and who knows how many Splicers will be changed how many times.

 

Oh and the Splicers are attached to a base form of an Activator, not to instances of a single base form. So I can't delete them when they aren't needed anymore without orphaning scripts in people's save files.

 

So, anyone have any better ideas?

[cdcooley]

OK, this is a bit of a hack and I'm probably going to regret sharing it if people starting thinking it's a good idea, but considering the situation it's at least as good as anything else and better than most things you might try.

 

If I'm understanding your situation the current script has this basic setup.

ScriptName SplicerScript extends Activator

ActorBase[] Property ActorList Auto

int[] Property EncLvlList Auto 

Event SomeEventOrFunction() ; there can be more than one of course 
    ; bits of code
    int i = EncLvlList.find(4)  ; or some other code referencing EncLvlList
    ; more bits of code
    ActorBase c = ActorList[i]  ; or some other code referencing ActorList
    ; even more bits of code
EndEvent

And you want to be able to refill those properties dynamically but you also have the ability to rewrite the scripts themselves.

 

Assuming you aren't trying to access those properties from other objects the simplest solution is to rewrite the script each time and fill a new property name.

 

In the simple case you just need two versions of the script. The first rewrite would rename the properties and all references to them but leave everything else the same.

ScriptName SplicerScript extends Activator

ActorBase[] Property ActorListAlt Auto

int[] Property EncLvlListAlt Auto 

Event SomeEventOrFunction() ; there can be more than one of course 
    ; bits of code
    int i = EncLvlListAlt.find(4)  ; or some other code referencing EncLvlList
    ; more bits of code
    ActorBase c = ActorListAlt[i]  ; or some other code referencing ActorList
    ; even more bits of code
EndEvent

Then if you need to change things again you revert to the first version of the script on the second time. Just alternating versions of the script will force the CK to keep re-initializing the properties and forgetting the old ones. There are a few big flaws in this implementation though. If one of the events or functions was running when a save was made and then the script is changed there could be errors because the old script code will be trying to use the old properties as the game is being reloaded. If the functions are short and you're fairly certain players won't be saving, upgrading, and reloading while they are running that won't be a problem. The other problem is that if the player runs the upgrade process twice without actually using the first one in the game you're back to the original script and changes to the properties won't be visible.

 

The slightly more complicated approach adds properties each time instead of replacing them. So if the first script has ActorList, the second script keeps ActorList and adds a new ActorList2 that's now the real list used by the functions. The third script will remove ActorList (to keep memory use sane), keep ActorList2, add an ActorList3, and switch all of the functions to start using the new ActorList3. (It's only when re-loading old saves that ActorList or ActorList2 could be used if you're on the third script.)

 

I suspect the simpler alternating version will probably be good enough. At the worst people might run your process twice and then need to run it one more time if changes don't appear in the game.

[DavidJCobb]

A mod can have a potentially unlimited number of Splicers, and they all share the same identical script. . . Oh and the Splicers are attached to a base form of an Activator, not to instances of a single base form. So I can't delete them when they aren't needed anymore without orphaning scripts in people's save files.

Based on what I'm reading, I'm not exactly sure how your system works right now, but I see two possibilities:

 

a) A client mod defines its own Activator, attaches your script to it, and then places an ObjectReference (with this activator as its base form) somewhere in the world. The ObjectReference is registered with your mod at run-time.

 

b) You provide a single base Activator with a script on it (possibly as an injected record). Client mods place an ObjectReference in the world and fill its script properties, and then the ObjectReference is registered with your mod at run-time.

 

I think I know what to do about this -- a solution that doesn't require creating a new script for every update to a client mod, or continually replacing different Splicer references, or disabling Splicer references every time a client mod needs to be updated.

 

Updating a mod using the old system, to use a new system. We need some way to render the data for the old system irrelevant. Accordingly, a Splicer ObjectReference using the old system should be disabled somehow, so that your mod will not pay attention to it. Attaching a new script to it (to call Self.Disable()) is messy but would get the job done.

 

I mean, I don't think you can edit the original script in use for Splicers: to avoid Papyrus warnings, client mods would have to ship their own copy of the PEX, and all copies of the PEX must be identical, no? So yes, you need to disable the reference, and then you'll simply have to modify your system and tell it to ignore disabled references. Other than that, you should make no other changes to the old system: you don't want to drop support for Splicer-References entirely.

 

Creating a new system for Splicers. In the old system, the Splicer ObjectReferences must have been passed to your mod somehow, yes? Well, in the new system, don't rely on ObjectReferences. Instead, have client mods define their own base Activator ("Splicer-Activator") and pass that to your mod instead; the client mods should attach a script (with the same properties as before) to it and set property values on the Splicer-Activator itself. When you need to check the Splicer-Activator's properties, have some reference PlaceAtMe it, check those properties on the newly-created Splicer-Reference, and then delete that Splicer-Reference.

 

Properties should only get saved on a per-ObjectReference basis: a newly-spawned ObjectReference should have whatever properties are attached to the base Activator (in the ESP) at the instant the reference is created; if you spawn these references, check them, and delete them, instead of retaining any of them, then you should always be able to see whatever values are defined in the client mods' ESPs at the current moment.

 

The end result of this system is that existing mods only need to disable a single Splicer-Reference; after that, they can begin using the new system. The new system should allow for changes directly to the Papyrus properties on a client mod's own Splicer-Activator, without needing any special handling to make those changes take effect; in particular, that Activator should be usable pretty much forever.

 

but, like, test it first, obvs, because i didn't

Edited September 22, 2017 by DavidJCobb

[opusGlass]

@cdcooley

 

Unfortunately my scripts are somewhat long and can mess things up if the user saves during execution, so don't think swapping scripts will work. And deleting existing properties on a script isn't safe for save files, so I can't alternate between deleting properties and recreating properties on the same script.

 

@DavidJCobb

 

The system is a bit different than you were speculating. I was hoping to avoid going into detail because it's quite complicated, but here's the gist:

 

-Clients install a FrameworkPlugin.esp that contains template versions of all the necessary Forms, the corresponding papyrus scripts with source, and xEdit scripts to incorporate the framework into their mod.

-Clients run an "Integrate Framework" xEdit script that (1) copies all of the necessary template Forms from FrameworkPlugin.esp into ClientMod.esp, (2) rename all papyrus scripts uniquely for the mod and recompile them to avoid version conflicts between mods, (3) edit ClientMod.esp's Forms in that mod to refer to their own versions of everything. Since everything has been copied into ClientMod.esp, it doesn't add any dependencies -- users don't need FrameworkPlugin.esp.

-Clients run several scripts to build the Forms needed for ClientMod.esp specifically, including the Splicers. Each Splicer has an entirely separate Activator form, with initial values for their accm_SplicerScript properties varying between Splicer forms. (This next part might surprise you but...) There are no ObjectReferences of any of the Splicers. It turns out that the engine stores, runs, and saves scripts for base forms (or at least for Activator base forms) without any need ObjectReferences. Even runtime changes to Properties are saved.

 

Now, because of that last point, your solutions won't quite work, but you actually gave me an idea for how to make it work:

 

-Add a const GlobalVariable "ModVersion." In my script, dynamically save the value of runtime.

-On each game load, check if the GlobalVariable is greater than the script-stored value of ModVersion. If so:

-For each Splicer, create an ObjectReference in the world. Hopefully, these references will get the plugin value of each property, instead of the base form's current property values. Then, copy the properties from the references back into the base forms. Then, delete the references.

 

It just might work. Thank you!

[cdcooley]

Your idea of creating a new object reference then deleting it should work. If you're already recompiling scripts why not use a version function with a "return 8" or similar statement as the body and compare that to a matching variable in the script instead of requiring an extra global? If the properties don't work like you hope you can always put the properties on some secondary object (or even an active magic effect).

 

But why do you think deleting properties isn't safe? The only issue I know of is that any instances of running functions at the time the mod was saved will be affected if the old properties or variables they are referencing have been removed when you reload that save. But after those functions finish running the old values will be unused and simply be skipped when you make the next save with no further errors or problems.

[opusGlass]

> If you're already recompiling scripts why not use a version function with a "return 8"

 

That is possible, I think it's a toss-up over which is easier. I just decided editing and compiling Papyrus from xEdit was a little more difficult than editing a GlobalVar ;)

 

> But why do you think deleting properties isn't safe?

 

Maybe my info on that is outdated. Pretty sure this page used to say something to the effect of "removing properties causes unpredictable erratic errors." SkyUI still says:

 

 

Based on [the article](http://www.creationkit.com/Save_File_Notes_(Papyrus\)) that was linked earlier in this section, when making changes to a script that has already been deployed in a release, we can define a few rules:

• ...
• Don't remove or rename variables or properties; only add new things.

 

and I've seen the same sentiment expressed in forums. But the page now says they will just be removed automatically from the save file, so perhaps that was more of a rumor.

[DavidJCobb]

Now, because of that last point, your solutions won't quite work, but you actually gave me an idea for how to make it work [...] It just might work. Thank you!

Glad to hear it! But since I'm just a fountain of ideas and (sometimes unwanted) suggestions: if I may,...

 

 

(This next part might surprise you but...) There are no ObjectReferences of any of the Splicers. It turns out that the engine stores, runs, and saves scripts for base forms (or at least for Activator base forms) without any need ObjectReferences. Even runtime changes to Properties are saved.

Oh, I see. Your script subclasses Activator and so on, rather than ObjectReference. I didn't think of that because it's not really something Bethesda intended for us to do. (If you were to spawn a reference, the game would try to attach the Activator subclass to the reference and then complain about the types not lining up. Attaching scripts to base forms is meant solely as a way to specify defaults for references.)

 

If it works, it works, and you should go with it; it's just not the "intended" usage.

 

 

(2) rename all papyrus scripts uniquely for the mod and recompile them to avoid version conflicts between mods

Hm. If you wish to switch to a completely new system, you may be able to improve this, but there may not be a strict need to.

 

Two of my mods, Cobb Positioner and Atronach Crossing, have to be able to talk to each other, and post-2.0 versions do this using an interface script. The idea is that Cobb Positioner offers a script with "stub" functions -- getters that return default values, event handlers that do nothing, and so on. Any other mod can bundle the PEX for the interface script, subclass the script, override the functions, and use the subclass on their forms. Papyrus treats all functions as virtual methods, so at run-time, (SomeSubclassInstance as Superclass).OverriddenMethod() and (SomeSubclassInstance as Subclass).OverriddenMethod() both call the subclass method; it seems the only way to call the superclass method is for the subclass to access it via Parent.

 

Which brings us to avoiding version conflicts. It turns out, you can just modify a Papyrus script's superclass whenever and it won't break existing saves at all... at least as long as the new superclass is a subclass of the old one. When Atronach Crossing 2.0 first shipped, it subclassed CobbPositionerSCRIPTInterface. When I needed to add new interface APIs to Cobb Positioner, I just added a subclass of the old interface, called that subclass CobbPositionerSCRIPTInterfaceV2, and then changed Atronach Crossing's scripts to subclass that instead -- literally just changed the extends portion in the existing scripts. (And of course, AC now needed to ship both the interface scripts' PEX files.) It's worked seamlessly.

 

If you wanted to, you could offer the same approach to client mods. This would reduce the number of different script names (i.e. string table usage) in users' savegames, and it also just kinda feels cleaner. However, I've not tested it for Skyrim Special at all; for all I know, that game would freak out completely if you retconned Papyrus class relationships for it.

 

Maybe my info on that is outdated. Pretty sure this page used to say something to the effect of "removing properties causes unpredictable erratic errors."

Checked the revision history on the page. It has only ever claimed that properties are discarded after an initial error, and I'm skeptical of that claim.

 

*tests*

 

Yep. I just reloaded a save from a profile that has had no changes to its mod list or mods, and it still got a bunch of warnings about removed Papyrus properties. Those things get stuck in the savegame forever, pretty much. I'm gonna go fix that CK article.

[cdcooley]

"removing properties causes unpredictable erratic errors."

 

That's referring to the behavior caused by the copies of running functions that get stored in a save game file and technically there's nothing unpredictable about it now that we understand the process. But when that was written we didn't realize that the game was storing the body of running functions in the save files.

 

 

Maybe my info on that is outdated. Pretty sure this page used to say something to the effect of "removing properties causes unpredictable erratic errors."

Checked the revision history on the page. It has only ever claimed that properties are discarded after an initial error, and I'm skeptical of that claim.

 

*tests*

 

Yep. I just reloaded a save from a profile that has had no changes to its mod list or mods, and it still got a bunch of warnings about removed Papyrus properties. Those things get stuck in the savegame forever, pretty much. I'm gonna go fix that CK article.

 

No, the wiki page is correct. That error message can mean two different things and you're seeing the other problem not we one we're talking about here.

 

Trust me, I've removed many properties from scripts across revisions of my mods and about 60 such properties for scripts in the Inigo mod. The unofficial patch has also used the technique repeatedly in its many updates.

 

The warning about removed properties can be triggered from both properties stored in your save game file and from entries in the ESP files you're using. The old properties from the saved game are definitely cleared out and not resaved, but only if the properties in question are also removed from the ESP files too!

 

As long as you still have something trying to fill (or refill) a property that's not currently in the script you'll get an error. If it's an ESP you'll keep getting the error every time you load the game, but if the error is coming from values stored in the save file you'll get the error once because the property won't get re-saved when the game knows the property doesn't exist in the script.

 

So if you're updating the script and the matching ESP at the same time you can clear out old properties without any problems. But if you're only modifying the script then you'll see errors because you made a mistake not because the game isn't clearing out old properties.

 

If you see those sort of errors, save, reload, and then see them again you have a mod who's author didn't go back and refill properties in the CK after changing the script or you have a mod conflict with one mod changing the same script used by other mods. Sometimes that's harmless and sometimes it's not.