How to completely delete an object (reference) from the game.

[LarannKiar]

Interesting fact: bethesda have clearly recognised the issue of LinkRef promoting target persistence as that is now an explicit parameter in the Starfield version of Papyrus:

 

Fallout4

SetLinkedRef(ObjectReference apRef, Keyword apKeyword = None)

Starfield

SetLinkedRef(ObjectReference akLinkedRef, Keyword apKeyword, Bool abPromoteParentRefr) 

WARNING FOR FUTURE ADHD INATTENTIVE TYPE SEARCHES: that is a Starfield Papyrus function, not Fallout4

 

I can hardly believe it.. I just started playing Starfield recently, I even haven't looked into its scripts.. well I set up the SFSE environment while the game was installing out of curiosity but there wasn't much to explore in the resources.. :) it's too early for that.

 

Wouldn't decompile all the scripts yet but I wonder for what reason they actually exposed this behavior to Papyrus.. beyond the assumption "in order to prevent unnecessary persistence if the refs in the link are always loaded" which seems... practical.

 

Lee3310/LarannKiar - do either of you have a program/script/mod that you have created to lists all the persistent objects (and type/providers etc)?

You have me very interested to see what mess my mods may be leaving behind (even though I am very rigourous at cleanup).

I suspect there is enough info in this discussion to make my own (using Garden of Eden functions) - but why re-invent the wheel?

 

Thanks!

 

I used some unreleased script extender functions to iterate and log these large amount of references actually.

[PJMail]

Well, I have used a mix of of GardenOfEden and FindAllReferencesWithKeyword functions to examine my 'deleted' actors - and, well, I don't understand this PP 'self' referencing...

I have an actor with NO scripts, created via a 'placeatme', and deleted via aActor.Delete(), yet it hangs around Deleted, but with PP of only itself.

Why? I have gone and waited in the root cellar for 'weeks' of in-game time and it still exists.

What am I doing wrong? Why won't it go away?

[hereami]

By the way, how serious is performance impact from Deleted entities stuck in Purgatory? Hopefully, scripts don't keep running like for Disabled (which is probably just an oversight still?) and they don't participate in background processing (packages and whatever else), right?

Edited October 26, 2023 by hereami

[PJMail]

Hereami - you could be right... Perhaps this always happens and now that I can see it I get worried.

I am only interested in case I am doing something wrong to cause deleted Actors to persist.

If it is 'normal' then I will forget about it, but at this point I am trying to finalize a (hopefully last) mod update so tying up loose ends.

[LarannKiar]

I just updated Garden of Eden recently, made a callable function:

   ; # Logs all persistent references based on the specified arguments
   ;    arrays must be initialized (e.g., "new Keyword[0]") even if they contain no elements
   ;    aiEP, aiPP: -1 = unspecified ("does not matter"). 0 = cannot have. 1 = must have. Both can't be set to 0 or -1 at the same time.
   ;    asSourcePlugins: array that contains the plugin where the ref originates
   ;    e.g., upon drag and dropping EncTurretTripod02 (BaseID: 001A0065) to Far Harbor, the created reference is still from "Fallout4.esm"
   ;    aiDeletedState, [D] flag in Console: -1 = unspecified ("does not matter"). 0 = not deleted. 1 = deleted.
   ;    akOrigoRef: if none, the player reference will be used if afDistance > 0
Bool Function LogPersistentRefs(int aiEditorPersistent, int aiPromotedPersistent, String[] asSourcePlugins, \
	Keyword[] akMustHaveKeywords, Keyword[] akMustNotHaveKeywords, \
	bool abMustHaveKeywordsModeAll = false, bool abMustNotHaveKeywordsModeAll = false, Form akBaseForm = None, \
	int aiDeletedState = -1, float afDistance = -1.0, ObjectReference akOrigoRef = None, bool abSuppressMessage = false) native global

It iterates the game's memory for REFR and ACHR records. I have tested it but not "that" very thoroughly (it's difficult to fully verify the results... :smile:). It can print even thousands of refs to the log within a second.

 

afDistance uses "aerial distance" (practically GetDistance() without the interior-exterior infinite distance). A messagebox also appears that shows how many persistent refs were logged, abSuppressMessage = True can hide it.

 

The log file is in My Games/Fallout4/F4SE/GardenOfEdenPapyrusScriptExtender.log.

 

The output is like:

[2023-10-26 06:21:01.413] [ F4SE log ] [info] [Objects.h:8143] LogPersistentRefs [1] -> 09000d6d. [EP] = 1. [PP] = 0. [Name] = Silver Pocket Watch. [BaseForm] = 00060e7d. [Deleted] = 0. [Distance] = 603.166321. [SourcePlugin] = Fallout4.esm
[2023-10-26 06:21:01.414] [ F4SE log ] [info] [Objects.h:8143] LogPersistentRefs [2] -> 09000c56. [EP] = 1. [PP] = 0. [Name] = Clean Striped Suit. [BaseForm] = 09000c50. [Deleted] = 0. [Distance] = 355.073730. [SourcePlugin] = cctosfo4001-neosky.esm
[2023-10-26 06:21:01.477] [ F4SE log ] [info] [Objects.h:8143] LogPersistentRefs [641] -> 09000c19. [EP] = 1. [PP] = 0. [Name] = High-Powered Microscope. [BaseForm] = 001c61c3. [Deleted] = 0. [Distance] = 855.076416. [SourcePlugin] = Fallout4.esm
[2023-10-26 06:21:01.477] [ F4SE log ] [info] [Objects.h:8143] LogPersistentRefs [642] -> 09000968. [EP] = 1. [PP] = 0. [Name] = Wood Shelf. [BaseForm] = 09000e16. [Deleted] = 0. [Distance] = 455.225067. [SourcePlugin] = cctosfo4001-neosky.esm
[2023-10-26 06:21:01.477] [ F4SE log ] [info] [Objects.h:8143] LogPersistentRefs [643] -> 09000f70. [EP] = 1. [PP] = 0. [Name] = Side Table. [BaseForm] = 0900085d. [Deleted] = 0. [Distance] = 209.167297. [SourcePlugin] = cctosfo4001-neosky.esm

For example:

Function LogEditorPersistentFarHarborNPCs()
	String[] asSourcePlugins = new String[1]
	asSourcePlugins = "DLCCoast.esm"
	Keyword[] akMustHaveKeywords = new Keyword[1]
	akMustHaveKeywords = ActorTypeNPC
	Keyword[] akMustNotHaveKeywords = new Keyword[0]
	LogPersistentRefs(1, 0, asSourcePlugins, akMustHaveKeywords, akMustNotHaveKeywords)
EndFunction

Also further improved IsPersistent(), added paramaters to filter the "persistence type".

   ; # Persistency check
   ;    optional paramaters for filtering; abMustMatch: if true, the persistence of akReference must match the [EP] and [PP] arguments
Bool Function IsPersistent(ObjectReference akReference, bool abEditorPersistent = true, bool abPromotedPersistent = true, bool abMustMatch = false) native global

Edited October 26, 2023 by LarannKiar

[lee3310]

Well, I have used a mix of of GardenOfEden and FindAllReferencesWithKeyword functions to examine my 'deleted' actors - and, well, I don't understand this PP 'self' referencing...

I have an actor with NO scripts, created via a 'placeatme', and deleted via aActor.Delete(), yet it hangs around Deleted, but with PP of only itself.

Why? I have gone and waited in the root cellar for 'weeks' of in-game time and it still exists.

What am I doing wrong? Why won't it go away?

Well, i continued the conversation on GOE mod page, what you are experiencing is the bug i was talking about. Even if DPPI shows nothing and the ref is flagged as temporary, the game can decide to keep it as Persistant. PP as LarannKiar explained means "Persistent Promoter" and in this case, the promoter is the ref it self. To avoid this behavior, you should never make a save with a temp ref still stored in a script var...(persistent). I found out that if i use "abDeleteWhenAble = true" with placeAtme to flag the ref as temp or GOE "SetTemporaryReference" function, clear all the vars/aliases/linkedRef... pointing at it then kill/disable the ref, it will get cleared as soon as the cell unloads. However, if you save the game with the ref still attached to something, there is a good chance that it will stick in game, even if you free it afterward (on game load for example)..

Hopefully you can get rid of PP (if DPPI returns empty) using "RemoveAllPersistentPromoters()". ForcePersistent() will also work but all script instances will stay attached and SetEditorPersistence(false, true) can cause CTD if there is something holding the ref ingame.

Never forget to clear script vars pointing at refs in active magic effect and reference alias scripts (any script that attaches to ref and receives same events), cause if the var is not cleared prior to quest shutdown / effect finish, there is a risk that you can't access the var ever again, even after recompiling the script and resurrect the ref... .

Edited October 27, 2023 by lee3310

[SKKmods]

Is it time for a simple big list of "all the things that can cause objects (especially actors) to be held persistent so undeletable and cause save bloat" ?

 

Reference Alias

RefCollectionAlias

FormList

Script Property

Script Variable (subclass: permanent ActiveMagicEffect via Cloak or Self).

Script Variable (subclass: local attached script with recursive "Self" as a variable).

LinkedRef Target (subclass temporary LinkedRef originator/promoter has been demised).

... add

... add

 

 

It may be 8 years since launch but much of this stuff will happily roll forward to Starfield and beyond ...

[LarannKiar]

SetLinkedRef Source can become persistent too.

 

Ref1 from "Ref1.SetLinkedRef(Ref2) may cause Ref1 to become ForcePersistent in some cases. When this happens, it can lead to save game bloating if one doesn't remove [PP] manually with ForcePersistent as the game doesn't appear to do it even on CellReset. I only managed to track down this bug when Ref2 is not [EP] (can still be promoted or not persistent at all). I can't state that it happens with every refs in a link, much more testing would be needed to make a statement like that but it can definitely happen.

 

It will be interesting to see how the Starfield version of SetLinkedRef affects persistence. The developers may have spotted this bug during development.

 

( Some terms I sometimes use:

Strict [EP] = has only editor persistence but not promoted.

Strict [PP] = has only promoted persistence but not editor set. )

 

 

Well, I have used a mix of of GardenOfEden and FindAllReferencesWithKeyword functions to examine my 'deleted' actors - and, well, I don't understand this PP 'self' referencing...

I have an actor with NO scripts, created via a 'placeatme', and deleted via aActor.Delete(), yet it hangs around Deleted, but with PP of only itself.

Why? I have gone and waited in the root cellar for 'weeks' of in-game time and it still exists.

What am I doing wrong? Why won't it go away?

Well, i continued the conversation on GOE mod page, what you are experiencing is the bug i was talking about. Even if DPPI shows nothing and the ref is flagged as temporary, the game can decide to keep it as Persistant. PP as LarannKiar explained means "Persistent Promoter" and in this case, the promoter is the ref it self. To avoid this behavior, you should never make a save with a temp ref still stored in a script var...(persistent). I found out that if i use "abDeleteWhenAble = true" with placeAtme to flag the ref as temp or GOE "SetTemporaryReference" function, clear all the vars/aliases/linkedRef... pointing at it then kill/disable the ref, it will get cleared as soon as the cell unloads. However, if you save the game with the ref still attached to something, there is a good chance that it will stick in game, even if you free it afterward (on game load for example)..

Hopefully you can get rid of PP (if DPPI returns empty) using "RemoveAllPersistentPromoters()". ForcePersistent() will also work but all script instances will stay attached and SetEditorPersistence(false, true) can cause CTD if there is something holding the ref ingame.

Never forget to clear script vars pointing at refs in active magic effect and reference alias scripts (any script that attaches to ref and receives same events), cause if the var is not cleared prior to quest shutdown / effect finish, there is a risk that you can't access the var ever again, even after recompiling the script and resurrect the ref... .

 

I added the requested IsTemporaryReference() which will make it easier to select the references for cleanup.

 

To get rid of [PP], if the only promoter is the ref itself, use ForcePersistent (console, toggle) or ForcePersistent(akRef, False) (from Garden of Eden). I switched from the custom ForcePersistent code to the native console ForcePersistent in either v14.1 or v14.2. It's more efficient.

 

Don't forget that [EP] is readded by the engine frequently.. for example, if you have a script variable that points to a ref (even a Papyrus function's temporary script variable), when the function finishes and the VM destructs the temporary variable, the game checks whether persistence should still be added to the ref. Actually, that's why IsPersistent() and SetEditorPersistence() now takes HexFormID instead of akReference, to make sure no VM related persistence can possibly "undo" the effects the functions after you call them. Might be a bit redundant but HexFormID can come in handy in some cases. (I usually refrain to change the signature of already released functions but now the new .txt reader, GetHexFormIDFromFile() for the "dump selected refs" logging mode of LogPersistentRefs() use HexFormID too).

 

Note that it's not possible to permanently remove [EP] from a ref using SetEditorPersistence(), the game will probably readd it later (if not sooner, on the next game launch). However, SetTemporaryReference() can remove those refs in the current game session even if the game code reapplied [EP] before the reference could unload. ( [EP] refs cannot actually unload anyway, even after SetEditorPersistence(False). The engine keeps them in the memory, preventing them from getting unloaded. Trying to interfere with this behavior from a script could lead to CTDs and it's not even necessary in order to delete the refs; the "cleanup functions" I mentioned earlier already bypasses the persistence check ).

 

To summarize [EP], [PP] and the console: the console text relies on other bytes (still ref data) and not the actual "persistence data" to display [EP] next to a RefID. Actually, it's possible to "apply" [EP] on a ref without making it persistent. ( This is why SetEditorPersistence(True) displays [PP] upon selecting the ref in the console and not [EP]. Would be possible to set [EP] for convenience but I haven't gotten around to that... ). [PP] is added to refs ingame, and refs which are only persistent due to having promoters ("Strict [PP] references") are loaded into the memory when one loads a save game. [EP] refs ( including the ones which are also promoted; this doesn't seem to matter ) are loaded when the game is launched. Both [PP] and [EP] can be added to both editor placed and ingame created refs though Strict [EP] created refs and Strict [PP] editor placed refs are rarer in the vanilla game.

Edited October 29, 2023 by LarannKiar