[Solution] Trick Papyrus engine into duplicating an array for you

[niston]

Sometimes, it may be necessary to create a copy/duplicate/clone of an Array.

Note that this is not the same as creating a second reference to your Array (see Posts below).

 

The obvious way to create a copy of an Array is:

SomeType[] sourceArray ; assumed this is already filled
SomeType[] destinationArray = new SomeType[0]
Int i = 0
While (i < sourceArray.Length)
    destinationArray.Add(sourceArray[i])
    i += 1
EndWhile

However, this will take a while, as Papyrus is going to process each and every element of the source Array, one by one. The more elements there are in the Array, the longer it takes. And Papyrus is slow by design.

If only Papyrus had a command to duplicate an entire Array at once!

 

Well... In fact, it does. Not a dedicated command or function, but you can "trick" the engine into duplicating almost any Array for you by doing this:

SomeType[] sourceArray ; assumed this is already filled
SomeType[] destinationArray = (sourceArray as Var[]) as SomeType[]

This is going to be orders of magnitude faster than the While loop method shown above.

Oh yes, and it can handle Arrays with more than 128 Elements just fine, where .Add() would fail !

 

But why does it work?

Because: When casting between Types, Papyrus creates a duplicate of the Array that is being casted, under the hood. And we can cast any Array through Var[] and back, without data loss.

 

Ka-Tching!

[dylbill]

If they are the same type you can just do:

SomeType[] SourceArray 
SomeType[] DestinationArray 

SomeEvent() 
    DestinationArray = SourceArray 
EndEvent

[niston]

Interesting point you're hitting there.

 

Because your method will not create a copy.

Instead, it will create a second reference to your Array.

 

This means that, if you change destinationArray after using destinationArray = sourceArray, say by removing an element, that element will be gone from sourceArray as well.

 

Try it.

 

Also the method I posted only works IF source and destination Array are of the same Type (or there might be data loss and besides, it wouldn't really make sense).

[dylbill]

Ah gotcha, that's good to know. Thanks!

[YouDoNotKnowMyName]

This is some interesting stuff ...

[SKKmods]

In learning the workshop system I have f***ed up the WorkshopParentScript.Workshops lists many many times by not realising I was only creating pointers to it (rather than a copy) and then modifying the array.

WorkshopScript[] Workshops = (pWorkshopParent as WorkshopParentScript).Workshops ; THIS IS A POINTER NOT A COPY  

But it is measurably faster for a script to work with a local pointer than directly on the remote script array.

 

This is not well documented/exposed and the cause of many workshop list script corruptions in mods.

 

ps props to you @Niston, very elegant ... how did you figure out that cast duplication ?

[YouDoNotKnowMyName]

Everybody probably knows about the "item duplication glitch" in older games (like Skyrim, Oblivion, maybe the older Fallouts?).

So there is also a "item duplication glitch" (or in this case "array duplication glitch") in Papyrus ...

 

:laugh:

 

(I know this is not really a "glitch" ...)

 

Also ...

Wait a second you can create arrays with more then 128 entries with this?

:woot:

 

Nice!

Edited October 14, 2020 by YouDoNotKnowMyName

[niston]

@SKK50 Believe it or not, it's in the documentation. Fo4 CK Wiki->Array Reference->Casting Arrays. I was quite amazed.

 

@YouDoNotKnowMyName No, you can't create large arrays with this. But it can copy them, for example if the game gives you an Array >128 elements, whereas a While loop using .Add() will fail after copying 128 elements.

 

But I am in fact using this to clone instances of an Event Buffer class I made. That class can hold up to 640 events, it's got 5 internal Arrays for which it does the banking. You can access the elements like so:

refBufferedEvent = eventBuffer.Entry(iElementIndex)

You can .Add() to it up to 640 times, after which Add() will return false and a buffer overflow error is printed to the log. And there is .Clear() of course and .Find().

It also supports indexed element removal, since a few hours ago. The tricky part there is to compact the bank Arrays after remove operations conclude, so that the indexed access doesn't run out of sync.

 

Why 640 Events? Because that will comfortably buffer up to 128 scheduler events per day (128 is the max. number of entries in a schedule for the CHRONOS event scheduler), for more than 4.125 days. Which equals to 99 hours, which is the maximum number of hours you can wait or sleep with reg2k's waiting menu. CHRONOS is capable of precisely tracking scheduled events - even if they did technically not occur because sleep/wait/fasttravel. And that's why it needs a way to buffer lots of events that did not happen. Sounds weird, but that's how it is. Should the buffer still overflow (you've been waiting for several thousand hours, somehow...), CHRONOS will clear the buffer, reinitialize itself and track missed events for just the current day.

 

But however that may be, here's the code I'm using to clone my buffer objects (this is inside the Event Buffer class):

; it is user's responsibility to destroy the clone once it is no longer used
F4MS_SchedulerEventBuffer Function Clone()
	; create new eventbuffer item object for clone
	F4MS_SchedulerEventBuffer bufferClone = Create()
	
	; safety check
	If (bufferClone == none)
		Debug.Trace(Self + ": ERROR - Clone() failed to create clone buffer object. Aborting.")
		Return none
	EndIf
	
	; fill clone banks with our banks' contents
	bufferClone.CloneFill(Bank1, Bank2, Bank3, Bank4, Bank5)

	; return newly created clone
	Return bufferClone
EndFunction

; fills this buffer's array banks with a CLONE (ie a copy and not a 2nd reference) of the array contents supplied as parameters
Function CloneFill(SchedulerEventBufferEntry[] aryBank1, SchedulerEventBufferEntry[] aryBank2, SchedulerEventBufferEntry[] aryBank3, SchedulerEventBufferEntry[] aryBank4, SchedulerEventBufferEntry[] aryBank5)
	If (aryBank1 != none)
		; casting to a different type forces papyrus engine to duplicate the array - we exploit this to have Papyrus make a clone of the source array for us
		Bank1 = (aryBank1 as Var[]) as SchedulerEventBufferEntry[]
	Else
		Bank1 = none
	EndIf
	If (aryBank2 != none)
		Bank2 = (aryBank2 as Var[]) as SchedulerEventBufferEntry[]
	Else
		Bank2 = none
	EndIf
	If (aryBank3 != none)
		Bank3 = (aryBank3 as Var[]) as SchedulerEventBufferEntry[]
	Else
		Bank3 = none
	EndIf
	If (aryBank4 != none)
		Bank4 = (aryBank4 as Var[]) as SchedulerEventBufferEntry[]
	Else
		Bank4 = none
	EndIf
	If (aryBank5 != none)
		Bank5 = (aryBank5 as Var[]) as SchedulerEventBufferEntry[]
	Else
		Bank5 = none
	EndIf
EndFunction

This can be used like so:

F4MS:F4MS_SchedulerEventBuffer myClonedBuffer = originalBuffer.Clone()

Convenient, eh?

It would of course provide even more convenience if that stupid 128 elements limit on Arrays didn't exist in the first place...

 

But I plan to further expand on the Array banking scheme to provide an "unlimited" storage system, leveraging the excellent Papyrus Compiler Struct Patch to provide up to 128 banks (an array of arrays) in a bucket class, and Reflinking to organize many buckets. It shall be known as Large Capacity Storage.

[SKKmods]

@Niston, you are of course totally right its there in black 'n white ... but not the cunningness of using var as an intermediary to flip back to the original.

 

FX: Sound of SKK updating 200 scripts ...

[niston]

@SKK Did you see the Compiler Patch?