[LE] Need help speeding up my script

[foamyesque]

AIUI, @cdcooley, candlepin's plan is to on-the-fly convert a selection of food and stuff into equivalent new alchemy ingredients but only if the player is, in fact, doing alchemy with them, and then return them back to the player as the originals for all other situations. That requires an exact counts of what was moved around in some manner and the swap on entering the alchemy crafting menu needs to be done as fast as manageable.

 

The reason for my approach was to reduce the stuff needed to happen on opening that menu to the absolute minimum I can think of: A RemoveItem call on the player, dumping to Chest A, and a RemoveAllItems call on Chest B, dumping to the player. The trick is in setting the contents of Chest B to be equal counts of the equivalent items that were added to Chest A.

 

That could also be done via inventory walking (candlepin's current approach, which will be accurate but slow) or by OnItemAdded event catching on Chest A to add items to Chest B, but since you also need to handle the reverse case you wind up with potential for an infinite loop of calls (that would have to be handled carefully). Player alias OnItemAdded and OnItemRemoved is a simpler approach since it's much easier to know exactly what state the script is currently in and whether it should be listening for them or not. The other problem with building that inventory from the events triggered on a RemoveItem or RemoveAllItems call is that it can lead to potential stack dumps.

[candlepin]

 

So that could be something like this?

 

script applied to player:

PlayerRef.RemoveItem(ListA, 999, true, SwapContainer)

script on SwapContainer:

Event OnItemAdded(Form akBaseItem, int aiItemCount, ObjectReference akItemReference, ObjectReference akSourceContainer)
     Int FormNumber ListA.Find(akBaseItem)
     ItemB = ListB.GetAt(FormNumber)
     PlayerRef.AddItem(ItemB, aiItemCount, true)
     SwapContainer.RemoveItem(akBaseItem, aiItemCount, true)
EndEvent     

 

Well I tested this approach using a persistent container and it really didn't make any huge improvement on the speed. Maybe it's a second faster?

 

Next on the testing block:

 

1) Split the script in two scripts and split the swap formlist into 2 formlist sets, each with half the items to swap. See if this improves performance noticeably.

 

2) Try foamyesque's suggestion of pre-loading the swap chest. This should definitely make the swap faster (it seems scrolling through the formlist is the slow part of what I'm trying to do), but if splitting the script into pieces makes it acceptably fast I might go that way so that there isn't a continuously running script on the player checking every time an item is added or removed from their inventory. But if splitting the formlists doesn't work, this approach seems promising.

[candlepin]

 

Next on the testing block:

 

1) Split the script in two scripts and split the swap formlist into 2 formlist sets, each with half the items to swap. See if this improves performance noticeably.

 

Interesting results. Splitting the list in half reduced the swap time by 50%. That was what I was hoping for.

 

That still puts me in a pickle though:

 

Assuming this reduction scales, I'd have to split my list into 45 different lists to get my swap down to 1 second (I'd consider a 1 second swap acceptable). That seems like a lot of tedious work, but I could grab another coffee (or forty) and brute force it if necessary.

 

Or I should I try foamyesque's approach but potentially at the cost of a little (perhaps not noticeable on it's own?) lag here and there.

 

Other than it being a stupid amount of tedious work for myself, is there any downside to the first of those approaches (splitting my list into a ton of smaller lists)? Is there a saturation point where too many scripts running at once, even simple ones, doesn't increase the speed? Or potentially even makes things worse?

Edited April 22, 2018 by candlepin

[cdcooley]

I'm confused by what you mean by "scrolling through the formlist". Do you still have some loop in your code? If not the "PlayerRef.RemoveItem(ListA, 999, true, SwapContainer)" call is probably what's taking the most time to complete and foamyesque's idea of putting the OnItemAdded check (with an appropriate inventory filter) on the player really is the best solution.

[candlepin]

"PlayerRef.RemoveItem(ListA, 999, true, SwapContainer)" --> I assume for this to execute the engine goes through each item in ListA, one by one, and transfers up to 999 of that item the SwapContainer. That's what I meant by scrolling through the formlist. And that is where I thought the bottleneck was.

 

I've been waffling back and forth about whether I want to split the formlists into tiny chunks and do the swaps that way. But it seems really tedious and I'm a little worried that if I try to run 40+ scripts from a single spell/magic effect that I may create a singularity in Skyrim and implode my laptop :tongue: But maybe I'm overreacting and the simplicity of these scripts would negate any chances of script-burst catastrophes? I would love to hear if this is a valid concern. Normally I would just try both approaches and see what is better but splitting the lists would be a crap ton of work and I really, really don't want to do all that work, have it fail miserably, and have someone turn around and say it was crazy/stupid to try it in the first place.

Edited April 22, 2018 by candlepin

[foamyesque]

"PlayerRef.RemoveItem(ListA, 999, true, SwapContainer)" --> I assume for this to execute the engine goes through each item in ListA, one by one, and transfers up to 999 of that item the SwapContainer. That's what I meant by scrolling through the formlist. And that is where I thought the bottleneck was.

 

It does, but it'll have to do the same thing even if you split it, and because the internal execution of the function isn't frame-linked (but the original calls are), splitting it into multiple form lists would tend to slow things down. I'm honestly surprised you say you saw a speed increase by dividing it in two -- my best guess for the cause there is that two separate RemoveItem calls allows the OnItemAdded events that're firing on the chest more time to complete and reduces the total number of suspended stacks.

 

 

I'll see about working up some full scripts for you to try out, but the total time required for the RemoveItem(formlist) / RemoveAllItems() shouldn't be more than a second or two. Just to check, though: Do you want to restore the originals or not? Because if you don't this is a much simpler problem.

[candlepin]

 

"PlayerRef.RemoveItem(ListA, 999, true, SwapContainer)" --> I assume for this to execute the engine goes through each item in ListA, one by one, and transfers up to 999 of that item the SwapContainer. That's what I meant by scrolling through the formlist. And that is where I thought the bottleneck was.

 

It does, but it'll have to do the same thing even if you split it, and because the internal execution of the function isn't frame-linked (but the original calls are), splitting it into multiple form lists would tend to slow things down. I'm honestly surprised you say you saw a speed increase by dividing it in two -- my best guess for the cause there is that two separate RemoveItem calls allows the OnItemAdded events that're firing on the chest more time to complete and reduces the total number of suspended stacks.

 

 

I'll see about working up some full scripts for you to try out, but the total time required for the RemoveItem(formlist) / RemoveAllItems() shouldn't be more than a second or two. Just to check, though: Do you want to restore the originals or not? Because if you don't this is a much simpler problem.

 

 

I should have been more explicit in saying that not only did I split the formlist, but I also put them in separate scripts that I attached to a separate magic effects (although I guess I could have attached them to the same magic effect with similar results). I attached both magic effects to the same spell and cast the spell. I assume that it cut the swap time in half since the two scripts were run simultaneously. Thus my comment about poor man's threading; by splitting the scripts and running them simultaneously it's effectively doing the same thing as threading.

Edited April 23, 2018 by candlepin

[candlepin]

 

Just to check, though: Do you want to restore the originals or not? Because if you don't this is a much simpler problem.

 

For my mod, I want to do the following:

 

1) Swap upon entering crafting menus, swap back upon leaving crafting menus to allow full utilization of the items. This may or may not be necessary based on the version the items are in upon entering the crafting menu (i.e. ingredient or not). This should be done automatically.

 

2) Swap whenever the player desires. Again, this is about utilizing the items. Maybe the player would like to eat the food version of a particular item. Or maybe they would like to eat the ingredient version to learn its alchemy effects. This should be triggered by casting a spell.

 

All the swapping events will be governed by a global variable. For the crafting menu swaps, the global will be changed and the items swapped upon entering the menu. When the menu is closed the global will be changed and the swap function will be called again. Since the global has been toggled, the swap will be back to the original version. Do you get what I'm going for now?

Edited April 23, 2018 by candlepin

[foamyesque]

That's what I thought I remembered your mod was trying to do -- you've talked about it before -- but I wanted to be sure.

[candlepin]

That's what I thought I remembered your mod was trying to do -- you've talked about it before -- but I wanted to be sure.

 

Yeah, I got this aspect of my mod working a while ago. Unfortunately I was initially only testing with a few items and everything seemed fine so I set it aside while I got other aspects of my mod to work. But recently I went back to update the formlists with all my items and tested the swap effect with pretty poor results; it worked but took way too long.

 

Any thoughts on whether it is worth trying to split my formlists and script into ~20 pieces and run those ~20 scripts off the same magic effect/separate magic effects? I think I read somewhere that someone was recommending limiting scripts to 10 threads. Either way I will also definitely try the method you recommended; making a container that adds and removes the swap version of items as the player adds and removes items from their inventory and then swapping the items when called upon.