info: [new-file-monitor] file list snapshot + info: memory usage growing fast usage --- add 1 minute to deploys?

[TennesseeTuxedo]

I am running BETA .12. 

This issue existed when I was in .10 and now in BETA .12.  I posted another thread believing this was caused from a Vortex error I had received and Vortex deploying every mod, but I just noticed comprehensive file deployment for the first time because I was staring at the screen trying to figure out why deploys are taking significantly longer now.  I am now thinking the issue may be version related and possibly by design.  If by design, it does add an overhead (for me 5x more time to deploy a mod).  So it would be nice to understand the purpose.

When installing a simple mod from the mod's Nexus Vortex link, several days ago it took me approximately ~10 seconds from clicking the link to having the mod deployed.  Today it's over a minute.  I attached an old log and a new log for two simple mod deploys like this. 

The main thing that stands out is the new logs have two new events and each takes tens of seconds to complete.  I don't see these events in my old logs.  What are they and when did they start appearing?  I don't have a problem if there are reasons, I'm just not sure if this is some sort of error state on my side, a misconfiguration in settings, a bug of a new version, or by design.

• (+ 22 seconds)    Fri, 22 Dec 2023 07:40:59 GMT - info: [new-file-monitor] file list snapshot saved filePath=C:\Users\xxxxxxxx\AppData\Roaming\Vortex\starfield\snapshots\snapshot.json, duration=996
• (+28 seconds)     Fri, 22 Dec 2023 07:41:30 GMT - info: memory usage growing fast usage=783.4 MB, previous=502.7 MB, max=4.00 GB

 

 

vortex 12-12-2023 was 9 seconds to install from link.zip vortex 12-22-2023 now 1 minute 5 seconds to install from link.zip

[TennesseeTuxedo]

I think the new log messages are related to:

[1.9.10] - 2023-11-30

• Support for future code signing certificate
• Updated design and styles for load order pages.
• More detailed log data when updating

 

Is it possible to turn the logging off?  An extra minute is not a big deal when I am installing a mod to use from Nexus.  It's more of an issue when working on a mod or troubleshooting them where you might install and uninstall frequently, it adds a minute to each install and uninstall.  Thanks for any help!

 

 

[showler]

Is it not related to them adding load order management to the Vortex Extension and therefore adding the snapshot functionality as part of that?

[TennesseeTuxedo]

I don't know if its part of that.  I have not transitioned to Vortex to manage my plugin load order at this point since I've been managing it on my own and the transition might be a little rocky.  Plus the loot folks have thrown up their hands about load ordering Starfield due inconsistencies in Starfield, so we're kind of on our own anyway, which actually has been nice.  I have been much more in-tuned to what each mods does managing my own order.  I even look at them in xedit to see what they are touching.

I think there needs to be another generation of tools, I was thinking of a loot like functionality that works real time as part of SFSE, that might give some ability to change the ordering on the fly.  Just re-imagine the whole approach.

But Vortex now takes an extra 50 seconds for any and every change, which normally isn't so bad unless you are working on mods and troubleshooting and need to disable/enable mods a bunch of times to resolve.  Then the 50 seconds adds up.

[Tannin42]

new-file-monitor is a utility that game extensions may use but don't have to. It's basically there to detect files added to the game directory by mods that Vortex can't automatically assign to the corresponding mod without help. Think mods that create a log file or save their configuration to a newly created file.

For Vortex to detect that files were added compared to the last time it ran it has to take note of which files were there before. This doesn't usually take that long, in your set up though it tracks over a million files apparently in at least 360 thousand directories. That seems - excessive. You might want to check if your setup actually contains that many files or whether there is some kind of directory link shenanigans that confuse it.

Otherwise it might also be safe to disable the new-file-monitor extension unless the Starfield extension uses it (one of the current Vortex devs should know) because previously I think it was only Stardew Valley that needed the information anyway.

Finally, my personal preference is to disable the automatic deployment anyway, only deploy once before actually starting the game - especially when setups start to get bigger.

[TennesseeTuxedo]

On 12/24/2023 at 9:04 AM, Tannin42 said:

new-file-monitor is a utility that game extensions may use but don't have to. It's basically there to detect files added to the game directory by mods that Vortex can't automatically assign to the corresponding mod without help. Think mods that create a log file or save their configuration to a newly created file.

For Vortex to detect that files were added compared to the last time it ran it has to take note of which files were there before. This doesn't usually take that long, in your set up though it tracks over a million files apparently in at least 360 thousand directories. That seems - excessive. You might want to check if your setup actually contains that many files or whether there is some kind of directory link shenanigans that confuse it.

Otherwise it might also be safe to disable the new-file-monitor extension unless the Starfield extension uses it (one of the current Vortex devs should know) because previously I think it was only Stardew Valley that needed the information anyway.

Finally, my personal preference is to disable the automatic deployment anyway, only deploy once before actually starting the game - especially when setups start to get bigger.

This is some great information for me to follow up on, thank you!  I didn't know how to read that "seenDirectories=367140" but it did look strange to me.  There is no way I have 367,140 directories.  To me that says some kind of symlink loop, which I have seen that kind of thing happen before.

A small possibility is it may coincide with unpacking .BA2s for modding, putting them deep into the Starfield folder structure.  The unpacking and decompiling instructions recommend unpacking and decompiling these resources into the folder structure where CK would look for them, which is in deeper subfolders than the game looks for resources.  It seems safe but I didn't know if this might mess with Vortex so I moved all the unpacked resources into a new subfolder under Starfield/Data/ that is not associated with the game.  But maybe this creates a problem.  Maybe cleaning out the Starfield folder or even re-installing the game and then re-applying mods will fix this.  It takes so long time to deploy mods now it might be worth it.

Thanks for the other tips of disabling new-file-monitor and auto deploy.  I should do those things too.  I was getting 10 seconds from click to deploy without disabling them, but there really is only need to deploy once before each play.