Unpacking BSAs to loose files

[saurusmaximus]

I have 171 of the suckers currently, and a surprising number are tiny, just a few KB. As often as not I'm finding scripts embedded in them, which someone else had recently told me was a bad practice (but didn't explain).

 

I even have a few that have just one file in them, and it's just a script. What's the motivation for doing that?

I don't know specifically if this is the case with the mods you are talking about but, for example, there are a couple scripts you can apply to keep static objects from flying around when you load into a cell; basically the script tells Havok to ignore that particular item. It's one tiny script that fires once when you first load into a cell, but if you have a lot of them, it does become a cumulative thing.

[VulcanTourist]

 

I have 171 of the suckers currently, and a surprising number are tiny, just a few KB. As often as not I'm finding scripts embedded in them, which someone else had recently told me was a bad practice (but didn't explain).

 

I even have a few that have just one file in them, and it's just a script. What's the motivation for doing that?

I don't know specifically if this is the case with the mods you are talking about but, for example, there are a couple scripts you can apply to keep static objects from flying around when you load into a cell; basically the script tells Havok to ignore that particular item. It's one tiny script that fires once when you first load into a cell, but if you have a lot of them, it does become a cumulative thing.

 

My intent in asking the question was to understand not why a mod would have just one script but rather why a mod would choose to put its one script inside a BSA rather than leaving it loose?

[FiftyTifty]

There are files that need to be loaded inside an archive, otherwise it breaks game functionality. The .egt files are such examples. Performance between having BSAs and loose files is massive. Even on NVME SSDs. You can see this in benchmarks comparing single sequential reads/writes to 4k IOPS reads/writes. CrystalDiskMark is great for that.

 

The ideal is to not extract files from the archives, but instead, use bsarch to create archives with no compression. That way, no CPU time is spent on decompressing the archives, but you get all the performance benefit of having large, singular files instead of hundreds of thousands of itty-bitty sized files.

[IsharaMeradin]

 

 

I have 171 of the suckers currently, and a surprising number are tiny, just a few KB. As often as not I'm finding scripts embedded in them, which someone else had recently told me was a bad practice (but didn't explain).

 

I even have a few that have just one file in them, and it's just a script. What's the motivation for doing that?

I don't know specifically if this is the case with the mods you are talking about but, for example, there are a couple scripts you can apply to keep static objects from flying around when you load into a cell; basically the script tells Havok to ignore that particular item. It's one tiny script that fires once when you first load into a cell, but if you have a lot of them, it does become a cumulative thing.

 

My intent in asking the question was to understand not why a mod would have just one script but rather why a mod would choose to put its one script inside a BSA rather than leaving it loose?

 

If the mod requires a plugin, using a BSA is smart. It keeps the assets in one location and loads it with the plugin. Change plugin load order and the assets change position with it. Leaving the files loose might be "easier" to start with but comes with an added risk of users doing something that causes the files to go missing thus causing the mod to break. Using BSAs help to avoid user created issues.

 

What you might have been told was to not include the PSC files as the game does not use them. However, as an author whose mods are mostly script related, I prefer that others include their source files. While Champoilion works to get a usable PSC from a PEX, it does not include any of the author's comments which can be extremely helpful in understanding their intention.

[VulcanTourist]

If the mod requires a plugin, using a BSA is smart. It keeps the assets in one location and loads it with the plugin. Change plugin load order and the assets change position with it.

Unless I have substantially misunderstood the behavior of BSAs and loose files as plugin dependencies, this is not true. The files referenced by a plugin always get loaded into memory and assigned the same mod index regardless of the physical origin of the files.

 

The only issue here is precedence. The files in a BSA are somewhat hidden from an otherwise transparent process, where a capable mod manager like Vortex can inform a player about file conflicts and help him make correct decisions about precedence, even irrespective of load order. If everything originated from a BSA, then load order - mod indices - would be the undisputed King of Precedence and nothing else would matter, no other executive decisions could be made aside from choosing load order.

 

I don't want load order to be the only deciding factor of file precedence in my game. Load order's value is in determining record precedence, which cannot be determined any other way, unlike file precedence.

 

Now, having files unique to a particular mod all grouped together and tied with a bow is a convenience, but it's a convenience that I can easily duplicate with loose files by seeking them out in the mods' staging directory. If I find a "deployed" loose file whose origin or purpose I cannot identify, I can just as easily find it in the staging directory and understand its purpose as I could if it was grouped in a BSA. The staging directories become my BSAs.

 

Leaving the files loose might be "easier" to start with but comes with an added risk of users doing something that causes the files to go missing thus causing the mod to break. Using BSAs help to avoid user created issues.

My concern is my convenience and workflow and not that of other mod authors. Loose files add little to no risk for me personally because I'm capable of avoiding stupid mistakes. I won't attempt to persuade others to abandon their own convenience for publishing and "tech support", but neither should they expect me to forsake my own if I find BSAs an impediment to my process.

[IsharaMeradin]

You misunderstand.

 

Loose files in the data folder will ALWAYS be available for use even if the associated plugin is inactive.

BSA files in the data folder will not be available for use if the associated plugin is inactive.

 

Any indexing of loose files and loading specifically with a plugin would be a feature provided by a mod manager and not behavior designed by Bethesda.

 

********************

I am not saying that one should ALWAYS use a BSA. It is definitely a hindrance to the development process. I am saying that one should consider using a BSA at release time for PC and definitely use one for consoles.

 

You can unpack every BSA in your game if you wish. If something does not work correctly because the loose order presented to the game is not what is expected by other files / records, that is on you to resolve.

[VulcanTourist]

You misunderstand.

 

Loose files in the data folder will ALWAYS be available for use even if the associated plugin is inactive.

BSA files in the data folder will not be available for use if the associated plugin is inactive.

I did misunderstand, if that was part of your intent. I wouldn't have thought of it because that is mostly if not entirely an obstacle created by trying to manage mods manually rather than with a management tool, and I wouldn't even consider modding this game without a manager.

 

In my original post, I was envisioning unpacking BSAs in the mod manager's staging folder to make the loose files part of what will be managed, in place of the BSA, not simply manually unpacking them myself into Data and deleting the BSAs from there. That tactic would cause Vortex, at least, to throw fits about the interference. What I had in mind would likely require reinstalling the unpacked mods, but I thought the mod manager would treat them normally beyond that (at least until an update). Do loose files from a mod whose plugin has been manually disabled still remain "deployed"?

Edited September 5, 2020 by VulcanTourist

[IsharaMeradin]

I do not know how Vortex handles the remaining files when a plugin is disabled. I would think that it would still deploy the loose files. But a test would be needed for that.

 

In the case of Mod Organizer, yes. Loose files are "deployed" so long as the mod is active even if the plugin is disabled.

[VulcanTourist]

I do not know how Vortex handles the remaining files when a plugin is disabled. I would think that it would still deploy the loose files. But a test would be needed for that.

 

In the case of Mod Organizer, yes. Loose files are "deployed" so long as the mod is active even if the plugin is disabled.

It's very rare for me to disable plugins. The only times I can recall doing it were for momentary troubleshooting or because Vortex/LOOT informed me that a plugin was "redundant". Even then, I need to know what the behavior is with respect to loose files in case it matters.

 

This might be enough to dissuade me from my unpacking project, if the loose files will not be managed as I expect.