.BSA Files, should I use them? & Game performance?

[MotokoKHammond]

Quite late to the party, but there's one other quite important point I haven't seen mentioned, yet.

 

Under original conditions the game knows no load order when it comes to BSAs. The order in which they're loaded is said to be erratic at best. Thus the replacer-type mods asked about won't even reliably work anymore, when served in BSAs. A BSA cannot override contents from another BSA. It was even an unwritten rule back then that a BSA of your own doing must not even contain the same files as another or Vanilla BSA.

 

However, with the introduction of some OBSE-loaded framework, bringing BSA handling up to speed with how it's done in the newer games, this is said to be no longer a problem. BSAs can override contents from other BSAs in Skyrim et al, and with said mod it is said to be the same in Oblivion then, too.

 

 

But I for one think the main motivation for Bethesda to put all Vanilla assets in BSAs instead of having them as loose files was 1) order: no game of a comparable size I know does "not" use archives for that, and 2) mods: that you can (almost) completely erase your entire data folder and the game will just jump back to Vanilla state, is definitely a welcome thing when it comes to handling mods.

 

 

To expand on Bethesda's intentions with BSAs, the original design for overwriting vanilla files was that newer files overwrite the older Bethesda files in the BSA. The DVD release would always be older than any mod, so any loose file will always overwrite. However, it quickly became apparent that this barely worked in any intended fashion, hence archive invalidation. This is especially important with newer digital releases that leave the file date of all the BSAs as the date they're downloaded. SkyBSA from DavidJCobb fixes this issue and makes Oblivion act more like Skyrim where any loose file that overwrites a file in a vanilla BSA will be used. The same mechanism works for BSAs now, so if you have SkyBSA you technically could package your textures into a BSAs, but as others have said it doesn't offer a ton of utility (BA2 archives are much more efficient so it'll affect things more). Oblivion's I/O situation is very inefficient, you'll have longer load times just from having more plugins or models to load in. I haven't noticed a drastic increase in load times when adding texture packs compared to plugins and LOD generation

 

Mod authors package things into BSAs so they don't have to deal with loose files

So by these arguements, would one be better to sort all new asset mods into .BSA files while leaving replacers as loose? If not for tangible in game benefits but purely based on keeping your installation orderly and less cluttered with new asset files bloating file's out?

[MotokoKHammond]

@KatsAwful + @DrakeTheDragon:

 

So by these arguements, would one be better to sort all new asset mods into .BSA files while leaving replacers as loose? If not for tangible in game benefits but purely based on keeping your installation orderly and less cluttered with new asset files bloating file's out?

Ooops just did this cus my post didnt post and then suddenly I got a double post lol.

Edited September 15, 2020 by MotokoKHammond

[MotokoKHammond]

I wonder about the difference between loading a BSA at game start (but does the game even load all the BSAs?) vs loading assets as they are called for to render on screen.

 

So an example of what I'm getting at ...

 

So Joes Cool Armor mod has unique meshes and unique textures for let's say two dozen different armors. If the textures (or maybe even meshes and textures) are packed into JoesCoolArmors.bsa does the game load all of that into memory when the game starts and JoesCoolArmors.esp is loaded? If instead the mod is distributed as loose files does the game load JoesCoolArmors.esp at game start and then only load the specific meshes and textures required to render the individual armor pieces as they are going to be rendered in game (so 23 armors not loaded but the one you took from the JoesCoolArmors chest and either put on or gave to a companion to wear gets loaded into memory and displayed on screen).

 

Subtle difference, maybe not important to people with ultra rigs, but it could have a difference for those on marginal setups.

This is a great question, and I would love to know the awnser aswell. Essentially it's a question of how does the game handle .BSA archives, as a single batch or does it read them and pick out the individual information as and when it is needed.

[snargel]

I seriously doubt Oblivion loads all .bsa files into memory at startup. Just looking at my own folder, the bsa's add up to 13.6 GB, and that's before they are unpacked. No way the Gamebryo engine could handle that, especially Oblivion's old version.

 

Windows does perhaps do some intelligent precaching, but I wouldn't count on it.

 

As a general rule, packaging replacers and mods that one is likely to uninstall as .bsas seems counterproductive, both because they could get overwritten and because it could complicate uninstallation unless it was prepped as part of a BAIN installation.

Edited September 15, 2020 by snargel

[KatsAwful]

Oblivion thrashes the data directory for any file, looking for BSAs to parse. This is why there's a disabled plugin limit, it can only handle 512 of these files in memory at once. I doubt that it loads the BSAs into memory, it would quickly run out of space, but it would definitely keep names of the files inside the archive so it doesn't have to read each BSA on the fly when loading a texture to memory

I did some quick testing, there wasn't much of a difference loading in OOO and its BSA (600MB file) over not loading it (the plugin and BSA weren't in the Data folder), just about 250MB of RAM and 200 MB of VRAM in either case. If it does cache something it would have to just be file names. I am not smart enough to know or figure out how Oblivion handles BSAs exactly, this is just my supposition

@OP replacers are best left outside of BSAs since the vanilla behavior won't allow those to be seen. Self contained large mods are recommended to be in BSAs so they're smaller in size and more manageable for users. Smaller standalone mods have no real benefit to being in BSAs

[Striker879]

Oblivion thrashes the data directory for any file, looking for BSAs to parse. This is why there's a disabled plugin limit, it can only handle 512 of these files in memory at once. I doubt that it loads the BSAs into memory, it would quickly run out of space, but it would definitely keep names of the files inside the archive so it doesn't have to read each BSA on the fly when loading a texture to memory

 

I did some quick testing, there wasn't much of a difference loading in OOO and its BSA (600MB file) over not loading it (the plugin and BSA weren't in the Data folder), just about 250MB of RAM and 200 MB of VRAM in either case. If it does cache something it would have to just be file names. I am not smart enough to know or figure out how Oblivion handles BSAs exactly, this is just my supposition

 

@OP replacers are best left outside of BSAs since the vanilla behavior won't allow those to be seen. Self contained large mods are recommended to be in BSAs so they're smaller in size and more manageable for users. Smaller standalone mods have no real benefit to being in BSAs

 

Makes sense.

 

So now, does the game decompress what files it needs to render on the fly? Sort of brings me back to my suspicion that loose files are a less intensive task overall, especially in this era of SSDs or faster. Or perhaps the uncompressed BSA theory is the go to solution ... game indexes the BSA contents for when it needs to retrieve them and no CPU cycles burned decompressing on the fly.

[KatsAwful]

Decompression does seem to be an issue sometimes. There's a program for Oblivion that uncompresses the vanilla BSAs and seems to help for lower end systems. No idea how well it actually works, doesn't seem to make a difference at all in Wine last time I tried (load times seem to be worse in Wine overall)

[snargel]

Decompression does seem to be an issue sometimes. There's a program for Oblivion that uncompresses the vanilla BSAs and seems to help for lower end systems. No idea how well it actually works, doesn't seem to make a difference at all in Wine last time I tried (load times seem to be worse in Wine overall)

Makes sense. A modern cpu and an SSD shouldn't be troubled overly much decompressing files, but it probably made a noticeable difference back when we were using Pentium 4 chips and hard disks for everything.

Edited September 16, 2020 by snargel

[Striker879]

 

Decompression does seem to be an issue sometimes. There's a program for Oblivion that uncompresses the vanilla BSAs and seems to help for lower end systems. No idea how well it actually works, doesn't seem to make a difference at all in Wine last time I tried (load times seem to be worse in Wine overall)

Makes sense. A modern cpu and an SSD shouldn't be troubled overly much decompressing files, but it probably made a noticeable difference back when we were using Pentium 4 chips and hard disks for everything.

 

 

Still, I'm of the mindset that any CPU cycles saved are CPU cycles that can be devoted to other tasks. Stingy old guy here.

[chambcra]

I've always made my own BSA files just for the organizational benefit. I've never used a mod manager and never will so I had to make BSA's because the Unoficial Oblivion Patch trashes the data folder. Here's what I think I've found, to be taken with a grain of salt.

1. Oblivion Mod Manager is good for browsing and extracting from BSA's but has a bug when creating if there is a single international character (à å ä ç č é è í í ò) in the file names. It acts like it made a good bsa file but it won't work.

2. I go along with what snargel said. I extracted Oblivion - Textures - Compressed.bsa and repacked it with no compression using BsaCommander and the tiny little bit of stutter I saw at the very beginning after starting the game has gone away. It's not going to hurt anything so when in doubt, why not.

3. With the OBSE plugin called SkyBsa I think you could pack all the loose files of the Unoficial Oblivion Patch into a bsa. You don't need UOP if all you play are quest mods so I haven't tried it.

4. I like Drake's point that Bethesda uses bsa files for a reason. Surely it's significantly easier to open and close a single file than 18,000 files.

5. I'm not sure if this goes against conventional wisdom but here's what happened. I've been using a map replacer that was a loose file. C:\Bethesda Softworks\Oblivion\Data\Textures\menus\map\world\cyrodiil_resized.dds. After I decompressed the textures bsa file the new file had a new modern date and the cyrodiil_resized.dds in the bsa was overriding the loose file.