The BSA Project

[dharh]

UPDATE 1 - Instead of 5 files I am doing 2.

UPDATE 2 - Tested, Approved, Uploaded

UPDATE 3 - removed, researching

UPDATE 4 - And i'm spent.

 

Project: Convert the many mods that I use heavily to use BSAs instead of copying hundreds of files into the data directory. Reason: So that files don't get constantly overwritten by the many many texture/mesh mods out there and thus make it far easier to remove a mod that replaces data files.

 

Phase One: Convert Unofficial Patches to use BSAs!

 

Files:

Unofficial Oblivion Patch.esp (normal, directly taken from the manual install .7z)

Unofficial Oblivion Patch.bsa (this contains ONLY those files used specifically by the esp all data files found within UOP archive)

 

UOP - Meshes.bsa (contains distantlod, meshes, trees; modeled after the oblivion meshes.bsa)

UOP - Sounds.bsa (contains sound/voice)

UOP - Textures.bsa (contains textures)

 

Results:

Failure [see update 4].

[dharh]

In case people are wondering, here is the load order for data files I hope to eventually accomplish.

 

Oblivion.esm -> Oblivion/ShiveringIsles/DLC*.bsa -> UOP/SIOP/UOMP.bsa (this is what ill create)-> <insert mod here>.bsa -> File found in the data directory.

[dharh]

After some further researching it looks like BSA files must be referenced in some way. The UOP files would need to go into the oblivion.ini file for them to get loaded. To avoid that i've decided that the Unofficial Oblivion Patch.bsa file will have ALL data files, even ones not specifically referenced by Unofficial Oblivion Patch.esp.

 

Since that esp loads directly after oblivion.esm I am hoping that the data files in the bsa will override vanilla data files. Will this work?

[David Brasher]

If the files are identical to master it is sort of a moot point whether of not they will override vanilla Oblivion. The result would usually be the same anyway, and the difference could not be detected.

 

I think that what you are trying to do will work. It will protect your vanilla files that you want to stay vanilla from being overwritten by mods. If you want to devise a test to see if things are working like you think they are, you should make a test BSA with files radically different from vanilla. So for instance, a certain robe might be blue in vanilla Oblivion but orange in your test BSA. You can play the game and see what color the robe is to see what files the game is referring to.

 

If you have two sets of things replacing meshes and textures, than perhaps you need a new design model or different design parameters. Like maybe you should simplify the setup so that vanilla only gets replaced once by that which you really want.

[dharh]

If the files are identical to master it is sort of a moot point whether of not they will override vanilla Oblivion. The result would usually be the same anyway, and the difference could not be detected.

 

I think that what you are trying to do will work. It will protect your vanilla files that you want to stay vanilla from being overwritten by mods. If you want to devise a test to see if things are working like you think they are, you should make a test BSA with files radically different from vanilla. So for instance, a certain robe might be blue in vanilla Oblivion but orange in your test BSA. You can play the game and see what color the robe is to see what files the game is referring to.

 

If you have two sets of things replacing meshes and textures, than perhaps you need a new design model or different design parameters. Like maybe you should simplify the setup so that vanilla only gets replaced once by that which you really want.

 

Did as you suggested. Seemed to work fine. Tested the files I made, everything seemed to work for that as well. I've uploaded the files.

[dharh]

Did a bunch more testing, the first test I did was dumb as I kept the pants that you start with in the data structure of the oblivion directory (I made the jail pants icon orange). Upon removing said icon and placing it inside the BSA. The icon revered back to the vanilla icon. It seems once a BSA claims a file path no other bsa, regardless of load order, can use the path. You can of course override this, in the same way that all replacers do, by putting the replacers in the normal file structure.

 

UPDATE:

 

I can't seem to let this rest. I came across another article about BSAs that suggest once again that a replacer mod using BSA is possible: http://elderscrolls.filefront.com/info/Oblivion_Mod_FAQs

 

It states though that archiveInvalidated problem, which I knew about and attempted to correct. I am wonder if _that_ is the issue I ran into which prevented me from loading the correct replacer icon.

 

It seems to me that I am not getting very many replies. Is it that no one is interested or that nobody knows?

 

UPDATE 2:

 

Fort those interested in the results of my researching the above mentioned stuff. It seems that archiveInvalidated is only used for when replacing textures/meshes using the normal directory structure. Something I previously thought worked all on its own. Looks like, unbeknown to me, that I OBMM set it up for me. This does not work for BSAs, only for replacements in real directories.

 

As a programmer this seems like poor design. It should have been trivial to allow the internal game database to recognize when a BSA is attempting to override a previous resource entry. But then again, they didn't fix the bug which makes archiveInvalidated necessary even for directory replacements.

Edited June 30, 2010 by dharh

[dharh]

Well this phase of the BSA project is over. It truly looks like there can be no replacers in BSA format without requiring esp redirects (IE, the esp points the reference to said object to the new BSA). Unless some other crazy person wants to do that, I see no other way to make the unofficial patches use BSA.

 

So, onto the next phase. Working on mods that are not replacers. heh.

 

As an aside. I learned quite a bit. And I also have custom versions of all the all the unofficial patches with optimized meshes, so not all bad.

[Tomlong54210]

Keep in mind that BSAs contribute to the limit of total BSAs and plugin files in the Data folder. For individual users, repacking can be very helpful, but adding many BSAs can push some setups toward that limit inefficiently. The other reason to leave that to individual users is that they are limited to 2GB in size. Modders packing mod into many groups of 100MB BSAs is not that helpful. Unless you know which mods a user is using, it would be hard for someone to upload ~2GB BSAs that are combinations of mods (which would be the only efficient packing method for and individual user.)

[dharh]

Keep in mind that BSAs contribute to the limit of total BSAs and plugin files in the Data folder. For individual users, repacking can be very helpful, but adding many BSAs can push some setups toward that limit inefficiently. The other reason to leave that to individual users is that they are limited to 2GB in size. Modders packing mod into many groups of 100MB BSAs is not that helpful. Unless you know which mods a user is using, it would be hard for someone to upload ~2GB BSAs that are combinations of mods (which would be the only efficient packing method for and individual user.)

I was mainly thinking about the most used packages. So I wouldn't be doing a bunch of 100MB BSAs. The original hope was to simplify the resources that get strewn all over the data directory, but alas I don't think its doable. Recently I put together a 500MB compilation package (HGEC, High Res HGEC, Natural Faces, Modular Beautiful People++, Working Eyelashes, lop-ear+mini, x117races). It's quite massive and when unpacked nears 2gig range. It'd be lovely if I could put the resources into a compressed BSA, but I can't since most of the resources are replacements rather than new files and i'm not sure its worth the time to change all those esp files to point to a new resources path, especially when some of those esps are still active projects by their respective authors.

 

In the end the best I can do is probably to maintain compilation packages that strive for no conflicts and ease of use. Right now with that package, and the UOP, USIP, UOMPs packages I made, it's very trivial to start from a scratch vanilla install and have a robust set of races for starts. Main problem is I can't really post that compilation package I just mentioned because some of the authors explicitly do not want their stuff on tesnexus.

[Tomlong54210]

I am not a fan of compilations either, really. The maintenance is always a question. I guess you already know about the possible cause of trouble when multiple versions of a file are packed in BSAs...