Converting an existing (and non-crashing) ESP worldspace to a version control ESM

[UndeadMartyr]

I am vaguely aware of the need to use version control to avoid the 16MB bug for large worldspace mods but not entirely clear on what this entails.

I have a functioning ESP with a new worldspace that I can load into my game and access with the console without crashing.  No LOD or Navmesh yet, but it doesn't crash.

As I am now over 16MB as of yesterday I want to convert to an ESM and start using version control for stability.

 

If I am understanding the VC tutorial properly, what I need to do is set up the folders and the ini, convert my esp to an esm, and then create a new empty esp and use version control to merge changes I make to that esp into the master file.  Is the remote master the .esm file I create or is that file the local master?

[madmongo]

I don't know what tutorial you are looking at.

Personally, I keep a backup of my "normal" ini files as well as my modified "network" ini files. That way I can easily switch between them if I want to make a normal mod vs. a version control mod.

Once you make the changes to your ini files, you'll end up with a network share somewhere. If you aren't actually in a networked development environment, this "network" share won't actually be on a networked machine, it will just be a folder on your local PC. It doesn't matter what you call this folder as long as it matches what you put in your ini files.

Make a blank ESP file and use FNVEdit to change it to an esm (make sure to rename it as .esm). Copy that file to your network folder in merging\data. You will also need to copy over FalloutNV.esm and any other master files that your mod depends on (HonestHearts.esm, etc. for example). Start up the GECK with your modified ini files. You'll probably have to start the GECK using Run As Administrator, otherwise you can end up with folder permission problems when you try to check things in. Go to File->Data and click on your new blank esm, then press the Details button. It will ask you if you want to use an empty list to save memory. Click Yes. When that comes up, press CTRL-SHIFT-B, because of course the GECK is as cryptic as it can possibly be and doesn't have a button for this anywhere (I've often said the GECK is a secret Vault-Tec experiment designed to test modder's frustration levels). This will generate the .fid .fud and .fvd files for your mod. Then close that form. Repeat this for all of your other esm files in your network folder (FalloutNV.esm, etc) to generate the .fid .fud and .fvd files for those as well.

Now restart the GECK and go to File-Data, select your blank esm along with whatever other esm files your mod depends on (FalloutNV.esm and any DLC esm files), and set your esp containing your worldspace as the active file.

Since you created your esp in normal mode, you need an extra step here. Once your mod is loaded, go into version control, click on Select All, and then click on Check Out. After that, you can click on Check In. You might need to restart the GECK between those two steps. This will copy everything from your worldspace mod into your blank mod. Your original worldspace mod will end up being a blank mod.

VERY IMPORTANT: Back up your worldspace mod to a different file or folder before you try any of this. If something goes wrong, the GECK can completely hose your mod beyond any hope of ever recovering it. If everything goes south, just delete all of the files, copy your backup copy back into your FNV folder, and start over.

Once you are done, copy the new worldspace file (originally your blank esm) back to your FNV folder.

From here on out, just open the GECK with your modified network ini files, select your new worldspace mod (originally your blank esm, but it now has your worldspace in it), and either use your original (now blank) esp as the active file or create a new esp as the active file. Make whatever changes you want to your worldspace, save your changes, then go into Version Control, select all, and Check In. Since you made the changes in networked mode, you don't need to check out those changes first. That's automatically done as you edit your mod.

After you are done checking in, always remember to copy the modified mod from your network folder back to your FNV folder. The GECK won't do that for you.

The files in your FNV folder are your "local" files.

 

 

[madmongo]

By the way, just for a better explanation of the 16 MB issue, since you are only vaguely aware of it. Everything in FNV that has some sort of ID uses the first two hex digits of that ID to indicate what mod it is from, and the last 6 hex digits as the offset into the file.

For example, if you look up the hex code for purified water, the code for it is 000151a3. That is mod 00 and offset 0151a3. Since the first two digits are used to indicate what mod it is from, the largest offset that you can have is FFFFFF, which is 16 MB. Let's say for example that your mod uses Honest Hearts. In my load order, Honest Hearts is the 6th esm to be loaded. If you look up the hex code for the .45 Auto Pistol, in the Fallout Wiki it is listed as xx008E1A. The xx is the mod number in your load order. So on my system, I could go into the console and type "player.additem 06008E1A 1" and it would add 1 auto pistol to my inventory.

Your worldspace is currently larger than 16 MB. So if you start the GECK in normal mode and add something that has any sort of ID, it's going to append that to the end of your mod at a file offset that is larger than 16 MB. But that doesn't fit into the 6 hex digits available. Since the GECK is a secret Vault-Tec experiment designed to test your frustration levels, it will happily add your new ID to your mod, and won't bother to tell you that the file offset doesn't fit into the 6 digits available for it. It won't give you any error at all. Instead, the GECK happily saves your mod, with a file offset that just doesn't work. Once you do this, your mod is forever broken. It will crash the game as soon as it encounters that bad file offset. And you can't fix it in the GECK because it will crash the GECK as soon as it encounters that bad file offset as well. Hope you have a backup of your mod from before you saved it with that bad file offset, or your mod is forever broken.

You can add a navmesh to your mod with no problem, because that doesn't use a file offset. You can generate LOD as well. But go in and create a new NPC, or clutter object, or static, or whatever, and you'll break your mod.

When you check in your changes using networked mode, it doesn't just append your changes to the end of the file like it does when you just edit the esp in normal mode. The GECK automatically places anything that uses a file offset well below the 16 MB limit in your network esm file. You can have multiple worldspaces and all kinds of NPCs, clutter objects, statics, etc. and you can always add to those in network mode.

[UndeadMartyr]

The issue (and source of my confusion) is that I have added items and neither the GECK nor the game has crashed when I enter the relevant cells.  I do keep regular backups.

 

If I'm understanding you correctly, I need to convert back to an .esp, make a blank .esp and convert that to an .esm, copy over the blank esm to merging, then generate the files for all the esms and start editing the esp.

Edited May 21, 2024 by UndeadMartyr

[madmongo]

Without knowing exactly what you did, I can't tell you why it didn't break your mod. All I can tell you is that FFFFFF hex is 16,777,215 in decimal. If your mod ever gets larger than that, and the GECK appends a record with an offset at the end of your mod, your mod will be forever broken.

I discovered the issue the hard way many years ago when I made my first worldspace. That was also when I discovered how to put the GECK into network mode and use Version Control to prevent the GECK from breaking my mods.

Worldspace stuff gets you into the buggiest parts of the GECK. Save often and make backups.

There is an automatic navmesh generator that claims to generate navmeshes for entire regions, so that you don't have to go cell by cell when creating navmeshes. Don't ever use it. I've managed to get it to create a functional navmesh once and only once. Every other time it has failed miserably. Sometimes it creates a navmesh for the entire worldspace OUTSIDE of the region you want instead of inside (and completely wipes out any navmeshes that you had created before). Sometimes it just gets locked up and constantly pops up an error message that says "Get Jean" over and over, and the only way to stop it is to go into Task Manager and kill the GECK task completely.

Lots and lots and lots of bugs. Eventually you'll get a feel for what you can get away with and what you can't. It just takes a lot of trial and error.

[UndeadMartyr]

I am not getting any prompt when I go into the details folder and CTRL+SHIFT+B does nothing.

 

Found the problem, forgot to add the relevant lines in geckCustom.ini.  However, I'm not getting any files in Merging/Data, and attempting to check out gives a "cannot find master on local network" error.

Edited May 24, 2024 by UndeadMartyr

[madmongo]

You probably have something set up incorrectly with the network paths.

Did you set up your folder as a network share? In other words, if you open File Explorer and put in \\127.0.0.1\Merging does this take you to your Merging folder?

Also, did you copy your esm files to \Merging\Data?

[UndeadMartyr]

Yes to both, I did find I had incorrect entries in GECKCustom which fixed the problem.  Most of the process is working, except that the generated Master file is blank, and the esp is not.

[madmongo]

If the ESP was created in the GECK in normal mode, then when you get to the check-in form you need to select all and then press Check Out. Once you have all of your changes checked out, then you can check them in to your new ESM.

If you do all of your editing in network mode, everything you change will automatically be checked out, so you won't have to do the extra step of checking out first.

[UndeadMartyr]

I've been doing that, yes, but the esm isn't being overwritten or changed, it's still blank, and the esp still contains everything.  What is checking out supposed to be doing?  It isn't altering any files.

Edited May 25, 2024 by UndeadMartyr