Version Control -- Setup and Clarification

[kyago8]

I don't see the GECK Version Control posted about much on here, so apologies if there's a specific section on the forum for that. 

Though my questions are about using Version Control with the GECK, I'm very uneducated in terms of computer science (I'm kind of a learn-as-you-go type of person) and my questions may have simple answers.

I am intending to create a mod that contains its own worldspace. Taking madmongo's words to heart, it's extremely easy to go past the 16mb cap for .esps and that's why I'm trying to get version control set up correctly, as well as clear up my general confusion about how Version Control is supposed to work.

Here are some of the sources I've used to help me so far:

GECK Wiki: https://geckwiki.com/index.php?title=Version_Control

Nexus: 

 

 

First off, I'm having an issue setting the local share correctly, as my access to the \\127.0.0.1\Merging and \\127.0.0.1\CheckinBackup is denied when I try to test the connection with a .txt document. I have given all users full control on the security tab in the properties of both folders, but nothing that I've tried has allowed me access to create a .txt doc within these folders. If someone can point me to a reliable article about setting up file share locally like must be done for VC or is able to tell me how to do this without too much inconvenience on your end, I will greatly appreciate it. Otherwise, I have all the .ini adjustments for the GECKCustom.ini and GECKPref.ini already input and saved. It should just be a matter of setting up local file sharing correctly -- or at least whatever issues there may be, I will come across later.

Secondly, I would like to clear up some of my confusion about Version Control and .esms/.esps in general.

If I understand it correctly, a mod's entire data can be included into an .esm without plugins attached (allows larger mods) and .esps are plugin files that rely on formIDs in .esms. Therefore, when using VC will I need to first input a blank .esm in the \Merging\Data\ folder alongside falloutNV.esm and other masters? I don't entirely understand the function on the file paths described by the wiki page, but I have them set up regardless. I understand that the function of VC (for my purposes) is to work on mods in .esp format, then merge them into an .esm in order to avoid the 16mb bug, I just don't want to override my base game masters.

I'm sure that I will have more questions and I have not found any active threads about Version Control or its setup, so I will post them here as they come up. It may come to be a great resource to people exploring the GECK's capabilities as there is very little documentation about running this version of the GECK. A user who posted a topic I referenced above stated that they would post a set up guide at some point, but it seems that they have been inactive for some time.

[kyago8]

I just realized that if the purpose of Version Control is to merge .esps into an .esm, why can't I just create a mod entirely through an .esm using the GECK Extender? I certainly must not be understanding something correctly...

[kyago8]

In the meantime, I should be fine to create the mod bit by bit with separate .esps (at least I think I should) by doing the worldspace and terrain in one .esp then adding objects and buildings with separate .esps until I can merge them into a singular master.

[madmongo]

Let's start with an explanation of what Version Control is. It's basically setting the GECK up to work in a networked configuration for team development. Let's say you have a team of a dozen or so developers, all working on creating a big mod. You're a "professional" team all in an office somewhere, and you need one machine to have the actual final product on it. So you set up a computer on your network, and it is what that article calls the "remote master". It has the actual esm that everyone is working on. Everyone creates esp files on their local computer, and checks their changes into the remote master.

So for our example, we'll call our project TheBigMod.esm. Developer Bob grabs a local copy of this mod and puts it on his machine, and makes an esp that depends on this esm. Let's call his local esp file BobsFile.esp. He makes his changes, then he uses the GECK's version control to check the changes from BobsFile.esp into TheBigMod.esm. When he's done checking the file in, TheBigMod.esm incorporates all of his changes, and his local BonsFile.esp ends up empty (since all of the changes are now in TheBIgMod.esm).

The important thing here is that when Bob checks his changes into TheBigMod.esm, the GECK doesn't just append things at the end of the file the way it does when you work single user on a mod in the GECK's normal mode. Instead, things that have any kind of reference ID and such get placed at the beginning of the file.

Why is this important? Let's say you create an item, called Bob's Wonderful Water (it has super-duper HP healing or something like that). This is stored with two hex digits indicating which mod it's from, and six digits for the offset into your file. The 16 MB bug comes from the fact that if you append something to a file that is larger than 16 MB, the offset is larger than six hex digits and bleeds over into the first two digits which are supposed to be used for the mod number.

Some real world examples might help clarify this. Lonesome Road contains a new weapon called the Arc Welder. If you look up the reference for this weapon, it is listed in the Fallout Wiki as xx00766B. On your particular system, the xx will be the mod index of the Lonesome Road esp. On my system, that is mod 8. But if I were to remove GRA and the Tribal Pack from my load order, Lonesome Road would end up being mod 6. So in the first case, the actual ID of the Arc Welder would be 0800766B, and in the second case, the ID would be 0600766B.

But what if I created a new weapon and added it to my mod that is larger than 16 MB? If you are just making a local file, the GECK is going to append your new weapon to the end of your file, and you could end up with a file offset of say 01234567 (which would be a bit larger than 19 MB). Notice that your offset now spills over into the first two digits, which is supposed to indicate the mod index, but it actually indicates the offset into your file. When the game encounters this screwed up offset, it crashes. If you try to open the file in the GECK to fix it, the GECK crashes. Hope you had a backup of your file before you added your new weapon, because there is no way to recover from this.

However, if you create a master esm out on the network share, and this master happens to be 19 MB in size, and you check in an esp that only contains your new weapon, the GECK is smart enough to place your new weapon down below the 16 MB boundary. Everything with an ID that indicates the mod number is placed at the beginning of the file when you check it in.

That is the important difference. The GECK in local mode simply appends your changes. The GECK when you check things in to a master will sort your references so that things that need to be below the 16 MB boundary will end up there.

But you're not Bob, and you're not part of a large development team, so what does all of this mean to you? Well, you can take advantage of the way that the GECK checks in esp files into a master to avoid the 16 MB bug  on mods that you create on your own computer.

To do this, instead of having a repository somewhere on the network, you instead create a local folder. But then you share that folder, so that it can be accessed as if it is out on the network somewhere. You do that through the "loopback" connection, which is an IP of 127.0.0.1. This IP always points to your own local computer, no matter what IP your computer actually has on the network. That is why it is called the "loopback".

Once you set this up, the GECK thinks that it is checking files into a network share somewhere on another computer. But actually, since that IP loops back to your own computer, the GECK gets tricked into checking files into a folder on your own computer.

A quick and easy way to demonstrate the 16 MB bug is to create a new worldspace. Define your worldspace, save and exit the GECK and restart it (because if you don't the GECK will crash at the next step), then open the heightmap editor. Generate a random landscape with a Base Offset of say 14,000 or so (you need your landscape above 14,000 or so to avoid floating tree LOD issues) and let's say a Frequency of 1000 and an amplitude of 1000. This will generate a reasonably random landscape. Exit out of that and save your mod. Your mod will now probably be around 12 to 13 MB in size. No problem, yet. Go under Regions and create a region that covers most of your new worldspace. Copy all of the objects from the RockTreeGen region into your region, and set up all of the object parameters to be the same as this region (both the Objects tab and the Obects (more) tab). Click on Generate and the GECK will happily create random trees all throughout your defined region.

Now you have a problem, because now when you save your mod, it's about 18 to 19 GB in size. So only about half of those trees are under the 16 MB boundary.

The GECK will happily save your mod, and give you NO ERROR INDICATIONS WHATSOEVER (because the GECK is really a secret Vault-Tec experiment designed to test your frustration levels). However, when you try to test your mod, the game crashes. When you try to load your mod into the GECK to see what is wrong with it, the GECK crashes. There is no way to fix your mod. It is forever broken.

This, incidentally, is basically how I discovered this bug for myself the first time. I was making a dead tree forest in a new worldspace.

Now let's do exactly the same thing, but we'll use Version Control instead. I make a new esp, create my worldspace, save and restart the GECK (stupid crash bug...), go into the heightmap editor, save that, and now I have an esp that is roughly 13 MB in size. At this point, I use Version Control, and check my changes into a new esm. Important note - always back up your files before you do this, because if it screws up somehow there is generally no way to recover from it.

I copy the new esm from my \\127.0.0.1\Merging\Data folder back to my FNV game folder, create a new esp that depends on this new esm, and now I create my new region, add all the objects etc. to my region, click on Generate, and make my big dead tree forest. Save that esp. Again, back up your files in case something goes south. Now I check in my esp into my master, and voila! I end up with an esm that contains my worldspace, my region, and all of those trees, but all of those trees are up near the beginning of the esm and aren't anywhere near the 16 MB boundary. There's no problem with any of the file offsets, I have avoided the 16 MB bug, and everything works.

Then I try out the auto-navmesher for regions and learn the hard way that it almost never works, and you should never, ever use it.

 

 

Edited February 13 by madmongo

[madmongo]

12 minutes ago, kyago8 said:

In the meantime, I should be fine to create the mod bit by bit with separate .esps (at least I think I should) by doing the worldspace and terrain in one .esp then adding objects and buildings with separate .esps until I can merge them into a singular master.

This works. But you will have to go through an extra step. Since you created the esp files in local mode, when you go into Version Control in the GECK you'll have to select all changed items and check them out first, then do the check-in. Otherwise the GECK will think that you don't have any changes to check in (stupid GECK).

Also, if you are having difficulty setting up the network shares on your local computer, Microsoft (who I believe is secretly working with Vault-Tec to test our frustration levels) has changed the way that they set up shared folders in recent years. You might have to use the command prompt to set up your share folders.

I personally had to set up a desktop shortcut to cmd.exe in Windows\System32 with the "Run as Administrator" box checked on the shortcut, otherwise I got "Access denied" errors. I also happen to use a folder named FNVNetwork on my D drive instead of the location that the article you are looking at uses. The name of the folder doesn't matter as long as you set the names the same in your ini files. On my system, I would use the following commands in the command prompt:

net share Merging=d:\FNVNetwork\Merging /grant:everyone,full

net share CheckInBackup=d:\FNVNetwork\CheckInBackup /grant:everyone,full

I also have to run the GECK from a shortcut with the "Run as Administrator" box checked.

I personally keep a backup set of ini files from before the changes for version control and a set of ini files from after the changes were applied, so that I can easily switch between the two modes by simply coping the appropriate set of ini files into my FNV folder in documents.

[kyago8]

30 minutes ago, madmongo said:

I personally keep a backup set of ini files from before the changes for version control and a set of ini files from after the changes were applied, so that I can easily switch between the two modes by simply coping the appropriate set of ini files into my FNV folder in documents.

Can this also be done by just changing GECKCustom.ini to bUseVersionMode=0? Also thank you, I finally got the folder sharing to work. I will continue through the rest of the article and return later.

[madmongo]

3 minutes ago, kyago8 said:

Can this also be done by just changing GECKCustom.ini to bUseVersionMode=0?

I don't know. I've never tried it. If you try it, let me know if it works.

[kyago8]

It seems that not even 10 minutes later I've run into another issue. It may even be the same issue, but I've mistakenly thought it was resolved. As per the wiki, I set the GECKcustom.ini with

bUseVersionControl=1

SNetwork Path=\\127.0.0.1\Merging\VersionBackup\

SNewVersionBackupPath=\\127.0.0.1\Merging\Data\

SLocalSavePath=Data\

SLocalBackupPath=Data\CheckInBackup\

However, when I attempt to generate the 3 files that should end up in \Merging\Data\, nothing is created. At some point during my attempts, they were created but ended up in the Fallout steam directory in the Data folder alongside the .esms, but now it seems like either the GECK is not generating these files or they're being sent to Big Mountain. Is there any troubleshooting for the Confirm Bit Array process?

[kyago8]

23 minutes ago, kyago8 said:

However, when I attempt to generate the 3 files that should end up in \Merging\Data\, nothing is created. At some point during my attempts, they were created but ended up in the Fallout steam directory in the Data folder alongside the .esms, but now it seems like either the GECK is not generating these files or they're being sent to Big Mountain. Is there any troubleshooting for the Confirm Bit Array process?

To make it clear, I am not experiencing the issue described in the wiki that happens when the directory in the GECKcustom.ini and GECKPrefs.ini doesn't match the UserAccountName. I experience both the "Do you want to use an empty list to save memory?" and the "Confirm Bit Array File update" popups. What is happening is that after I hit "Yes" on to create the 3 files, the pop up instantly closes with no buffer time. When I click "OK" on the data window, it takes some more time than normal to buffer, but at the end of it, the .fid .fud and .fvd are no where to be found.

[madmongo]

You need to copy FalloutNV.esm and any DLC esm files that you want to use to your \\127.0.0.1\Merging\Data folder.

Run the GECK as administrator. Even before you select your data files, you should be able to click on File and see an option for Version Control. That lets you know you at least have Version Control set up in the ini files.

Now go to File -> Data and for each of the files that you copied over to your \\127.0.0.1\Merging\Data folder, select the esm, click on Details, then CTRL-Shift-B. When I get the "Do you want to use an empty list to save memory?" prompt I always select Yes. It's never caused an issue that I can see. Some references say this may take a while, but it always finishes pretty much immediately for me. The 3 files (.fid, .fud, .fvd) will be created in your \\127.0.0.1\Merging\Data folder.

If you don't use Run as Administrator when you start the GECK, it probably won't create the files. I can't remember if it gives you an Access Denied error or not for those particular files or if it just silently fails.