Multiple Skyrim setups without "profiles"

[Lord Garon]

This topic is a continuation of a Skyrim-General Discussions thread where, in retrospect, I provided some ill-considered advice. I would like to continue it here, with the following disclaimer:

 

WARNING: The advice in this thread has the potential to destroy your game completely, mess up your operating system file structure, and cause the Sun to stop rising in the morning. USE AT YOUR OWN RISK. If you've never backed up your game before changing something, now is the time to start.

 

I am an OCD control freak regarding my computers. I don't tolerate, to the maximum extent possible, my machines doing something I don't like and/or don't understand. This thread explains the way that I created several (6) independent Skyrim game SETUPS on my Windows 7 64bit game machine.

 

The Issue: I want several completely independent Skyrim game SETUPS in order to play different mod loads, to test mods, and to do strange things to Skyrim without affecting the other games. I want Steam and utility programs to see a standard game and operate normally when they run on any particular SETUP. (I capitalize SETUP because that's what I call an independent game installation.) I don't want to use a mod manager that I don't understand.

 

The Main Problems: Several file locations are (seemingly) hard-coded into various components of Skyrim. Most utilities (Loot, etc) keep track of ONE Skyrim install and game. Windows 7 (NTFS file system) cannot create dynamic symlinks (more on that later). The Windows Registry. The Windows ACLs.

 

Proposed Solution One: Multiple Skyrim directory locations are created with unique names to hold mods, savegames, and config info. Each location is renamed to the standard Skyrim directory name when a particular SETUP is desired.

 

Problems with the Proposed Solution: With no utility programs, only a couple. Works fine. Simple directory re-naming, or a batch file doing the same thing, switches SETUPS easily. Main problem is file duplication. Each SETUP requires core game files and mod files used in every SETUP. For instance, all my game setups have common STEP mods, SKSE, and a couple use the same ENB. Eats up disk space after a while.

Utility programs (mod managers, Loot, etc) cause a BIG headache. Consider a mod manger (mm); you install mods (mm remembers), activate/deactivate mods (mm remembers), and sort the load order (Loot remembers). Everything is fine until you switch SETUPS. Then what the mm and Loot remember are not related the new SETUP. The problem is getting each utility program to remember each game SETUP, as well.

 

(END Wall-o-Text One)

Edited October 27, 2014 by Lord Garon

[Lord Garon]

(BEGIN Wall-o-Text Two)

 

Specifics on Proposed Solution One: Base vanilla Skyrim maintains game information in three (primary) storage locations:

 

...\Steam\SteamApps\common\Skyrim. This is the base game file and data location. The ellipsis (...) refers to the Drive and install location on your machine. This entire PATH is stored in the registry so that Steam and other programs can locate the game, and that becomes a major PITA later on.

 

%LOCALAPPDATA%\Skyrim. First, the percent signs delimit an environment variable name, a system variable in this case, whose contents contain the actual path portion prior to \Skyrim. If you don't understand this concept, this thread will be difficult to follow and I wouldn't recommend doing the things I'm promoting without understanding this concept. This is the AppData folder location where loadorder.txt (sorted mod list) and plugins.txt (active game mod list) are kept. Typically, the full pathname resembles

%HOMEDRIVE%\Users\%USERNAME%\AppData\Local\Skyrim. The actual location will be determined by the name of your logon account, %USERNAME%, and the drive ID containing Windows user profiles, %HOMEDRIVE%.

 

%USERPROFILE%\My Documents\My Games\Skyrim. If you moved "My Documents", off of an SSD for example, you may have "Documents" instead of "My Documents". ("My Documents" is just an earlier Windows version compatibility junction pointing to "Documents" in Win7.) This location contains ini files and savegames (in a vanilla game).

 

NOTE: each of the above PATHNAMES ends in "Skyrim". We will use this fact next. My initial SETUPS went like this:

 

1. Make copies of each of the three "Skyrim" folders above. I named the first set of folder copies SETUP2, but it doesn't matter what you name them.

 

2. When I wanted to switch game setups, I would just re-name the three Skyrim directories to SETUP1 and the SETUP2 directories to Skyrim. Presto! A setup switch.

 

This makes two identical game SETUPS at first. You can then just change either game the way you want; add mods, remove them all and start over, change ini settings, install a new ENB, etc. When you switch to the other game setup, all your changes will remain in the other folders, safe and secure. That is, as long as you make the changes manually. If you use a mod manager like NMM, say, to do the mod changes, it will remember the status and changes you made the last time it was run. If you switch setups, NMM remembers what was in the "other" game, not the current one. I feel a slight throbbing in the back of my skull.

 

Renaming directories (folders) is the basic idea in having multiple game setups, but the problem with mod managers and other utility programs "remembering" one game setup at a time gets real old, real fast. Especially if you have a lot of mods and really NEED a mod manager. We'll look at that later.

 

(END Wall-o-Text Two)

Edited October 27, 2014 by Lord Garon

[Lord Garon]

(BEGIN Wall-o-Text Three)

 

It's more complicated than it seems, or, Why they invented Mod Organizer: Its almost a miracle that we can mod Skyrim to the extent we can. Steam tries its best to thwart piracy and necessarily interferes with modding in the process. Skyrim itself is full of quirks and odd behaviors. Windows, the World's Greatest Kludge (look it up), throws obstacles at us at every turn. It took me years to get a stable, consistent, modded Skyrim working how I wanted it to, which was just to simply work.

 

On the face of it, making some directory copies and renaming them to have multiple game setups seems to be a fairly easy task. In some ways it is. I went months with three simple game setups and had no issues at all, that is, no issues that weren't expected. However, there's considerable inefficiencies in doing the copy/rename process. I want to mention some basic problems I ran into at this point, not to dissuade people, but as a partial explanation as to why I ended up where I did and why Mod Organizer Profiles are a much better way to do this kind of thing. Unless you're OCD like me, and simply HAVE to do it your way.

 

The first issue I hit was the Registry. It stores Skyrim install info. Ah Ha!, I thought, easy enough to just switch a registry entry and have multiple base game installs. Nope. The launcher resets that entry every time its run. Well, I launch with SKSE, maybe that's not so bad. Oh oh, Skyrim won't run from a User account; Steam says it needs more permissions to correct things (the registry entry). A can of worms is coming; giving Steam more permissions defeats the purpose of User account permissions (not able to change too much stuff). Put the registry entry back to default and Steam lets things run. That throbbing in my skull is a dull ache.

 

Oh well, file duplication across the SETUPS is getting out of hand now and I need to work on that any way. Decide to move some core files to their own directory and point to them in the SETUP folders with symlinks. Perfect! Saving a lot of space. Now I'll move common data files (mods) out to a common mods directory and link to them. CTD on load. What the heck? Okay, I have BSA files in RAMDISK and on a flash drive already, maybe its the esp or esm files? Sure enough, symlinking a plugin crashes Skyrim. Much flailing until I discover that a file hardlink works, but a softlink doesn't; Skyrim accesses plugins differently than bsa files. Whooda thunk. Man, my head is really pounding now.

 

Okay, Loot only remembers metadata changes for one setup, let's work on that. Alrighty, Loot keeps that info in an AppData folder in a yaml (!?) file. But the frickin report files I have to remember are in the main Loot folder. Should I make more symlinks, or switch the entire AppData Loot folder. Maybe I'll move the folder into the Skyrim AppData folder so it switches automatically. Maybe I'll move ALL Loot files to a Skyrim folder. Hell, it'd be just as easy to have six Loot installations inside six different Skyrim setup folders. Then I could move common Loot files to my common core files directory... But, I could do that with one Loot install. Cripes, I can't think with this headache. Think I'll go look at some Star Citizen ships and relax for a while.

 

Moral of the story: I had a sudden realization as to the impetus behind Mod Organizer. MO does this stuff "automagically" and now I understand why. I still don't understand MO, but I understand why they built some of it. Well, 413,687 can play at this game; I'll automate MY setups with a script. That'll make things a lot quicker, at least. And get rid of all those tupos typos I make.

 

A multiple setup Skyrim with heavily modded games requires some automated support. The preceding is all true, but PALES in comparison to maintaining, updating, and organizing 470 mods across four actual gaming setups (two setups are for testing). Upcoming Walls-o-Text will contain some batch scripts for helping with this stuff. In reality, my current scripts are either VBS or Powershell scripts, but I don't understand either well enough to talk about them. I should be able to hack a couple useful batch files together, though.

 

Next time, back on track.

 

(END Wall-o-Text Three)

Edited October 27, 2014 by Lord Garon

[Lord Garon]

(BEGIN Wall-o-Images Four)

I installed a Kingston 120GB SSD a couple days ago (it was on sale at Microcenter for $60.00) and put a new Win7 install on it. Pretty nice, the machine boots in about 7 seconds. Moved some busy folders off the SSD and on to a platter drive and decided to re-do the HDD drives, including my Skyrim setups. Jumped straight to what I'd been thinking of doing with the setups.

I still have 6 each of the 3 main Skyrim storage locations, but now a directory symlink target address selects the desired active setups. My batch file just deletes and re-creates the symlinks with different target addresses to switch setups. Thought I'd post an Explorer picture to break the monotony and show what I'm rambling on about. First, a capture of the SteamApps\common directory:

 

 

http://static-1.nexusmods.com/15/images/110/1545930-1414473521.png

 

The folder panel shows the six SETUP directories, the common file and data store directories, and the script folder where the create and switch batch files live. Notice the re-direct arrow on the Skyrim symlink. Its pointing at SETUP2 at the moment. On the right panel are the contents of SETUP2, which is being cleaned up for a new mod load (Gotta push that new i5). The Skyrim Mods folder holds BASH installers (mod archive zip files) and mod info for this setup and automatically switches with the setup. I highlighted a couple SKSE file symlinks; they point to the real files living in the CommonGameFiles folder. Note that they show 0 bytes in size. Moving stuff to the Common folders cuts a vanilla+DLC SETUP folder from over 13GB to less than 1GB each; most of that size drop is from moving vanilla+DLC bsa files. That's with no other mods, of course. I think the STEP REGS pack (SETUP4) is the biggest; just over 60GB, with a lot of stuff moved out.

I just now finished "translating" my VBS hta switch script to a cmd.exe batch file. My poor coding skills are available for your entertainment in the spoiler tag below.

 

 

 

 

@echo off
SETLOCAL enabledelayedexpansion

:: This script switches the active Skyrim game setup.
:: Currently, the active setup is determined by symlink target addresses.
:: This routine will also update Current_Game.txt to set the global state value
:: of the active game setup.
::
:: Usage: switch N
:: N is a single integer digit, 1-9. Absence of N is an error and will terminate
:: switch.bat with no action taken. If N=Current setup number, no action is taken.
:: If SETUP[N] does not exist, the user is notified.

:: Vars

set _SKYLINK=F:\Steam\SteamApps\common\Skyrim
set _SKYTARGET=F:\Steam\SteamApps\common\SETUP
set _DATALINK=%LOCALAPPDATA%\Skyrim
set _DATATARGET=%LOCALAPPDATA%\SETUP
set _SAVELINK=%USERPROFILE%\My Documents\My Games\Skyrim
set _SAVETARGET=%USERPROFILE%\My Documents\My Games\SETUP

:: Get current game setup number from Current_game.txt. Bail on errors.

for /f "delims=" %%i in (current_game.txt) do set _CURRENTGAME=%%i

if not defined _CURRENTGAME (
:: Bail out
echo.
echo Could not get current game setup number. Exiting...
echo.
goto :eof
)

:: Get command line argument (N)

set _request=%1

if not defined _request (

echo.
echo.
echo You must include a desired setup number, 1-9.
echo.
echo Usage: switch N
echo.
echo.
echo Exiting...
echo.
goto :eof
)

:: Is the requested setup the same as the current setup?

if %_request%==%_CURRENTGAME% (

echo.
echo.
echo SETUP%_request% is already the active game.
echo.
echo Exiting...
goto :eof

)

echo.
echo.
echo Game switch to SETUP%_request% requested...
echo.
echo Checking SETUP%_request% availability...
echo.
)

:: **************** SETUP if exist Block ******************

if not exist "..\SETUP%_request%\" (

echo.
echo.
echo SETUP%_request% not found. Run create.bat to create new game setup.
echo.
echo.
echo Exiting...
goto :eof

)

echo SETUP%_request% found, switching...
echo.

:: Okay, rewrite the links

rmdir %_SKYLINK%
mklink /d %_SKYLINK% %_SKYTARGET%%_request%
echo.
echo.
rmdir %_DATALINK%
mklink /d %_DATALINK% %_DATATARGET%%_request%
echo.
echo.
rmdir "%_SAVELINK%"
mklink /d "%_SAVELINK%" "%_SAVETARGET%%_request%"
echo.
echo.

del Current_game.txt
echo %_request% >current_game.txt

echo Switch completed.
echo.

 

 

 

(END Wall-o-Text Four)

Edited October 28, 2014 by Lord Garon

[Deleted5770650User]

A few things to add here.

 

The following assumes you have the default NMM folder structure of Skyrim\Mods for mods and Skyrim\Install Info.

If you use NMM it has a folder where it store all the mods it can to use. For me this is C:\Games\Nexus Mod Manager\Skyrim If I change this folder all the mods NMM sees and "knows" are installed to a new folder for a second install NMM would also keep track of the keep the mods for each install correct and you could also use NMM to install and remove mods from all installs.

[Lord Garon]

A few things to add here.

 

The following assumes you have the default NMM folder structure of Skyrim\Mods for mods and Skyrim\Install Info.

If you use NMM it has a folder where it store all the mods it can to use. For me this is C:\Games\Nexus Mod Manager\Skyrim If I change this folder all the mods NMM sees and "knows" are installed to a new folder for a second install NMM would also keep track of the keep the mods for each install correct and you could also use NMM to install and remove mods from all installs.

 

I'm certain that works; I just use BASH and am unfamiliar with NMM. However, NMM has a GREAT feature that I am dying to use and that is it's ability to automatically recognize mod updates on the Nexus. I update mods manually now and its a mega-PITA with six games worth of mods (read that: I rarely update mods).

 

If I installed NMM (again), would you be willing to answer some NMM questions? I am a noob NMM user.

[Lord Garon]

(BEGIN Another Wall-o-Text Five)

 

One more WoT, then I'll talk about utilities and end this.

 

This is a capture of the Skyrim\Data folder (symlinked SETUP2) and what the system "sees" when I run Skyrim on this SETUP:

 

 

http://static-2.nexusmods.com/15/images/110/1545930-1414520703.png

 

 

Just want to point out a couple things here. I'm reverting SETUP2 to vanilla+DLC for a new mod load and ENB. The right-hand pane shows the contents of the Data folder; almost every file in there is a symlink of one type or another. All the "real" files are located in the CommonDataFiles folder. ALL the vanilla and DLC bsa files are standard soft links (mklink with no options). I have never had a problem with a soft link and a bsa file. I actually have some bsa files (optimized textures and 3DNPC bsa's) in a RAMDISK and on a USB 3.0 flash drive in order to even out disk I/O queues with my OLD, SLOW hard drives. I can't say the same about esm and esp files; soft linking some esm's causes Skyrim to CTD during loading on my machine. A hard link (mklink /h), however, does not crash. There's subtle differences between hard and soft links (apparently) which may cause issues depending on how Win7 is told to access the files. I guess the most common file access method (a system call thing) has no problem with soft links, but some file access methods specifically requested by an application will fail on soft links. All the esm files in the Data folder are hard links. A hard link is indistinguishable from a normal file entry. That's why I have those [Whatever].esm.is.a.hardlink files; they remind me whether I've linked an esm or not. The hard link file size is also mis-leading; they don't take up that amount of space, its just a report on the size of the original file. You can have multiple hard links to the same file, but the storage requirement for the file itself doesn't increase. Unfortunately, you can only hard link to a file on the same volume; they don't span drives.

 

All those links point to original (TES5Edit cleaned esm's) files in the CommonDataFiles directory and all SETUPs use the same "core" data files. The same for mods and exe/dll files which I install in all, or many, SETUPs. It significantly reduces drive storage requirements with this many game setups.

 

Sometimes, though, my symlink spider web confuses even me, especially the hard links if I don't remember they are hard links. You really can't see it here, but all my disk volumes are created relatively small (64-256 GB). That lets me quickly check volumes for errors and repair them, do fast defrags, have small backups, and things like that. My whole Skyrim install is larger than my biggest drive volume; I have symlinked Skyrim, and Oblivion, stuff ALL OVER THE PLACE. I should just rename my computer "Tamriel". Point is, symlinking requires some sort of PLAN and some sort of way to maintain your spider web. Scripts (batch files) are your friend. If I have to do something twice, I'll try to script it so the third time is easier. If you put comments in your scripts, you can document what you are doing. Just a hard-learned lesson I thought I'd mention.

 

(END Wall-o-Text Five)

Edited October 28, 2014 by Lord Garon

[Deleted5770650User]

 

I'm certain that works; I just use BASH and am unfamiliar with NMM. However, NMM has a GREAT feature that I am dying to use and that is it's ability to automatically recognize mod updates on the Nexus. I update mods manually now and its a mega-PITA with six games worth of mods (read that: I rarely update mods).

 

If I installed NMM (again), would you be willing to answer some NMM questions? I am a noob NMM user.

 

 

I'll do what I can, but I'm not active on the nexus anymore. From my experiance the update dection with NMM is not very usefull. For many mods the version that works with your mod list is a different version, but NMM tell you there is an update when there really is not one.

[Lord Garon]

^^ Okay, thanks for the offer.

 

I've had problems with the NMM "Update" facility before. I really should say that I've had problems with the Nexus mod page naming conventions, which NMM follows. The displayed "Mod Names" are really (as I understand it) a Nexus "container" name and do not correspond to any actual mod files. Mod authors can set up their naming system any way they desire. But even a "hint" that there is a possible mod update available would help.

[bben46]

The problem is not with NMM, but with the very inconsistent way that modders label their mods. There is no standard way to version mods - and no way to get modders to follow any standard method. :rolleyes:

 

I have seen the first version with no version number, then a update with again no version number, then a second update with a v1.01 ( there is no V1.0, but 2 with no version number at all) And, not every mod maker even tries t use the same method - one I remember had 3 variants of their mod - use one only was stated in their read me They were labeled v1, v1.2 and v1.3. Then the next version combined all 3 and was labeled NEW v1.0 Try explaining that to a computer program that is trying to check for a actual newer version. - Go by the date? Then they will roll back to an earlier version and re version it when a newer one has problems - so the proper one to use is actually the older date AND the newer version number. Which is explained it the documentation that no one seems to bother to read. :wallbash:

 

The little delta symbol under updates IS a HINT! that their might be a newer version, But should not mean you should absolutely install that update just because the symbol is present - READ the documentation to see if there really is an update - and whether you really need to update if there is. :thumbsup: