We have a name! And a Q&A session with Tannin regarding the new mod manager.

[Reidlos]

Any New eta?

[NexusPros]

In response to post #50976092. #50976292, #50979777, #50981122 are all replies on the same post.

RavenMind wrote: I hope this hasn't been asked/answered before. I couldn't find it using the forum search. Will Vortex be able to do things like merge aspects of esp's into something like a bashed patch they way Wrye Bash does, or have some sort of similar feature to reduce the number of esp's?
 
Thanks for your hard work on this!

Ethreon wrote: Really hope not. That is not the job of a mod manager. And you can already do that with proper, dedicated tools.

J.O.D. wrote: That would be a waste of developers' time, IMO. There are tools for that. Vortex must not be a Swiss army knife. You know, if you do everything, you are not doing it well.

RavenMind wrote: Hmm, good points. It's also been my experience that all-in-ones rarely do everything well. Looks like I'll still be using Wrye Bash in tandem with a mod manager. Or just WB like I'm doing now. Still will have to try the new mod manager though. Can't wait to see what the team comes up with!

Merge feature wanted.

[Stiffon]

Just wondering how the alpha testing is going?

[zAzAaZOZO]

Will there still be a 135 mod limit or will that increase with the better file keeping? Also the mod order and the mod conflict tools have never worked properly in FOMM. I know that with thousands of mods, keeping track of every potential conflict between all of them its a herculean task. Still if visualization is a better mousetrap, then perhaps Vortex will find better order of mods and alert users to potential conflicts. I've noticed as Tannin as stated, that when I delete a mod other mods disappear from the list as well. I hope visualization of files helps this linking problem and the limit of mods will be raised appreciably and everything runs smoother. Edited June 29, 2017 by zAzAaZOZO

[Target1213]

Still playing/mod tweaking OldRim. Watching intently for The opportunity to start my virgin SkyrimSE using the shiny new VorteX, the mod manager that is like a force of nature in the electronic realm. Like an eight year old waiting for Christmas. Thank you Tannin and crew ( not to mention any Dark forces involved...) for giving me something to anticipate again. Do the magic that you do!

[Tannin42]

In response to post #51243917.

ermodk wrote: I've been using MO (oldrim) and NMM (TW3) for a few years now and have come to appreciate both. I've also studied Comp.Sci. and modded a few games for fun, so I do know my arse from my elbow when it comes to programming, even though I'm not employed as a software developer.

@Tannin42:

One thing that struck me when thinking about the vfs (imaginary filesystem from the game .exe's point of view) vs. virtualization (hard/symbolic links instead of files), is that maybe the overall best approach is actually to adopt something akin to what Plan 9 out of Bell Labs did:

Turn the problem on its head and think of the original game as a 'file store' or 'source' instead of as a 'target', and link files from said 'file store' into a new, unique 'profile', which acts as a 'world view' and can 'bind' files from both the 'file store' and the various 'mod stores'. This allows you to keep the original install location *completely pristine* like MO does, yet still reap the benefits provided by symbolic/hard links like NMM 0.6x does.

The only thing you might need here is some registry magic, where you trick the game .exe into believing you when you tell it that the game is installed in the 'profile' location. But since every game accesses the registry through a similar Windows API, this *might* not be such a big deal?

==
EXAMPLE

This outlines the case where we have a single game profile configuration called 'my_profile' (there could be an arbitray number of these):

- Game was originally installed to D:\Steam\SteamApps\common\Fallout 4 <- %FILE_STORE% (shared between profiles)
- Mod archives are downloaded to D:\Vortex\FO4\downloads\ <- %MOD_DOWNLOAD_DIR% (shared between profiles)
- Mod archives are unpacked to subfolders under D:\Vortex\FO4\unpacked\ <- %MOD_STORE% (shared between profiles)
- Mod profiles live in subfolders under D:\Vortex\FO4\profiles\ <- %GAME_PROFILE_DIR% (each profile is unique)

Now, each mod profile would be initialized with hardlinks of every single original game file (assuming that hardlinks are indistinguishable from the original files) placed into the %GAME_PROFILE_DIR%\my_profile\ folder.

After that, Vortex would populate the %GAME_PROFILE_DIR%\my_profile\ folder with symbolic links from the %MOD_STORE% folder according to the user's vishes per GUI checkboxes and whatnot, making it easy to distinquish between which files are from the %FILE_STORE% and which are from the %MOD_STORE%. You could potentially allow 'normal' override files as well.

When a modder develops his mod, he creates a 'my_mod_environment' profile and then creates his mod as %MOD_STORE%\my_mod and asks Vortex to add it to his %GAME_PROFILE_DIR%\my_mod_environment\ folder.

Each time he makes a change in %MOD_STORE%\my_mod, he asks Vortex to re-populate %GAME_PROFILE_DIR%\my_mod_environment (this is the DevOps way -- you reset state every time you deploy for consistency's sake) and does the rinse-repeat dance until he's satisfied that he can cut a release of my_mod.

This way, both the user and the developer get the freedom they need and absolutely NOTHING is changed in the original install location.

END EXAMPLE
==

Thoughts?

Sorry for the late reply, I only just saw this.
I thought about something this and it's something I'd like to maybe try at a later time but there is one big problem with this solution: All the desktop links, startmenu links, registry keys, ... would still be pointing to the original game directory so while Vortex could start the modded game, all external tools, like loot, that consult the registry to determine where the game is installed, would still use the unmodded game directory.
Some games might also throw a fit when they find the game isn't being run from the location they were installed to.

What would probably be less problematic is to have the "my_profile" dir alongside the original game so you'd have:
D:\Steam\SteamApps\common\Fallout 4.original
D:\Steam\SteamApps\common\Fallout 4.profile1
D:\Steam\SteamApps\common\Fallout 4.profile2

Each as a fully functional game directory through hard-links of all the game files like you described.
then, when the user picks a profile we simply create a single junction point (directory link)
named D:\Steam\SteamApps\common\Fallout 4 to the appropriate profile dir.
Now that folder is the modded game and all the links and registry keys would still point to it, while the vanilla game is still available at D:\Steam\SteamApps\common\Fallout 4.original and can always be restored.
But then: what happens when Steam comes along and updates the game?

[SetArk]

Hail!
A question i wanted to make, i knwo that may be already asked a LOT of times but, welp, here it goes again.
There is, any kind of timetable, any ETA for the beta or anything like that?

[ermodk]

In response to post #51243917.

 

 

 

 

ermodk wrote:

 

 

 

 

I've been using MO (oldrim) and NMM (TW3) for a few years now and have come to appreciate both. I've also studied Comp.Sci. and modded a few games for fun, so I do know my arse from my elbow when it comes to programming, even though I'm not employed as a software developer.

 

@Tannin42:

 

One thing that struck me when thinking about the vfs (imaginary filesystem from the game .exe's point of view) vs. virtualization (hard/symbolic links instead of files), is that maybe the overall best approach is actually to adopt something akin to what Plan 9 out of Bell Labs did:

 

Turn the problem on its head and think of the original game as a 'file store' or 'source' instead of as a 'target', and link files from said 'file store' into a new, unique 'profile', which acts as a 'world view' and can 'bind' files from both the 'file store' and the various 'mod stores'. This allows you to keep the original install location *completely pristine* like MO does, yet still reap the benefits provided by symbolic/hard links like NMM 0.6x does.

 

The only thing you might need here is some registry magic, where you trick the game .exe into believing you when you tell it that the game is installed in the 'profile' location. But since every game accesses the registry through a similar Windows API, this *might* not be such a big deal?

 

==

EXAMPLE

 

This outlines the case where we have a single game profile configuration called 'my_profile' (there could be an arbitray number of these):

 

- Game was originally installed to D:\Steam\SteamApps\common\Fallout 4 <- %FILE_STORE% (shared between profiles)

- Mod archives are downloaded to D:\Vortex\FO4\downloads\ <- %MOD_DOWNLOAD_DIR% (shared between profiles)

- Mod archives are unpacked to subfolders under D:\Vortex\FO4\unpacked\ <- %MOD_STORE% (shared between profiles)

- Mod profiles live in subfolders under D:\Vortex\FO4\profiles\ <- %GAME_PROFILE_DIR% (each profile is unique)

 

Now, each mod profile would be initialized with hardlinks of every single original game file (assuming that hardlinks are indistinguishable from the original files) placed into the %GAME_PROFILE_DIR%\my_profile\ folder.

 

After that, Vortex would populate the %GAME_PROFILE_DIR%\my_profile\ folder with symbolic links from the %MOD_STORE% folder according to the user's vishes per GUI checkboxes and whatnot, making it easy to distinquish between which files are from the %FILE_STORE% and which are from the %MOD_STORE%. You could potentially allow 'normal' override files as well.

 

When a modder develops his mod, he creates a 'my_mod_environment' profile and then creates his mod as %MOD_STORE%\my_mod and asks Vortex to add it to his %GAME_PROFILE_DIR%\my_mod_environment\ folder.

 

Each time he makes a change in %MOD_STORE%\my_mod, he asks Vortex to re-populate %GAME_PROFILE_DIR%\my_mod_environment (this is the DevOps way -- you reset state every time you deploy for consistency's sake) and does the rinse-repeat dance until he's satisfied that he can cut a release of my_mod.

 

This way, both the user and the developer get the freedom they need and absolutely NOTHING is changed in the original install location.

 

END EXAMPLE

==

 

Thoughts?

 

 

 

Sorry for the late reply, I only just saw this.

 

I thought about something this and it's something I'd like to maybe try at a later time but there is one big problem with this solution: All the desktop links, startmenu links, registry keys, ... would still be pointing to the original game directory so while Vortex could start the modded game, all external tools, like loot, that consult the registry to determine where the game is installed, would still use the unmodded game directory.

Some games might also throw a fit when they find the game isn't being run from the location they were installed to.

 

What would probably be less problematic is to have the "my_profile" dir alongside the original game so you'd have:

D:\Steam\SteamApps\common\Fallout 4.original

D:\Steam\SteamApps\common\Fallout 4.profile1

D:\Steam\SteamApps\common\Fallout 4.profile2

 

Each as a fully functional game directory through hard-links of all the game files like you described.

then, when the user picks a profile we simply create a single junction point (directory link)

named D:\Steam\SteamApps\common\Fallout 4 to the appropriate profile dir.

Now that folder is the modded game and all the links and registry keys would still point to it, while the vanilla game is still available at D:\Steam\SteamApps\common\Fallout 4.original and can always be restored.

 

But then: what happens when Steam comes along and updates the game?

 

 

Nice of you to get back to me. =)

 

Just to clarify:

 

Perhaps I didn't flesh out my idea of the "registry magic" well enough, as this "virtualized registry spoof" is the linchpin that enables the idea of the original pristine "file store" to work in theory.

 

The point here was for Vortex to -- in effect -- become the new "base environment".

 

While I think I understand your concerns, I take the view that what the shortcuts and the registry point to is (IMHO) not terribly relevant insofar as the "Virtualized Registry Spoof" works in the "base environment" or "Matrix" from which the tool in question is launched.

 

Let's take LOOT as an example and show the layers in the stack with Windows at the bottom:

 

LOOT.exe

=== "Matrix" boundary ===

%VIRTUAL_REGISTRY_SPOOF%

%MY_PROFILE%

%MOD_STORES%

Vortex.exe

== "Vortex" boundary ==

%FILE_STORE%

%WINDOWS%

 

The idea here is for the VRS to encapsulate and transparently intercept registry calls from LOOT.exe such that LOOT reads from/writes to %MY_PROFILE%. You could say that we're essentially sandboxing LOOT into a carefully controlled matrix without LOOT knowing about it.

 

Granted, LOOT is open source and could thus be modified to work in tandem with Vortex, but it should also be trivial (famous last words...) to use the appropriate tools in the Sysinternals Suite to see which calls are executed to read from the registry when LOOT is fired up.

 

The same techniques could in theory be used to voodoo other, non-open-source tools into taking the blue pill and staying in the "Matrix" we've oh-so-carefully defined and constrained from within Vortex.

 

Again, I'm not saying that what I propose is actually possible, just that it strikes me as an elegant solution to a somewhat thorny problem in theory.

 

And can I just add that I very much look forward to seeing how Vortex (and its story) will evolve once it is released.

[Tannin42]

In response to post #51504172.

ermodk wrote:

In response to post #51243917.

ermodk wrote:

I've been using MO (oldrim) and NMM (TW3) for a few years now and have come to appreciate both. I've also studied Comp.Sci. and modded a few games for fun, so I do know my arse from my elbow when it comes to programming, even though I'm not employed as a software developer.

@Tannin42:

One thing that struck me when thinking about the vfs (imaginary filesystem from the game .exe's point of view) vs. virtualization (hard/symbolic links instead of files), is that maybe the overall best approach is actually to adopt something akin to what Plan 9 out of Bell Labs did:

Turn the problem on its head and think of the original game as a 'file store' or 'source' instead of as a 'target', and link files from said 'file store' into a new, unique 'profile', which acts as a 'world view' and can 'bind' files from both the 'file store' and the various 'mod stores'. This allows you to keep the original install location *completely pristine* like MO does, yet still reap the benefits provided by symbolic/hard links like NMM 0.6x does.

The only thing you might need here is some registry magic, where you trick the game .exe into believing you when you tell it that the game is installed in the 'profile' location. But since every game accesses the registry through a similar Windows API, this *might* not be such a big deal?

==
EXAMPLE

This outlines the case where we have a single game profile configuration called 'my_profile' (there could be an arbitray number of these):

- Game was originally installed to D:\Steam\SteamApps\common\Fallout 4 <- %FILE_STORE% (shared between profiles)
- Mod archives are downloaded to D:\Vortex\FO4\downloads\ <- %MOD_DOWNLOAD_DIR% (shared between profiles)
- Mod archives are unpacked to subfolders under D:\Vortex\FO4\unpacked\ <- %MOD_STORE% (shared between profiles)
- Mod profiles live in subfolders under D:\Vortex\FO4\profiles\ <- %GAME_PROFILE_DIR% (each profile is unique)

Now, each mod profile would be initialized with hardlinks of every single original game file (assuming that hardlinks are indistinguishable from the original files) placed into the %GAME_PROFILE_DIR%\my_profile\ folder.

After that, Vortex would populate the %GAME_PROFILE_DIR%\my_profile\ folder with symbolic links from the %MOD_STORE% folder according to the user's vishes per GUI checkboxes and whatnot, making it easy to distinquish between which files are from the %FILE_STORE% and which are from the %MOD_STORE%. You could potentially allow 'normal' override files as well.

When a modder develops his mod, he creates a 'my_mod_environment' profile and then creates his mod as %MOD_STORE%\my_mod and asks Vortex to add it to his %GAME_PROFILE_DIR%\my_mod_environment\ folder.

Each time he makes a change in %MOD_STORE%\my_mod, he asks Vortex to re-populate %GAME_PROFILE_DIR%\my_mod_environment (this is the DevOps way -- you reset state every time you deploy for consistency's sake) and does the rinse-repeat dance until he's satisfied that he can cut a release of my_mod.

This way, both the user and the developer get the freedom they need and absolutely NOTHING is changed in the original install location.

END EXAMPLE
==

Thoughts?

 

Sorry for the late reply, I only just saw this.

I thought about something this and it's something I'd like to maybe try at a later time but there is one big problem with this solution: All the desktop links, startmenu links, registry keys, ... would still be pointing to the original game directory so while Vortex could start the modded game, all external tools, like loot, that consult the registry to determine where the game is installed, would still use the unmodded game directory.
Some games might also throw a fit when they find the game isn't being run from the location they were installed to.

What would probably be less problematic is to have the "my_profile" dir alongside the original game so you'd have:
D:\Steam\SteamApps\common\Fallout 4.original
D:\Steam\SteamApps\common\Fallout 4.profile1
D:\Steam\SteamApps\common\Fallout 4.profile2

Each as a fully functional game directory through hard-links of all the game files like you described.
then, when the user picks a profile we simply create a single junction point (directory link)
named D:\Steam\SteamApps\common\Fallout 4 to the appropriate profile dir.
Now that folder is the modded game and all the links and registry keys would still point to it, while the vanilla game is still available at D:\Steam\SteamApps\common\Fallout 4.original and can always be restored.

But then: what happens when Steam comes along and updates the game?

 

Nice of you to get back to me. =)

Just to clarify:

 

Perhaps I didn't flesh out my idea of the "registry magic" well enough, as this "virtualized registry spoof" is the linchpin that enables the idea of the original pristine "file store" to work in theory.

The point here was for Vortex to -- in effect -- become the new "base environment".

While I think I understand your concerns, I take the view that what the shortcuts and the registry point to is (IMHO) not terribly relevant insofar as the "Virtualized Registry Spoof" works in the "base environment" or "Matrix" from which the tool in question is launched.

Let's take LOOT as an example and show the layers in the stack with Windows at the bottom:

LOOT.exe

=== "Matrix" boundary ===
%VIRTUAL_REGISTRY_SPOOF%
%MY_PROFILE%

%MOD_STORES%

Vortex.exe

== "Vortex" boundary ==

%FILE_STORE%
%WINDOWS%

The idea here is for the VRS to encapsulate and transparently intercept registry calls from LOOT.exe such that LOOT reads from/writes to %MY_PROFILE%. You could say that we're essentially sandboxing LOOT into a carefully controlled matrix without LOOT knowing about it.

Granted, LOOT is open source and could thus be modified to work in tandem with Vortex, but it should also be trivial (famous last words...) to use the appropriate tools in the Sysinternals Suite to see which calls are executed to read from the registry when LOOT is fired up.

 

The same techniques could in theory be used to voodoo other, non-open-source tools into taking the blue pill and staying in the "Matrix" we've oh-so-carefully defined and constrained from within Vortex.

Again, I'm not saying that what I propose is actually possible, just that it strikes me as an elegant solution to a somewhat thorny problem in theory.

And can I just add that I very much look forward to seeing how Vortex (and its story) will evolve once it is released.

Yeah, this would work, but it would also introduce a lot of the problems that MO had, like incompatibility with other software that hooks into api calls (the registry access) and AV software throwing a fit.
And adjusting individual application when they are open source would be way too tedious since we want to support more than just the Bethesda games. And even within that environment we have tools like FNIS that aren't OSS.

[ermodk]

 

In response to post #51504172.

 

 

 

ermodk wrote:

In response to post #51243917.

 

 

 

ermodk wrote:

 

 

 

 

I've been using MO (oldrim) and NMM (TW3) for a few years now and have come to appreciate both. I've also studied Comp.Sci. and modded a few games for fun, so I do know my arse from my elbow when it comes to programming, even though I'm not employed as a software developer.

 

@Tannin42:

 

One thing that struck me when thinking about the vfs (imaginary filesystem from the game .exe's point of view) vs. virtualization (hard/symbolic links instead of files), is that maybe the overall best approach is actually to adopt something akin to what Plan 9 out of Bell Labs did:

 

Turn the problem on its head and think of the original game as a 'file store' or 'source' instead of as a 'target', and link files from said 'file store' into a new, unique 'profile', which acts as a 'world view' and can 'bind' files from both the 'file store' and the various 'mod stores'. This allows you to keep the original install location *completely pristine* like MO does, yet still reap the benefits provided by symbolic/hard links like NMM 0.6x does.

 

The only thing you might need here is some registry magic, where you trick the game .exe into believing you when you tell it that the game is installed in the 'profile' location. But since every game accesses the registry through a similar Windows API, this *might* not be such a big deal?

 

==

EXAMPLE

 

This outlines the case where we have a single game profile configuration called 'my_profile' (there could be an arbitray number of these):

 

- Game was originally installed to D:\Steam\SteamApps\common\Fallout 4 <- %FILE_STORE% (shared between profiles)

- Mod archives are downloaded to D:\Vortex\FO4\downloads\ <- %MOD_DOWNLOAD_DIR% (shared between profiles)

- Mod archives are unpacked to subfolders under D:\Vortex\FO4\unpacked\ <- %MOD_STORE% (shared between profiles)

- Mod profiles live in subfolders under D:\Vortex\FO4\profiles\ <- %GAME_PROFILE_DIR% (each profile is unique)

 

Now, each mod profile would be initialized with hardlinks of every single original game file (assuming that hardlinks are indistinguishable from the original files) placed into the %GAME_PROFILE_DIR%\my_profile\ folder.

 

After that, Vortex would populate the %GAME_PROFILE_DIR%\my_profile\ folder with symbolic links from the %MOD_STORE% folder according to the user's vishes per GUI checkboxes and whatnot, making it easy to distinquish between which files are from the %FILE_STORE% and which are from the %MOD_STORE%. You could potentially allow 'normal' override files as well.

 

When a modder develops his mod, he creates a 'my_mod_environment' profile and then creates his mod as %MOD_STORE%\my_mod and asks Vortex to add it to his %GAME_PROFILE_DIR%\my_mod_environment\ folder.

 

Each time he makes a change in %MOD_STORE%\my_mod, he asks Vortex to re-populate %GAME_PROFILE_DIR%\my_mod_environment (this is the DevOps way -- you reset state every time you deploy for consistency's sake) and does the rinse-repeat dance until he's satisfied that he can cut a release of my_mod.

 

This way, both the user and the developer get the freedom they need and absolutely NOTHING is changed in the original install location.

 

END EXAMPLE

==

 

Thoughts?

 

Sorry for the late reply, I only just saw this.

 

I thought about something this and it's something I'd like to maybe try at a later time but there is one big problem with this solution: All the desktop links, startmenu links, registry keys, ... would still be pointing to the original game directory so while Vortex could start the modded game, all external tools, like loot, that consult the registry to determine where the game is installed, would still use the unmodded game directory.

Some games might also throw a fit when they find the game isn't being run from the location they were installed to.

 

What would probably be less problematic is to have the "my_profile" dir alongside the original game so you'd have:

D:\Steam\SteamApps\common\Fallout 4.original

D:\Steam\SteamApps\common\Fallout 4.profile1

D:\Steam\SteamApps\common\Fallout 4.profile2

 

Each as a fully functional game directory through hard-links of all the game files like you described.

then, when the user picks a profile we simply create a single junction point (directory link)

named D:\Steam\SteamApps\common\Fallout 4 to the appropriate profile dir.

Now that folder is the modded game and all the links and registry keys would still point to it, while the vanilla game is still available at D:\Steam\SteamApps\common\Fallout 4.original and can always be restored.

 

But then: what happens when Steam comes along and updates the game?

 

Nice of you to get back to me. =)

 

Just to clarify:

Perhaps I didn't flesh out my idea of the "registry magic" well enough, as this "virtualized registry spoof" is the linchpin that enables the idea of the original pristine "file store" to work in theory.

 

The point here was for Vortex to -- in effect -- become the new "base environment".

 

While I think I understand your concerns, I take the view that what the shortcuts and the registry point to is (IMHO) not terribly relevant insofar as the "Virtualized Registry Spoof" works in the "base environment" or "Matrix" from which the tool in question is launched.

 

Let's take LOOT as an example and show the layers in the stack with Windows at the bottom:

 

LOOT.exe

=== "Matrix" boundary ===

%VIRTUAL_REGISTRY_SPOOF%

%MY_PROFILE%

%MOD_STORES%

Vortex.exe

== "Vortex" boundary ==

%FILE_STORE%

%WINDOWS%

 

The idea here is for the VRS to encapsulate and transparently intercept registry calls from LOOT.exe such that LOOT reads from/writes to %MY_PROFILE%. You could say that we're essentially sandboxing LOOT into a carefully controlled matrix without LOOT knowing about it.

 

Granted, LOOT is open source and could thus be modified to work in tandem with Vortex, but it should also be trivial (famous last words...) to use the appropriate tools in the Sysinternals Suite to see which calls are executed to read from the registry when LOOT is fired up.

The same techniques could in theory be used to voodoo other, non-open-source tools into taking the blue pill and staying in the "Matrix" we've oh-so-carefully defined and constrained from within Vortex.

 

Again, I'm not saying that what I propose is actually possible, just that it strikes me as an elegant solution to a somewhat thorny problem in theory.

 

And can I just add that I very much look forward to seeing how Vortex (and its story) will evolve once it is released.

Yeah, this would work, but it would also introduce a lot of the problems that MO had, like incompatibility with other software that hooks into api calls (the registry access) and AV software throwing a fit.

And adjusting individual application when they are open source would be way too tedious since we want to support more than just the Bethesda games. And even within that environment we have tools like FNIS that aren't OSS.

 

 

I was sort of hoping that limiting the voodoo to intercepting a few simple registry calls would prove easier than it was to make MO's approach work.

 

Then again, perhaps this idea (along with the other ideas you've explored) might end up morphing into something surprisingly useful in your head over time... =)

Edited July 5, 2017 by ermodk