Package Management ?

[simsrw73]

Is something like this on the horizon? An incredibly vague answer is fine.

The only cool ideas for plugins I can come up with so far all seem like they are things likely to be implemented (package management, user + Nexus tags, loose file conflict viewer: dds side-by-side compare, pex decompile diff, etc.) For package management, there are probably dozens of ways to do it. Below is my first thoughts.

First, terminology. I'm probably showing my ignorance here, but I'm not certain of the proper terms, so I'm making stuff up. Four concepts seem evident, plus another bonus one for convenience:

• Mod Repository (aka NexusMods) that contains mod vaults(?!?!)
• Mod Vault - A mod page on the nexus that contains one or more Mod Packages/files.
• Mod Package - An individual archive file to download from a Mod Vault that can be classified as in the list of package types below. Contains zero or more Mod Plugins. This term is still ambiguous as a downloaded mod package is not equivalent to an installed mod package because of merges, edits, etc.
• Mod Plugin - esm, esp file.
• Super Mod - An xml configuration file that lists a group of Mod Packages to install. Like Chocolatey's packages.config file, but with rules. (https://chocolatey.github.io/usage.html#the-packagesconfig-file)

Ideally, Nexus' upload procedure would need to be updated to record the type of mod package file uploaded:

• Main package - primary mod package or most popular, default configuration
• Alternate package(s) - different configuration(s) of the main package, easy/hard, full/lite, 2k/4k
• Update package - this is either
  - a package to bring your older installed version up to the current release version
  - a package that provides an incremental update to the Main package
  (which of the above are the case for a mod can be determined algorithmically by comparing versions)
• Add-On package - Add on features that may have extra requirements or offer a different experience
• Optional package - Texture replacers, etc. (not sure if Optional should be separate from add-on)
• Patch - patch to make this mod compatible with another mod
• Edits/IniTweaks - New type of download file that can patch game/mod ini files.

Nexus' upload procedure would also need to record on a mod package/file level the list of dependencies (possibly a listed of suggested/recommended mods), and/or any special load before/after instructions that can be selected by auto complete list or mod vault index. Maybe a list of known incompatibilities. Trying to minimize the amount of info the mod authors need to enter and make it as easy as possible. I think the above is fairly reasonable and is the most accurate way to get the metadata. Biggest problem is inactive authors and un-updated mod vaults/packages.

There's surely more. Eg. I've completely skipped over version issues. Does a patch apply to only one version of mods or a range of versions. Punt on versions and assume the latest patch applies to the most current available versions, apply only to the exact versions available at the time the patch is uploaded, or dump a lot of extra book keeping on mod authors.

BUT, WE can't do anything about the above. That's most definitely up to Dark0ne & co. An alternative approach might be to create a plugin to (temporarily) allow users to enter the above metadata. Maybe. Users could define a mod vault. List the file ids of the mod packages in the vault along with category and requirements. This information might even be shareable between users, stored in a master list/db.

With the above metadata available, the following functions are suggested:

• Download/Install alternate - display a list of alternates so the user can select one or more for download.
• Switch alternate - remove the current main file and install an alternate.
• Install suggested/recommended - If a mod package includes a listed of author suggested mods, list them and let the user choose if/which to install.
• Install update - Merge with builtin update to install incremental update if available over downloading the full version package
• Remove update - If an incremental update has been installed, allow it to be removed. Included for completeness, but I don't know if it's necessary or even feasible because it can get complicated keeping up with what other operations have been performed on the mod package.
• Download/Install Add-On/Optional component - Show user a list of optional components to install and install along with any requirements.
• Auto Patch - every time a new mod package is installed, see if it is tagged in another mod packages patch list OR if a mod package it contains a patch for an already installed mod package OR if there is a different mod vault on nexus whose main file is a patch file with requirements on the mod package installing and any other mod package installed. Ok, the last one is a long stretch.
• Auto Install Dependencies - With per package file dependencies, automated install of dependencies should be easier to do correctly.

I honestly have no idea if I'm going anywhere with any of this or just tossing ideas. But I am curious if anything like this is already in the pipes?

Package management and super mods would rely a lot on Rules, and auto sorting of mod plugin, etc. to function as reliably as possible, so I can imagine any plugin that disabled auto sorting would disable a host of advanced features in Vortex like possible package management? Or is that something we'd have to contend with?

[simsrw73]

Is some of this what I see in the DOWNLOADS action menu Edit Meta Data? Hard to figure out from the dialogs. No one mentioned this area that I recall during tested; I should have waited until I got my hands on Vortex before making plans for it. I'll see what the code says, but this is a confusing, empty dialog.

[HomerSlated]

Hello, former package maintainer and QA tester for Red Hat and Gentoo here. Well versed in RPM and ebuild packaging, dependency tracking and conflict resolution. Built and maintained my own packages, maintained third party packages, did QA and bug reporting to distro maintainers and upstream, patch submission and testing, liaising with developers and handling user tech support.

 

I'm new to modding, but from what I can tell your "mods" are essentially packages minus the metadata. You'll need to standardise variables for versioning, dependencies and install locations. We actually used a large number of fields, given the complexity of a typical Linux install process. You'll also need a per-profile generated database of this metadata for querying purposes, to track dependencies, verify package integrity, handle file conflicts and perform housekeeping during updates and uninstalls.

 

The biggest obstacle here is, unlike in the open source world, 99% of the content is proprietary, redistribution and further modification is often explicitly prohibited, and usually no sources are available, making the job of dependency tracking rather laborious. Mod authors can submit this information themselves, but it will still need to be verified, and unfortunately there's no way to do that programmatically under these circumstances. As you've already noted, this is even more of a problem when the authors abandon their work.

 

The prospect of a small handful of Nexus admins creating and maintaining tens of thousands of packages, manually checking all their dependencies, is simply not viable.

 

But there is a solution. In a typical Linux distro package maintenance is handled by thousands of volunteers (like me). The package maintainer does not need to be a skilled programmer or know anything much about the actual technical workings of the packages he maintains, he only needs to understand the package manager software and the standard operating procedures for submission and package maintenance. Maintainers typically volunteer on the basis of their interest in the particular package they've volunteered to maintain. In my case it tended to be security software. That way volunteers are not lumbered with a chore they dread, but active participation in something they enjoy and usually have a specific need for. I chose security packages that I needed but were unavailable because they'd been orphaned by departing maintainers. For me this was the easiest and quickest way to ensure that the packages I wanted were available in the repo.

 

So having standardised the package format, metadata and created the necessary Vortex plugin to handle this data and subsequent operations, you'll need to recruit volunteers to do all the actual legwork to create and maintain these packages, and a separate group of volunteers to QA their submissions. Then you can sign off on the results, digitally sign the packages and publish them. Given the propriety nature of the submissions, I'd recommend that you also scan them for malware and have another group dedicated to reviewing licensing compliance.

 

Oh, and sorry to necro this thread, but it still seems relevant.