Peer to Peer mod distribution via IPFS

[infosphereIT]

Hi All,

 

I've been following and using Nexus for a number of years now and while I've never posted a mod to Nexus I am an appreciative consumer of the talented and dedicated people who make the games I play more enjoyable.

I currently work on IPFS (https://ipfs.io/) which is an open source peer to peer protocol that helps people securely find and distribute data without needing to care exactly where the data is hosted. In short people can add a file to a locally running IPFS node and they will get out a long digital fingerprint (i.e. a hash like SHA256) that uniquely identifies the content. When someone in the future wants to find that content they ask the network "hey does anyone know where I can find the file with this digital fingerprint?" and if so they can download the data from them.

I've been thinking that it might be helpful if mods could be tagged with their IPFS digital fingerprints then it would be possible for mod managers like Vortex to support downloading mods over IPFS as well as from public server infrastructure like Nexus. There are a number of reasons this could be pretty convenient but a few that are currently on my mind are:

1. If your friend comes over and you try and download a mod they already have you won't waste time or bandwidth going to Nexus for the mod since you'll automatically find that the data is accessible over your home network
2. Nexus' bandwidth costs could come down which could help out the team maintaining this fantastic services. If Nexus' costs were to ever exceed their inbound revenue then this great service would go away. I suspect there are a number of users who might be willing to donate their bandwidth to keep the mods they use alive but would find it more difficult to actually give money to a paid service.
3. Mod packs. This technique could make it easier for people who want to document the exact set of mods they're using (e.g. streamers) to put together this list and make it easier for people to download the entire set of mods even if they come from various different websites
4. Making mods easier to get even if a particular distribution point is unavailable (e.g. website down, DDoS, website blocked in your country, etc.).

I understand that the modding community has varied opinions on how mods should be found and distributed. For those of you who have been active in the community longer or who are involved with maintaining Nexus or Vortex what do you think about this idea? If I were to work on a Vortex extension that would simply try to fetch the data via IPFS if there was some nexus metadata tag containing the digital fingerprint of the data would that be welcome? Are there any folks here who might be interested in helping me out a bit on the Nexus/Vortex side of things, as that might make my life a little easier :smile: ?

Thanks and looking forward to your feedback

[HadToRegister]

People get banned from Modding Websites and Modding Web-pages for a reason.

This essentially sounds like a Napster or Torrent distribution of people's Mods without their control or permission.

 

-1 to this idea, people already have to file DMCA notices from sites that steal and distribute mods.

[infosphereIT]

Could you give me a little more background on the types of problems modders who distribute their work freely on Nexus tend to run into in terms of people stealing their work? Some examples I could see people running into are:

• People literally copy-pasting a modder's work and removing the copyrights + attributions
• People including the modder's work within their larger work and not giving credit
• People reposting the mod on another site without the modder's permission but with full credit + attribution (i.e. they have not licensed their work for redistribution in non-Nexus places)
• People including the modder's work within their larger work and giving credit, but not respecting the license of the work

While some of these issues can crop up as a result of decentralized technologies being harder to censor than a simple DMCA request some of them are not that big a deal. For example, game mods are code and it's generally pretty important that you know what code you download is doing. While you can download data from anywhere the only way to know if you're getting the "real" thing is to check the digital fingerprint (or hash) of the data stored on Nexus itself. This means that if you are given a link to the mod code but with the attributions removed and license changed that you're getting data that's different from that on Nexus with no real way of telling how similar they are without downloading both versions since the digital fingerprints of the files have changed. This should push people towards keeping the credit where it is due and distributing the licenses with the mod.

Note: there's also a nice bonus here which is that the more people have downloaded a mod the faster you will be able to retrieve it. Aside from the nice performance benefits this gives it also highly incentivizes people to download the correct version of the mod instead of an altered one since there should be more people who have the correct version of the mod which will cause it to download faster.

I've worked on projects with various sorts of open and closed source licenses in the past and I relate to the pain of people stealing your work. IMO the benefits to the modding community are worthwhile due to the added flexibility and resiliency it gives to mod distribution (e.g. there are people who are interested in building version control tools and package managers that work IPFS for similar reasons) and I'd definitely like to learn more about the particular concerns that are prevalent here.

Edited September 8, 2020 by infosphereIT

[agc93]

Could you give me a little more background on the types of problems modders who distribute their work freely on Nexus tend to run into in terms of people stealing their work? Some examples I could see people running into are:

• People literally copy-pasting a modder's work and removing the copyrights + attributions
• People including the modder's work within their larger work and not giving credit
• People reposting the mod on another site without the modder's permission but with full credit + attribution (i.e. they have not licensed their work for redistribution in non-Nexus places)
• People including the modder's work within their larger work and giving credit, but not respecting the license of the work

While some of these issues can crop up as a result of decentralized technologies being harder to censor than a simple DMCA request some of them are not that big a deal. For example, game mods are code and it's generally pretty important that you know what code you download is doing. While you can download data from anywhere the only way to know if you're getting the "real" thing is to check the digital fingerprint (or hash) of the data stored on Nexus itself. This means that if you are given a link to the mod code but with the attributions removed and license changed that you're getting data that's different from that on Nexus with no real way of telling how similar they are without downloading both versions since the digital fingerprints of the files have changed. This should push people towards keeping the credit where it is due and distributing the licenses with the mod.

Note: there's also a nice bonus here which is that the more people have downloaded a mod the faster you will be able to retrieve it. Aside from the nice performance benefits this gives it also highly incentivizes people to download the correct version of the mod instead of an altered one since there should be more people who have the correct version of the mod which will cause it to download faster.

 

I've worked on projects with various sorts of open and closed source licenses in the past and I relate to the pain of people stealing your work. IMO the benefits to the modding community are worthwhile due to the added flexibility and resiliency it gives to mod distribution (e.g. there are people who are interested in building version control tools and package managers that work IPFS for similar reasons) and I'd definitely like to learn more about the particular concerns that are prevalent here.

 

Just to add to some of the discussion we had on Discord earlier: I'm an open-source developer, work for an open-source organisation and have spent a lot of time working on modding tools at this point. The very first thing I noticed was "game mods are code" because that is definitely not always true. A massive quantity of game mods are creative assets with zero code involved.

 

One of the main lessons I've learned is that the demands of "traditional" open-source environments and modding are surprisingly different. Couple of examples:

 

- Licenses are frequently not distributed in-file because these files are going directly into a game's directory which you probably don't want to be full of license files. That means that mod files from different sources can have the same hash even when one of them is breaching license/permissions.

- Licensing for mod source and permissions for mod files are not the same thing. My *code* is open-source (MIT), but I specifically don't allow my binaries to be uploaded without permission (in Nexus's permissions)

- The target users of your files are frequently not familiar with the development side of your work. I can explain the specific terms of the MIT license to users all day, but they just want to install skins in their game

- a lot more, but its hard to think of them all!

 

That's also not even going into the fact that bypassing Nexus and downloading files from other users also takes potential earnings away from authors using the DP system, since that is measured from Nexus's download numbers.

 

I understand your motivations (they're clear and reasonable!) and what you're looking for is *probably* doable with Vortex and Nexus with enough work, but I personally would also not recommend it and would be ensuring my files weren't being distributed through it (which admittedly doesn't really apply for most of my files).

[infosphereIT]

@agc93 thanks again for the feedback here and on Discord.

Could you tell me a little more about why you don't allow my binaries to be uploaded without permission? I'd like to understand more about the underlying concerns here.

> That's also not even going into the fact that bypassing Nexus and downloading files from other users also takes potential earnings away from authors using the DP system, since that is measured from Nexus's download numbers.

Decentralized tracking of user metrics is something that's been bothering me as well, but there are stop gap solutions. I suspect that if this were to start gaining traction there would be a way to flag Nexus with an "I downloaded this" flag associated with the users' account. Perhaps I'm being overly generous to people, but I having a behind the scenes notification to Nexus that I downloaded a mod so that the author can be compensated seems like a thing users of this extension would be happy with.

> Licenses are frequently not distributed in-file because these files are going directly into a game's directory which you probably don't want to be full of license files. That means that mod files from different sources can have the same hash even when one of them is breaching license/permissions.

This could be problematic. I've seen plenty of mods with readme and license files, but you're right many mods don't include them in the downloaded files. Maybe this is a good argument towards avoiding dealing with the hashes as they're currently stored on Nexus and relying on authors posting the IPFS identifiers for their content which could come with licenses. It seems to me like license files should probably be downloaded with mods anyway, perhaps having some standard structure where the licenses are downloaded but not actually transferred into the game directory would be helpful here.

> A massive quantity of game mods are creative assets with zero code involved.

Absolutely, totally my bad there. I tend to think of all assets injected into a running program as "code" in that they're able to manipulate the program I interact with. However, that's a very particular view that I suspect is not shared by the majority of the people on this forum .

> a lot more, but its hard to think of them all!

Absolutely, as decentralization oriented technologies start to take off we're going to have to figure out what the edge cases look like here. I'm interested in seeing what some of the models look like going forward and always open to hearing more GOTCHAs as they come.

Edited September 8, 2020 by infosphereIT

[agc93]

> I suspect that if this were to start gaining traction there would be a way to flag Nexus with an "I downloaded this" flag associated with the users' account.

I certainly hope not! Exposing any programmatic way to increase download counts for files sounds like it's ripe for exploitation, no matter how carefully the tracking would be done. That's not an unsolvable problem, but it's a pretty big one given the problem it's solving is, you know, paying authors for their work.

 

> Licenses etc.

 

Your points are all very sound, and make sense for someone who is used to working with open source code. Mod files, however, don't follow the same rules. The reason lots of mods don't include licenses is because lots of mods don't *have* licenses. Mods don't follow the same standardised method of licensing assets that software does so all the conveniences of things like SPDX lists just don't exist.

 

You're right that authors *could* post IPFS identifiers but you're now back to the problem of having to parse some arbitrary text fields (the only API-exposed text fields per-file are the name, description and changelog), which is far from ideal.

 

> code vs non-code mods

 

Yeah, that ones a common misconception that also tripped me up early on. For an example, the most recent game I've been modding was Ace Combat 7. It's skin mods are literally image files, packed in a proprietary format and installed as-is. Mod creators never see a line of code, and their only real option for effective licensing would be the CC family, which is definitely not what most of them would choose.

 

---

 

As for why I don't allow external uploads: the primary reason is support. Every time I get a user reporting something wrong with one of my tools, I want to know that those files came from a known source that I trust. While hashes can help with this, I don't want to have to ask users to start checksumming files just so I know that no-one has messed with the files or that they're not hilariously out-of-date (honestly the bigger concern of the two). It also lets me more effectively control the impact/reach if I accidentally introduce a critical bug in an update.

 

If someone wants to compile it themselves, they a) have some idea of what they're getting into and b) should understand that I can't necessarily help them.

[HeyYou]

Also, the tool simply reporting to Nexus that X file was downloaded by Y user, in order to deal with the DP for the mod...... simply won't work. Nexus NEEDS those users to actually VISIT the site, in order to generate revenue. (ads, premium membership, and such) They have zero motivation to support awarding DP to a mod that wasn't actually downloaded from their site. DP only works with downloads from Nexus. Doesn't matter how many downloads you get at some other source, Nexus ain't gonna pay anyone for that.

[HadToRegister]

The site also has had people make numerous accounts in order to "Endorse" their own mods and download them for DP.
So, imagine that someone figures out how to anonymously download their mods over and over again en masse?

[infosphereIT]

> Nexus NEEDS those users to actually VISIT the site, in order to generate revenue. (ads, premium membership, and such)

I get what you're saying and it's a good point, but I think it's more complicated then you're making it out to be. Bandwidth is a cost and in the p2p case Nexus no longer has to eat as much of that cost. I suspect users will still spend plenty of time on Nexus looking at ads, etc. because they will spend time looking at the mods. I know that when I use Nexus I spend way more time browsing mods and reading docs and posts then I do sitting on the download screen, but maybe I'm an outlier ‍.

> The site also has had people make numerous accounts in order to "Endorse" their own mods and download them for DP.
> So, imagine that someone figures out how to anonymously download their mods over and over again en masse?

So... my strawman suggestion was to tie the "I have downloaded a mod" trigger to particular accounts. That makes this problem almost the same as the one that already exists. I'm sure there are plenty of ways to bikeshed this particular attribution problem in general that have nothing to do with p2p technologies (e.g. even in the existing system it could be beneficial to care about the number of user endorsements post download instead of just number of downloads) and IMO is worthy of its own topic independent from this one.

Edited September 9, 2020 by infosphereIT

[HeyYou]

If the users are coming here to browse mods, why not just download them from here as well, instead of resorting to a third party tool.....

 

This particular topic has come up before as well. (like, every couple of years, since morrowind released.....) Never has really gained any traction. Most of the mod managers will interact directly with the sites, making downloads/installations fairly easy. A couple clicks, and down the road you go. Most folks simply aren't interested in adding yet another layer of complexity to modding their games, (or, installing yet another program......) It's an interesting idea, but, I don't see Nexus integrating support for it into Vortex.