NexusMods needs a new metric to access mod quality

[ModdingXModder]

In academic publishing a metric that we use to validate an author's work quality, productivity and impact is H-index. https://en.wikipedia.org/wiki/H-index

I've seen several low quality mods that have high unique and total download numbers and non existent endorsements.Several groups of users in NexusMods publish or support collections and consequently their own included mods get high download numbers. Collections are helpful for the modding naive players, but this practice affects some newer quality mods in many games that don't get the needed exposure and promotion.The guys behind NexusMods need urgently to develop a new metric system that could help the newer quality mods to shine. Mods that are more stable, well written for the latest game editions. As a premium member (and willing to support the site further) I'd expect a better metric system in accessing mod quality.There are many modders that provide their excellent and time consuming work for free in hope that it could help them in their work progression (artists in graphics, music and programmers). A new metric is urgently needed to give these excellent people a further motivation to keep contributing. The mistakes done in past with the collections policy that drove excellent modders out of NexusMods, should not repeat in other instances. Please consider the adoption of a new objective metric to promote excellent modders' work and making easier for us to find quality work instead of trash.Endorsments are subjective. The adoption of such a system would be a motive personally for me to keep supporting the site as a subscriber.

[showler]

How, and be precise, would that work?

 

Also, explain how Collections are unfairly affecting download numbers in any way?

[Ethreon]

Nah, we don't.

[Guest deleted34304850]

ha ha ha

 

yeah. no.

[JimboUK]

What makes a good mod is largely subjective so an objective metric isn't possible. I noticed you have no files uploaded which didn't come as a surprise, try making something yourself and then see how keen you are on having it graded by some arbitrary system.

[BigBizkit]

What metric could that be? What are your thoughts on the success rating collections use?

 

The H-index is interesting, but I'd argue scholarly work by definition is very different as it's looking for and arguing for the objective truth. Mods are closer to artwork in nature, but I appreciate there are some objective criteria to how "good" a mod is such as "does it even work?", "does it work with this or that version of the game?" etc.

[Zanderat]

The collections model is flawed. Many people vote "doesn't work" because of their own incompetence or errors without providing feedback or asking for help. Truly there is no perfect metric for measuring what is ultimately subjective or taste.

[gnarly1]

In academic publishing a metric that we use to validate an author's work quality, productivity and impact is H-index. https://en.wikipedia.org/wiki/H-index

I've seen several low quality mods that have high unique and total download numbers and non existent endorsements.Several groups of users in NexusMods publish or support collections and consequently their own included mods get high download numbers. Collections are helpful for the modding naive players, but this practice affects some newer quality mods in many games that don't get the needed exposure and promotion.The guys behind NexusMods need urgently to develop a new metric system that could help the newer quality mods to shine. Mods that are more stable, well written for the latest game editions. As a premium member (and willing to support the site further) I'd expect a better metric system in accessing mod quality.There are many modders that provide their excellent and time consuming work for free in hope that it could help them in their work progression (artists in graphics, music and programmers). A new metric is urgently needed to give these excellent people a further motivation to keep contributing. The mistakes done in past with the collections policy that drove excellent modders out of NexusMods, should not repeat in other instances. Please consider the adoption of a new objective metric to promote excellent modders' work and making easier for us to find quality work instead of trash.Endorsments are subjective. The adoption of such a system would be a motive personally for me to keep supporting the site as a subscriber.

1. Most of your post is anecdotal and very likely informed by confirmation bias

2. H-index is not without its own deficiencies, including being prone to manipulation and therefore it is not an objective measure either

3. The half-life of scientific consensus (the time it takes for half of the 'accepted' wisdom in an academic field to become obsolete/disproved) across the various disciplines/domains is often alarmingly short. This is not dissimilar to modding where mods that were once considered staples of most load orders fall out of favour due to issues that aren't discovered until later, or mod authors creating mods that do similar things but in objectively better ways

4. We all know why some mods get massive attention and other mods, that contribute more 'value' to the community, are often unappreciated in comparison. That's due to the nature of the community, not due to a lack of an objective measure of quality

5. I don't doubt that there is some method of objectively and reliably measuring mod quality but perhaps the most difficult part will be finding a method that is resilient against being manipulated.

Edited March 21, 2023 by gnarly1

[ScytheBearer]

There is only one truly objective measure of mod quality. Does it work as advertised? Any other metric is based on opinion and is subject to manipulation. Furthemore ...

 

... using the general user's perceptions as a metric simply turns the evaluation process into a beauty contest, much as already exists.

 

... the age of a mod is no measure of a mods quality. Some very simple mods are quite old, but because of their simplicity still meet the singular criteria. they work.

 

... judging the artistic quality of a mod is always subjective, regardless of who is evaluating the mod. One persons beauty is another persons stomach turner.

 

... the larger and more complex a mod, the higher probability that it will have it's detractors. This negativity is generally based on one or more issues which the reviewer may have with the mod, based solely on the reviewers personal preference.

 

... expectations front loaded on mod makers based on their past performance (or lack thereof) often affects the perception of the quality of their work. See any movie review for a sequel for evidence of this expectation bias.

 

... an authors' stance on how and where their mods are to be used often affects the perception of the quality of their mods. Mod quality is denigrated because a mod is only supported on a specific version of the game or on specific platforms.

 

... an authors' stance on the sharing of their work as a resource for others (re: the Parlour v Cathedral debate) will often affect the perception of the quality of the work. Parlour mods are viewed as "closed" or bad, while Cathedral mods are viewed as "open" or good.

 

I will close by repeating my opening argument. Any evaluation of a mod other than "does it work" is subjective. Beyond that simple criteria, and just as is in the evaluation of any work of art, what constitutes quality is in the mind of the reviewer.

 

EDIT: Observable, testable, measurable, repeatable; the four criteria for an objective metric. Any metric proposed for evaluating a mod's quality which does not fulfill all four of these criteria is simply seeking an opinion. And we all know what opinions are like.

Edited March 24, 2023 by ScytheBearer

[gnarly1]

There is only one truly objective measure of mod quality. Does it work as advertised?

Demonstrably wrong.

 

A mod may work as advertised yet it might, for example, have a deficiency which causes save game bloat.

 

You couldn't call that a quality mod.

 

To avoid further embarrassment, I recommend you at least attempt to falsify your own assertions before committing them to pixels.