Rating system overhaul

[Dark0ne]

As promised both on the forums and in recent news posts the rating system in the downloads area has been completely reworked and changed to suit the underlying rules and principles that this site revolves around. This update has been long in the coming and I've based my design choices on the feedback of literally hundreds of users on this site who have had their say on how they'd like to see the rating system changed (or in some cases, not changed).

 

 

The story so far...

First of all let me address why this update was necessary. Since Oblivion's launch TESNexus has gathered a very strong following of both forum regulars, file database regulars and general users who like to upload and download from the site. These users have grown to understand how this site operates and how to conduct themselves properly within the Elder Scrolls community, which I personally believe is a unique community compared to other modding communities. The Elder Scrolls community is not only productive, but constructive and full of avid modders who are willing to share and extend their knowledge to other users. It has always been important to foster this virtue within the community and as such Nexus sites have always been advocates of the "constructive criticism" policy; that is if you have something to say you word it in such a way that it is constructive to the author of the work.

 

Unfortunately my views of "constructive criticism" no matter how much I tried to explain it to users often fell on deaf ears, especially to new users who had come from communities that didn't share the same views as myself or were new to modding communities in general and who couldn't be bothered to read the rules of the site. This problem was further compounded by the 1-to-10 rating system that really simply became the 1-or-10 rating system. Any number other than 10 was seen as a negative rating due to the way in which the top 100 was calculated. The underlying problem with the top list system was how you create a top list of downloads on a site that can accommodate the fact that not all files are going to be as popular as each other? The weighted system was the best system irrespective of its many flaws.

 

So in light of these problems it became apparent that a change was needed. Either the 1-to-10 rating system needed to be better enforced or a complete change was necessary. As far as further enforcement goes I don't care what anybody says; the moderation team at Nexus (Buddah, LHammonds, BBen and Slof) have done a brilliant job over the years and they've gone through various high-profile and not so high-profile banning runs that would take its toll on anyone. Rather than ask them to keep up or step-up their current efforts, as well as take on more moderation crew, I decided to rework the rating system to better fit in to my views of "constructive criticism". Many users of the site wanted a multi-tiered rating 1-to-10 rating system that would see users rate various aspects of a file, unfortunately the problem with this is that some users of the site have a problem with only one thing to rate, so to give them many different things to rate would seem counter productive!

 

 

So what's new

A rating system needs to be able to present people interested in downloading a particular file valuable feedback from the community on whether they think that file is actually worth downloading.

 

As mentioned the 1-to-10 system was always really the 1-or-10 system and as such the ratings were always very black or white, 1 or 0, you either liked it or you didn't like it. The new system follows but expands on the image share thumbs-up rating system that had a successful integration last year. But simply having a "I endorse this" feature on the site only presents one side of the picture, and a popular file might still have its issues. In the past the problem was that people didn't know how to be tactful in how they down-rated files; instead of providing that all important "constructive criticism" they simply lambasted the author of the file in question, never really addressing what the problem was and instead expressing themselves in such a way that it made the author feel like crap and then the ban-hammer swiftly came out. That's exactly what I didn't want.

 

In response to this the new system is a thumbs up, thumbs down system with a twist. The thumbs down system doesn't negatively rate a file; it doesn't affect the new top lists of the site and instead is used to explain to an author why exactly you haven't decided to endorse the file in question. What's more you can't leave a comment with your negative rating and instead you choose from a set list of reasons why you haven't endorsed the file. Why have I done this? Well, to address the issue of people not being able to play nice and to save the Nexus moderation team from sifting through countless people who haven't read the rules. It's the new "let me hold your hand through this process" system for those people who don't like to read before they rate.

 

With the new rating system comes a new top list system. To offset the popularity contest that is "whoever has the most endorsements wins" more features have been added to the list, along with a top 25 that shows the most rated files in the past two weeks, which provides a surprise; it might not be the files you think! Probably most asked for is the ability to generate lists based on adult and non-adult file preferences along with lists for files within individual categories. These features are now all available.

 

 

Now with added protection

Added to this are new security checks built in to the rating system. You can no longer rate a file without downloading it, and similarly you cannot rate a file within 3 hours of downloading it. This insures that users have at least taken the time to try files out before they rate.

 

New members have a 24 hour cooldown period that means they cannot rate files positively or negatively for the first 24 hours of their account life. This should help to reduce those "special" individuals who create multiple accounts in order to down-rate other user's files.

 

These features go hand-in-hand with the updated download history page (accessible from the right hand navigation) that will now show you which files you have rated, and how you have rated them.

 

 

But this isn't the end....

These changes are in the forefront of further changes to the download section of the site that should both improve the functionality of the site and increase usefulness and interactivity.

 

Following on from this rating system rehaul work has begun on a new commenting system for all files that will change the current mono-thread commenting system into a multi-threaded system that will be moderated not only by the moderation staff here but also by the file authors themselves. Thus file pages will become self-contained "mini-forums" where users can create multiple threads to post in, rather than all posts being placed in a single thread, with everything fully controlled by the file owner who can set whether they want other users to be able to create new threads or just themselves, as well as being able to remove posts and threads made by other users. And of course the file owners will be able to pin selected threads to the top of the list as well.

 

 

But for now

Take a look at the new system and see what you think and maybe hold back judgement until you see the affect it has on the site. Of course all the old ratings have been converted over to this new system and ratings of 7 or more on files have been changed in to endorsements for the file in question. Ratings of less than 7 have not been changed in to endorsement feedback because a PHP script cannot ascertain the reasons why someone has rated low.

 

Feedback, as ever, is appreciated. Just remember; constructive criticism.

[pierrejaquet]

I see that the new system has just been put in place. I like the new system better than the old one. But i also see that you can only rate 3 hours after you have downloaded the file. It was already extremly difficult to make people vote for a file. On 300000 downloads there is like 45 votes. Now they will probably be no ratings at all. ^^ maybe the 3 hours block should be reduced a bit. Edited July 5, 2009 by pierrejaquet

[Usop]

I think there should be something more to define the quality of a mod.

 

the way it is now, the older the file, the more chances it has to stay in the top100 forever.

 

especially considering that voting now is more difficult.

 

 

I think you have introduced the top25 to balance this effect, but I fear it won't be enough.

 

 

But in general good work! in particular I think the possibility to see a top list for each category is brilliant.

Edited July 5, 2009 by Usop

[buddah]

Meet me at the corner pub, I owe you a guiness! ! ! ! !

[Zarrastro]

OK. Nice. I like the Add-Ons. I hope the mini forum update will not take much time! it's a nice idea

 

Good work!

[bben46]

I will join with Buddah in buying whatever libation you prefer. And as many as you can handle.

[Eiries]

Ooo, very nice. You're right, the rating systems were much too black and white; now it will be for sure. People getting upset over a 9 rating were becoming all too common.

 

That being said, I do agree with pierrejaquet. I understand the point of trying out a mod before rating it, but too few a person do it now to begin with. The chances of someone coming back to the mod's page three hours later just to rate it may be just too slim.

 

However, I can also understand why you may not want to change the 3 hour limit. I suppose over time we'll see how the new system plays out.

[hfffoman]

I agree with the logic of the new system but not the way the old ratings have been grandfathered into the new score. Counting a 7 as a positive endorsement is too generous. Had everyone strictly followed the ratings guidelines it would have been appropriate but in practice people didn't. People rating at 7, and often at 8 or 9, had serous reservations about the mod.

 

I used to look hard at the ratings. If a mod had 25 people all rating at 10, that was a strong indication. If it had the same number of people rating at an average of 9 that meant a lot of reservations. The difference would be enough to dissuade me from downloading at least until I had read all the comments. I think it's a pity this information has been lost.

[nintai83]

Awesome, I admit I had no idea this was under works "been playing the sims 3 a lot" but I am 100% behind this idea and I think it will get the job done. Hopefully these huge changes will also stir additional activity in the creation department for the authors. Maybe give it "that new car smell" around here lol.

 

I laughed out loud when I read the part...

 

"Many users of the site wanted a multi-tiered rating 1-to-10 rating system that would see users rate various aspects of a file, unfortunately the problem with this is that some users of the site have a problem with only one thing to rate, so to give them many different things to rate would seem counter productive!"

 

That's common sense and we need more these days.

[Tekuromoto]

Super! A ratings system overhaul was definitely one of the few things Nexus still needed.

 

I'd like to chime in about the 3-hour limit, too. I've always thought it was weird that people would say "Great mod, 10 out of 10! I'll download it and have a look!" Pfft! You're rating an idea and screenshots at that point. But on the other hand, hoping that players will come back on their own accord and remember to rate that mod they downloaded last weekend is kind of wishful thinking.

 

So I'm wondering if it would be possible to add an eBay-style reminder system: a page you can bring up and/or a messagebox that pops up when you first get to Nexus that says something to the effect of "You've downloaded but not rated these files... would you like to rate them now?"

 

[Edit] Okay, I see the Download History tab; musta missed that part in the OP. What about a way to make it a little more "in your face", like a reminder box that pops up once the first time you come to Nexus in a day?

 

[Edit2] I take it that eventually the list we see when we manage our uploaded files we get updated to thumbs up/down instead of the now-obsolete-but-still-being-displayed 1 to 10 ratings.

Edited July 6, 2009 by Tekuromoto