Jump to content

Photo

On the internet, trolls, and finally creating an effective rating syst


  • Please log in to reply
186 replies to this topic

#31
KDStudios

KDStudios

    The Master of Puppets

  • Supporter
  • PipPipPipPipPip
  • 3,593 posts
Dark0ne,

I am certainly looking forward to this new effort to sort out the file rating system.
Everything seems to be hitting the objective right on the head.
The only problem like you stated will be getting people to understand more that this will be a much more fair system.

Trolling has become a large disease now all over the web, the nexus staff and moderating team have done a great job in rooting out the worst of the bad egg's
These family of sites are litterally the best I have ever come across and its all down to you; the staff, moderators and the good people of the community.

I'm proud to be a part of this and am excited to see this new implementation unfold =]
Best of luck to you and I am 110% agreeing with all of this.

#32
Terralventhe

Terralventhe

    Fan

  • Banned
  • PipPipPip
  • 456 posts
"The problem is when impressionable idiots start bleeding out from these sites and thinking that they can act like that and share similar content on any site on the internet. These types intermingle with the gaming community, often taking root in the more devolved side of the gaming community such as the stereotypical whiney, voice hasn't broken yet, underage Call of Duty players and they start spreading across the internet spouting out old 4chan memes like they were written yesterday."

Just for this alone, I continued on and read the whole thing. I agree with this point THAT much. It's not that websites like 4chan are a problem, it's that these days a bunch of underaged idiots (who aren't even old enough to browse 4chan in the first place) go there, annoy the everloving s*** out of everyone on 4chan itself, then herpderp their way all over the rest of the internet. 4chan isn't the problem, this current generation of trolling muppets is.

#33
jesserevis

jesserevis

    Enthusiast

  • Supporter
  • PipPip
  • 118 posts
quote: "As a result myself and the moderation team here are what you might call "ban heavy" to compensate. We've been doing this a long time now and we like to think we can spot these types from a mile off."

I'd believe it. Its not hard to spot them at all, we got trust in you guys. Things have always been run well here, from my own perspective anyway, and I have downloaded mods from here for years. Never once have I had a virus or any other issues due to the site or staff. So what you guys think will work best, probably will.

#34
shiholude

shiholude

    Faithful poster

  • Premium Member
  • 1,106 posts
To all at nexus,

Good st00f here Dark0ne!

I cannot begin to imagine the difficulty and stress involved with running a site like this. I have worked jobs that have aged my body by 10 years in a 5 year span, and now live a very low-stress lifestyle - so I have been there to an extent, and moved on from similiar situations. But the internet is a unique beast, and anyone willing to wrestle with it is a man's man...

If/when you get back around to "restarting" the rating system, I wanted to let you know that I look forward to it as a modder myself. I come from a quality background in automotive (that former high-stress stuff above), and the idea of continuous improvement is built into my very being. I welcome all critisms (moreso when they are structured and logical and READ-ABLE, lol), as each word of advice or quirky request helps me become a better modder.

If I could suggest anything, it would be to have the rating system tied to a pop-up window - much like how the "track file" button works. This way, it is more-likely to be used (ease of use) than a new pop-up window (IMO). Within this window, I would suggest to have a section to select the rating, a section that shows how to rank things, and of course a section that requires X number of words or characters as input from the user. Obviously most trollers would hardly take the time to fill out such forms in spite, and those malicious enough to take the time would still be predictable in thier responses. :)

For the ranking system (odd numbers to be filled in as appropriate):
2 = Highly problematic, many errors (details below)
4 = Good, but has some glitches (see comments)
6 = Works as stated, but could be improved (see comments)
8 = Operates without glitches and as promised, it might benefit from the following improvements (see comments)
10 = Absolutely flawless, nothing can be improved upon (see praises below)

Again, this only lists 5 of the possible 10 ranks. I'd be interested to see documn's suggestion built in somehow as well - good feedback there.

I don't say it enough, but thank you very much to ALL OF YOU for keeping these sites up and running! Your time and dedication and occasional tigger-happy banhammer is always welcome!! :)

-Shiholude

[edit] Cleaned up some and I wanted to add that some other suggestions here are very good as well (documn - questiosn to ask yourself before rating, Thandal - suggested ratings, Stewb - X number of posts, pictures, financial contributions to keep your review capabilities)

Edited by shiholude, 19 November 2010 - 06:54 AM.


#35
pintocat

pintocat

    Faithful poster

  • Premium Member
  • 1,565 posts

As with any of Dark0ne's posts, if I'm reading it, I'm reading all of it!

Love the idea of a "filtered" Review system.

Support keeping it simple, I'd suggest no more than five grades:

Excellent
Above Average (works well)
Average (works)
Below Average (problems, esp. compatibility or installation)
Don't Bother

I don't believe there should be a minimum number of words in the reviews. But for Average-or-below ones, there should be a requirement to include some easily understood terms to identify the shortcomings, much like the current remarks for Strikes and Bans; "Flaming", "Spam", "Piracy". So; "Unclear install instructions", "Requires multiple additional mods", "Poor quality art/sound/editing", etc. I'd bet the most common reasons will become rapidly apparent. :laugh:

[Edited to add another thought.]

It's not clear to me whether this would REPLACE the "Thumbs Up/Down" option. I thought there would still be the rapid, and dead-simple, feedback provided that way, PLUS more in-depth Reviews by a (somewhat more mature than average) group.


You totally didn't get what Dark0ne was saying. The numerical rating should NOT be based on how well it works or how well it does what it claims to, but how good it is.

This has always been my problem... why should a simple mod that is nothing but a single field edit of a GMST warrant a perfect 10 when an incredible scripted mod that performs at 98% but has 1012 lines of code, get a 7 or 8 because the script is complex enough that it cannot be "perfect"?


Here's the plan in the original post:

Gone will be the old appeasement policy of "rate on what it says on the tin, not on anything else". Essentially the old system asked you to rate the file based on what the mod author said it did. If you made an awesome tin-opener object to place in to the game and said as much in the mod description then you'd expect to get a 10 for it. With the new system I want to promote something very different, essentially saying "Great, you've added a tin opener, but where are the tins to use the tin-opener on?". Your file might be good, but it might have a whole lot more potential, it might need tweaking, it might lack an actual purpose. Some mods get added "just because I can". That's great, but why is someone going to want to download a file that replaces all the posters in the game with pictures of you on your holiday? Should such a mod get a 10 rating just because it says what it does? Of course not; the rating and review system should tell you about files you really need to download as well as how good these files are.



#36
Leyic

Leyic

    Newbie

  • Members
  • Pip
  • 15 posts
Does the new system need ten rating bins or even use numbers when you were able to state a few bins verbally? Just use "Bad, Below average, Average, Good, Amazing, Perfect", or some such; then there's no need to explain what a number means, and just enough bins for all of them to be meaningful. That said, the new system shouldn't enforce any one bin as "average". Rather, the baseline should be something like "not bad but not good" or "so-so", letting the average come out through statistics only.

Also, I like the "rate on what it says on the tin" policy. I can read mod descriptions to determine if a given mod might interest me or not, and then use the endorsements to compare it to comparable mods. Under the new system, should I rate down a mod simply because it doesn't interest me? Or should I take a view that's removed from my feelings about the mod, rating it up if I acknowledge it as a quality piece of work, even though it's something I'd never use? Either way, it doesn't feel particularly honest or meaningful. To use an analogy, it'd be like a movie reviewer known for disliking horror movies rating a horror movie negatively; should anyone care about that review and add it to the aggregate? I'd say no.

I understand you're not exactly satisfied with the current system, and I try not to be someone who dislikes any and all change, but the endorsement system mostly works for me as is: If a mod increases my enjoyment of the game, I endorse it, otherwise I leave it alone. Granted, not everyone uses the system this way, but that's true of all systems where the reviewers don't go through rigorous standardization.

#37
pintocat

pintocat

    Faithful poster

  • Premium Member
  • 1,565 posts
What used to happen when there was a 1-10 went basically:

  • Mod A changed one line of one thing. It had no room for bugs, so did this one minor thing perfectly. Because it did some minor thing perfectly, it was rated a 10.
  • Mod B was a script mod that did things nobody knew the engine could do... but being a script, it might have had a bug somewhere. Or it might have had a bug in version 1 that was later fixed. A low rating from a bug that was later fixed, would haunt it forever. A low rating because it couldn't be perfect because it was more complex, meant that it would not get a 10, but instead be rated lower than the single field-edit mod.

Do you actually think that a mod that changes the camera FOV should be rated HIGHER score than a mod that adds like, a flying, firebreathing dragon to a game which has no flight built into the game engine, just because the latter mod, because of the innate complexity, is not CAPABLE of being 100% flawless?


This was how it used to work. So then, because Mod A got a 10, Mod B, which arguably was a better mod in any metric, had to be rated a 10 or a mod that actually deserved the recognition, would drift down below other mods. And then we got into the cycle of everything must be either, a 1 or 10.

#38
homaredou

homaredou

    Fan

  • Banned
  • PipPipPip
  • 375 posts
+10 Sincere Respect to the 'dark one'.

#39
Leyic

Leyic

    Newbie

  • Members
  • Pip
  • 15 posts
@pintocat

The flying dragon mod is more complex than the FOV mod, but comparing them is like comparing coffebreak hack-n-slash roguelikes to modern big-budget FPSRPGs: they can both be enjoyable in their own right. Why should enjoyable mods be condemned to have lower ratings due to their simplicity? Plus, if someone wants to find a simple mod, they'd have to look through the lower rated ones, not knowing if the low rating was due to the mod's simplicity or due to genuine flaws within the mod.

#40
pintocat

pintocat

    Faithful poster

  • Premium Member
  • 1,565 posts
Why should a mod that has 100 hours put into development be scored lower than a mod that anyone can make in less than 30 seconds of CS/GECK use?

The numeric score has to be based on something.

If it's based on "how well" it works, it slants in favor of simple mods... kind of discouraging to any kind of mod that requires effort to create.

If it's based on "what it does", it doesn't favor mods based on complexity alone, but how good useful they are.

We know from past experience the LAST time we had 1 - 10, using "how well", what happens. In order for a "complex" mod to compete with a mod with a mod that has no way of failing, it HAS to be voted a 10. And we're left in the age old dilemma once more.

The rating has to be based on actual merit. A 5 is NOT "bad". It's average. Average means... average. The 1 or 2's would be mods that have no real purpose. It is subjective... but really. We all know how many dozen "Dog--launching Fatman" mods there are on Fallout3nexus. Just because it's a mod doesn't mean it's worth anybody's time, especially if it's the 15th copy of the same thing. They serve a purpose... they're probably someone's tutorial project. But just because the 15th Dog Launching Fatman mod does, indeed, launch dogs, does not mean it's worth a 10.




IPB skins by Skinbox
Page loaded in: 0.950 seconds