Jump to content

Photo

On the internet, trolls, and finally creating an effective rating syst


  • Please log in to reply
186 replies to this topic

#41
Dark0ne

Dark0ne

    Webmaster

  • Admin
  • 21,489 posts
Pintocat is hinting the real crux of the issue here. You've got to remember the review and endorsement system are going to compliment each other. There's no point in the review system working on the old "rate on what it says it does" because that's basically what the endorsement system does. The endorsement system tells you how many people liked the file, and in a way, how many use it. The review system needs to tell you something different, it needs to tell you how good the file is in the grand scheme of things.

If you take a file like Midas Magic, or AlexScorpion's weapons and then compare them to a simple couple of lines of code bug fix is it fair that the former files have the same rating as the latter? Of course not. That bug fix might be extremely helpful but it's still only an average file. And here-in is where the review and endorsement systems work together: you'd endorse both Midas Magic and the bug fix, but you'd give Midas Magic a higher rating in the review system because it's grander, it's thought outside the box, it's taken more time, more effort, more blood sweat and tears and it's still been executed perfectly. Both mods are great but one IS greater and the review system needs to be able to show this.

#42
Rasana

Rasana

    Fan

  • Premium Member
  • 464 posts
Dark0ne, I applaud your efforts at making the site better and the ideas being tossed around. After reading all that (yes, I read all of it, :P), however, I did not see what is going to change the modder's or the rater's mentalities of "1" for suck, or "10" for good, and no rating in between is acceptable.

Limiting the amount of subscribers who are allowed to post ratings is a step in the right direction on getting "mature" raters, but I didn't read anything in your post that would do the same for getting "mature" Modders.

How are you going to get the modders themselves, who you admit you had many issues with, to accept anything less than a 10 in the ratings...and daresay they get 5 or 6's? Won't it just go back to the same old problem before Endorsements came around?

I dunno...just some thoughts. I guess you really won't know if you can change a culture/mentality until you try our a new idea to facilitate that change.

#43
TheCalliton

TheCalliton

    King of the Lounge

  • Members
  • PipPipPipPipPip
  • 2,548 posts
hey Dark0ne
i hope i dont sound like im full of myself, but im pretty good at knowing what is OK on the site (i have had my... moments... but they are over hopefully) so if you need help, i guess im availible... ive always drempt of being a mod here


edit: a new rating system could you can only rate if you downloaded it over 3 days ago, no troll is that patient

double edit: well some trolls may be that patient

triple edit: most trolls dont download the things they flame though, right?

#44
chronosmagus

chronosmagus

    Enthusiast

  • Premium Member
  • 130 posts

a new rating system could you can only rate if you downloaded it over 3 days ago, no troll is that patient


Nor will a very large part of the community be. Looking at myself, every once in a while I spend an entire evening looking for new mods to try and improve my gameplay. For most mods the first impression in-game will say enough for a constructive comment (not counting advanced quests and such), to have to wait for several days will probably end up in forgetting to rate at all.

There's a balance that needs to be found and making the process too hard will lead to very few ratings for the less common files.. making them even harder to find. Of course quality over quantity is good, but we still need a certain quantity to get a good average.

#45
Wildangel7

Wildangel7

    Fan

  • Members
  • PipPipPip
  • 285 posts
actually i think there could be something to find good mods faster o.o since there are really good ones out there and normaly people like me search mods with highest endorsements first , but since some file-authors blocked endorsements on files its sometimes not easy to find the really good ones =P
maybe a search thingy based on some other rating then endorsement would be good =P
would fit nice to that rating system that whole talk is about =O (sorry if my english is not thaat good)

#46
Zaldir

Zaldir

    Moderator

  • Moderators
  • 12,717 posts
Well, that was a long post, Dark0ne! But absolutely worth reading. :)

I agree with you on most, if not all topics:

- The new review system sounds really great, if made the right way of course, but from what I know of you, you make everything great.. so I don't think we have to worry about that. And I can say that as a modder I would much rather get a 5, with an explanation of what is bad than a 10 with just "Great mod!". I think most would agree with me on that - at least the mature (Not in Age, but personality) modders.
I've used the Amazon review system a little, and it really works like a charm, and I think it would work the same here. Maybe having a 1-5 star would be better than a 1-10, as that makes it easier to rate a mod something else than 1 or 10, but that is just my personal opinion.

- Allowing only those with an Activity Level of over 12, post count of over 50 or being a supporter to review a mod is a really good idea, as, as you said, it will rule out most trolls and have mostly mature reviews.

- Keeping the Thumbs up system is also a good thing to do, as it's, as you said a way for people to say they like it, even though it wasn't perfect. And removing the Thumbs Down is also a good idea I think, as it will not be of much use with the review system. As stating you don't like it doesn't really help the modder, but stating that you don't like it with a review as well, does.

Thank you very much Dark0ne! :D

#47
HellKnightX88

HellKnightX88

    Master Hashshashin

  • Premium Member
  • 2,486 posts
I agree with Rasana.

This new rating system has the potential of going back to the old ways where anything but a 10 was considered a slap in the face by some people. If I'm not mistaken the old system used the "if it works as intended then it's perfect" rating method, now this new system would be something like I like it but it could be improved this way or that way so I'll rate it a 7. The more "sensitive" modder instead of seeing it as an above average/room-for-improvement rating sees it as a way of penalizing him/her and thus we're back to the old system were sometimes people give an honest opinion and authors throw a fit.

Some end-users give bad ratings because they're too ignorant or mean and some modders fail to understand that <10 isn't the end of the world. You can have "distinguished" members rating the mods thus creating a friendlier environment for modders but what about those authors that are a bit more sensitive to how people see their work? They'll agree to use the rating system or not but it can still lead to unwanted situations of modders leaving because of communication failures.

The image of everyone being on the same page regarding how ratings are supposed to work is utopic IMO. If it workes like it should it will be great, but it probably won't (it is the Internet afterall).

With the current system (which I understand you won't remove and I'm happy to hear that) I ask myself these questions: am I using the mod and would I recommend it to other people? If the answer is yes then I endorse. Easy as that. Then I give feedback on how to improve it if I think of any possible improvements. If the answer is no then I might give some suggestions if I see ways of improving it but if it's something I don't like then I won't say anything (I don't use negative endorsements).

With the 1-10 system I doubt I'd use anything bellow 5 because some people simply take such ratings very badly (some people might even frown upon a rating of 8 or 9) so again I'd have to be careful to not step on any toes with my rating even though it's my honest opinion.

If one person rates a mod with 7 and and some other more vocal people strongly disagree then all hell breaks loose. I've seen it happening in the past.

You owe me kudos btw!

Edited by }{ellKnight, 19 November 2010 - 02:46 PM.


#48
LadyMilla

LadyMilla

    Resident poster

  • Moderators
  • 15,300 posts
Interesting idea, although I am a bit worried that it might open the door to biased reviews that will be thumb-upped by those who share the same sentiments. When you toss out the objective "whether or not the mod performs as advertised in the Description" criteria, a lot of subjective opinions, views, likes or dislikes will factor in. For example, each people have a personal tolerance limit when it comes to lore-friendliness or unfriendliness, whether a mod is just overpowered or a cheat/god item, or when it is about endorsing, accepting, tolerating or shunning anime style races, sex mods, modern weapons in a retro-futuristic game and so on. For a mod that is meant to be lore-unfriendly (e.g. an anime conversion, or a desert reforestation project for New Vegas), it would be unfair if lore-fans came flocking in to give a bad review or thumb up a review that downrates the mod due to being non-lore friendly. Obviously, such a comment/review may be buried under a heap of thumbs-downs given by other reviewers, but in the case of a mod like KDS Green World (for Vegas), the complaints may sound objective and still, they are unfair because the author targeted a different group of users who are sick of the desert and prefer a patch of green here and there. The same goes for body replacers. Originality is also a difficult thing to judge: there can be huge differences between various implementations of the same idea (e.g. player homes), or sometimes simply re-texturing a mod may do wonders.

Some of the game review sites use a rating system based on multiple criteria (graphics, gameplay, learning curve, stability, etc.), each presented as a slider. I think the final score given by the reviewer could be calculated from a set of objective (mandatory) criteria (e.g. the quality of the description/installation instructions, no missing components, quality of implementation vs complexity, quality of textures (they are not shifted, misaligned, etc), quality of meshes (no gaps, etc.), and so on) , and a set of subjective criteria (originality of the idea, aesthetically pleasing visuals/textures/models, etc) that may be limited based on what tags have been ticked and confirmed for the mod. (E.g. if it is marked as non-lore friendly, then the reviewers would not be able to use the lore-friendliness rating slider, etc.).

Just a few thoughts.

#49
XTR3M368

XTR3M368

    Foe Hammer

  • Supporter
  • PipPipPipPip
  • 2,344 posts
I think leaving the top 100 and other "tiered" lists ranked by the number of endorsements and supplementing that system with the new one would work best IMHO. People do not take the time to endorse mods that they believe are mediocre or "not their style" very often. I think that having the proposed "review" add-on would be a fine thing for someone to look at to decide if it is right for them. That would help for the author's that just do the "quick and ugly" description of "just drop this file in your <insert place determined by mod and game it is for here> folder". What I have noticed is that the mods that have 50+ endorsements almost never do that. They usually will have a very nice presentation of the mod in the description with good install instructions and nice screenies.

Why even have a "negative" type rating number at all? That is just an open invitation for legal trolling. How about having it just start at "good" and go up and then re-enforce the old policy of, "if this mod is not for you, don't post (or review in this case) and move on". How will the community benefit from reading a review from a person that thinks all female to male armor conversions are ridiculous and gives the file a bad or mediocre rating? I guess, at the very least, these "reviews" should have a "report" button for them.

I have seen a lot of good people, but I also know human nature. Positive, constructive input will benefit the community. Low ratings and negative criticism will not. It would be better to not say anything IMO. People will get the picture if the mod has no endorsements and no "reviews".

Had to edit this lol. I was even doing the "TLDR" to myself. :tongue:

#50
razorwire

razorwire

    Fan

  • Supporter
  • PipPipPip
  • 484 posts
I would agree with XTR3M368

Whats nice about the endorsement system is its a much more gradual rating system, a review system sounds like a good way to weed different people out, not just modders, as well as highlight some of the qualities on offer.

Whatever you chose to implement, I think its important to keep it constructive and fair, were everyone feels like its worth their time contributing, whether your just the little guy whos still learning, the seasoned veteran, or the team overflowing with talent, no one starts out a pro or likes being rated as inferior.

A lot of could-be-good artists simply give up because they cant take artistic criticism or feel like reaching that next platau just isnt worth the hardship and personal investment.

Edited by razorwire, 19 November 2010 - 05:49 PM.





IPB skins by Skinbox
Page loaded in: 1.112 seconds