A Quality label

[kaf]

Well, then the author (the submitter) should at least have an option to exclude his problem from the rating system. Maybe some checkbox to mark/unmark?
Or separating real problems, made by masters, from "i-kind-of-played-a-game-and-think-that-everyone-should-know-about-that" would be nice too.

[adum]

as tails said, the ratings are for all registered users, and are permanent (though of course users are free to change their rating at any time).

as the system is fairly new, i wouldn't get too concerned when a problem has only two ratings and you disagree with the average. let's see how the ratings balance out after some more time.

the Stats page shouldn't just show the problems with the highest average as Top Rated, but should do something like a confidence interval:
http://www.evanmiller.org/how-not-to-so ... ating.html
this is my fault since i always had it as an average.

excluding a subset of submissions from a rating system is probably not going to happen. if something gets submitted to digg or StackOverflow or any other social site i can think of, it suffers the same trial by fire as any other submission.

[kaf]

Thanks, Adum. I just overreacted, as usually. That "1" set to my favorite problem tipped me off. Nevertheless, the "most high-rated problems" list is a face of the site, if you will. So, I think the idea of placing a problem in this list only after at least 10 people voted, is not bad. Evidently, any invalid problem rated "5" by its creator will top the list and deface the whole site, until somebody downrates it.

[tails]

Hmm... I thought it was rather a good thing that the list is unstable, in that new problems have more chance to get listed, to be viewed, to get voted and so to settle into appropriate values. But that seems wrong.

Adum, can I re-implement the highest and lowest rated lists using the function in the page you wrote above? The detailed ideas are:

• Calculate a lower bound for each problem, and use it to order the highest rated list.
• Calculate an upper bound for each problem, and use it to order the lowest rated list.
• Users can't see those values (because the values don't make much sense humanly), and users still can see average and number of votes.

Or should we have 20 highest rated problems instead of 10 highest and 10 lowest in that page?

[kaf]

That's an excellent idea, tails. I could never understand the purpose of the lowest rated problems list.

[adum]

i agree with your suggestions, tails and kaf.

and lowest rated probably is not necessary.

[tails]

I have removed the lowest rated problems list. Please give me some more time for ordering problems by the confidence interval function. The page shows a function for two-value votes, and I must remake it for multi-value votes.

[tails]

I have made a test page that shows how a problem will get scored in the new ordering system.
http://www.goproblems.com/test/wilson/wilson.php
Please note that "1" voters have more impact on the new ordering system because it uses the lower bound of the interval.
I think I will wait for opinions for some days before implementing the new list.

[kaf]

What I'm trying to recommend is excluding from the list problems with only one vote. That should take care of garbage. Although I've learned statistics while ago, I remember that one cannot make statistics out of a single point.

[tails]

A single 5-star vote will give only 1.83 Wilson score to a problem, which is lower than the score three 3-star votes give, so the problem should be unlikely in the list. Isn't that satisfy your request?

I think the good point of Wilson score (compared to the normal approximation) is that it can give a good score even for a small number of votes.