discuss: Re: Beginning of outline for policies
Subject:
Re: Fwd: Beginning of outline for policies
From:
David Lawyer ####@####.####
Date:
21 May 2002 07:09:42 -0000
Message-Id: <20020521000946.B300@lafn.org>
On Mon, May 20, 2002 at 04:47:52PM -0500, David Merrill wrote:
> The approach in Lampadas is to allow all registered users to rate a
> document on a 1-10 scale. If enough people rate documents, which seems
> likely since I expect many users, the values should be fairly
> reliable.
From what I've seen from a review system where readers posted short
reviews of books and also rated them, the ratings may be very unreliable.
In one case a book was terrible, but friends of the author rated it
excellent. The opposite case can happen too if the author has enemies.
What we can do is to take a close look at a doc that receives both very
high and very low ratings. One of the above situations may be
happening. One might also want to investigate the people who falsified
their evaluation to try to guess why they did it. But that would likely
be too much work for us.
Another problem is that if a doc reads well, many readers may rate it
well even though it contains serious technical errors. This is
especially true if the errors are for situations that the evaluators
don't encounter.
David Lawyer
