On Tuesday 30 August 2011 00:34:50 Dan Asimov wrote:

I don't know. The idea of a bonus for getting all the answers right seems fine, but why not just award that to anyone who gets all the answers right?

Well, I just gave a reason: don't you like it? (The reason, to recap, is that the all-wrong trick provides a way to distinguish "got all the answers right" from "got all the answers right, and knew it", and on a true/false test those can be very different. It's *not* just a matter of rewarding people who get all the answers right.)

I once took an epidemiology course from Olli Miettinen at Harvard School of Public Health, where tests were not only true-false +-1, but each question also had a space where you were supposed to enter how sure you are that your answer is right -- a number between 0 and 1. The +-1 score was then multiplied by this factor for each question before the numbers were summed.

Oh, that's a nice idea. But with this scheme, if what you want is maximal expected score, you should always rate every answer at 1. There are slight variations on this theme that make the optimal test-taking strategy be to give a rating that exactly corresponds to your estimated probability of having the answer right. One of the simplest goes like this: if you get a question right with confidence q, then you get log(1+q) points; if you get it wrong, you get log(1-q). (Note that the latter is negative, and may be extremely large if q is close to 1.) Many, many other formulae give the same probability-revealing effect, and some don't penalize confident wrong answers so severely. -- g