[math-fun] testing coins (and actresses and bishops?)
From: Eugene Salamin <gene_salamin@yahoo.com>
That's a complete misrepresentation of what I said. The number N of tosses is not fixed.? You can toss all you want, until your decision procedure returns fair, unfair, or undecided.
? --? Gene
--if you, say, apply your test for some N, then 2N, then 4N, then 8N, ... (or maybe N+1, N+2, N+3,...; whatever), until some test says "unfair", then that was not a legitimate testing procedure in the sense that the claimed confidence will be incorrect. Any repeated test whatever, where individual test has confidence C of being right, and you repeat until get some sought result, will no longer have confidence C for that result. Repetition of tests ruins confidences. This is a well known banana peel in statistics. Now you could try to correct for that by altering confidences to try to restore safety, but then you'd be inefficient. Or you could not-do that, in which case you'd be incorrect. So hopefully you now see why I want a procedure designed from the start to involve N continually incrementing, and with confidences correctly based on the failed-termination probability from that whole infinite sequence of experiments.
It's beginning to look like we don't agree on what Bayesian analysis is. I'm saying that the posterior probability after n tosses updates and supersedes the posterior after k < n tosses. The posterior provides everything that can be known about x, the probability that the coin comes up heads. Bayes can't tell you if the coin is fair; that's for you to decide knowing the posterior. -- Gene From: Warren D Smith <warren.wds@gmail.com> To: math-fun@mailman.xmission.com Sent: Saturday, March 5, 2016 2:12 PM Subject: [math-fun] testing coins (and actresses and bishops?)
From: Eugene Salamin <gene_salamin@yahoo.com>
That's a complete misrepresentation of what I said. The number N of tosses is not fixed.? You can toss all you want, until your decision procedure returns fair, unfair, or undecided.
? --? Gene
--if you, say, apply your test for some N, then 2N, then 4N, then 8N, ... (or maybe N+1, N+2, N+3,...; whatever), until some test says "unfair", then that was not a legitimate testing procedure in the sense that the claimed confidence will be incorrect. Any repeated test whatever, where individual test has confidence C of being right, and you repeat until get some sought result, will no longer have confidence C for that result. Repetition of tests ruins confidences. This is a well known banana peel in statistics. Now you could try to correct for that by altering confidences to try to restore safety, but then you'd be inefficient. Or you could not-do that, in which case you'd be incorrect. So hopefully you now see why I want a procedure designed from the start to involve N continually incrementing, and with confidences correctly based on the failed-termination probability from that whole infinite sequence of experiments.
Re NJA Sloane remark about coin tossing mechanism, etc... Arggh. Regard the entire blasted experiment, coin, coin-tosser, actresses, bishops, and all, as a black box which returns a boolean every time you ask it to. I allow you to assume each black box output is identically and independently distributed from every other. OK? So now I want to test whether said box is an "ideal fair coin toss" or not, I want the test to be efficient, and I want it to tell the truth, meaning always with correctness probability >= 1-K (for user-specified K). I can't believe all this jive feedback. 1. It is not a matter of opinion whether black box is an ideal fair coin. No opinion whatever is involved. 2. No physics is involved. No physical assumption re involved. This is a pure computational question. 3. Any statement that "here is some test involving N tosses for some particular N, and here is a way to compute its confidence value" is not an adequate answer. Then saying "oh, and by the way, you could run my test again with a different N" is not an adequate answer. Then saying "oh, and you could do that for some infinite sequence of different choices of N" still is not an adequate answer. 4. No testing for pseudorandom patterns like HTHTHT... is needed. Your test can be based solely on the head and tails counts so far, intentionally forgetting all other information, as you go. Due to the i.i.d. assumption, that forgetting cannot hurt you. 5. If you disagree with any of 1,2,3,4 you are just wrong. -- Warren D. Smith http://RangeVoting.org <-- add your endorsement (by clicking "endorse" as 1st step)
From: Warren D Smith <warren.wds@gmail.com> To: math-fun@mailman.xmission.com Sent: Saturday, March 5, 2016 2:30 PM Subject: Re: [math-fun] testing coins (and actresses and bishops?) Re NJA Sloane remark about coin tossing mechanism, etc... Arggh. Regard the entire blasted experiment, coin, coin-tosser, actresses, bishops, and all, as a black box which returns a boolean every time you ask it to. I allow you to assume each black box output is identically and independently distributed from every other. OK? So now I want to test whether said box is an "ideal fair coin toss" or not, I want the test to be efficient, and I want it to tell the truth, meaning always with correctness probability >= 1-K (for user-specified K). I can't believe all this jive feedback. 1. It is not a matter of opinion whether black box is an ideal fair coin. No opinion whatever is involved. Gene: If you want to declare the coin to be fair or unfair, the opinion enters in the choice of K. If you make a probabilistic statement involving K, then I'm OK with it. 2. No physics is involved. No physical assumption re involved. This is a pure computational question. 3. Any statement that "here is some test involving N tosses for some particular N, and here is a way to compute its confidence value" is not an adequate answer. Then saying "oh, and by the way, you could run my test again with a different N" is not an adequate answer. Then saying "oh, and you could do that for some infinite sequence of different choices of N" still is not an adequate answer. Gene: I don't understand what you're saying. What else do you propose to do with the coin but toss it, and the number of tosses is some integer n. Well, you could measure its uniformity of shape and mass density, but that's not what what we're discussing. 4. No testing for pseudorandom patterns like HTHTHT... is needed. Your test can be based solely on the head and tails counts so far, intentionally forgetting all other information, as you go. Due to the i.i.d. assumption, that forgetting cannot hurt you. 5. If you disagree with any of 1,2,3,4 you are just wrong. -- Warren D. Smith http://RangeVoting.org <-- add your endorsement (by clicking "endorse" as 1st step)
Wald's sequential probability ratio test provides specified alpha/beta errors which can be interpreted as one-sided confidences. Brent On 3/5/2016 2:12 PM, Warren D Smith wrote:
From: Eugene Salamin <gene_salamin@yahoo.com> That's a complete misrepresentation of what I said. The number N of tosses is not fixed.? You can toss all you want, until your decision procedure returns fair, unfair, or undecided.
? --? Gene --if you, say, apply your test for some N, then 2N, then 4N, then 8N, ... (or maybe N+1, N+2, N+3,...; whatever), until some test says "unfair", then that was not a legitimate testing procedure in the sense that the claimed confidence will be incorrect.
Any repeated test whatever, where individual test has confidence C of being right, and you repeat until get some sought result, will no longer have confidence C for that result. Repetition of tests ruins confidences. This is a well known banana peel in statistics.
Now you could try to correct for that by altering confidences to try to restore safety, but then you'd be inefficient. Or you could not-do that, in which case you'd be incorrect.
So hopefully you now see why I want a procedure designed from the start to involve N continually incrementing, and with confidences correctly based on the failed-termination probability from that whole infinite sequence of experiments.
_______________________________________________ math-fun mailing list math-fun@mailman.xmission.com https://mailman.xmission.com/cgi-bin/mailman/listinfo/math-fun
participants (3)
-
Brent Meeker -
Eugene Salamin -
Warren D Smith