September 2005 - mathfun@mailman.xmission.com

RE: [math-fun] Trimmed means and Multi-dimensional medians
by dasimov＠earthlink.net 29 Sep '05

29 Sep '05

Joshua writes: << . . . Here is another question about the median: is there a median that makes sense in two or more dimensions? Suppose (X,Y) ~ f(x,y) where f(x,y) is the continuous joint pdf of the random variables X and Y. Is there a reasonable quantity to call the median? >> For a *finite uniform* distribution, a reasonable way to generalize the 1D median (on R) to R^n is to use "convex peeling": do Take the convex hull of the data, then remove data on the boundary of its convex hull; while data remains. When eventually a removal leaves no remaining data, put these last points back and let their vector mean be the median. (For a continuous distribution, I suspect that by taking an arbitrarily large finite sample, the result of convex peeling should converge, with probability 1, to a point depending only on the original distribution. Better yet, there's probably some differential equation that implements this limit without resorting to samples. WPT ?) --Dan

2 1

[math-fun] Re: Trimmed means and Multi-dimensional medians
by Thomas Colthurst 29 Sep '05

29 Sep '05

> Here is another question about the median: is there a median that makes > sense in two or more dimensions? Yes, the maximium likelihood point estimate of the center of the Lapace distribution I gave in my previous email makes sense in any dimension, and with suitable (positive definite) choice of S, will satisfy sum_i | x_i - mu | / ( x_i - mu ) = 0 > Here's an interesting question: suppose we have data X_1, ..., X_n drawn > from a Gaussian distribution with unknown mean > mu and known variance 1. We wish to estimate mu with a guess muhat. > Virtually everyone uses the sample mean of the dataset as an estimate of mu, > but note that mu is also the *median* of the distribution. Under what > circumstances would we be justified in prefering the sample median of the > data to estimate mu? If the data was drawn from a Laplace rather than a Gaussian distribution. > Since the sample average is a sufficient statistic, the > answer might be never, but I'm not sure. Might it be the case that that the > sample median is preferable if we are using L1 loss, i.e., seeking to > minimize E_mu |mu - muhat| ? As others have pointed out, the posterior distribution of mu is symmetric about the average, so any reasonable loss function (including L1 loss) is going to be minimized at the average. -Thomas C

1 0

Re: [math-fun] NYTimes: melting ice cubes cause water to rise ?
by Henry Baker 29 Sep '05

29 Sep '05

The first line of the article says "The >>floating<< cap of sea ice on the Arctic Ocean shrank this summer..." At 07:48 AM 9/29/2005, ed pegg wrote: >Floating ice displaces an equal amount of water, true. > >However, the Arctic ice is partially anchored, and not >floating. > >Ed Pegg Jr > >--- Henry Baker <hbaker1(a)pipeline.com> wrote: > >> Let me see -- melting floating ice makes the water level rise? Isn't >> this a standard SAT question? I guess Mr Revkin's verbal SAT scores >> exceed his math SAT scores.

1 0

[math-fun] Trimmed means and Multi-dimensional medians
by joshua sweetkind-singer 29 Sep '05

29 Sep '05

While we're at it, the "trimmed mean" is another way to combine the robustness of the median with the acuity of the mean. the k-trimmed-mean throws out the k most extreme data points (if I remember correctly, and probably k should be even) then takes the mean of the rest. The pure mean is then equivalent to the 0-trimmed mean, and the pure median is equivalent to the (n-1)-trimmed mean (where n is the number of data points in the set). Here's an interesting question: suppose we have data X_1, ..., X_n drawn from a Gaussian distribution with unknown mean mu and known variance 1. We wish to estimate mu with a guess muhat. Virtually everyone uses the sample mean of the dataset as an estimate of mu, but note that mu is also the *median* of the distribution. Under what circumstances would we be justified in prefering the sample median of the data to estimate mu? Since the sample average is a sufficient statistic, the answer might be never, but I'm not sure. Might it be the case that that the sample median is preferable if we are using L1 loss, i.e., seeking to minimize E_mu |mu - muhat| ? Here is another question about the median: is there a median that makes sense in two or more dimensions? Suppose (X,Y) ~ f(x,y) where f(x,y) is the continuous joint pdf of the random variables X and Y. Is there a reasonable quantity to call the median? -Joshua On 9/29/05, Mike Speciner <speciner(a)ll.mit.edu> wrote: > > So, if y(x) is the histogram, the median is the m such that > > integral(x<m) y(x) = integral(x>m) y(x) > > while the mean is the m such that > > integral(x<m) |x-m|*y(x) = integral(x>m) |x-m|*y(x) > > This suggests a whole family of averages (using various functions of > (x-m) for the weighting), though what use they might have escapes me. > > --ms > > > David Gale wrote: > > > > > Jim, what is the Propp median if there are m zeros and m fives (and zero > > everything else)? > > Dan, if you're going to bring in averages at all then why not go all the > > way and use THE average? But maybe the CDC was using some sort of hybrid > > like the one you suggest. > > > > D > > > > At 09:14 PM 9/28/2005, you wrote: > > > >> The picture was supposed to show a rectangle of width 1 and height 2 > >> whose bottom is centered at x=1, and to the right of it, a rectangle > >> of width 1 and height 1 whose bottom is centered at x=2. > >> > >> The base of the first rectangle goes from x=1/2 to x=3/2, and the base > >> of the second rectangle goes from x=3/2 to x=5/2. > >> > >> The total area under the histogram is (1)(2)+(1)(1) = 3. > >> > >> The area to the left of the line x=5/4 is (5/4-1/2)(2) = 3/2, which > >> is half of the total area. So x=5/4 is the "median". > >> > >> Jim > >> > >> _______________________________________________ > >> math-fun mailing list > >> math-fun(a)mailman.xmission.com > >> http://mailman.xmission.com/cgi-bin/mailman/listinfo/math-fun > > > > > > > > > > _______________________________________________ > > math-fun mailing list > > math-fun(a)mailman.xmission.com > > http://mailman.xmission.com/cgi-bin/mailman/listinfo/math-fun > > > > _______________________________________________ > math-fun mailing list > math-fun(a)mailman.xmission.com > http://mailman.xmission.com/cgi-bin/mailman/listinfo/math-fun >

2 2

Re: [math-fun] Sex, Lies and Government Conducted Surveys
by Richard Schroeppel 29 Sep '05

29 Sep '05

(forwarded, with headers eviscerated -- Rich) From: Dylan Thurston <dthurston(a)barnard.edu> Date: Thu, 29 Sep 2005 11:37:07 -0400 On Wed, Sep 28, 2005 at 02:50:45PM -0400, Michael B Greenwald wrote: > .... For the purpose of a mathematical example, and not to cater to > stereotypical prejudices, let's assume that the men want to, and > willingly do, pair with every possible partner (the 20 women for the > lone heterosexual man, and the other 18 homosexual men for the 19 > homosexual men). The 20 monogamous women, each partner with the one > heterosexual man (presumably unaware that he's not monogamous). The > homosexual men each pair up with every possible homosexual partner. > Then we'd have a mean of 1 partner for the women, but a mean of > about 18.1 partners for the men. David quote the report imprecisely: if you look at the report, it asks for the number of female partners of males, and the number of male partners of females. The point about an unequal total population might still work, but wouldn't put the averages off by a factor of 2. Of course they're reporting median, not average. Peace, Dylan

1 0

[math-fun] Relativity Challenge
by Thane Plambeck 29 Sep '05

29 Sep '05

Please pass on by if you're not interested in cranks. > Scientists and researchers have scrutinized Einstein's work and have been unable to definitively find anything wrong with it... until now! > Best of all, you don't have to have a PhD to understand and verify Einstein's mistake for yourself. > All that's required is an understanding of basic Algebra, access to Einstein's papers, and information to guide the way.... > http://www.relativitychallenge.com/index.htm -- Thane Plambeck http://www.plambeck.org/ehome.htm

3 3

Re: [math-fun] another kind of median (trying again)
by dasimov＠earthlink.net 29 Sep '05

29 Sep '05

Jim writes: << >But: For very small samples, the median doesn't generally convey much information. >For a large sample, however, this "bar-graph" median can be unduly influenced >by outlying data -- much as an average can. Really? It seems to me that the bar-graph median is very close to the "true" median. In one case, your probability measure on R is a combination of point-masses; in the other case, it's a combination of uniform distributions on disjoint intervals. But it's only near the median that it makes any difference which you're using. >> I agree that if Jim's idea is translated into the corresponding thing for a continuous probability density, it's exactly thr usual median of a continuous distribution. I don't think it's the same for a discrete distribution. (There's a big difference between a discrete distribution and one that's a combination of uniform continuous distributions on intervals.) For example, given just one datum for each n, 1 <= n <= 100, one gets the bar-graph median at x = 50.5. But now say you just replace the single datum of 100 by a datum of 1000. The vertical line bisecting the area under the bar graph is at x = 999.9505 . --Dan

2 1

RE: [math-fun] Relativity Challenge
by dasimov＠earthlink.net 29 Sep '05

29 Sep '05

Thane wrote: << Please pass on by if you're not interested in cranks. <<< Scientists and researchers have scrutinized Einstein's work and have been unable to definitively find anything wrong with it... until now! Best of all, you don't have to have a PhD to understand and verify Einstein's mistake for yourself. All that's required is an understanding of basic Algebra, access to Einstein's papers, and information to guide the way.... >>> http://www.relativitychallenge.com/index.htm >> Actually, Einstein's first paper (more of an appendix) on E = mc^2 -- which I was just looking at yesterday -- contains several errors (as I learned from the footnotes), though these may all be chalked up to typos. Most likely there are similar errors in his other early papers. (These errors in the 1905 E = mc^2 were all found & corrected in print soon after.) The guy behind "Relativity Challenge" seems to have merely caught some of these early typos. --Dan

1 0

[math-fun] Re: Median
by Thomas Colthurst 29 Sep '05

29 Sep '05

I have a general philosophy that descriptive statistics such as the median are only interesting when viewed through the lens of inferential statistics. Thus, averages are important because they are the maximum likelihood point estimates of the center of a Gaussian distribution, and medians are important only so far as the maximum likelihood point estimate of the center of the Laplace distribution pi^(n/2) n Gamma(n) ------------------- e^( - | S^(-1) ( x - mu ) | ) (n/2)! det S is important. In one dimension (n=1), any mu satisfying sum_i | x_i - mu | / ( x_i - mu ) = 0 <=> sum_i sign( x_i - mu ) = 0 will maximize the Laplace distribution's likelihood. For the "hard" cases such as the median of {1, 2} which have a continuum of maximum likelihood point estimates, the use of the "median average" 1.5 is easy to motivate: it is the loss minimizing point estimate for any symmetric cost function. Since this is math-fun and we are talking about medians, here's a question I've had for a while. Say you have n real numbers, x_1 ... x_n. If we define sum_j x_j / | u_i - x_j | u_{i+1} = --------------------------- ( j = 1 .. n ), sum_j 1 / | u_i - x_j | and assuming that u_i is never equal to any x_j, then does the sequence { u_i } always converge to a median of the x_j ? -Thomas C

1 0

[math-fun] Median
by Eugene Salamin 29 Sep '05

29 Sep '05

This idea of fudging the median by introducing some kind of averaging bothers me in a more fundamental way. If a distribution Y exists over a set X, then for the median to be defined, it suffices that X be an ordered set. There is no necessity that X possess an addition, although of course Y must have an addition so that you know when it's split in half. For example, if I define the order: mouse < cat < bear, and I have a mouse and 2 cats, then my median pet is a cat. But if I adopt a bear, what would the median become? Perhaps the pair (cat, bear); that's much more sensible than (cat+bear)/2. Even for distributions over the real line, medians and quantiles are often preferred to averages and standard deviations, since the former always exist while the latter need not. Example: the Cauchy distribution 1/(1+x^2). Suppose one has a distribution p(x), and one wishes to describe it by a single number, a central value, c, of x. Under the criterion of minimizing the mean square error int(p(x) (x-c)^2 dx), assuming the integral exists, we know that c is the mean. But here is something perhaps not as familiar. If instead we wish to minimize the mean absolute error, int(p(x) |x-c| dx), again assuming that the integral exists, then c is the median. Gene __________________________________ Yahoo! Mail - PC Magazine Editors' Choice 2005 http://mail.yahoo.com

1 0