how little do we know



Assume that you have two finite samples X1, X2, ..., Xn and Y1, Y2, ...Ym independently drawn from two distributions. Don't worry we know that both distributions are normal, no fat tails and no other complications. All we want to know is the probability p, given the Xs and Ys, that the two distributions have the same mean. We don't know and we don't care about the variances.

You would think that statisticians have a test ready for this, an algorithm which takes the Xs and Ys and spits out p, and you would think they have figured out the best possible algorithm for this simple question.

You would be wrong. Behrens-Fisher is one of the open problems in statistics.