One reason I like thinking about probability and statistics is that my raw intuition does not fit well with the theory, so again and again I find myself falling into the same pits and enjoying the same revelations. As an impetus to start blogging again, I thought I should share some of these pits and revelations. So, for your amusement and instruction, here are three statistics questions with wrong answers which I came across during the last quarter. The mistakes are well known, and in fact I am sure I was at some level aware of them, but I still managed to believe the wrong answers for time spans ranging from a couple of seconds to a couple of weeks, and I still get confused when I try to explain what’s wrong. I will give it a shot some other post.

** — Evaluating independent evidence —**

The graduate students in a fictional department of economics are thrown to the sharks if it can be proved at significance level that they are guilty of spending less than eighty minutes a day on reading von Mises `Behavioral econometrics’. Before the penalty is delivered, every student is evaluated by three judges, who each monitors the student in a random sample of days and then conducts a statistical hypothesis testing about the true mean of daily minutes the student spend on von Mises:

The three samples are independent. In the case of Adam Smith, a promising grad student, the three judges came up with p-values . Does the department chair have sufficient evidence against Smith ?

**Wrong answer: **Yup. The p-value in every test is the probability of failing the test under the null. These are independent samples so the probability to end up the three tests with such p-values is . Therefore, the chair can dispose of the student. Of course it is possible that the student is actually not guilty and was just extremely unlucky to get monitored exactly on the days in which he slacked, but hey, that’s life or more accurately that’s statistics, and the chair can rest assured that by following this procedure he only loses a fraction of of the innocent students.

** — The X vs. the Y —**

Suppose that in a linear regression of over we get that

where is the idiosyncratic error. What would be the slope in a regression of over ?

**Wrong answer: **If then , where . Therefore the slope will be with being the new idiosyncratic error.

** — Omitted variable bias in probit regression —**

Consider a probit regression of a binary response variable over two explanatory variables :

where is the commulative distribution of a standard normal variable. Suppose that and that and are positively correlated, i.e. . What can one say about the coefficient of in a probit regression

of over ?

**Wrong answer: **This is a well known issue of omitted variable bias. will be larger than . One way to understand this is to consider the different meaning of the coefficients: reflects the the impact on when increases and *stays fixed*, while reflects the impact on when increases without controlling on . Since and are positively correlated, and since has positive impact on (as ), it follows that .

## 2 comments

April 11, 2012 at 12:37 am

Jonathan WeinsteinIn the first one, there is in fact a valid and reasonable test which would throw Smith to the sharks based on your data. Namely, let . I am sure you mean that the test which uses is invalid. This is a cute point.One could avoid your trap by the “reductio ad absurdum” of having *lots* of independent p’s averaging .5, which multiply to something tiny; something must be wrong if you reject someone there.

April 12, 2012 at 1:25 am

edI once wrote up a problem essentially the same as your third example, and put it on the econometrics comprehensive exam at at top Econ department. I don’t think anyone got it completely right.