This column by David Leonhardt cites the recent Supreme Court ruling as an example showing that prediction markets aren’t so great. After all, Intrade was putting the chances of the individual mandate being found unconstitutional at 75%.
But concluding anything from this about the prediction market being “wrong” is, well, wrong. If *everything* that Intrade said would happen with probability 75% indeed happened, *that* would be a clear sign of a market inefficiency. If only half of such events happened, that would constitute an inefficiency also. But a single event with market probability .75 failing to happen is no more a sign that the market is inefficient or “wrong” than the Yankees losing a game to the Royals, however much more important the event may be.
What should we expect from Intrade? Barring insider trading, we can expect a reasonably efficient aggregation of publicly available information. If there is little public information, we will see probabilities that tend more towards .5 than to the extremes, and they will often be “wrong” if one defines this as being on the wrong side of .5, but non-extreme predictions are supposed to be “wrong” a substantial minority of the time. Of course, if you were able to seduce a Supreme Court clerk, you might have been able to make a better prediction. This doesn’t expose a market flaw any more than the potential profits from insider trading expose a flaw in the conventional stock market.
Roberts’ vote was no more predictable to an outside observer than a baseball pitcher’s unexpectedly good or bad game, however many reasons we may find for it ex post. Unpredictable things happen. This is why the probability was .75 and not 1. Many pundits acted as if striking down the law was near-certain, so I would say the market showed some wisdom in placing the probability at only .75.
4 comments
July 7, 2012 at 11:28 pm
Jonathan Weinstein
I just saw that Robin Hanson posted something along my lines earlier today, but with a meaner subject heading (should I be taking lessons in how to attract attention in the blogosphere?): http://www.overcomingbias.com/2012/07/leonhardt-blows-it.html
July 8, 2012 at 12:11 am
Gobanian (@Gobanian)
it’s nonsense to claim the market “offered some wisdom” in putting the probability at 0.75. Because you could make the same arguments in defence of markets for any number. For example, “if nothing with a 10% probability on Intrade happened, that would clearly be wrong.”
All we can say is that one event doers not allow us to judge.
July 8, 2012 at 12:23 am
Jonathan Weinstein
I didn’t mean to imply that .75 was an especially brilliant number. In this particular case, it was (ex post) better than any higher number, and worse than any lower number, and reasonable independent observers could have produced either. I hope the earlier sections in the post made it clear (as you point out) no one should judge anything either way by one prediction. Perhaps the final sentence should have read more like “As easily as one could claim that .75 was a bad prediction, I could claim it was a good prediction compared to the punditocracy. Everything is relative. But even better would be to make no claims about a single prediction.”
July 8, 2012 at 8:17 pm
rvohra
Dear J.
There was another irritating line that column:
My colleague Nate Silver, whose book on prediction comes out later this year, has found that a simple average of well-known economic forecasts is substantially more accurate than individual forecasts.
It gives to Silver the credit for something that has been observed and noted for at least half a century by others and formalized by Hannan and Blackwell among others.