Really Stupid Statistics, Chapter MMMDCCC

During last night’s NBA finals game, the announcer stated that Manu Ginobli has a better career winning percentage than the other members of the Spurs’ “Big Three” (Duncan, Parker, Ginobli.) This means, I presume, that the Spurs have won a higher percentage of games in which Ginobli plays than games in which Duncan plays, or games in which Parker plays. If so, this is a Really Stupid Statistic. I will leave the reason “as an exercise” and assume a commenter can explain why; if not, I’ll post tomorrow.

Game Theory at the Polymath Project

The “polymath project” is a blog where serious unsolved problems in math are attacked in a massively collaborative way. A problem is posted and hundreds of users suggest lines of attack. Of course, a lot of the progress is made by a few superstars, but it is still a cool idea. The cynical economist in me must point out that while the supposed m.o. of the project is to harness the beauty of collaboration and cooperation in the age of the web, surely the fuel behind the engine is the competitive instincts of the participants. Nothing wrong with that, though; if not for the competitive drive we might still all live in the jungle.

Also, once a year they run a “mini-polymath” project where they attack a problem from the International Math Olympiad. Of course this is very different from an unsolved problem; everyone knows there is a solution that can reasonably be found by a single person in an hour or two. Still a fun idea. This year’s problem, posted yesterday, is phrased as a game. I’ve played with it for a few minutes and I suspect that knowledge of specific theorems or concepts in game theory will not be useful, though of course the habit of thinking strategically will. (Rubinstein would say that this is true of most real-life applications of game theory as well.) Try your hand, or look at the comments, which surely have spoilers by now as it has been up for about a day.

Prediction Markets and The Health Care Ruling

This column by David Leonhardt cites the recent Supreme Court ruling as an example showing that prediction markets aren’t so great. After all, Intrade was putting the chances of the individual mandate being found unconstitutional at 75%.

But concluding anything from this about the prediction market being “wrong” is, well, wrong. If *everything* that Intrade said would happen with probability 75% indeed happened, *that* would be a clear sign of a market inefficiency. If only half of such events happened, that would constitute an inefficiency also. But a single event with market probability .75 failing to happen is no more a sign that the market is inefficient or “wrong” than the Yankees losing a game to the Royals, however much more important the event may be.

What should we expect from Intrade? Barring insider trading, we can expect a reasonably efficient aggregation of publicly available information. If there is little public information, we will see probabilities that tend more towards .5 than to the extremes, and they will often be “wrong” if one defines this as being on the wrong side of .5, but non-extreme predictions are supposed to be “wrong” a substantial minority of the time. Of course, if you were able to seduce a Supreme Court clerk, you might have been able to make a better prediction. This doesn’t expose a market flaw any more than the potential profits from insider trading expose a flaw in the conventional stock market.

Roberts’ vote was no more predictable to an outside observer than a baseball pitcher’s unexpectedly good or bad game, however many reasons we may find for it ex post. Unpredictable things happen. This is why the probability was .75 and not 1. Many pundits acted as if striking down the law was near-certain, so I would say the market showed some wisdom in placing the probability at only .75.

Desperate Times?

Last night, with about 8 minutes to go in the Indiana-Miami NBA playoff game, Danny Granger for Indiana picked up his 5^th foul. Loyal readers will know that I was rooting hard for Granger to be left in (especially since, like so many fans, I would like nothing better than for Miami’s stars to take their talents home for a long offseason.) This time, my hope was actually fulfilled as the coach did not substitute. Just when the evening appeared to be a triumph for rationality, announcer Mike Breen said “Well, you have to leave him in…it’s an elimination game” (meaning a loss would eliminate Indiana.) Now, finding inane comments by sportscasters is like shooting garrulous fish in a barrel, but I think there are some common and important fallacies at work here, so let’s dissect why Breen would have said what he did. As in most fallacious thinking, he was applying reasoning which would apply in closely related situations, but not here.

1. Desperate times call for desperate measures. In some situations, this intuition can be invaluable. If you are down 3 points with time expiring, you had better try a 3. But it only applies to the state of the series if decisions have spillover effects from game to game. Now if we were talking about players becoming injured, or extremely fatigued, there might be spillover effects to the next game, and then you actually should consider the state of the series. But not for fouls (in basketball, unlike soccer.) Unless your decision affects the next game, you play to maximize your chance of winning the current game, whether ahead or behind in the series. The proverbial “one game at a time” really does apply here.

2. Leaving a player in with 5 fouls is a risky move. It’s easy to think this way, but wrong. I won’t rehash my earlier post. Incidentally, the argument I posted here appeared (independently, apparently) in the book Scorecasting, an enjoyable compendium of insights that go against sports conventional wisdom.

Krugman on Hypocrisy

I am a big fan of Paul Krugman, but I think he missed something important in today’s column.

First, let’s go back to November. In this blog post, Krugman pointed out correctly that it is silly to perceive an inconsistency or “hypocrisy” when wealthy individuals promote progressive taxation. If Warren Buffett believes that the good of the nation is against his self-interest, this is civic-minded unselfishness, the furthest thing from hypocrisy. Furthermore, the belief that the wealthy should pay more should never be confused with hatred of the wealthy, or self-hatred; one does not imply the other.

Today, Krugman points out a supposed contradiction between (a) conservative opposition to the social safety net and (b) the fact that social programs send more benefits to red states (which tend to be poorer) than blue states. What, conservatives can’t be civic-minded? Just as it is not inconsistent for Mr. Buffett to fail to volunteer higher tax payments, it is not inconsistent to accept benefits while believing they shouldn’t exist. If I find myself playing Monopoly with the horribly misguided house rule that money is placed under Free Parking, am I morally compelled to refuse the money, on the grounds that I believe with all my heart it shouldn’t be there? Clearly, no. It would be hypocritical to complain about other players’ accepting the Free Parking money while I do so myself, but not hypocritical to advocate that the rule itself be changed so that no one receives money including me.

(Side note: One shouldn’t actually conclude from a state-by-state correlation that the individuals receiving benefits are the ones opposing them. It is equally possible, based only on such data, that seeing one’s red-state neighbors receive benefits leads one to oppose them.)

Finally, while I think failing to note the analogy with his previous post was a big omission today, I do think that other evidence in the column tends to recover Krugman’s point. He wants to show that support for conservative austerity measures is based not on principle and willingness to fairly share sacrifice, but on perceived self-interest, which is in some cases misperceived. The column cites a study by Suzanne Mettler stating that over 40% of those receiving benefits from Social Security, unemployment and Medicare believe they “have not used a government program.” This does suggest the possibility that some conservatives are opposed to such programs only “for other people.” A lack of self-awareness can facilitate the cloaking of self-interest in a purported principled stand, which would indeed be hypocrisy. In contrast, Mr. Buffett could hardly miss the fact that a “Buffett tax” would fall heavily on him.

The Muddle of the Mode

It is well-known that a non-linear transformation of a random variable does not transform the mean in an entirely straightforward way. That is, for a random variable X and function f, we can easily have . In our intro decision science courses, we call this the “flaw of averages,” a term coined by Sam Savage. See his book of that title for many examples of how one can, often inadvertently, fall into the false assumption that it suffices to replace a random variable by its average.

What if instead of the average, we talk about the mode, or most likely outcome? Denote this M(X). Surely, if f is a one-to-one function, the most likely value of f(X) must be f(M(X))? Amazingly, this can be false as well! It is much “closer to true,” in that it is true for all discrete distributions, but we run into trouble with continuous distributions. Here is an example:

Let X follow a standard normal distribution, and let . The distribution of Y is called “lognormal.” Here is a graph of its density:

Notice that while the standard normal distribution is peaked at 0, this distribution is not peaked at ! What is going on?

We can work this out with algebra and calculus, but here is a conceptual way of looking at it. The key is that probability densities differ fundamentally from probabilities, and the precise definition of the mode is different for continuous than discrete distributions. Saying that 0 is the “most likely” value for a standard normal variable X isn’t quite right. Any particular exact value, of course, has probability zero. What we really mean is that X is more likely to be near 0 than near any other value. Fine, so then why isn’t more likely to be near 1=exp(0) than any other value? Because the exponential transformation does funny things to “near.” Nearby values get stretched out more for larger X. So, while more realizations of X are near 0 than near -1, they are spread out more thinly when we exponentiate, so that the maximum density of Y occurs not at exp(0) but at exp(-1). It takes some algebra to find the exact value, but this argument makes it fairly clear it should be less than exp(0).

Why is this important? One reason is that in regression analysis we often use a model which predicts ln y rather than y. Then we need to convert the predicted value for ln y to a predicted value for y. It is well-known (and has been taught for years in our intro stats course) that if we are predicting the average y, it does not suffice to exponentiate; a correction must be made. But for predicting an individual observation of y, we all teach that no correction is necessary. It only occurred to me last week, after teaching the course for 6 years, that this is problematic if we seek the “most likely” value of y.

What is the resolution? First of all, there is no distortion for the median. If k is the point prediction for ln y, then we can conclude that y has a 50% chance of being above exp(k). So the method we have been teaching is a fine way to estimate the median value of y. Our lectures and textbook haven’t really said in precise language whether we are predicting a median or modal value, so I am glad to report we haven’t been teaching anything unambiguously wrong. Secondly, the problem goes away if we work with prediction intervals rather than single-value predictions, as we often encourage students to do. If we are 95% confident that ln y is in [a,b], we can certainly conclude that we are 95% confident y is in .

Most importantly, we should reinforce the lesson from the “flaw of averages” that any single number – mean, median or mode – is a poor summary of our knowledge of a random variable. This is especially true for a variable that is lognormal (or any asymmetric distribution) rather than normal, in which case all three values are usually different.

Postscript: To learn more about “the muddle of the mode,” here are two basic PhD-level exercises:

1. Show “Reverse Jensen’s inequality for modes”: If f is an increasing convex function, X is a continuous random variable, and both X and f(X) have a unique mode (point of maximum density), then mode(f(X)) <= f(mode(X)) . If f is strictly convex and X has continuously differentiable density, the inequality is strict.

2. (Based on a suggestion from Peter Klibanoff.) Let X have a continuously differentiable density and a unique mode, and Y=exp(X). Define the density of Y “on a multiplicative scale” by

Show that g is maximized at exp(mode(X)). Note that the above formula is similar to the standard density, but with having replaced . That is, if we consider Y to be measured on a multiplicative scale, with multiplicative errors, there is no distortion in the mode.

Must Polls be Plurality-Based, Too?

There are many ways to hold an election with more than two candidates. For most major elections in this country, we are stuck with perhaps the worst, the familiar plurality voting, in which each voter picks one candidate and the highest vote total wins. If you are unfamiliar with the deficiencies of plurality voting, one place to look is this excellent blog post, written by mathematician Tim Gowers in the context of Britain’s recent (sadly unsuccessful) referendum to change their system. If this seems like a mere technical issue, one should look here at a notorious historical election in which the winning party carried 33% of the vote.

Plurality voting is especially bad in a crowded field where no single candidate has strong support, as in the current Republican presidential primary race. We open our daily paper and see that Herman Cain is a front-runner with a whopping 25% of the vote, but cannot tell if he is the second choice or last choice of the other 75%. This surely has an impact on his chances of success, both as the primary field narrows and in the general election.

Now, changing an electoral system is a very difficult thing, not least because those in power got there via the current system. But what of polls? These do not suffer from any similar institutional inertia, and there is no monopoly on polls; Gallup, Rasmussen, etc. are each free to ask whatever questions they see fit. Surely we would get a better picture of the mood of the electorate by, at minimum, asking each pollee to approve/disapprove of each candidate, or, better still, rank each candidate on a numerical scale of 1 to 5 or 10, or simply rank all candidates in order of preference. No single method is perfect, so some variety might be ideal. Since polls have an impact on real elections, improving polls might reduce some of the idiosyncrasies that arise from plurality voting. Of course, it would also be the responsibility of attention-span-deficient news outlets to report not just the familiar plurality-based polls, but also those which give a more detailed picture of the electorate’s preferences.

Disagreement Point

In bargaining theory, a “disagreement point” or “threat point” is the policy which is implemented if no agreement is reached. Typically, it is bad for both sides, but may be worse for one. The disagreement point has a profound impact on the outcome of negotiations, even if it never comes to pass. (In theory-land, say in Nash or Rubinstein bargaining, there is never disagreement, but the threat of disagreement is a crucial determinant of the outcome.)

 

Obviously, in our government’s current situation, the disagreement point is a shutdown. TV pundits have been speculating for weeks about which side this hurts more. But this isn’t the only imaginable disagreement point. The system could decree that, say, last year’s budget gets extended automatically if no new budget is passed. This would drastically impact the negotiations (favoring the Democrats in this particular case.) My question to anyone who knows the history of government better than I do is: How did we come to have a system where the government shuts down unless a new budget is passed each year? And do other countries differ on this point?

Fun with Alice and Bob

This puzzle, which appeared on the blog associated with the cute webcomic xkcd, would make a good exercise in a game-theory course:

Alice secretly picks two different real numbers by an unknown process and puts them in two (abstract) envelopes.  Bob chooses one of the two envelopes randomly (with a fair coin toss), and shows you the number in that envelope.  You must now guess whether the number in the other, closed envelope is larger or smaller than the one you’ve seen.

Is there a strategy which gives you a better than 50% chance of guessing correctly, no matter what procedure Alice used to pick her numbers?

A good start is to notice that the possibility that Alice may randomize is a red herring. You need a strategy with a greater than 50% chance of winning against any pair she may choose. If you can do this, it takes care of her randomized strategies for free. (A familiar argument to those who enjoy computing values of zero-sum games.)

Reasoning too Coarsely

“Everything should be made as simple as possible, but no simpler.” – Einstein.

This is one of my favorite maxims. It is also a very difficult rule to follow, giving it a certain humor which was probably not lost on Einstein. How simple is too simple? Well, if adding just a little more detail reverses the analysis completely, it was too simple. Here’s an example from this weekend’s diversion, the NFL playoffs.

With 4:08 left in their game against the Packers, the Eagles faced 4^th and goal, inches from the goal line. They trailed by 11. To win, they would either need to score a touchdown with two-point conversion and field goal (in either order), then win in overtime, or score two touchdowns. Eagles coach Andy Reid called timeout to think it over. What should he have done?

Announcer Troy Aikman argued that they should kick the field goal, saying that “you have to cut it to a one-possession game.” This is a classic example of what decision theorists call coarse decision-making. Aikman knows that it’s better to cut it to 5 or 3 than 8, but he simplifies the decision by considering 3, 5 and 8 to be similar to one another (in each case, one possession may suffice to tie or win) while they are very different from 11. Is his analysis good enough? No, it turns out he made it too simple, and in the wrong way.

Let me simplify it differently. I won’t even need the language of probability; I can put it purely in terms of verbal logic, perhaps a kind you could even explain on TV (almost). First, assume that if we score a touchdown now, the 2-point conversion will fail and the deficit will be 5 (this goes against the decision I’m advocating, so it’s an acceptable simplification.) Now, since we are going to be down at least 5, assume that we will get the ball back and score a touchdown; otherwise, our decision now is irrelevant. This makes it very easy to see which option is best right now:

 

If we kick a field goal, victory will depend on later converting the 2-pointer (from the 2-yard line), then winning in overtime.

If we go for it now, victory will depend only on making it now from inches away.

 

Would you rather try to score from 2 yards out for a tie, or try to score from inches away for a win? Looked at this way, it’s a total no-brainer. And of course, I ignored the fact that you might cut it to 3 if you score now, which gives you an even better chance to win. Andy Reid got this decision right (though one colleague, an Eagles fan, is harsh on him for needing a timeout to think about it, I find this forgivable.) The Eagles indeed scored a touchdown to cut the lead to 5, then were in position to go for the win later but were thwarted by an interception in the end zone.

What went wrong with Aikman’s simplification, or “coarsening”? One way to look at it is that he lumped the wrong scores together. His way of thinking comes very naturally, especially since one-possession vs. two-possession corresponds roughly to alive vs. dead, and this is how we tend to categorize probabilities. But in terms of win probability, an 8-point deficit is actually closer to 11 than it is to 5! If t is the probability of scoring a later touchdown, then if the deficit after this possession is

5, the probability of winning is t

8, the probability of winning is (roughly) .25t

11, the probability of winning is roughly 0.

This is based on a 50/50 chance at a two-point conversion (published figures vary) and a 50-50 chance at winning in overtime. These figures mean that it would be worthwhile to try for a touchdown even if it only worked 25% of the time (the actual figure is over 70%), and even if you never successfully cut the deficit to 3. It turns out it would be better to think of an 8-point deficit as “not quite dead” rather than “alive.”

This situation also relates to my previous entry “Risky Move.” In principle, all strategies are risky, and you should pick the one with the best chance of success. In practice, people think of the decision which resolves more of the uncertainty immediately as “risky,” and are biased against this decision. They sometimes then choose a slower, but more certain, death.