When you toss a fair coin infinitely many times, the frequency of heads will be exactly 1/2 with probability (aka `almost surely’). That’s the strong law of large numbers, and it has a precise mathematical formulation. What this law means is less clear. A sequence of outcomes with different frequency of heads is not logically impossible, so what does it mean that the probability of getting such a sequence is zero ? Is this an assertion about the physical world ? Is there some law of nature that says that the frequency of heads must turn out to be exactly 1/2 ?

The issue of interpretation of probability is puzzling from several aspects. The very fact that mathematical assertions about probability call for interpretation is remarkable: Unlike other creatures that appear in math theorems, such as Hilbert spaces and Lie groups, we don’t view coins and probabilities as exclusively mathematical concepts. When we say that the frequency of heads will be almost surely , we feel that we say something about a situation in which a coin is tossed, but it’s hard to formalize what exactly it is that we say.

I don’t know a satisfactory answer to this perplexity, but in this post I want to sketch one approach — the game theoretic foundations of probability, which I only recently became aware of. I learned it from the works of Shafer and Vovk. This approach says that an event has probability zero iff it is possible to design a gambling strategy that will keep your fortune nonnegative regardless of what happens, and make you infinitely rich if occur. Let me exemplify this for the event `the coin will always land on heads': Suppose that a gambler starts with 1$, and before every coin toss he bets all his money that the outcome will be heads. This gambler will never get into debt — when he loses all his money he stops gambling — and he becomes infinitely rich if the sequence of outcomes is always heads.

Here is the same idea with more latex code (thanks, Luca !): A *gambling strategy* is a function : After observing the finite sequence of outcomes , you bet an amount that the next outcome will be heads. Here is the set of finite sequences of Heads () and Tails (), including the empty sequence. If you start with 1$ and the realized sequence of outcomes of the coin tosses is then after stage you have

dollars. The following theorem was first formulated by Jean Ville

TheoremLet be the probability measure over which corresponds to i.i.d tosses of a fair coin. Then the following conditions are equivalent:

- .
- There exists a gambling strategy such that for every and , and for every .

That the second condition implies the first is an easy martingale argument. The other direction, which is also not difficult to prove, is the new part (at least new for readers like me, who studied probability from the measure theoretic approach that governs in math departments). In a way, the game theoretic approach to probability views martingale as a more fundamental notion than probability: Occurrence of a probability zero event is reduced to the possibility of becoming infinitely rich via fair gambling. Note that in the second condition there is no reference to probabilities — it is based on the notion of gambling strategies, which is assumed to be less conceptually problematic.

## 5 comments

July 14, 2009 at 12:41 pm

Dean FosterActually you can do this using fairly classical probability. Basically you want the conditional distribution P(.|E). This is a measure which will be different from the IID measure. Using it for betting will now generate the desired martingale. The hard part is the good old fashioned measure theory–namely how do you define the conditional probability without having to divide by zero? :-)

July 14, 2009 at 9:37 pm

EranNot sure this works. Suppose for example that E is the set of all sequences that are heads until some point and then always tails, so E is a countable set and intuitively the conditional distribution given E should be uniform on E, but of course there is no such a thing. The good old fashioned measure theory can do miracles when it comes to conditioning given a sub-sigma algebra, but i don’t think you can define a posterior probability given E, for a fixed null event E.

April 4, 2010 at 11:53 am

Jonathan WeinsteinI would try to prove Ville’s theorem as follows: a measure-0 event A can be contained in a union B of dyadic intervals of arbitrarily small total mass \mu(B)=\epsilon. I think it’s easy to design a strategy that makes 1/\epsilon whenever B occurs — namely, your wealth at any moment should be the conditional probability of B divided by \epsilon, a martingale. Now…as you consider a sequence of B’s with \mu(B)->0, can you unify these strategies into the desired one for A, i.e. do they converge?

April 4, 2010 at 9:59 pm

Eranso you describe a strategy with which you can start with some amount of money and if A occurs then at some point you will multiply your wealth by 10 (if epsilon = 1/10). Now, when that happy moment arrives and you find yourself inside one of these cylinders, let’s call it J, you can start all over again: cover the set A with a new union B’ of cylinders such that \mu(B’ | J) < 1/10 and apply the same strategy to multiply your money by 10 again, and then again and again and again and again

April 6, 2010 at 2:07 am

Jonathan WeinsteinNice, that sounds right.