2. What is probability?
Here is a number: 0.7. And here is a claim — the probability of rain tomorrow is 0.7. Everyone nods along. But what, precisely, does that number mean?
It is not obvious. Tomorrow happens exactly once. It will either rain or it will not. So what is being measured, and by what?
Before any axiom, before a single formula, this course owes you an answer. There are three, they are genuinely different, and each one fails somewhere the others hold.
1. The classical reading: count the outcomes
The oldest answer is the simplest. Count the outcomes you want, divide by the total:
P(A) = \frac{\text{number of outcomes in } A}{\text{total number of outcomes}}
A die has six faces, three of them even, so P(\text{even}) = 3/6 = 1/2.
This is clean, and it built the subject — Cardano, Pascal and Fermat were all essentially counting. But it smuggles in an assumption: that the outcomes are equally likely. Load the die, and the counting still returns 3/6, and the answer is now simply wrong.
Worse, the assumption is circular. "Equally likely" means "of equal probability" — so this definition of probability quietly uses the word probability inside itself.
And it has nothing whatever to say about rain tomorrow, which has no symmetric faces to count.
2. The frequentist reading: repeat the experiment
The second answer throws out symmetry and replaces it with repetition. Flip a coin many times and watch the running fraction of heads. It lurches about at first, then settles down:
P(A) = \lim_{n \to \infty} \frac{\text{number of times } A \text{ occurred}}{n}
This is a genuinely different idea, and it's the one most working scientists carry around. It needs no symmetry at all — only an experiment you can repeat. Load the die as heavily as you like; roll it ten million times and the frequencies will tell you the truth about it.
But look at the cost. Probability is now a property of a repeatable experiment, defined by a limit you can never actually reach. And it still says nothing about rain tomorrow — because tomorrow is not repeatable. There is no long run.
3. The subjective reading: how much would you bet?
The third answer gives up on the world and turns to look at you. A probability is a degree of belief, made precise by the bets you would accept.
If you would pay 70 cents for a contract that pays $1 if it rains, then your probability of rain is 0.7.
And this works for anything — a horse race, an election, a defendant's guilt, a theory being true. One-off events included.
The obvious objection is that it's personal: two reasonable people can hold different beliefs about the same event. But it is not arbitrary. If your degrees of belief break the rules, someone can offer you a collection of bets that you accept one by one, and that lose you money collectively, whatever happens.
The Dutch book
Suppose you claim P(\text{rain}) = 0.7 and P(\text{no rain}) = 0.5. Those sum to 1.2, which "feels" merely sloppy. Watch what it costs you.
By your own prices you'll pay 70c for a contract paying $1 on rain, and 50c for one paying $1 on no rain. I sell you both, and collect $1.20.
- If it rains, I pay you $1.
- If it doesn't rain, I pay you $1.
Either way I pay $1 and keep 20c. Not on average — every single time, whatever the weather does. You've been Dutch booked.
The only way to be immune is for your prices on a set of mutually exclusive, exhaustive outcomes to sum to exactly 1. Avoiding a sure loss forces your beliefs into precisely the shape probability requires.
The striking part: they all obey the same rules
Three very different answers — counting, frequency, belief — and all three obey the same three rules:
- A probability is never negative.
- Something must happen, so the total is 1.
- For outcomes that cannot occur together, probabilities add.
That is what Kolmogorov did in 1933. He stopped asking what probability is, and wrote down instead how it behaves. The axioms are deliberately silent about meaning, and that silence is exactly why they work for everybody.
So the argument about interpretation never touches the arithmetic. A frequentist and a Bayesian compute conditional probabilities identically. What they disagree about is which things may be assigned a probability at all — and that argument surfaces much later, in statistical inference (§7). The mathematics ahead of you is common ground.
Worked example
Which reading is doing the work in each statement?
"The probability this fair coin lands heads is 1/2." Classical. Two outcomes, assumed symmetric, count them. You could also defend it as frequentist, and the two agree — which is exactly why coins are the example everyone reaches for.
"The probability a component fails within 1000 hours is 0.03." Frequentist. It comes from testing many components and recording the fraction that failed. There is no symmetry to count, but there is a repeatable experiment.
"The probability this startup succeeds is 0.15." Subjective, necessarily. This startup runs once. There are no equally likely faces and no long run. The number encodes what an investor believes, and it is testable only in the sense that consistently bad numbers lose money.
"The probability of rain tomorrow is 0.7." Subjective in form, but a good forecaster's number is calibrated against frequency: across all the days they said 0.7, it should rain about 70% of the time. This is the useful hybrid in practice — a degree of belief, disciplined by a track record.
Doing it in Python
The frequentist claim is one you can watch happen. Flip a coin and track the running fraction of heads:
import random
random.seed(0) # so the run is reproducible
heads = 0
for n in range(1, 100_001):
heads += random.random() < 0.5
if n in (10, 100, 1_000, 10_000, 100_000):
print(f"after {n:>7,} flips: {heads / n:.4f}")
The fraction lurches early and settles late. Note what the frequentist definition actually claims: not that the fraction reaches 0.5, but that it converges to it. We'll make "converges" precise in §3, where this becomes the Law of Large Numbers.
Now the Dutch book. Coherence is just a sum:
# Your stated prices for tomorrow's weather (mutually exclusive, exhaustive).
prices = {"rain": 0.7, "no rain": 0.5}
total = sum(prices.values())
print("your prices sum to", round(total, 2))
# A bookmaker sells you every contract, collects `total`, and pays out exactly
# $1 whichever outcome occurs.
print("bookmaker's guaranteed profit:", round(total - 1, 2))
Your turn
1. A die is loaded so that 6 comes up half the time and the other five faces share the rest equally. What does the classical definition say P(\text{even}) is? What is it really?
2. Someone offers prices of 0.4 on "India wins", 0.4 on "England wins" and 0.1 on "draw" for a match with no other outcome. Can you Dutch book them, and in which direction?
3. Why can't the frequentist reading assign a probability to "there is life on Europa"?
Solutions
1. The classical definition counts three even faces out of six and returns 3/6 = 1/2. It cannot do anything else — counting can't see the loading.
Really: P(6) = 1/2, and the remaining 1/2 is split equally among five faces, so each of 1–5 has probability 1/10. Then
P(\text{even}) = P(2) + P(4) + P(6) = \tfrac{1}{10} + \tfrac{1}{10} + \tfrac{1}{2} = 0.7
The classical answer is off by 0.2. This is the precise sense in which the equally-likely assumption isn't a technicality — it's the whole content.
2. The prices sum to 0.4 + 0.4 + 0.1 = 0.9, which is less than 1, so the Dutch book runs the other way: you buy nothing, you sell. Buy all three contracts for 90c total; exactly one pays out $1. You profit 10c whatever happens.
The general rule: prices summing to more than 1 let someone profit by selling you everything; summing to less than 1 let someone profit by buying everything. Only exactly 1 is safe. (Real bookmakers deliberately price above 1 — the excess is their margin, called the overround.)
3. Because there is no repeatable experiment. Europa exists once; there is no ensemble of Europas to sample, and no long-run frequency to converge to. A strict frequentist must say the question is not one probability can answer.
A subjectivist has no such trouble — the number describes a state of belief given current evidence, and it updates when a probe returns data. This is exactly the divide that resurfaces in §7 as Bayesian vs. frequentist inference: it is not a disagreement about arithmetic, but about what is permitted to have a probability.
Check yourself in code
Write a coherence checker. Given a dictionary of stated prices for mutually exclusive, exhaustive outcomes, print the total and the verdict.
Print exactly this:
total 0.90
incoherent: buy all contracts for a sure 0.10
Round both numbers to 2 decimal places. Print coherent if the total is
exactly 1; if the total is below 1 print the buy line as above; if above 1
print incoherent: sell all contracts for a sure X instead.
prices = {"India": 0.4, "England": 0.4, "draw": 0.1}
total = sum(prices.values())
print("total", f"{total:.2f}")
# Decide the verdict and print the second line.
prices = {"India": 0.4, "England": 0.4, "draw": 0.1}
total = sum(prices.values())
print("total", f"{total:.2f}")
if total == 1:
print("coherent")
elif total < 1:
print(f"incoherent: buy all contracts for a sure {1 - total:.2f}")
else:
print(f"incoherent: sell all contracts for a sure {total - 1:.2f}")
Probability, three ways: a ratio of equally likely cases, a long-run frequency, or a coherent degree of belief. Each fails somewhere the other two don't.
What they share is their behaviour: never negative, totalling one, additive over outcomes that can't happen together. That shared behaviour is the entire foundation of the subject.
Next: the vocabulary those rules are written in — sample spaces, events, and the three axioms in full.