Sample spaces, events and the axioms · Probability & Statistics

Every probability you will ever compute rests on just three rules. But first, two pieces of vocabulary: the sample space and the event.

The sample space

The sample space, written \Omega (capital omega), is the set of every possible outcome of an experiment.

Roll a die and the sample space is

$\Omega = \{1, 2, 3, 4, 5, 6\}$

Flip two coins and it's \Omega = \{HH, HT, TH, TT\}. Measure someone's height and it's an interval of real numbers — sample spaces need not be finite.

The one requirement: the outcomes must be exhaustive (something in \Omega always happens) and mutually exclusive (exactly one of them happens).

Events

An event is any subset of the sample space. That's the whole definition.

Take "rolling an even number". That's just the set

$A = \{2, 4, 6\} \subseteq \Omega$

An event is nothing more than a collection of outcomes we've decided to care about. Because events are sets, all the set operations mean something:

Set notation	Reads as	Happens when
A \cup B	A or B	at least one occurs
A \cap B	A and B	both occur
A^c	not A	A does not occur
A \cap B = \varnothing	A, B disjoint	they can't both occur

The three axioms

Here is the entire foundation, due to Andrey Kolmogorov (1933). A probability is a function P assigning a number to each event, such that:

Axiom 1 (non-negativity). For any event A, $P(A) \ge 0$

Axiom 2 (normalisation). Something always happens: $P(\Omega) = 1$

Axiom 3 (additivity). If A and B cannot happen at the same time — they are disjoint — their probabilities add: $A \cap B = \varnothing \implies P(A \cup B) = P(A) + P(B)$

That's it. That is the entire axiomatic foundation of probability theory.

Axiom 3 is stated for countably many disjoint events in full generality (P(\bigcup_i A_i) = \sum_i P(A_i)), which matters for infinite sample spaces. We'll come back to why "countably" is exactly the right word in §9.

Everything else is a consequence

Notice how little was assumed. Every familiar fact now has to be derived.

The impossible event has probability zero. \Omega and \varnothing are disjoint and their union is \Omega, so by Axioms 2 and 3, 1 = P(\Omega) = P(\Omega) + P(\varnothing), hence $P(\varnothing) = 0$

The complement rule. A and A^c are disjoint and cover \Omega: $P(A^c) = 1 - P(A)$

This is worth more than it looks. "At least one" problems are usually far easier as 1 - P(\text{none}).

Monotonicity. If A \subseteq B, then B splits into the disjoint pieces A and B \cap A^c, so P(B) = P(A) + P(B \cap A^c) \ge P(A): $A \subseteq B \implies P(A) \le P(B)$

A consequence: no probability can exceed 1, since every A \subseteq \Omega.

The inclusion–exclusion rule. Axiom 3 needs disjointness. When events overlap, adding them double-counts the overlap, so subtract it once: $P(A \cup B) = P(A) + P(B) - P(A \cap B)$

Worked example

Let's check that last rule on a fair die, where every outcome has probability 1/6.

Let A = \text{"even"} = \{2, 4, 6\}, so P(A) = 3/6. Let B = \text{"greater than 3"} = \{4, 5, 6\}, so P(B) = 3/6.

Their overlap is A \cap B = \{4, 6\}, so P(A \cap B) = 2/6.

By inclusion–exclusion:

$P(A \cup B) = \frac{3}{6} + \frac{3}{6} - \frac{2}{6} = \frac{4}{6}$

Check it directly: A \cup B = \{2, 4, 5, 6\}, which is 4 outcomes out of 6. It matches.

Had we naively added 3/6 + 3/6 = 1, we'd have claimed the event was certain — but rolling a 1 or a 3 falsifies that immediately. The overlap \{4, 6\} was counted twice, once in each set.

Doing it in Python

Sets in Python are close enough to the mathematics that you can write the axioms almost literally. For a finite sample space with equally likely outcomes, probability is just relative size:

omega = {1, 2, 3, 4, 5, 6}

def P(event):
    """Classical probability on a finite, equally likely sample space."""
    return len(event & omega) / len(omega)

A = {2, 4, 6}          # even
B = {4, 5, 6}          # greater than 3

print("P(A)        =", P(A))
print("P(B)        =", P(B))
print("P(A and B)  =", P(A & B))
print("P(A or B)   =", P(A | B))
print("inclusion-exclusion:", P(A) + P(B) - P(A & B))
print("complement: P(not A) =", P(omega - A), "= 1 - P(A) =", 1 - P(A))

Because &, | and - are genuine set operations, this doubles as a way to check yourself: compute an event two ways and see whether they agree.

Your turn

1. Two fair coins are flipped. Write down \Omega, then find the probability of at least one head — twice, once by counting and once with the complement rule.

2. In a group, P(\text{speaks Hindi}) = 0.7, $P(\text{speaks Tamil}) = 0.4$, and P(\text{both}) = 0.2. What fraction speaks neither?

3. Someone claims P(A) = 0.6, P(B) = 0.5 and P(A \cap B) = 0.1. Is that possible? What if they said P(A \cap B) = 0.05?

Solutions

1. \Omega = \{HH, HT, TH, TT\}, four equally likely outcomes.

By counting: "at least one head" is \{HH, HT, TH\}, so P = 3/4.

By the complement: the opposite of "at least one head" is "no heads at all", which is the single outcome TT. So

$P(\text{at least one H}) = 1 - P(TT) = 1 - \tfrac{1}{4} = \tfrac{3}{4}$

Both give 3/4. With two coins the counting is easy; with twenty coins the complement is the only sane route — "at least one head" has 1,048,575 outcomes and "no heads" still has exactly one.

2. First the union, by inclusion–exclusion:

$P(H \cup T) = 0.7 + 0.4 - 0.2 = 0.9$

"Neither" is the complement of "at least one":

$P((H \cup T)^c) = 1 - 0.9 = 0.1$

So 10% speak neither language. Note that adding 0.7 + 0.4 = 1.1 > 1 is not a contradiction — it just proves the two events must overlap, which they do.

3. The first is fine. Check the union: 0.6 + 0.5 - 0.1 = 1.0 \le 1, and the overlap 0.1 is no bigger than either event. Consistent.

The second is impossible. It would give

$P(A \cup B) = 0.6 + 0.5 - 0.05 = 1.05 > 1$

which violates monotonicity (A \cup B \subseteq \Omega, so its probability can't exceed P(\Omega) = 1).

There's a general constraint hiding here. Since P(A \cup B) \le 1,

$P(A \cap B) \ge P(A) + P(B) - 1$

For these numbers the overlap must be at least 0.1 — two events covering 60% and 50% of the space simply cannot avoid each other.

Check yourself in code

Build the sample space for rolling two dice and verify inclusion–exclusion on it. Let A be "the sum is even" and B be "the sum is greater than 8".

Print exactly this:

P(A) = 0.5
P(B) = 0.2778
P(A and B) = 0.1111
P(A or B) = 0.6667
matches: True

Round every probability to 4 decimal places, and print matches: True if the directly counted P(A \cup B) equals the inclusion–exclusion value.

from itertools import product

omega = set(product(range(1, 7), repeat=2))   # 36 equally likely pairs

def P(event):
    return len(event) / len(omega)

A = {(i, j) for (i, j) in omega if (i + j) % 2 == 0}
B = {(i, j) for (i, j) in omega if i + j > 8}

print("P(A) =", round(P(A), 4))
# print P(B), P(A and B), P(A or B), then compare the two routes

from itertools import product

omega = set(product(range(1, 7), repeat=2))   # 36 equally likely pairs

def P(event):
    return len(event) / len(omega)

A = {(i, j) for (i, j) in omega if (i + j) % 2 == 0}
B = {(i, j) for (i, j) in omega if i + j > 8}

print("P(A) =", round(P(A), 4))
print("P(B) =", round(P(B), 4))
print("P(A and B) =", round(P(A & B), 4))
print("P(A or B) =", round(P(A | B), 4))
print("matches:", round(P(A | B), 4) == round(P(A) + P(B) - P(A & B), 4))

Three simple rules: never negative, totalling one, additive when disjoint. Every other fact in probability theory is built out of them.

Next: how to count the outcomes in the first place, when there are far too many to list.

3. Sample spaces, events and the axioms