13. Common discrete distributions

Six discrete distributions come up again and again. Learn their stories — the physical situation each one describes — and a huge amount of applied statistics starts to feel familiar rather than arbitrary.

The skill isn't memorising formulas. It's recognising which story you're in.

Bernoulli — one trial

The atom. A single trial with two outcomes: success (1) with probability p, failure (0) otherwise.

p(1) = p, \qquad p(0) = 1 - p

E[X] = p, \qquad \operatorname{Var}(X) = p(1-p)

We derived both from the MGF two lessons ago. The variance is largest at p = 0.5 — maximum uncertainty — and vanishes at p = 0 or 1, where there's nothing left to be uncertain about.

Story: one coin flip, one click-or-not, one patient recovering.

Binomial — count successes in n trials

Run n independent Bernoulli trials with the same p and count the successes. Literally a sum of n Bernoullis.

P(X = k) = \binom{n}{k}p^k(1-p)^{n-k}, \qquad k = 0, 1, \dots, n

The formula reads straight off the story: p^k(1-p)^{n-k} is the probability of one specific arrangement with k successes, and \binom{n}{k} counts how many such arrangements exist.

E[X] = np, \qquad \operatorname{Var}(X) = np(1-p)

Both follow instantly from summing n independent copies: expectations always add, and variances add because of independence.

Story: 10 coin flips, how many heads? 500 users see an ad, how many click?

Assumptions that matter: fixed n, independent trials, constant p. Break any one and the Binomial is the wrong model.

Poisson — rare events in a fixed window

Counts events in a fixed interval of time or space, when they occur independently at a constant average rate \lambda.

P(X = k) = \frac{\lambda^k e^{-\lambda}}{k!}, \qquad k = 0, 1, 2, \dots

E[X] = \lambda, \qquad \operatorname{Var}(X) = \lambda

Mean and variance are equal. That's the Poisson's signature, and it's a testable prediction: if your count data has variance much larger than its mean, it is overdispersed and Poisson is wrong.

Where it comes from: the Poisson is the limit of the Binomial as n \to \infty and p \to 0 with np = \lambda held fixed — many trials, each very unlikely, with a stable expected count:

\binom{n}{k}p^k(1-p)^{n-k} \longrightarrow \frac{\lambda^k e^{-\lambda}}{k!}

That's why it models "rare events": it is the many-rare-trials limit. It also explains why k has no upper bound — there's no n left to cap it.

Story: emails per hour, typos per page, decays per second, goals per match.

Geometric — wait for the first success

Repeat independent trials until the first success. X = the number of trials up to and including it.

P(X = k) = (1-p)^{k-1}p, \qquad k = 1, 2, 3, \dots

Read it directly: k-1 failures, then a success.

E[X] = \frac{1}{p}, \qquad \operatorname{Var}(X) = \frac{1-p}{p^2}

The mean is pleasingly intuitive — if success has probability 1/6, expect to wait 6 trials.

Memorylessness. The Geometric is the only discrete distribution with

P(X > m + n \mid X > m) = P(X > n)

Having already failed 10 times tells you nothing about how much longer you'll wait. The coin has no memory of your bad luck — this is the precise mathematical refutation of the gambler's fallacy.

Convention warning. Some texts define X as the number of failures before the first success, giving P(X = k) = (1-p)^k p for k = 0,1,\dots and mean \frac{1-p}{p}. SciPy's geom uses the trials-including-the-success version above. The two means differ by exactly 1 — always check which a source means.

Negative Binomial — wait for the r-th success

The Geometric generalised: X = trials up to and including the r-th success.

P(X = k) = \binom{k-1}{r-1}p^r(1-p)^{k-r}, \qquad k = r, r+1, \dots

The \binom{k-1}{r-1} arranges the first r-1 successes among the first k-1 trials. The k-th trial must be a success, so it isn't free to move — that's why the binomial coefficient uses k-1 and not k.

E[X] = \frac{r}{p}, \qquad \operatorname{Var}(X) = \frac{r(1-p)}{p^2}

Set r = 1 and everything collapses back to the Geometric exactly.

Convention warning, again — and this time SciPy disagrees with us. Exactly as with the Geometric, some texts count the failures before the r-th success rather than the total trials, giving E[X] = \frac{r(1-p)}{p}. Unlike the Geometric, SciPy's nbinom uses that other convention, so nbinom.mean(5, 0.3) returns 11.67, not the \frac{r}{p} = 16.67 this lesson's formula gives. The two differ by exactly r — add r to SciPy's answer to get ours. The Python section below does this explicitly rather than letting the mismatch pass silently.

Story: how many sales calls to close 5 deals? It's also the standard overdispersed Poisson — when count data has variance exceeding its mean, the Negative Binomial is the usual replacement.

Hypergeometric — sampling without replacement

The Binomial needs constant p, which requires replacing what you draw. Draw without replacement from a finite pool and the probability shifts after every draw.

From N items of which K are successes, draw n without replacement:

P(X = k) = \frac{\binom{K}{k}\binom{N-K}{n-k}}{\binom{N}{n}}

Pure counting: choose k successes from the K available, n-k failures from the N-K others, out of all ways to choose n from N.

E[X] = n\frac{K}{N}, \qquad \operatorname{Var}(X) = n\frac{K}{N}\left(1 - \frac{K}{N}\right)\frac{N-n}{N-1}

The mean is identical to the Binomial's with p = K/N. The variance is smaller, by the finite population correction \frac{N-n}{N-1} — sampling without replacement is more informative, because you never waste a draw re-examining an item you've already seen.

When N \gg n that correction \to 1 and Hypergeometric \to Binomial. This is why polling 1,000 people out of a billion can be treated as independent.

Story: defective items in a shipment, card hands, capture–recapture.

How they relate

Distribution	Question it answers	Parameters
Bernoulli	Did it succeed?	p
Binomial	How many successes in n?	n, p
Poisson	How many events in a window?	\lambda
Geometric	How long until the 1st success?	p
Negative Binomial	How long until the r-th?	r, p
Hypergeometric	Successes in n draws, no replacement	N, K, n

The connections are worth knowing as well as the definitions:

• Binomial = sum of n Bernoullis
• Poisson = Binomial limit (n \to \infty, p \to 0, np = \lambda)
• Geometric = Negative Binomial with r = 1
• Hypergeometric \to Binomial when N \gg n

One distinction organises the whole table: the Binomial fixes the number of trials and lets the successes vary; the Negative Binomial fixes the successes and lets the trials vary. They are mirror images of each other.

Worked example

A factory produces items with a 2% defect rate. In a batch of 100:

(a) Probability of exactly 3 defects?

Fixed n = 100, constant p = 0.02, independent items — Binomial:

P(X = 3) = \binom{100}{3}(0.02)^3(0.98)^{97} \approx 0.182

(b) Approximate it with a Poisson.

n is large and p small, so take \lambda = np = 2:

P(X = 3) \approx \frac{2^3 e^{-2}}{3!} = \frac{8 \times 0.1353}{6} \approx 0.180

Within 0.002 of exact — and far easier by hand.

(c) Expected number of items inspected before the first defect?

The question is now a waiting time, so it's Geometric with p = 0.02:

E[X] = \frac{1}{0.02} = 50

Same factory, three different distributions — because the question changed, not the process. That's the real skill here.

Doing it in Python

All six live in scipy.stats behind the same interface — .pmf(), .cdf(), .mean(), .var(), .rvs():

from scipy.stats import bernoulli, binom, poisson, geom, nbinom, hypergeom

print("Bernoulli(0.3):      mean", bernoulli.mean(0.3), " var", round(bernoulli.var(0.3), 4))
print("Binomial(100, 0.02): mean", binom.mean(100, 0.02), " var", round(binom.var(100, 0.02), 4))
print("Poisson(2):          mean", poisson.mean(2), " var", poisson.var(2), " (equal!)")
print("Geometric(0.02):     mean", geom.mean(0.02), " var", round(geom.var(0.02), 1))
# nbinom counts FAILURES before the r-th success (mean r(1-p)/p = 11.67), while
# this lesson counts TRIALS including it (mean r/p = 16.67). Differ by exactly r.
print("NegBinom(r=5,p=0.3): mean", round(nbinom.mean(5, 0.3) + 5, 4),
      " <- SciPy's own value is", round(nbinom.mean(5, 0.3), 4), "(failures only)")
print("Hypergeom(50,10,5):  mean", round(hypergeom.mean(50, 10, 5), 4))

The factory example, all three parts:

from scipy.stats import binom, poisson, geom

print("(a) exact Binomial P(X=3) =", round(binom.pmf(3, 100, 0.02), 4))
print("(b) Poisson approx        =", round(poisson.pmf(3, 2), 4))
print("(c) expected wait         =", geom.mean(0.02))

Watch the Poisson approximation tighten as n grows and p shrinks with np = 2 fixed — the limit theorem happening in front of you:

from scipy.stats import binom, poisson

lam = 2
print(f"{'n':>8} {'p':>9} {'Binomial P(X=3)':>18} {'Poisson':>10} {'error':>10}")
for n in (10, 50, 100, 1_000, 100_000):
    p = lam / n
    b, q = binom.pmf(3, n, p), poisson.pmf(3, lam)
    print(f"{n:>8} {p:>9.5f} {b:>18.6f} {q:>10.6f} {abs(b - q):>10.6f}")

And the finite population correction, which is easiest to simply see:

from scipy.stats import binom, hypergeom

n, frac = 10, 0.2
print(f"{'N':>10} {'Hypergeom var':>15} {'Binomial var':>14}")
for N in (20, 100, 1_000, 100_000):
    K = int(N * frac)
    print(f"{N:>10} {hypergeom.var(N, K, n):>15.5f} {binom.var(n, frac):>14.5f}")

Your turn

1. A player makes 70% of free throws. In 10 attempts, what's the probability of exactly 8 makes?

2. A call centre gets 5 calls/hour on average. What's the probability of no calls in a given hour?

3. You're dealt 5 cards from a standard deck. Probability of exactly 2 hearts? Which distribution — and why not the Binomial?

Solutions

1. Fixed 10 attempts, constant 0.7, independent — Binomial:

P(X = 8) = \binom{10}{8}(0.7)^8(0.3)^2 = 45 \times 0.05765 \times 0.09 \approx 0.2335

About 23%. (The single most likely value is 7, since E[X] = np = 7.)

2. Poisson with \lambda = 5; at k = 0 the formula collapses:

P(X = 0) = \frac{5^0 e^{-5}}{0!} = e^{-5} \approx 0.0067

Under 1% — a completely silent hour would be genuinely surprising.

3. Hypergeometric, with N = 52, K = 13 hearts, n = 5 drawn:

P(X = 2) = \frac{\binom{13}{2}\binom{39}{3}}{\binom{52}{5}} = \frac{78 \times 9139}{2598960} \approx 0.2743

Not Binomial, because the draws are neither independent nor constant-p. Before any card, P(\text{heart}) = 13/52 = 0.25; after drawing one heart the next card is a heart with probability 12/51 \approx 0.235. The deck has changed.

The Binomial answer, \binom{5}{2}(0.25)^2(0.75)^3 \approx 0.2637, is close but wrong. It would be fine for a shoe of ten decks, where removing one card barely matters — the N \gg n limit again.

Check yourself in code

Compute the factory example and show the Poisson approximation is close to the exact Binomial.

Print exactly this:

binomial P(X=3) = 0.1823
poisson  P(X=3) = 0.1804
absolute error  = 0.0018
expected wait   = 50.0

Round the three probabilities to 4 decimal places and the wait to 1.

from scipy.stats import binom, poisson, geom

n, p = 100, 0.02
lam = n * p

exact = binom.pmf(3, n, p)
print("binomial P(X=3) =", round(exact, 4))

# Print the Poisson approximation, the absolute error between them,
# and the Geometric expected wait for the first defect.

from scipy.stats import binom, poisson, geom

n, p = 100, 0.02
lam = n * p

exact = binom.pmf(3, n, p)
approx = poisson.pmf(3, lam)

print("binomial P(X=3) =", round(exact, 4))
print("poisson  P(X=3) =", round(approx, 4))
print("absolute error  =", round(abs(exact - approx), 4))
print("expected wait   =", round(geom.mean(p), 1))

---

Bernoulli is one trial. Binomial sums many of them. Poisson is the Binomial's limit when trials are many but rare. Geometric waits for the first success, Negative Binomial for the r-th. And Hypergeometric is the Binomial without replacement.

Next: the same catalogue for continuous variables — including the three distributions that all of hypothesis testing is built on.