58. Convergence theorems

The question this section has been building toward:

\lim_{n\to\infty}\int f_n\,d\mu \;\overset{?}{=}\; \int \lim_{n\to\infty}f_n\,d\mu

When can you swap a limit and an integral? Equivalently, when does E[X_n] \to E[X] follow from X_n \to X?

Not always — and the three theorems that say when are the workhorses of probability theory.

It really can fail

Take f_n = n\,\mathbb 1_{(0, 1/n)} on [0,1]: a spike of height n on a window of width 1/n.

Pointwise, f_n \to 0 everywhere. Fix any x > 0; once n > 1/x the spike has moved past it, so f_n(x) = 0 from then on. And f_n(0) = 0 always.

But every integral is 1:

\int_0^1 f_n\,d\lambda = n \times \frac{1}{n} = 1 \quad \text{for every } n

\lim_n \int f_n = 1 \;\ne\; 0 = \int \lim_n f_n

The mass escapes. It doesn't vanish; it gets squeezed into an ever-narrower window while growing taller. The area is conserved even as the function collapses pointwise to zero.

This is the phenomenon all three theorems are designed to rule out.

1. Monotone convergence theorem (MCT)

If 0 \le f_1 \le f_2 \le \cdots and f_n \to f pointwise, then

\lim_{n\to\infty}\int f_n\,d\mu = \int f\,d\mu

Conditions: non-negative and increasing. Nothing else — the limit may be +\infty, and that's fine.

Why the counterexample is excluded: the spikes aren't monotone. f_2 is taller than f_1 near 0 but zero where f_1 was positive, so neither dominates the other.

MCT is what justifies the very construction of the Lebesgue integral (approximate from below by simple functions), and it gives the useful corollary that for non-negative terms, \sum_n \int g_n = \int \sum_n g_n — you may always interchange a sum and an integral when everything is positive.

2. Fatou's lemma

For any non-negative f_n — no convergence assumption at all:

\int \liminf_{n\to\infty} f_n\,d\mu \;\le\; \liminf_{n\to\infty}\int f_n\,d\mu

The weakest and most general of the three. It says mass can be lost in the limit but never created.

Check it against the spikes: \liminf f_n = 0, so the left side is 0, and the right side is 1. Indeed 0 \le 1. ✓ Fatou correctly predicts the direction of the failure.

3. Dominated convergence theorem (DCT)

If f_n \to f pointwise and there is an integrable g with |f_n| \le g for all n, then

\lim_{n\to\infty}\int f_n\,d\mu = \int f\,d\mu

This is the one you'll use most. The dominating function g acts as a lid: it prevents any f_n from growing tall enough to smuggle mass away.

Why the counterexample is excluded: any g dominating all the spikes must satisfy g(x) \ge n whenever x < 1/n, which forces g(x) \ge 1/x near zero — and \int_0^1 \frac{dx}{x} = \infty. No integrable dominating function exists.

Bounded convergence is the special case that covers most probability applications: on a probability space, if |X_n| \le M for a constant M, then E[X_n] \to E[X]. A constant is integrable when the total measure is finite, which is why probability is a friendlier setting than general measure theory.

Which to reach for

Theorem	Requires	Gives
MCT	f_n \ge 0, increasing	equality
Fatou	f_n \ge 0 only	inequality (\le)
DCT	\vert f_n \vert \le g integrable	equality

Rough rule: monotone → MCT. Bounded → DCT. Neither → Fatou, and accept an inequality.

Where they're used

The strong law of large numbers (§3) needs DCT in its proof.

Differentiating under the integral sign — the step that produced the score having mean zero in §4's Fisher information derivation — is DCT applied to difference quotients. That step was flagged as "under regularity conditions"; this is what those conditions are.

Fubini's theorem, for swapping the order of a double integral, is proved via MCT.

Continuity of expectation: showing E[X_n] \to E[X] for a sequence of estimators is a DCT argument almost every time.

Worked example

X_n is n with probability 1/n and 0 otherwise. Does $E[X_n] \to E[\lim X_n]$?

Pointwise limit. For any fixed outcome, P(X_n = n) = 1/n \to 0, so X_n \xrightarrow{p} 0. (And along a suitable construction, X_n \to 0 almost surely.)

The expectations.

E[X_n] = n \times \frac1n + 0 \times \left(1 - \frac1n\right) = 1 \quad \text{for every } n

So \lim_n E[X_n] = 1, while E[\lim_n X_n] = E[0] = 0.

They disagree. Same structure as the spikes: a vanishing probability of an exploding value, with the product held constant.

Why no theorem applies. MCT needs monotonicity — the sequence isn't monotone. DCT needs a dominating integrable g with |X_n| \le g; such a g would need g \ge n on a set of probability 1/n, and E[g] \ge \sum of those contributions diverges. Fatou applies and gives 0 \le 1, which is true and uninformative.

The practical moral: convergence in probability does not imply convergence of expectations. Heavy tails, or rare extreme values, break the link — which is also why §4 distinguished consistency (about the estimator) from unbiasedness (about its expectation). They're different properties for exactly this reason.

Adding uniform integrability — a slight strengthening that rules out escaping mass — restores the implication.

Doing it in Python

The escaping-mass counterexample, made concrete:

import numpy as np

print("f_n = n on (0, 1/n), zero elsewhere\n")
print(f"{'n':>8} {'f_n(0.1)':>10} {'f_n(0.01)':>11} {'integral':>10}")
for n in (1, 5, 10, 100, 1000, 100_000):
    at_01 = n if 0.1 < 1/n else 0
    at_001 = n if 0.01 < 1/n else 0
    print(f"{n:>8} {at_01:>10} {at_001:>11} {n * (1/n):>10.4f}")

print("\nPointwise the function goes to 0 everywhere.")
print("Every integral equals 1. The limit and the integral do NOT commute.")

MCT working, because the sequence increases:

import numpy as np
from scipy.integrate import trapezoid

# f_n(x) = x^(1/n) on [0,1] increases to 1 (for x > 0)
grid = np.linspace(1e-9, 1, 200_000)

print("f_n(x) = x^(1/n), increasing to 1 on (0,1]\n")
print(f"{'n':>8} {'f_n(0.5)':>12} {'integral':>12}")
for n in (1, 2, 5, 20, 100, 1000):
    f = grid ** (1 / n)
    print(f"{n:>8} {0.5**(1/n):>12.6f} {trapezoid(f, grid):>12.6f}")

print("\nlimit function is 1, whose integral is 1.0 -- MCT holds. ")
print("(Exact: integral of x^(1/n) is n/(n+1) -> 1.)")

DCT working, and the domination that makes it work:

import numpy as np
from scipy.integrate import trapezoid

# f_n(x) = sin(n x) / (1 + x^2) on [0, 10]. Dominated by g(x) = 1/(1+x^2),
# which is integrable. The functions oscillate wildly but the integrals behave.
grid = np.linspace(0, 10, 500_000)
g = 1 / (1 + grid**2)

print(f"dominating function integral: {trapezoid(g, grid):.6f} (finite -> DCT applies)\n")
print(f"{'n':>8} {'max |f_n|':>12} {'integral':>12}")
for n in (1, 5, 20, 100, 1000):
    f = np.sin(n * grid) / (1 + grid**2)
    print(f"{n:>8} {np.abs(f).max():>12.6f} {trapezoid(f, grid):>12.6f}")

print("\nEvery |f_n| stays under the same integrable lid, so the integrals")
print("converge (here to 0, by oscillation).")

The probabilistic version — convergence in probability without convergence of means:

import numpy as np

rng = np.random.default_rng(0)

print("X_n = n with probability 1/n, else 0\n")
print(f"{'n':>10} {'P(X_n != 0)':>14} {'E[X_n]':>10} {'simulated':>12}")
for n in (10, 100, 1_000, 10_000):
    draws = np.where(rng.random(400_000) < 1/n, n, 0.0)
    print(f"{n:>10} {1/n:>14.5f} {1.0:>10.4f} {draws.mean():>12.4f}")

print("\nX_n -> 0 in probability (the spike gets rarer),")
print("but E[X_n] = 1 forever. Consistency does not imply unbiasedness.")

And uniform integrability restoring the link:

import numpy as np

rng = np.random.default_rng(1)

# Y_n = 1 with probability 1/n, else 0 -- BOUNDED, so DCT applies
print("Y_n = 1 with probability 1/n (bounded by 1, an integrable dominator)\n")
print(f"{'n':>10} {'E[Y_n]':>10} {'simulated':>12}")
for n in (10, 100, 1_000, 10_000):
    draws = (rng.random(400_000) < 1/n).astype(float)
    print(f"{n:>10} {1/n:>10.5f} {draws.mean():>12.5f}")

print("\nHere E[Y_n] -> 0 = E[lim Y_n]. The only change from the previous")
print("block is capping the value at 1 -- that cap IS the dominating function.")

Your turn

1. f_n = \frac{1}{n}\mathbb 1_{[0,n]} on [0,\infty). Does \int f_n \to \int \lim f_n?

2. Which theorem applies to f_n(x) = x^n on [0,1]?

3. Why does the DCT need g to be integrable rather than just finite?

Solutions

1. No.

Pointwise, f_n(x) = 1/n \to 0 for every x (once n > x, and even before that the height is shrinking). So \lim f_n = 0 and \int \lim f_n = 0.

But each integral is

\int_0^\infty f_n\,d\lambda = \frac{1}{n} \times n = 1

so \lim \int f_n = 1 \ne 0.

This is escaping mass again, in the other direction. Rather than a spike growing taller on a shrinking set, we have a plateau flattening over a spreading set. The mass runs off to infinity instead of collapsing to a point.

No dominating integrable g exists: g would need g \ge 1/n on [0,n] for every n, forcing g \ge something with infinite integral over the half-line. (Note this requires an infinite measure space — on a probability space, mass cannot escape this way, which is one reason probability is more forgiving.)

2. Dominated convergence (or bounded convergence, its special case).

The functions converge pointwise to

f(x) = \begin{cases}0 & 0 \le x < 1\\ 1 & x = 1\end{cases}

and |x^n| \le 1 on [0,1], where the constant 1 is integrable since the interval has finite measure. So DCT applies:

\int_0^1 x^n\,dx = \frac{1}{n+1} \to 0 = \int_0^1 f\,d\lambda

The limit function is 1 only at the single point x = 1, a null set, so its integral is 0. ✓

MCT does not apply — the sequence is decreasing on [0,1), not increasing. (There's a decreasing version of MCT, but it needs an integrable first term, which happens to hold here.)

3. Because merely being finite everywhere doesn't stop the mass from escaping — \int g\,d\mu has to be finite too.

Consider f_n = n\mathbb 1_{(0,1/n)} again. A valid pointwise bound is g(x) = 1/x, which is finite at every x > 0. But

\int_0^1 \frac{1}{x}\,dx = \infty

so g is not integrable, DCT does not apply, and indeed the conclusion fails.

The dominating function has to control the total mass, not just the height at each point. That's exactly what integrability means, and it's the precise condition ruling out the escaping-mass failure.

Intuitively: g is a lid on the whole family at once. If the lid itself encloses infinite area, it isn't constraining anything.

Check yourself in code

Demonstrate the escaping-mass counterexample and confirm that a bounded version behaves properly.

Print exactly this:

unbounded E[X_n] at n=10000 1.0
limit of E[X_n] 1.0
E[lim X_n] 0.0
theorems fail: True
bounded E[Y_n] at n=10000 0.0001

For the unbounded case, X_n = n with probability 1/n and 0 otherwise; for the bounded case, Y_n = 1 with probability 1/n. Compute the expectations exactly rather than by simulation, and round to 4 decimal places.

n = 10_000

E_Xn = n * (1 / n)
print("unbounded E[X_n] at n=10000", round(E_Xn, 4))
print("limit of E[X_n]", 1.0)

# Print E[lim X_n] (the pointwise limit is 0), whether the two disagree,
# and E[Y_n] for the bounded version Y_n = 1 with probability 1/n.

n = 10_000

E_Xn = n * (1 / n)
print("unbounded E[X_n] at n=10000", round(E_Xn, 4))
print("limit of E[X_n]", 1.0)
print("E[lim X_n]", 0.0)
print("theorems fail:", bool(1.0 != 0.0))
print("bounded E[Y_n] at n=10000", round(1 / n, 4))

---

Limits and integrals do not always commute — mass can escape by growing tall on a shrinking set or by flattening over a spreading one. Monotone convergence handles increasing non-negative sequences, dominated convergence handles anything capped by an integrable function, and Fatou's lemma always applies but only gives an inequality. These are the conditions hiding behind every "regularity condition" invoked earlier in the course.

Next: the derivative that relates one measure to another, and what conditional expectation really is.