Continuous random variables, PDF and CDF · Probability & Statistics

Some random variables don't jump between whole numbers — they can land anywhere on a continuum. A person's exact height, the precise time a bus arrives, the temperature at noon.

That single change breaks the machinery from the last lesson, and the repair introduces the most misunderstood object in probability: the density.

From here on, we integrate. The first eight lessons needed nothing beyond arithmetic; this one and everything after it use calculus — integrals to get probabilities from densities, derivatives to go back the other way, and the Fundamental Theorem of Calculus to connect them. If that is unfamiliar, work through a basic calculus course first and return: the statistics here is not harder than what came before, but it is written in calculus and cannot be paraphrased out of it.

Every exact value has probability zero

Let X be the exact height of a randomly chosen adult, in metres.

What is P(X = 1.75)? Not "about 1.75" — exactly 1.75000000..., every decimal place zero forever.

The answer is zero. Here's the argument, and it's worth following because the conclusion sounds absurd:

Suppose instead P(X = 1.75) = \varepsilon for some \varepsilon > 0. Heights are spread over a range, and by symmetry there's nothing special about 1.75 — plenty of other values would have to carry at least as much. But there are uncountably many candidate values, and if even $\lceil 1/\varepsilon \rceil + 1$ of them each carried mass \varepsilon, the total would exceed 1, violating Axiom 2. So no individual value can carry positive mass.

$P(X = c) = 0 \quad \text{for every real } c$

This is where a genuinely uncomfortable idea enters: probability zero does not mean impossible. X certainly takes some value, and whatever it takes had probability zero beforehand. "Impossible" and "probability zero" come apart permanently here, and §9 is where we build the machinery that makes peace with it.

But asking for a range still works perfectly well. P(1.70 < X < 1.80) is an ordinary, positive number. Intervals carry probability; points do not.

The probability density function

If points carry nothing, what describes the distribution? A density:

$P(a \le X \le b) = \int_a^b f(x)\,dx$

The function f is the probability density function. Three things about it that trip people up:

1. f(x) is not a probability. It's a probability per unit of x — a rate, like density in kg/m³. To get a probability you must multiply by a width, which is what the integral does.

2. f(x) can exceed 1. A uniform distribution on [0, 0.5] has f(x) = 2 everywhere on that interval. Nothing is wrong: 2 \times 0.5 = 1. Only the area is capped at 1, never the height.

3. Only area is probability. Shade the region under the curve between two points and that shaded area is the probability of landing in the range.

The two rules mirror the discrete case, with the sum replaced by an integral:

$f(x) \ge 0 \;\text{ for all } x, \qquad \int_{-\infty}^{\infty} f(x)\,dx = 1$

A useful way to read the density: for a small width dx,

$P(x < X < x + dx) \approx f(x)\,dx$

So f(x) tells you the relative plausibility of landing near x. If f(3) = 2f(5), values near 3 are twice as likely as equally narrow windows near 5 — even though both exact points have probability zero.

A consequence worth pocketing

Because individual points carry no mass, endpoints are free:

$P(a \le X \le b) = P(a < X < b)$

The strict-vs-inclusive distinction that mattered so much for discrete variables completely vanishes here. That's one of the few places where continuous is easier.

The CDF

The CDF is defined exactly as before — the definition never changed, only the way we compute it:

$F(x) = P(X \le x) = \int_{-\infty}^{x} f(t)\,dt$

It's the running total of area from the far left up to x. And since points carry no mass, F is now continuous — no staircase, no jumps, just a smooth curve rising from 0 to 1.

Now the elegant part. The Fundamental Theorem of Calculus, run in reverse:

$f(x) = \frac{d}{dx}F(x)$

The PDF and CDF are each other's derivative and integral. Differentiate the accumulated probability and you get back the density you started with.

This makes the CDF the more fundamental object of the two. It always exists, for discrete and continuous variables alike; the density exists only when F is differentiable. When a problem gets confusing, retreating to the CDF is almost always the move — you'll see this used directly in transformations (§2) and order statistics.

$P(a \le X \le b) = F(b) - F(a)$

Worked example

X has density f(x) = 3x^2 for 0 \le x \le 1, and 0 elsewhere. Verify it's a valid density, find the CDF, and compute P(0.5 \le X \le 0.8).

Valid? It's non-negative on [0,1], and

$\int_0^1 3x^2\,dx = \big[x^3\big]_0^1 = 1 - 0 = 1 \quad\checkmark$

Note f(1) = 3 > 1, which is fine — heights aren't capped.

CDF. For 0 \le x \le 1:

$F(x) = \int_0^x 3t^2\,dt = x^3$

and F(x) = 0 below 0, F(x) = 1 above 1. Sanity checks: F(0) = 0, F(1) = 1, and F increases on [0,1]. ✓

The probability. Straight from the CDF:

$P(0.5 \le X \le 0.8) = F(0.8) - F(0.5) = 0.512 - 0.125 = 0.387$

And differentiating back: \frac{d}{dx}x^3 = 3x^2 = f(x). ✓

The density rising as x^2 means values near 1 are far more likely than values near 0 — the top 20% of the range, [0.8, 1], carries 1 - 0.512 = 48.8\% of all the probability.

Doing it in Python

SciPy's continuous distributions expose .pdf(), .cdf() and .ppf() (the inverse CDF). Note there is no .pmf() — that's the discrete/continuous split showing up in the API:

from scipy.stats import norm

# Standard normal: mean 0, sd 1
print("f(0)      =", round(norm.pdf(0), 4))       # a density, not a probability
print("F(0)      =", round(norm.cdf(0), 4))       # 0.5 by symmetry
print("P(-1<X<1) =", round(norm.cdf(1) - norm.cdf(-1), 4))
print("P(-2<X<2) =", round(norm.cdf(2) - norm.cdf(-2), 4))

# Any exact value has probability zero:
print("P(X = 0)  =", norm.cdf(0) - norm.cdf(0))

# ppf inverts the CDF: which x has 97.5% of the mass below it?
print("ppf(0.975)=", round(norm.ppf(0.975), 4))

Those numbers — 68% within one standard deviation, 95% within two — are worth memorising; they underpin every confidence interval in §4.

You can also confirm a density integrates to 1 numerically:

from scipy.integrate import quad

f = lambda x: 3 * x**2          # our example density on [0, 1]

total, _ = quad(f, 0, 1)
chunk, _ = quad(f, 0.5, 0.8)
print("total area:", round(total, 10))
print("P(0.5 <= X <= 0.8):", round(chunk, 4))
print("matches F(0.8) - F(0.5):", round(0.8**3 - 0.5**3, 4))

Your turn

1. f(x) = 2x on [0, 1]. Find P(X > 0.5).

2. X is uniform on [0, 4]. Write its density and find P(1 < X < 2.5).

3. Why is it fine for a density to be 5 at some point, but never fine for a PMF value to be 5?

Solutions

1. First confirm it's a density: \int_0^1 2x\,dx = [x^2]_0^1 = 1. ✓

The CDF is F(x) = x^2 on [0,1], so

$P(X > 0.5) = 1 - F(0.5) = 1 - 0.25 = 0.75$

Three quarters of the mass sits in the upper half of the range, because the density grows linearly — twice as tall at x = 1 as at x = 0.5.

2. A uniform density is constant, and the constant is forced by the total-area rule. Over a width of 4:

$f(x) = \tfrac{1}{4} \text{ for } 0 \le x \le 4, \qquad 0 \text{ otherwise}$

Probability is then just (width) × (height):

$P(1 < X < 2.5) = (2.5 - 1) \times \tfrac{1}{4} = \tfrac{1.5}{4} = 0.375$

3. Because they are different kinds of quantity.

A PMF value p(x) is a probability — it's P(X = x) directly. Axiom 2 caps every probability at 1, so p(x) = 5 is an immediate contradiction.

A density f(x) is a probability per unit length. It only becomes a probability after being multiplied by a width. f(x) = 5 is perfectly legal so long as the region where it's that tall is narrow enough that the total area stays 1 — for instance uniform on [0, 0.2], where f = 5 everywhere and 5 \times 0.2 = 1.

The clean way to say it: a PMF is capped in height, a PDF is capped in area.

Check yourself in code

Verify that f(x) = 3x^2 on [0,1] is a valid density, then compute a range probability two ways — by integrating the density, and from the closed-form CDF F(x) = x^3 — and confirm they agree.

Print exactly this:

total area: 1.0
by integration: 0.387
by CDF: 0.387
agree: True

Round the area to 1 decimal place and both probabilities to 3.

from scipy.integrate import quad

f = lambda x: 3 * x**2
F = lambda x: x**3

total, _ = quad(f, 0, 1)
print("total area:", round(total, 1))

# Compute P(0.5 <= X <= 0.8) by integrating f, then from F, and compare.

from scipy.integrate import quad

f = lambda x: 3 * x**2
F = lambda x: x**3

total, _ = quad(f, 0, 1)
print("total area:", round(total, 1))

by_int, _ = quad(f, 0.5, 0.8)
by_cdf = F(0.8) - F(0.5)

print("by integration:", round(by_int, 3))
print("by CDF:", round(by_cdf, 3))
print("agree:", round(by_int, 3) == round(by_cdf, 3))

A single point has probability zero. Density only becomes probability once you integrate it, so its height is unbounded but its area is not. And the CDF and PDF are simply each other's integral and derivative — with the CDF the safer of the two to reason with.

Next: the two numbers that summarise any distribution, discrete or continuous — where its centre is, and how far it typically strays.

9. Continuous random variables, PDF and CDF