Expectation, variance and standard deviation · Probability & Statistics

Two questions about any random variable: where is its centre, and how far does it typically stray from that centre? Those questions have names — expectation and variance — and between them they summarise most of what you need to know about a distribution.

In §0 we computed a mean and a standard deviation from data. Now we compute them from a distribution, before any data exists. The formulas will look familiar; the meaning is different, and the difference matters in §4.

Expectation

Roll a fair die over and over. What's the average result in the long run?

Expectation answers that with a probability-weighted average — each value counted according to how likely it is:

$E[X] = \sum_x x\,p(x) \qquad \text{(discrete)}$

$E[X] = \int_{-\infty}^{\infty} x\,f(x)\,dx \qquad \text{(continuous)}$

For the fair die, each face has probability 1/6:

$E[X] = \frac{1+2+3+4+5+6}{6} = \frac{21}{6} = 3.5$

You can never roll a 3.5. It isn't an outcome at all. The expectation is not a value you expect to see — it's the long-run average over many repetitions, and it needn't be attainable. (§3 makes "long-run average" precise as the Law of Large Numbers.)

A useful physical reading: E[X] is the balance point of the distribution. Put the PMF's bars on a weightless ruler and E[X] is where it balances.

Linearity — the property that does all the work

$E[aX + b] = a\,E[X] + b$

$E[X + Y] = E[X] + E[Y]$

The second is the remarkable one: it holds whether or not X and Y are independent. No other summary statistic is this well behaved, and it makes otherwise brutal problems trivial.

For the sum of two dice, instead of building the 11-value PMF and summing \sum x\,p(x), just write S = X_1 + X_2 and read off E[S] = 3.5 + 3.5 = 7.

Beware the reverse: E[g(X)] \ne g(E[X]) in general. In particular E[X^2] \ne (E[X])^2 — a gap we're about to give a name to. To compute E[g(X)], weight the transformed values by the original probabilities: E[g(X)] = \sum_x g(x)p(x).

Variance

How spread out is X around its mean? Take the distance from the mean, square it, and average:

$\operatorname{Var}(X) = E\big[(X - \mu)^2\big], \qquad \mu = E[X]$

Why squared? Because the raw deviations always average to exactly zero — E[X - \mu] = E[X] - \mu = 0 — so they carry no information. Squaring removes the signs and, deliberately, punishes far-away points much harder than near ones.

The computational shortcut

Expand the definition using linearity:

$\operatorname{Var}(X) = E[X^2 - 2\mu X + \mu^2] = E[X^2] - 2\mu E[X] + \mu^2 = E[X^2] - \mu^2$

$\boxed{\operatorname{Var}(X) = E[X^2] - (E[X])^2}$

Much easier in practice: one pass for E[X], one for E[X^2], subtract. This also shows E[X^2] \ge (E[X])^2 always, since variance can't be negative.

Scaling

$\operatorname{Var}(aX + b) = a^2\operatorname{Var}(X)$

Two things to notice. Adding b changes nothing — shifting the whole distribution moves its centre but not its spread. And multiplying by a multiplies the variance by a^2, not a, because variance is in squared units.

Standard deviation

Those squared units are awkward: if X is in rupees, \operatorname{Var}(X) is in rupees². So take the square root:

$\sigma = \operatorname{SD}(X) = \sqrt{\operatorname{Var}(X)}$

Now we're back in the original units, and \sigma reads as roughly the typical distance from the mean.

Worked example

Compute the variance and standard deviation of a fair die roll.

We have \mu = E[X] = 3.5. Now the second moment:

$E[X^2] = \frac{1^2 + 2^2 + 3^2 + 4^2 + 5^2 + 6^2}{6} = \frac{91}{6} \approx 15.167$

Apply the shortcut:

$\operatorname{Var}(X) = \frac{91}{6} - (3.5)^2 = 15.1\overline{6} - 12.25 = 2.91\overline{6}$

$\sigma = \sqrt{2.91\overline{6}} \approx 1.708$

So a typical roll lands about 1.7 away from 3.5 — which passes the smell test, since rolls range from 2.5 below to 2.5 above.

Note how badly E[X^2] \ne (E[X])^2 here: 15.17 against 12.25. That gap of 2.92 is the variance. The two are never equal unless X is constant.

Doing it in Python

Straight from the definitions, so nothing is hidden:

from fractions import Fraction

values = range(1, 7)
p = Fraction(1, 6)

mean = sum(x * p for x in values)
second = sum(x**2 * p for x in values)
var = second - mean**2

print("E[X]      =", mean, "=", float(mean))
print("E[X^2]    =", second, "=", round(float(second), 4))
print("Var(X)    =", var, "=", round(float(var), 4))
print("SD(X)     =", round(float(var) ** 0.5, 4))
print()
print("E[X^2] == (E[X])^2 ?", second == mean**2)   # emphatically not

SciPy gives the same numbers for named distributions, plus .stats() for both at once:

from scipy.stats import randint, norm

die = randint(1, 7)          # uniform on {1,...,6}; upper bound is exclusive
print("die  mean, var:", die.mean(), round(die.var(), 4))
print("die  sd:", round(die.std(), 4))

# For a continuous example, a Normal has its parameters as mean and sd directly
X = norm(loc=10, scale=3)
print("norm mean, var:", X.mean(), X.var())
print("Var(2X + 5) should be 4*9 = 36:", norm(loc=2*10+5, scale=2*3).var())

And a quick empirical confirmation that linearity doesn't need independence — here Y = -X, about as dependent as it gets:

import numpy as np

rng = np.random.default_rng(0)
X = rng.integers(1, 7, size=200_000)
Y = -X                                    # perfectly (negatively) dependent

print("E[X]      =", round(X.mean(), 4))
print("E[Y]      =", round(Y.mean(), 4))
print("E[X + Y]  =", round((X + Y).mean(), 4), " (exactly 0)")
print("E[X]+E[Y] =", round(X.mean() + Y.mean(), 4))
print()
print("Var(X)     =", round(X.var(), 4))
print("Var(X + Y) =", round((X + Y).var(), 4), " (0 -- variances do NOT add here)")

Expectations add regardless. Variances do not, unless the variables are uncorrelated — the missing term is the covariance, which arrives in §2.

Your turn

1. X takes value 0 with probability 0.7 and 10 with probability 0.3. Find E[X], \operatorname{Var}(X) and \sigma.

2. X is uniform on [0, 1] with density f(x) = 1. Find E[X] and \operatorname{Var}(X).

3. A game costs ₹50 to play. You win ₹200 with probability 0.2, otherwise nothing. Should you play?

Solutions

1. Expectation:

$E[X] = 0(0.7) + 10(0.3) = 3$

Second moment — square the values, keep the original probabilities:

$E[X^2] = 0^2(0.7) + 10^2(0.3) = 30$

$\operatorname{Var}(X) = 30 - 3^2 = 21, \qquad \sigma = \sqrt{21} \approx 4.58$

Note \sigma = 4.58 while X only ever takes the values 0 and 10. Standard deviation is a summary of typical distance, not a bound on it.

2. For a continuous variable, integrate:

$E[X] = \int_0^1 x \cdot 1\,dx = \left[\tfrac{x^2}{2}\right]_0^1 = \tfrac{1}{2}$

$E[X^2] = \int_0^1 x^2\,dx = \left[\tfrac{x^3}{3}\right]_0^1 = \tfrac{1}{3}$

$\operatorname{Var}(X) = \tfrac{1}{3} - \tfrac{1}{4} = \tfrac{1}{12} \approx 0.0833$

So \sigma \approx 0.289. The general uniform result is \operatorname{Var} = \frac{(b-a)^2}{12}, which this confirms with b - a = 1.

3. Compute the expected net result per play:

$E[\text{net}] = 0.2(200 - 50) + 0.8(0 - 50) = 30 - 40 = -10$

You lose ₹10 per play on average, so no — not if you're trying to make money.

But note what expectation alone doesn't tell you. The standard deviation here is large. The two net outcomes are +150 (probability 0.2) and -50 (probability 0.8), so the gap between them is 200, not 250 — the stake is already inside both figures. For a two-point variable \sigma = |a - b|\sqrt{p(1-p)}:

$\sigma = 200 \times \sqrt{0.2 \times 0.8} = 200 \times 0.4 = ₹80$

(Directly: E[\text{net}^2] = 0.2(150^2) + 0.8(50^2) = 6500, so \operatorname{Var} = 6500 - (-10)^2 = 6400 and \sigma = 80 ✓.)

At \sigma = ₹80 against a mean of -₹10, single plays are wildly variable and a few wins prove nothing. It's the Law of Large Numbers (§3) that guarantees the -10 is what actually happens over many plays — and that guarantee is exactly what a casino is selling.

Check yourself in code

Compute E[X], E[X^2], \operatorname{Var}(X) and \sigma for a fair die directly from the definitions, and confirm the shortcut formula agrees with the original definition E[(X-\mu)^2].

Print exactly this:

E[X] = 3.5
Var(X) = 2.9167
SD(X) = 1.7078
shortcut matches definition: True

Round the variance and SD to 4 decimal places.

values = range(1, 7)
p = 1 / 6

mean = sum(x * p for x in values)
print("E[X] =", round(mean, 4))

# Compute the variance two ways:
#   shortcut:   E[X^2] - mean^2
#   definition: E[(X - mean)^2]
# Print Var(X) and SD(X) to 4 dp, then whether the two routes agree.

values = range(1, 7)
p = 1 / 6

mean = sum(x * p for x in values)
print("E[X] =", round(mean, 4))

shortcut = sum(x**2 * p for x in values) - mean**2
definition = sum((x - mean) ** 2 * p for x in values)

print("Var(X) =", round(shortcut, 4))
print("SD(X) =", round(shortcut**0.5, 4))
print("shortcut matches definition:", round(shortcut, 10) == round(definition, 10))

Expectation is the probability-weighted centre, and it's linear — even for dependent variables. Variance is the average squared distance from that centre, computed most easily as E[X^2] - (E[X])^2, and it scales by a^2. Standard deviation puts the spread back into interpretable units.

Next: these two turn out to be the first entries in an infinite family, and a single function that packages all of them at once.

10. Expectation, variance and standard deviation