Joint, marginal and conditional distributions · Probability & Statistics

Most of the world isn't a single random variable — it's several at once. The weather and your commute. A patient's age and their blood pressure. Two quantities, varying together.

Everything you can ask about a pair lives in one object: their joint distribution. Marginal and conditional distributions are both derived from it, by two different operations that are easy to confuse.

The joint distribution

For two discrete variables, the joint PMF is

$p(x, y) = P(X = x, \; Y = y)$

where the comma means and. Here it is as a table — rows are the weather, columns are how you get to work:

	walk	bus	bike	row total
sunny	0.15	0.10	0.25	0.50
rainy	0.05	0.40	0.05	0.50
col total	0.20	0.50	0.30	1.00

Each cell is the probability of that exact combination — sunny and biking is 0.25. And because the table covers every possible pair, all cells sum to 1:

$\sum_x \sum_y p(x, y) = 1$

For continuous variables the same idea holds with a joint density f(x,y), a surface over the plane whose total volume is 1:

$\iint f(x,y)\,dx\,dy = 1, \qquad P((X,Y) \in A) = \iint_A f(x,y)\,dx\,dy$

Marginal — sum a variable away

Suppose you only care about the weather and want to forget the commute entirely. Add up each row:

$p_X(x) = \sum_y p(x, y)$

Those row totals — 0.50 and 0.50 — are the marginal distribution of the weather. Add up each column instead and you get the marginal of the commute: walk 0.20, bus 0.50, bike 0.30.

Continuous version, with the sum becoming an integral:

$f_X(x) = \int_{-\infty}^{\infty} f(x, y)\,dy$

The name is literal: these totals were historically written in the margins of the table.

The key point: a marginal is still a genuine probability distribution over one variable — it sums to 1 — and getting it is a summing-away operation. The other variable isn't fixed at anything; it's been averaged out over all its possibilities.

Conditional — fix a variable, then rescale

Conditioning is a different operation, and this is where people slip.

You're told it's raining, and you ask how you'll travel. Keep only the rainy row and throw the rest of the table away:

	walk	bus	bike
rainy	0.05	0.40	0.05

But that row sums to 0.50, not 1 — it isn't a distribution yet. Divide every entry by that 0.50:

$P(\text{walk} \mid \text{rain}) = 0.10, \quad P(\text{bus} \mid \text{rain}) = 0.80, \quad P(\text{bike} \mid \text{rain}) = 0.10$

Now it sums to 1. Compare with the unconditional marginal (0.20, 0.50, 0.30): walking has halved, biking has collapsed, and the bus has taken over. Learning about the weather genuinely changed the picture.

In general:

$p_{Y|X}(y \mid x) = \frac{p(x, y)}{p_X(x)}, \qquad f_{Y|X}(y \mid x) = \frac{f(x, y)}{f_X(x)}$

which is exactly the conditional probability definition from §0, with the marginal supplying the denominator.

The one thing to remember: marginal = sum away; conditional = slice, then rescale. Both turn a two-variable object into a one-variable one, but a marginal averages over the other variable while a conditional fixes it.

For continuous variables, conditioning means taking a slice through the density surface and rescaling that slice back to area 1. Note f_{Y|X}(y\mid x) is a legitimate density in y even though P(X = x) = 0 — the ratio of densities is well defined wherever f_X(x) > 0.

Putting them back together

The definitions rearrange into two facts you'll use constantly:

$f(x,y) = f_{Y|X}(y \mid x)\,f_X(x) \qquad \text{(chain rule)}$

$f_Y(y) = \int f_{Y|X}(y \mid x) f_X(x)\,dx \qquad \text{(law of total probability)}$

That second one is §0's law of total probability again, now with a continuous partition. It's how you build hierarchical models: specify a marginal for X and a conditional for Y given X, and the joint follows.

Worked example

From the weather/commute table, find (a) the marginal of commute, (b) P(\text{sunny} \mid \text{bus}), and (c) P(\text{bus}) via total probability.

(a) Sum each column:

$p(\text{walk}) = 0.15 + 0.05 = 0.20, \quad p(\text{bus}) = 0.10 + 0.40 = 0.50, \quad p(\text{bike}) = 0.25 + 0.05 = 0.30$

Total = 1.00. ✓

(b) Now condition the other way — fix the column:

$P(\text{sunny} \mid \text{bus}) = \frac{p(\text{sunny, bus})}{p_{\text{commute}}(\text{bus})} = \frac{0.10}{0.50} = 0.20$

So seeing someone on the bus makes it only 20% likely to be sunny, against a 50% base rate. That's Bayes at work: the observation is evidence about the weather.

Note P(\text{bus} \mid \text{sunny}) = 0.10/0.50 = 0.20 happens to equal it here, purely because both marginals are 0.50. In general the two conditionals are different, and confusing them is the base-rate error from §0.

(c) Via the law of total probability, partitioning on weather:

$P(\text{bus}) = P(\text{bus} \mid \text{sunny})P(\text{sunny}) + P(\text{bus} \mid \text{rainy})P(\text{rainy})$ $= (0.20)(0.50) + (0.80)(0.50) = 0.10 + 0.40 = 0.50 \quad\checkmark$

Same answer as summing the column, as it must be.

Doing it in Python

A joint distribution is naturally a 2-D array, and marginals are just sums along an axis:

import numpy as np

#            walk   bus   bike
joint = np.array([[0.15, 0.10, 0.25],    # sunny
                  [0.05, 0.40, 0.05]])   # rainy

weather = ["sunny", "rainy"]
commute = ["walk", "bus", "bike"]

print("total mass:", joint.sum())

p_weather = joint.sum(axis=1)     # sum away commute -> marginal of weather
p_commute = joint.sum(axis=0)     # sum away weather -> marginal of commute

print("\nmarginal of weather:", dict(zip(weather, p_weather)))
print("marginal of commute:", dict(zip(commute, p_commute.round(4))))

Conditioning is slicing plus a divide — and keepdims makes the broadcasting do the rescaling for you:

import numpy as np

joint = np.array([[0.15, 0.10, 0.25],
                  [0.05, 0.40, 0.05]])
commute = ["walk", "bus", "bike"]

# P(commute | weather): divide each ROW by its own total
cond = joint / joint.sum(axis=1, keepdims=True)

for i, w in enumerate(["sunny", "rainy"]):
    print(f"P(commute | {w}) =", dict(zip(commute, cond[i].round(4))),
          " sums to", round(cond[i].sum(), 10))

# And the other direction: P(weather | commute) divides each COLUMN by its total
cond2 = joint / joint.sum(axis=0, keepdims=True)
print("\nP(sunny | bus) =", round(cond2[0, 1], 4))
print("P(bus | sunny) =", round(cond[0, 1], 4))
print("-> different questions, and in general different answers")

Rebuilding the joint from a marginal and a conditional (the chain rule) is the same arithmetic run backwards:

import numpy as np

p_weather = np.array([0.5, 0.5])                    # marginal
cond = np.array([[0.30, 0.20, 0.50],                # P(commute | sunny)
                 [0.10, 0.80, 0.10]])               # P(commute | rainy)

rebuilt = cond * p_weather[:, None]                 # f(x,y) = f(y|x) f(x)
print(rebuilt.round(4))
print("matches original:", np.allclose(rebuilt,
      np.array([[0.15, 0.10, 0.25], [0.05, 0.40, 0.05]])))

Your turn

1. Two fair coins. X = number of heads, Y = 1 if the first is heads. Write the joint PMF as a table.

2. From that table, find the marginal of X and P(X = 1 \mid Y = 1).

3. A joint density is f(x,y) = 4xy on the unit square. Find f_X(x).

Solutions

1. Four equally likely outcomes: HH, HT, TH, TT, each with probability 1/4. Map each to (X, Y): HH \to (2,1), HT \to (1,1), TH \to (1,0), TT \to (0,0).

	Y=0	Y=1
X=0	1/4	0
X=1	1/4	1/4
X=2	0	1/4

The two zeros are structural, not accidental: you cannot have 0 heads if the first coin was heads, nor 2 heads if it wasn't.

2. Marginal of X — sum each row:

$p_X(0) = \tfrac14, \qquad p_X(1) = \tfrac12, \qquad p_X(2) = \tfrac14$

which is Binomial(2, 0.5), as expected.

For the conditional, first the denominator: $p_Y(1) = 0 + \tfrac14 + \tfrac14 = \tfrac12$. Then

$P(X = 1 \mid Y = 1) = \frac{p(1, 1)}{p_Y(1)} = \frac{1/4}{1/2} = \tfrac{1}{2}$

Given the first coin is heads, X = 1 exactly when the second is tails — probability 1/2. ✓

Here this happens to equal the marginal p_X(1) = 1/2, which might suggest X and Y are independent. They are not: check a different value and it fails immediately, since P(X = 0 \mid Y = 1) = 0 while p_X(0) = 1/4. Independence has to hold for every pair of values, not one lucky one — which is exactly the next lesson.

3. Integrate y away, holding x fixed:

$f_X(x) = \int_0^1 4xy\,dy = 4x\left[\frac{y^2}{2}\right]_0^1 = 4x \cdot \tfrac12 = 2x$

for 0 \le x \le 1. Check it's a density: \int_0^1 2x\,dx = 1. ✓

By symmetry f_Y(y) = 2y. And notice f_X(x)f_Y(y) = 4xy = f(x,y) — the joint factors into its marginals, which is exactly the definition of independence coming next.

Check yourself in code

From the weather/commute joint table, compute both marginals and the two conditionals that are easy to confuse.

Print exactly this:

marginal commute: [0.2 0.5 0.3]
P(bus | sunny) = 0.2
P(sunny | bus) = 0.2
rows sum to 1: True

Round the marginal array to 4 decimals and each conditional to 4.

import numpy as np

#            walk   bus   bike
joint = np.array([[0.15, 0.10, 0.25],    # sunny
                  [0.05, 0.40, 0.05]])   # rainy

print("marginal commute:", joint.sum(axis=0).round(4))

# P(bus | sunny): divide the sunny ROW by its total.
# P(sunny | bus): divide the bus COLUMN by its total.
# Then confirm every row of P(commute | weather) sums to 1.

import numpy as np

joint = np.array([[0.15, 0.10, 0.25],
                  [0.05, 0.40, 0.05]])

print("marginal commute:", joint.sum(axis=0).round(4))

by_row = joint / joint.sum(axis=1, keepdims=True)
by_col = joint / joint.sum(axis=0, keepdims=True)

print("P(bus | sunny) =", round(by_row[0, 1], 4))
print("P(sunny | bus) =", round(by_col[0, 1], 4))
print("rows sum to 1:", bool(np.allclose(by_row.sum(axis=1), 1)))

One joint object holds the whole story of two variables. Sum a variable away to get a marginal; fix a variable and renormalise to get a conditional. Everything else in this section is built on that distinction.

Next: the special case where conditioning changes nothing at all.

15. Joint, marginal and conditional distributions