20. The chain rule
This is the one you'll use most. Almost every function in the wild is a composition — \sin(x^2), e^{-t/\tau}, \sqrt{1+x^3}, and every neural network ever trained — and the chain rule is how you differentiate all of them.
\frac{d}{dx}f(g(x)) = f'(g(x))\cdot g'(x)
Derivative of the outer function, evaluated at the inner function, times the derivative of the inner function.
Why it looks like cancellation
In Leibniz notation, with y = f(u) and u = g(x):
\frac{dy}{dx} = \frac{dy}{du}\cdot\frac{du}{dx}
The du's appear to cancel. They aren't fractions and it isn't cancellation, but Leibniz designed the notation so that it would look like this, and the mnemonic is reliable.
The intuition it encodes is a chain of rates. If y changes 3 times as fast as u, and u changes 5 times as fast as x, then y changes 15 times as fast as x. Rates multiply through a chain.
A concrete version: a car uses fuel at 8 litres per 100 km, and you're driving at 120 km per hour. Litres per hour is the product, \frac{8}{100} \times 120 = 9.6. The units cancel exactly the way the du's appear to.
The proof, and its one subtlety
The natural argument is:
\frac{f(g(x+h)) - f(g(x))}{h} = \frac{f(g(x+h)) - f(g(x))}{g(x+h)-g(x)}\cdot\frac{g(x+h)-g(x)}{h}
The second factor tends to g'(x). The first is a difference quotient for f at g(x), with step g(x+h)-g(x) \to 0, so it tends to f'(g(x)). Multiply.
The gap: if g(x+h) = g(x) for some h arbitrarily close to 0, we divided by zero. That's not hypothetical — it happens for g(x) = x^2\sin(1/x) near 0, which equals its value at 0 infinitely often in every neighbourhood.
The repair defines an auxiliary function that is continuous at the problem point:
\Phi(t) = \begin{cases}\dfrac{f(t) - f(g(x))}{t - g(x)} & t \neq g(x)\\[2mm] f'(g(x)) & t = g(x)\end{cases}
\Phi is continuous at g(x) precisely because f'(g(x)) exists, and f(t)-f(g(x)) = \Phi(t)(t - g(x)) holds for all t including the bad one. Substituting t = g(x+h), dividing by h, and letting h \to 0 gives the rule with no division by zero anywhere. \blacksquare
Worth knowing the standard proof has a hole, and worth knowing it's patchable. Most textbooks present the cancelling version without comment.
Using it: identify outer and inner
The mechanical skill is decomposition. Ask: what would I do last if I were evaluating this at a number? That's the outer function.
| Function | Outer | Inner | Derivative |
|---|---|---|---|
| \sin(x^2) | \sin(\square) | x^2 | \cos(x^2)\cdot 2x |
| (3x+1)^5 | \square^5 | 3x+1 | 5(3x+1)^4\cdot3 |
| e^{x^2} | e^\square | x^2 | e^{x^2}\cdot 2x |
| \sqrt{1+x^3} | \sqrt\square | 1+x^3 | \frac{1}{2\sqrt{1+x^3}}\cdot3x^2 |
| \ln(\cos x) | \ln\square | \cos x | \frac{1}{\cos x}\cdot(-\sin x) = -\tan x |
The error to avoid is forgetting the second factor. $\frac{d}{dx}\sin(x^2) = \cos(x^2)$ is wrong; the 2x is not optional. A quick check: at x=0 the true derivative should be 0 (the function is roughly x^2 there), and only the version with 2x gives that.
Nesting
Three layers means three factors, working outside in:
\frac{d}{dx}\sin^3(2x) = \frac{d}{dx}\left[\sin(2x)\right]^3 = 3\sin^2(2x)\cdot\cos(2x)\cdot2
Outer: cube. Middle: sine. Inner: 2x. Each contributes one factor, evaluated at whatever is inside it.
Arbitrarily deep nesting is fine, and the pattern is always a product of one factor per layer:
\frac{d}{dx}f(g(h(x))) = f'(g(h(x)))\cdot g'(h(x))\cdot h'(x)
This is backpropagation. A neural network is a deep composition, its gradient is this product, and computing it right-to-left rather than left-to-right is the entire reason training large models is feasible. §14.3 and §14.4 make that precise.
The special cases worth memorising
Because they come up constantly:
\frac{d}{dx}\left[g(x)\right]^n = n\left[g(x)\right]^{n-1}g'(x) \qquad\text{(the "general power rule")}
\frac{d}{dx}e^{g(x)} = e^{g(x)}g'(x), \qquad \frac{d}{dx}\ln g(x) = \frac{g'(x)}{g(x)}
\frac{d}{dx}\sin(kx) = k\cos(kx), \qquad \frac{d}{dx}e^{kx} = ke^{kx}
That last pair is why a frequency or a decay rate always drops out front — and why §0.4 insisted that using degrees would poison every trig derivative with \pi/180.
Doing it in Python
Decomposition, checked layer by layer:
from math import sin, cos, sqrt, exp
def numerical(f, x, h=1e-6):
return (f(x + h) - f(x - h)) / (2 * h)
cases = [
("sin(x^2)", lambda x: sin(x**2), lambda x: cos(x**2) * 2*x),
("(3x+1)^5", lambda x: (3*x+1)**5, lambda x: 5*(3*x+1)**4 * 3),
("e^(x^2)", lambda x: exp(x**2), lambda x: exp(x**2) * 2*x),
("sqrt(1+x^3)", lambda x: sqrt(1+x**3), lambda x: 3*x**2 / (2*sqrt(1+x**3))),
]
x = 0.8
print(f"{'function':>14} {'chain rule':>14} {'numerical':>14} {'match':>8}")
for name, f, df in cases:
r, n = df(x), numerical(f, x)
print(f"{name:>14} {r:>14.8f} {n:>14.8f} {abs(r-n) < 1e-5!s:>8}")
Forgetting the inner factor, priced:
from math import sin, cos
def numerical(f, x, h=1e-6):
return (f(x + h) - f(x - h)) / (2 * h)
print(f"{'x':>6} {'truth':>14} {'with 2x':>14} {'without 2x':>14}")
for x in (0.0, 0.5, 1.0, 2.0):
truth = numerical(lambda t: sin(t**2), x)
print(f"{x:>6} {truth:>14.8f} {cos(x**2)*2*x:>14.8f} {cos(x**2):>14.8f}")
print("\nthe third column is only right where 2x happens to equal 1")
Deep nesting, and the product of one factor per layer:
import sympy as sp
x = sp.Symbol('x')
expr = sp.sin(sp.exp(sp.sqrt(x**2 + 1)))
print(f"f(x) = {expr}")
print(f"f'(x) = {sp.simplify(sp.diff(expr, x))}")
# assemble it by hand, outside in
inner = sp.sqrt(x**2 + 1)
by_hand = (sp.cos(sp.exp(inner)) # d/d(exp) of sin
* sp.exp(inner) # d/d(sqrt) of exp
* sp.diff(inner, x)) # d/dx of sqrt(x^2+1)
print(f"by hand = {sp.simplify(by_hand)}")
print(f"identical: {sp.simplify(sp.diff(expr, x) - by_hand) == 0}")
The chain of rates, in units:
litres_per_km = 8 / 100
km_per_hour = 120
print(f"fuel use : {litres_per_km} litres per km")
print(f"speed : {km_per_hour} km per hour")
print(f"burn rate : {litres_per_km * km_per_hour} litres per hour")
print("\n(L/km) * (km/h) = L/h -- the km cancel, exactly as the du's appear to")
Worked example
Differentiate y = \sqrt{\sin(3x^2 + 1)}.
Three layers. Peel from the outside:
- Outermost: square root of something.
- Middle: sine of something.
- Innermost: 3x^2+1.
\frac{dy}{dx} = \underbrace{\frac{1}{2\sqrt{\sin(3x^2+1)}}}_{\text{d(sqrt)}} \cdot \underbrace{\cos(3x^2+1)}_{\text{d(sin)}} \cdot \underbrace{6x}_{\text{d(inner)}}
= \frac{6x\cos(3x^2+1)}{2\sqrt{\sin(3x^2+1)}} = \frac{3x\cos(3x^2+1)}{\sqrt{\sin(3x^2+1)}}
Each factor is the derivative of one layer, evaluated at everything inside it. Note the middle factor is \cos(3x^2+1) and not \cos of anything else — the argument stays untouched as you move outward.
A domain caveat worth stating. The square root needs \sin(3x^2+1) \ge 0, and the derivative additionally needs it > 0 (the denominator). At points where the sine hits zero, the function has a vertical tangent — the failure mode from §2.1 — and the formula correctly reports division by zero rather than a slope.
Your turn
1. \dfrac{d}{dx}(x^3+2x)^7
2. \dfrac{d}{dx}\cos(5x^2)
3. \dfrac{d}{dx}\left[x^2\sin(3x)\right] — which rules, in what order?
4. \dfrac{d}{dx}\sqrt{\dfrac{x+1}{x-1}}
Solutions
1. General power rule: outer is \square^7, inner is x^3+2x.
7(x^3+2x)^6\cdot(3x^2+2) = \boxed{7(3x^2+2)(x^3+2x)^6}
2. Outer \cos, inner 5x^2:
-\sin(5x^2)\cdot 10x = \boxed{-10x\sin(5x^2)}
3. Product rule first, because at the top level this is a product of two things. The chain rule then handles the second factor.
\frac{d}{dx}\left[x^2\sin(3x)\right] = 2x\sin(3x) + x^2\cdot\underbrace{3\cos(3x)}_{\text{chain}}
= \boxed{2x\sin(3x) + 3x^2\cos(3x)}
The general principle: the outermost operation decides which rule you apply first. Here the last thing you'd do evaluating at a number is multiply, so product rule leads.
4. Two routes, and one is much better.
Direct. Outer \sqrt\square, inner the quotient (needing the quotient rule):
\frac{d}{dx}\frac{x+1}{x-1} = \frac{(x-1) - (x+1)}{(x-1)^2} = \frac{-2}{(x-1)^2}
\frac{dy}{dx} = \frac{1}{2\sqrt{\frac{x+1}{x-1}}}\cdot\frac{-2}{(x-1)^2} = \frac{-1}{(x-1)^2}\sqrt{\frac{x-1}{x+1}}
= \boxed{\frac{-1}{(x-1)^{3/2}(x+1)^{1/2}}}
Better route. Take logs first — logarithmic differentiation, §2.6:
\ln y = \tfrac12\left[\ln(x+1) - \ln(x-1)\right]
Differentiating both sides turns the product/quotient structure into a sum, and the algebra is markedly cleaner. Worth remembering once you've met the technique.
Check yourself in code
Verify the chain rule against numerical differentiation, and show what dropping the inner factor costs.
At x = 0.8, for \sin(x^2), (3x+1)^5, e^{x^2}, and \sqrt{1+x^3}: print the chain-rule value and the central difference (h=10^{-6}), each to 8 decimals. Then print \frac{d}{dx}\sin(x^2) at x=0.8 with and without the 2x.
Print exactly this:
sin(x^2) chain 1.28335321 numeric 1.28335321
(3x+1)^5 chain 2004.50400000 numeric 2004.50400015
e^(x^2) chain 3.03436941 numeric 3.03436941
sqrt(1+x^3) chain 0.78072006 numeric 0.78072006
with 2x 1.28335321
without 2x 0.80209576
from math import sin, cos, sqrt, exp
def numerical(f, x, h=1e-6):
return (f(x + h) - f(x - h)) / (2 * h)
cases = [
("sin(x^2)", lambda x: sin(x**2), lambda x: cos(x**2) * 2*x),
("(3x+1)^5", lambda x: (3*x+1)**5, lambda x: 5*(3*x+1)**4 * 3),
("e^(x^2)", lambda x: exp(x**2), lambda x: exp(x**2) * 2*x),
("sqrt(1+x^3)", lambda x: sqrt(1+x**3), lambda x: 3*x**2 / (2*sqrt(1+x**3))),
]
x = 0.8
for name, f, df in cases:
print(f"{name:<13} chain ... numeric ...")
# then sin(x^2) with and without the inner 2x
from math import sin, cos, sqrt, exp
def numerical(f, x, h=1e-6):
return (f(x + h) - f(x - h)) / (2 * h)
cases = [
("sin(x^2)", lambda x: sin(x**2), lambda x: cos(x**2) * 2*x),
("(3x+1)^5", lambda x: (3*x+1)**5, lambda x: 5*(3*x+1)**4 * 3),
("e^(x^2)", lambda x: exp(x**2), lambda x: exp(x**2) * 2*x),
("sqrt(1+x^3)", lambda x: sqrt(1+x**3), lambda x: 3*x**2 / (2*sqrt(1+x**3))),
]
x = 0.8
for name, f, df in cases:
print(f"{name:<13} chain {df(x):.8f} numeric {numerical(f, x):.8f}")
print(f"with 2x {cos(x**2) * 2*x:.8f}")
print(f"without 2x {cos(x**2):.8f}")
\frac{d}{dx}f(g(x)) = f'(g(x))g'(x): differentiate the outer function, leave its argument alone, and multiply by the derivative of what's inside. Rates multiply through a chain, which is why the Leibniz form looks like cancellation. Deeper nesting just means more factors — one per layer — and that product, computed efficiently, is backpropagation.
Next: the trig derivatives, which finally cash in the two limits from §1.4 and §1.5.