28. Linear approximation and differentials
§2.0 called the derivative "the best linear approximation". Here's what that buys you: a way to estimate values of hard functions using nothing but arithmetic, and a precise account of how wrong the estimate is.
The linearization
Near x = a, replace the curve by its tangent line:
L(x) = f(a) + f'(a)(x-a)
f(x) \approx f(a) + f'(a)(x-a) \qquad \text{for } x \text{ near } a
L is called the linearization of f at a. Geometrically: zoom in far enough on a differentiable function and it looks straight, so over a short interval the tangent line is a decent stand-in for the curve.
That "zoom in and it looks straight" property is arguably what differentiability means, and it's the version that survives into higher dimensions (§10.3) and into the definition used in §15.7.
Using it
Estimate \sqrt{101}.
Choose a = 100 — near the target, and easy to evaluate. With f(x) = \sqrt x and f'(x) = \frac{1}{2\sqrt x}:
f(100) = 10, \qquad f'(100) = \frac{1}{20} = 0.05
\sqrt{101} \approx 10 + 0.05(1) = 10.05
True value: 10.049875\ldots — off by 0.000124, about 0.001%. From one multiplication.
Choosing a is the only judgement call, and the rule is: pick the nearest point where you can evaluate f and f' exactly. For roots that means perfect squares/cubes; for trig, multiples of \frac\pi6; for logs and exponentials, x=1 and x=0.
How wrong is it?
The error is second order, and you can say so precisely. Taylor's theorem (§8.2) gives
f(x) - L(x) = \frac{f''(c)}{2}(x-a)^2
for some c between a and x. Three consequences worth having:
The error shrinks quadratically. Halve the distance, quarter the error. Reduce it tenfold and the error drops a hundredfold — which is exactly the 10^{-4}, 10^{-6}, 10^{-8}, 10^{-10} ladder you'll see in the code below.
Concavity gives the direction. If f'' > 0 the curve lies above its tangent, so the linearization underestimates. If f'' < 0 it overestimates. For \sqrt x, f'' < 0, so 10.05 should be too big — and it is ✓. You get the direction of the error for free, without computing anything.
A bound is available. If |f''| \le M near a, the error is at most \frac{M}{2}(x-a)^2. That turns an estimate into a guarantee.
Differentials
Same idea, different notation, and it's the notation §4 will use constantly.
Write dx for a small change in input and dy for the corresponding change along the tangent line:
dy = f'(x)\,dx
Meanwhile \Delta y = f(x+\Delta x) - f(x) is the true change along the curve. With dx = \Delta x:
\Delta y \approx dy, \qquad \text{error} = \Delta y - dy = O\!\left((\Delta x)^2\right)
The distinction is the whole content: dy is what the tangent predicts, \Delta y is what actually happens, and they agree to first order.
Historically dy and dx were "infinitesimals" — quantities smaller than any positive number but not zero. That's incoherent in the real numbers, which is why 19th-century analysis replaced them with limits. (Non-standard analysis rehabilitated them rigorously in the 1960s, but the limit definition won.) The notation survived because it's excellent, and \frac{dy}{dx} = \frac{dy}{du}\frac{du}{dx} is why.
Error propagation
The most useful application outside of estimating square roots.
You measure a quantity with some uncertainty. How uncertain is anything computed from it? Differentials answer this directly.
Absolute error: dy = f'(x)\,dx.
Relative error: \dfrac{dy}{y} = \dfrac{f'(x)}{f(x)}dx — the logarithmic derivative from §2.6.
Example. A sphere's radius is measured as 10 \pm 0.1 cm. How uncertain is the volume?
V = \tfrac43\pi r^3, \qquad dV = 4\pi r^2\,dr = 4\pi(100)(0.1) \approx 125.7\text{ cm}^3
Relative: $\frac{dV}{V} = \frac{4\pi r^2}{\frac43\pi r^3}dr = \frac{3\,dr}{r} = 3(0.01) = 3\%$.
A 1% error in radius becomes a 3% error in volume, because volume scales as r^3. The general rule falls straight out of \frac{d}{dr}\ln(r^n) = \frac nr:
y = x^n \implies \frac{dy}{y} = n\frac{dx}{x}
Relative errors get multiplied by the exponent. Cubing triples your uncertainty; taking a square root halves it. This is the single most useful thing in this lesson for anyone doing experimental work.
Doing it in Python
The quadratic error law, visible:
from math import sqrt
a, fa, fpa = 100.0, 10.0, 0.05
print(f"{'dx':>10} {'linear':>16} {'true':>16} {'error':>12} {'error/dx^2':>12}")
for dx in (10.0, 1.0, 0.1, 0.01, 0.001):
approx = fa + fpa * dx
true = sqrt(a + dx)
err = true - approx
print(f"{dx:>10} {approx:>16.10f} {true:>16.10f} {err:>12.2e} {err/dx**2:>12.6f}")
print("\nthe last column is constant: the error really is proportional to dx^2")
print("f''(100)/2 = -1/(8*1000) =", -1/8000)
Concavity predicting the direction of the error:
from math import sqrt, exp, sin
def report(name, f, fp, fpp, a, dx=0.1):
approx = f(a) + fp(a) * dx
true = f(a + dx)
side = "under" if fpp(a) > 0 else "over"
print(f"{name:<12} f''={fpp(a):>+9.5f} -> predicts {side:<6} "
f"actual error {true - approx:>+.2e}")
report("sqrt at 4", sqrt, lambda x: 0.5/sqrt(x), lambda x: -0.25*x**-1.5, 4.0)
report("e^x at 0", exp, exp, exp, 0.0)
report("sin at 1", sin, lambda x: __import__('math').cos(x),
lambda x: -sin(x), 1.0)
print("\nf'' > 0: curve above the tangent, so the linearization undershoots")
Error propagation, and the exponent rule:
from math import pi
r, dr = 10.0, 0.1
V = 4/3 * pi * r**3
dV = 4 * pi * r**2 * dr
print(f"radius : {r} +/- {dr} ({dr/r:.1%} relative)")
print(f"volume : {V:.2f} +/- {dV:.2f} ({dV/V:.1%} relative)")
print(f"\nratio of relative errors: {(dV/V) / (dr/r):.4f} -- exactly the exponent 3")
print("\nthe rule dy/y = n dx/x, checked:")
for n in (2, 3, 0.5, -1):
print(f" y = x^{n:<5}: a 1% error in x becomes {abs(n) * 1:.1f}% in y")
dy versus \Delta y:
def f(x):
return x ** 3
x = 2.0
print(f"{'dx':>10} {'dy (tangent)':>16} {'delta y (curve)':>18} {'difference':>14}")
for dx in (1.0, 0.5, 0.1, 0.01):
dy = 3 * x**2 * dx
delta = f(x + dx) - f(x)
print(f"{dx:>10} {dy:>16.8f} {delta:>18.8f} {delta - dy:>14.8f}")
print("\ndelta y - dy = 3x*dx^2 + dx^3: second order, so it vanishes faster than dx")
Worked example
Estimate \sqrt[3]{8.1} and bound the error.
f(x) = x^{1/3}, a = 8 (a perfect cube — that's why we chose it).
f(8) = 2, \qquad f'(x) = \tfrac13x^{-2/3}, \qquad f'(8) = \tfrac13\cdot\tfrac14 = \tfrac1{12}
\sqrt[3]{8.1} \approx 2 + \tfrac{1}{12}(0.1) = 2 + 0.00833\overline{3} = 2.008333
Which way is it wrong?
f''(x) = -\tfrac29 x^{-5/3} < 0 \text{ for } x > 0
Concave down, so the tangent lies above the curve: the estimate is too big.
By how much? On [8, 8.1], |f''| is largest at x=8:
|f''(8)| = \tfrac29 \cdot 8^{-5/3} = \tfrac29\cdot\tfrac{1}{32} = \tfrac{1}{144}
|\text{error}| \le \frac{M}{2}(x-a)^2 = \frac{1}{288}(0.1)^2 = \frac{0.01}{288} \approx 3.5\times10^{-5}
So \sqrt[3]{8.1} = 2.008333 \pm 0.000035, and we know it's on the low side of that. The true value is 2.0082988\ldots — error 3.4\times10^{-5}, right at the bound ✓.
That's the whole workflow: estimate from the tangent, get the direction from the sign of f'', get the size from a bound on |f''|. It converts a guess into a result with error bars.
Your turn
1. Estimate \sqrt{4.1} using linearization at a=4.
2. Estimate \sin(0.1), and compare to the true value.
3. A cube's edge is measured as 5 \pm 0.05 cm. Estimate the absolute and relative error in its volume.
4. Estimate (1.02)^{10}.
Solutions
1. f(x) = \sqrt x, f(4) = 2, f'(4) = \frac{1}{2\sqrt4} = 0.25:
\sqrt{4.1} \approx 2 + 0.25(0.1) = \boxed{2.025}
True: 2.024846. Error -1.5\times10^{-4}, and negative as concavity predicts (f''<0, so we overestimated) ✓.
2. f(x) = \sin x at a = 0: f(0)=0, f'(0) = \cos 0 = 1:
\sin(0.1) \approx 0 + 1(0.1) = \boxed{0.1}
True: 0.0998334. Error about -1.7\times10^{-4}.
This is the small-angle approximation \sin x \approx x from §0.4, and now you can see why it's good and how good: the error is $\frac{|f''(c)|}{2}x^2 \le \frac{x^2}{2} = 0.005$ — and in fact much better than that bound, because f''(0) = -\sin 0 = 0. The first correction is actually third order, -\frac{x^3}{6}, which is why the small-angle approximation is unreasonably accurate.
3. V = s^3, dV = 3s^2\,ds = 3(25)(0.05) = \boxed{3.75\text{ cm}^3}.
Relative: \frac{dV}{V} = \frac{3.75}{125} = 3\%. Or directly from the exponent rule: \frac{ds}{s} = \frac{0.05}{5} = 1\%, times the exponent 3 ✓.
4. f(x) = x^{10} at a = 1: f(1) = 1, f'(1) = 10:
(1.02)^{10} \approx 1 + 10(0.02) = \boxed{1.2}
True: 1.21899. Error about 1.6% — noticeably worse than the earlier estimates, because f'' = 90x^8 is large, and 0.02 raised to the tenth power amplifies things.
This one is worth slowing down for, because the obvious bound is wrong. The rule above says |f''| \le M near a — so M is the maximum of |f''| across the whole interval [1, 1.02], not its value at a. Take M = f''(1) = 90 and you get \frac{90}{2}(0.02)^2 = 0.018, while the true error is 0.018994. The "bound" is violated. Nothing went wrong with the arithmetic; M was simply sampled at the wrong end. Since f'' = 90x^8 is increasing, its maximum sits at the far endpoint:
M = f''(1.02) = 90(1.02)^8 = 105.45 \implies \text{error} \le \frac{105.45}{2}(0.02)^2 = 0.0211
and 0.018994 \le 0.0211 ✓. A bound that holds only at the basepoint is not a bound at all.
This is also the "10% growth for 10 years ≈ 100% total" fallacy: linearization ignores compounding, and compounding is precisely the second-order term.
Check yourself in code
Verify that the linearization error is proportional to (\Delta x)^2.
For f(x) = \sqrt x linearized at a=100 (so L(x) = 10 + \frac{\Delta x}{20}), print for \Delta x = 1, 0.1, 0.01, 0.001: the approximation and the true value to 10 decimals, and the error in scientific notation with 2 decimals.
Print exactly this:
dx=1.0 approx=10.0500000000 true=10.0498756211 err=-1.24e-04
dx=0.1 approx=10.0050000000 true=10.0049987506 err=-1.25e-06
dx=0.01 approx=10.0005000000 true=10.0004999875 err=-1.25e-08
dx=0.001 approx=10.0000500000 true=10.0000499999 err=-1.25e-10
from math import sqrt
for dx in (1.0, 0.1, 0.01, 0.001):
# tangent-line estimate at a=100, then the true sqrt, then the error
print(f"dx={dx:<7} ...")
from math import sqrt
for dx in (1.0, 0.1, 0.01, 0.001):
approx = 10 + dx / 20
true = sqrt(100 + dx)
print(f"dx={dx:<7} approx={approx:.10f} true={true:.10f} err={true-approx:.2e}")
f(x) \approx f(a) + f'(a)(x-a) replaces a curve by its tangent, and the error is \frac{f''(c)}{2}(x-a)^2 — second order, so halving the step quarters the error, and the sign of f'' tells you which way you're wrong before you compute anything. In differential notation dy = f'(x)dx is the tangent's prediction and \Delta y is the truth. Applied to measurement, relative errors multiply by the exponent: a 1% error in a radius is a 3% error in a volume.
Next: turning the tangent line into a root-finder that beats bisection by a mile.