25. Adv: the derivative as a linear map; the inverse and implicit function theorems

This course opened, in §2.0, with the derivative as a slope — a single number. §10.3 upgraded it to a tangent plane, and §14.2's Jacobian upgraded it again to a matrix. This closing lesson names what all three really were, all along: the derivative is the best linear approximation to a function near a point, made fully rigorous with an actual limit rather than the informal "\approx" every earlier module leaned on — and that single reframing is what finally explains, with genuine proof rather than a formula taken on faith, why the inverse function derivative (§2.7) and implicit differentiation (§2.9, §10.4) work at all.

The rigorous definition

f:\mathbb R^n\to\mathbb R^m is differentiable at \vec a if there exists a linear map L such that

\lim_{\vec h\to\vec0}\frac{|f(\vec a+\vec h)-f(\vec a)-L(\vec h)|}{|\vec h|}=0

In one variable, L(h)=f'(a)\cdot h — an ordinary number acting as a 1\times1 linear map — and the condition becomes \lim_{h\to0}\frac{|f(a+h)-f(a)-f'(a)h|}h=0, which is exactly §2.0's difference-quotient definition, rearranged: \frac{f(a+h)-f(a)}h\to f'(a) is algebraically identical to this limit vanishing.

In several variables, L is represented by the Jacobian matrix (§14.2) — this single definition is the rigorous version of §10.3's tangent-plane approximation and §14.2's linear-approximation claim, finally justified with a genuine limit rather than asserted by analogy. The derivative is not a number, and not even fundamentally a matrix — it is the linear map that best approximates f near \vec a, with the number or matrix being merely how that map gets written down in coordinates.

The Inverse Function Theorem

If f is continuously differentiable near a and f'(a)\ne0 (equivalently, in higher dimensions: the Jacobian at \vec a is an invertible matrix), then f has a differentiable inverse defined on some neighborhood of f(a), and \big(f^{-1}\big)'(f(a))=\big[f'(a)\big]^{-1}

This is §2.7's inverse-function derivative formula, \frac{dx}{dy}=\dfrac1{dy/dx}, now stated with an actual existence guarantee attached: §2.7 only ever showed what the derivative of an inverse would be, assuming the inverse already existed; this theorem proves the inverse genuinely exists (at least locally, near the point in question) whenever the linear approximation f'(a) is invertible. In higher dimensions, "invert a number" becomes "invert a matrix" — the formula's whole structure survives unchanged, only the arithmetic of inversion gets promoted from division to matrix inversion.

Why invertibility of the derivative is the right condition: the derivative is the best linear approximation to f, so if that linear approximation is invertible, f itself should be "invertible enough" close to a — a linear map with no inverse collapses some direction to zero width, and no nearby nonlinear perturbation of it can un-collapse that direction either.

The Implicit Function Theorem

If F(x,y) is continuously differentiable near a point (a,b) with F(a,b)=0 and F_y(a,b)\ne0, then near (a,b), the equation F(x,y)=0 implicitly defines y as a differentiable function of x, with \dfrac{dy}{dx}=-\dfrac{F_x}{F_y}.

This is exactly §10.4's implicit-differentiation formula, derived there from the multivariable chain rule under the silent assumption that y(x) already exists as a genuine function — this theorem supplies the missing existence proof, and it does so via a clever reduction to the Inverse Function Theorem just proven: define G(x,y)=(x,F(x,y)), a map from \mathbb R^2 to \mathbb R^2. Its Jacobian at (a,b) is \begin{pmatrix}1&0\\F_x&F_y\end{pmatrix}, with determinant F_y(a,b) — invertible exactly when F_y(a,b)\ne0. The Inverse Function Theorem then guarantees G has a local inverse, and unpacking that inverse's second component produces precisely the function y(x) the implicit function theorem promises.

Both theorems, together, close the final gap this entire course has carried since §2.7 and §2.9: every time this course differentiated an inverse or an implicitly-defined curve, it was legitimate — and now, finally, provably so.

Doing it in Python

Confirming the rigorous linear-map definition holds for f(x)=x^2 at a=2 — the error ratio genuinely vanishes as h\to0:

def f(x):
    return x**2

a, f_prime_a = 2, 4   # f'(2) = 4, from the power rule

print(f"{'h':>8} {'|f(a+h)-f(a)-f_prime_a*h| / |h|':>32}")
for h in (0.1, 0.01, 0.001, 0.0001):
    ratio = abs(f(a+h) - f(a) - f_prime_a*h) / abs(h)
    print(f"{h:>8} {ratio:>32.8f}")
print("\nratio -> 0, confirming f'(2)*h is the correct linear approximation")

Confirming the Inverse Function Theorem numerically for g(x)=x^3+x: computing g^{-1} near b=2=g(1) via Newton's method (§3.2), and checking (g^{-1})'(2)=\frac1{g'(1)}:

def g(x):
    return x**3 + x

def g_prime(x):
    return 3*x**2 + 1

def newton_inverse(y_target, x0=1.0, steps=50):
    x = x0
    for _ in range(steps):
        x = x - (g(x) - y_target) / g_prime(x)
    return x

b = 2.0
epsilon = 1e-5
x_at_b = newton_inverse(b)
x_at_b_plus = newton_inverse(b + epsilon)

derivative_estimate = (x_at_b_plus - x_at_b) / epsilon
print(f"g^-1(2) = {x_at_b}")
print(f"estimated (g^-1)'(2) = {derivative_estimate:.6f}")
print(f"1 / g'(1) = {1/g_prime(1):.6f}")

Confirming the Implicit Function Theorem's derivative formula for F(x,y)=x^2+y^2-25 (a circle), checked against direct implicit differentiation from §2.9:

import sympy as sp

x, y = sp.symbols('x y')
F = x**2 + y**2 - 25

F_x, F_y = sp.diff(F, x), sp.diff(F, y)
implicit_derivative = -F_x / F_y

point = {x: 3, y: 4}   # on the circle: 9+16=25
print(f"F_y at (3,4) = {F_y.subs(point)}   (nonzero -- implicit function theorem applies)")
print(f"dy/dx = {implicit_derivative.subs(point)}")

Worked example

Verify the Inverse Function Theorem for g(x)=x^3+x at a=1: confirm g'(1)\ne0, and check (g^{-1})'(g(1))=\dfrac1{g'(1)} numerically.

g'(x)=3x^2+1

g'(1)=3(1)+1=4\ne0

Since g'(1)\ne0, the Inverse Function Theorem guarantees g^{-1} exists and is differentiable near g(1)=1+1=2, with

\big(g^{-1}\big)'(2)=\frac1{g'(1)}=\frac14=\boxed{0.25}

Numerical confirmation: using Newton's method to actually compute g^{-1}(2) and g^{-1}(2+\varepsilon) for tiny \varepsilon, then estimating the derivative by a difference quotient (§2.0), gives \approx0.24999953 — matching 0.25 to five decimal places.

Sanity check. g(x)=x^3+x has g'(x)=3x^2+1\ge1>0 for every real x — never zero anywhere, meaning the Inverse Function Theorem's hypothesis actually holds globally here, not just locally at a=1 (consistent with g being strictly increasing everywhere and therefore genuinely invertible on all of \mathbb R, not merely "locally invertible" near one point). The near-perfect numerical match — five correct decimal digits from a Newton's-method-plus-finite-difference estimate — is exactly the kind of independent confirmation this course has run at the end of nearly every worked example since Module 1, now applied to close its very last one. ✓

Your turn

1. For f(x)=e^x at a=0, state what the Inverse Function Theorem guarantees, and verify f'(0)\ne0.

2. For F(x,y)=x^3+y^3-6xy (the folium of Descartes, from §10.4's own "Your turn," problem 2) at the point (3,3) — check that F(3,3)=0 and that F_y(3,3)\ne0, confirming the Implicit Function Theorem applies there.

3. True or false: the Inverse Function Theorem guarantees a global inverse for f whenever f'(a)\ne0 at a single point a.

Solutions

1. f'(x)=e^x, so f'(0)=e^0=1\ne0. The Inverse Function Theorem guarantees e^x has a differentiable inverse near x=0 (which is, of course, \ln y, already known from §2.6 — but now with a rigorous existence guarantee rather than an assumed fact), with

\big(f^{-1}\big)'(f(0))=\frac1{f'(0)}=\frac11=1

matching \frac{d}{dy}\ln y\big|_{y=1}=\frac11=1 exactly.

2. F(3,3)=27+27-6(9)=54-54=0 ✓ — the point lies on the curve. F_y=3y^2-6x, so F_y(3,3)=3(9)-6(3)=27-18=9\ne0 ✓.

\boxed{\text{the Implicit Function Theorem applies at }(3,3)}

(consistent with §10.4's own computation there, which found \frac{dy}{dx}=\frac{2y-x^2}{y^2-2x} well-defined at this exact point, since the denominator y^2-2x=9-6=3\ne0 too.)

3. False. The theorem is explicitly local — it guarantees an inverse exists on some neighborhood of a, not on all of f's domain. A classic counterexample: f(x)=x^2 (§0.2) has f'(1)=2\ne0, so the theorem correctly guarantees a local inverse near x=1 (namely \sqrt y, valid for y near 1) — but f has no global inverse on all of \mathbb R, since f(-1)=f(1)=1 violates the one-to-one requirement any inverse needs. Non-zero derivative at one point says nothing about the function's behavior far away.

Check yourself in code

For g(x)=x^3+x, verify g'(1)\ne0 and confirm the Inverse Function Theorem's prediction (g^{-1})'(g(1))=\dfrac1{g'(1)} numerically.

Print exactly this:

g'(1) = 4
predicted (g^-1)'(2) = 0.25
numerical estimate = 0.250000...

def g(x):
    return x**3 + x

def g_prime(x):
    return 3*x**2 + 1

print("g'(1) = ...")
print("predicted (g^-1)'(2) = ...")

def newton_inverse(y_target, x0=1.0, steps=50):
    x = x0
    for _ in range(steps):
        x = x - (g(x) - y_target) / g_prime(x)
    return x

b, epsilon = 2.0, 1e-5
x_at_b = newton_inverse(b)
x_at_b_plus = newton_inverse(b + epsilon)
estimate = (x_at_b_plus - x_at_b) / epsilon
print(f"numerical estimate = {estimate:.6f}...")

def g(x):
    return x**3 + x

def g_prime(x):
    return 3*x**2 + 1

print(f"g'(1) = {g_prime(1)}")
print(f"predicted (g^-1)'(2) = {1/g_prime(1)}")

def newton_inverse(y_target, x0=1.0, steps=50):
    x = x0
    for _ in range(steps):
        x = x - (g(x) - y_target) / g_prime(x)
    return x

b, epsilon = 2.0, 1e-5
x_at_b = newton_inverse(b)
x_at_b_plus = newton_inverse(b + epsilon)
estimate = (x_at_b_plus - x_at_b) / epsilon
print(f"numerical estimate = {estimate:.6f}...")

The derivative, defined rigorously, is the linear map L making f(\vec a+\vec h)\approx f(\vec a)+L(\vec h) error-free in the limit — the single idea underlying §2.0's slope, §10.3's tangent plane, and §14.2's Jacobian all along. The Inverse Function Theorem upgrades §2.7's inverse-derivative formula from an assumed fact to a proven one, guaranteed whenever that linear approximation is invertible, and the Implicit Function Theorem does the same for §2.9's implicit differentiation and §10.4's -F_x/F_y formula, by reducing to the inverse function theorem through a clever change of coordinates.

That closes this course. Sixteen modules ago, a limit was defined as a number a function approaches; every subsequent idea — derivatives, integrals, series, vectors, multivariable calculus, differential equations, machine learning — was built by asking what that single idea could measure next, and this final module went back to confirm, with full rigor, that the foundation underneath all of it was solid the whole time.