BrevFeed
Sign in

34. Positive definite / semidefinite matrices

📖 Reading · 10 min
💡 Every code box below is live — edit it and hit Run.

§21.0 guaranteed real eigenvalues for every symmetric matrix, but said nothing about their sign. This lesson studies the case where they're all positive — a condition with a strikingly simple algebraic test, and the property that makes optimization problems well-behaved (Module 24 and the statistics course's regression chapter both lean on it directly).

Definitions

A symmetric matrix A is:

  • Positive definite (PD) if \vec x^TA\vec x>0 for every nonzero \vec x\in\mathbb{R}^n.
  • Positive semidefinite (PSD) if \vec x^TA\vec x\ge0 for every \vec x (equality allowed, unlike PD).
  • Negative definite/semidefinite — the mirrored conditions, \vec x^TA\vec x<0 or \le0.
  • Indefinite — none of the above; \vec x^TA\vec x takes both signs depending on \vec x.

\vec x^TA\vec x is a quadratic form (§21.3 studies these in their own right); this lesson only needs its sign.

The eigenvalue test

A (symmetric) is positive definite \iff every eigenvalue of A is positive. (PSD \iff every eigenvalue is \ge0; the negative cases mirror this exactly.)

Proof sketch, using §21.0 directly: write A=Q\Lambda Q^T. Let \vec y=Q^T\vec x (so \vec x=Q\vec y, and \vec x\neq\vec0\iff\vec y\neq\vec0 since Q is invertible). Then

\vec x^TA\vec x=\vec x^TQ\Lambda Q^T\vec x=\vec y^T\Lambda\vec y=\sum_i\lambda_iy_i^2

A sum of \lambda_iy_i^2 terms is positive for every nonzero \vec y exactly when every \lambda_i>0 (if some \lambda_i\le0, choosing \vec y=\vec e_i makes the sum \le0; if all \lambda_i>0, any nonzero \vec y has some nonzero y_i^2 term, all contributing positively). This is the single cleanest characterization available — diagonalizing turns an n-variable positivity question into n independent single-variable ones.

Other equivalent tests

  • All leading principal minors positive (the determinants of the top-left 1\times1, 2\times2, …, n\times n submatrices) — Sylvester's criterion, useful for hand-checking small matrices without finding eigenvalues at all.
  • A=B^TB for some matrix B with independent columns — and conversely, B^TB is always PSD for any B (§20.5's normal equations relied on exactly this: \vec x^T(B^TB)\vec x=\|B\vec x\|^2\ge0 always, with equality only when B\vec x=\vec0, i.e. strictly PD exactly when B's columns are independent — the same invertibility condition §20.5 needed for a unique least-squares solution).
  • Cholesky decomposition exists (§22.0): A=LL^T for a real triangular L — this is the practical test software actually uses, since it's cheaper than computing eigenvalues.

Why it matters

A PD matrix's quadratic form \vec x^TA\vec x traces out an ellipsoid (never a saddle-shaped or unbounded surface) — exactly the shape that guarantees a unique minimum in optimization (§14.2's Hessian test for calculus, restated: a critical point is a strict local minimum exactly when the Hessian is PD there). Covariance matrices in statistics are always PSD by construction (they're B^TB-type objects built from centered data), which is why variance can never come out negative — a PD requirement hiding in plain sight throughout the statistics course.

Doing it in Python

import numpy as np

def classify(A):
    eigvals = np.linalg.eigvalsh(A)
    if np.all(eigvals > 0): return "positive definite"
    if np.all(eigvals >= 0): return "positive semidefinite"
    if np.all(eigvals < 0): return "negative definite"
    if np.all(eigvals <= 0): return "negative semidefinite"
    return "indefinite"

A1 = np.array([[2., 1.], [1., 2.]])
A2 = np.array([[1., 1.], [1., 1.]])   # rank 1: singular, PSD not PD
A3 = np.array([[1., 2.], [2., 1.]])   # indefinite

for name, A in [("A1", A1), ("A2", A2), ("A3", A3)]:
    print(f"{name}: eigenvalues={[round(v,4) for v in np.linalg.eigvalsh(A)]} -> {classify(A)}")

A1: eigenvalues=[1.0, 3.0] -> positive definite
A2: eigenvalues=[0.0, 2.0] -> positive semidefinite
A3: eigenvalues=[-1.0, 3.0] -> indefinite

Confirming B^TB is always PSD, and PD when B's columns are independent:

import numpy as np

B_full_rank = np.array([[1., 0.], [0., 1.], [1., 1.]])   # independent columns
B_deficient = np.array([[1., 2.], [2., 4.], [3., 6.]])     # column 2 = 2 * column 1

for name, B in [("full rank", B_full_rank), ("rank deficient", B_deficient)]:
    A = B.T @ B
    eigvals = np.linalg.eigvalsh(A)
    print(f"{name}: B^T B eigenvalues = {[round(v, 4) for v in eigvals]}")

full rank: B^T B eigenvalues = [1.0, 3.0]
rank deficient: B^T B eigenvalues = [0.0, 70.0]

Worked example

Classify A=\begin{pmatrix}5&2\\2&3\end{pmatrix} using Sylvester's criterion.

Leading principal minors: the 1\times1 minor is A_{11}=5>0. The 2\times2 minor is \det A=5(3)-2(2)=15-4=11>0. Both positive.

\boxed{A\text{ is positive definite}}

Sanity check via eigenvalues: \operatorname{tr}A=8, \det A=11. \lambda^2-8\lambda+11=0\Rightarrow\lambda=\frac{8\pm\sqrt{64-44}}2=\frac{8\pm\sqrt{20}}2=4\pm\sqrt5. Both 4-\sqrt5\approx1.76>0 and 4+\sqrt5\approx6.24>0 ✓ — both eigenvalues positive, confirming PD independently via the eigenvalue test, agreeing with Sylvester's criterion despite using an entirely different calculation.

Your turn

1. Classify A=\begin{pmatrix}-2&0\\0&-3\end{pmatrix} using the eigenvalue test (immediate for a diagonal matrix).

2. Use Sylvester's criterion to classify A=\begin{pmatrix}1&2\\2&1\end{pmatrix}.

3. True or false: every diagonal matrix with positive diagonal entries is positive definite.

Solutions

1. Diagonal, so eigenvalues are -2,-3 — both negative. Negative definite.

2. 1\times1 minor: 1>0. 2\times2 minor: \det A=1-4=-3<0 — fails Sylvester's criterion (needs all leading minors positive), so not positive definite. (Eigenvalues: \operatorname{tr}=2,\det=-3, \lambda^2-2\lambda-3=0\Rightarrow\lambda=3,-1 — one positive, one negative, confirming indefinite, consistent with Sylvester failing rather than indicating a different definite category.)

3. True. For diagonal A=\operatorname{diag}(d_1,\dots,d_n), \vec x^TA\vec x=\sum d_ix_i^2 — a sum of nonnegative terms (x_i^2\ge0) each scaled by a positive d_i, so the whole sum is positive whenever some x_i\neq0, i.e. whenever \vec x\neq\vec0. (This also follows immediately from the eigenvalue test: a diagonal matrix's eigenvalues are its diagonal entries, §19.1's shortcut.)

Check yourself in code

Classify A=\begin{pmatrix}4&2\\2&5\end{pmatrix} using its eigenvalues.

Print exactly this:

eigenvalues: [2.4384, 6.5616]
classification: positive definite

import numpy as np

A = np.array([[4., 2.], [2., 5.]])
eigvals = np.linalg.eigvalsh(A)
print("eigenvalues:", [round(v, 4) for v in eigvals])
# print "classification: positive definite" if all eigenvalues > 0

import numpy as np

A = np.array([[4., 2.], [2., 5.]])
eigvals = np.linalg.eigvalsh(A)
print("eigenvalues:", [round(v, 4) for v in eigvals])

if np.all(eigvals > 0):
    print("classification: positive definite")
else:
    print("classification: not positive definite")

A symmetric matrix is positive definite exactly when every eigenvalue is positive — equivalently, when every leading principal minor is positive (Sylvester), or when A=B^TB for full-column-rank B. Positive definiteness is what makes a quadratic form's graph a bounded ellipsoid, guaranteeing unique minima in optimization and legitimate (nonnegative) variance in statistics.

Next: the object whose sign this lesson just studied — the quadratic form \vec x^TA\vec x itself, and how eigenvalues classify its geometric shape in full.

← Previous Next →