BrevFeed
Sign in

35. Quadratic forms, classification via eigenvalues

📖 Reading · 9 min
💡 Every code box below is live — edit it and hit Run.

§21.2 asked only whether \vec x^TA\vec x was always positive, always negative, or neither. This lesson looks at the whole shape that expression traces out, and shows the Spectral Theorem turns any quadratic form — however tangled its cross terms look — into the simplest possible expression: a sum of squares.

What a quadratic form is

A quadratic form on \mathbb{R}^n is Q(\vec x)=\vec x^TA\vec x for a symmetric matrix A. Expanded,

Q(\vec x)=\sum_ia_{ii}x_i^2+2\sum_{i<j}a_{ij}x_ix_j

— diagonal entries give pure squared terms, off-diagonal entries give cross terms (each counted twice, once from A_{ij} and once from A_{ji}=A_{ij}, which is exactly why the factor of 2 appears, and why A is taken symmetric in the first place: any quadratic expression can be written this way with a unique symmetric A).

Example: Q(x,y)=3x^2+4xy+y^2 corresponds to A=\begin{pmatrix}3&2\\2&1\end{pmatrix} — the cross-term coefficient 4 splits evenly into two off-diagonal 2s.

Diagonalizing a quadratic form

By §21.0, A=Q\Lambda Q^T for orthogonal Q. Substituting \vec x=Q\vec y (a change of variables, §17.3, using an orthonormal basis of eigenvectors):

\vec x^TA\vec x=(Q\vec y)^TQ\Lambda Q^T(Q\vec y)=\vec y^TQ^TQ\Lambda Q^TQ\vec y=\vec y^T\Lambda\vec y=\lambda_1y_1^2+\cdots+\lambda_ny_n^2

using Q^TQ=I twice. Every cross term vanishes in the rotated coordinates \vec y — this is called the principal axes of the quadratic form, and it's the exact same calculation §21.2 already used to prove the eigenvalue test, now kept for its own sake rather than just its sign.

Classifying the shape

The signs of \lambda_1,\dots,\lambda_n classify Q(\vec x)=c's level sets completely, extending §21.2's PD/PSD/indefinite language into actual geometric shapes (in \mathbb{R}^2):

  • All \lambda_i>0 (PD): level sets are ellipses — bounded, closed curves.
  • All \lambda_i<0 (negative definite): also ellipses, just for Q(\vec x)=c<0 instead.
  • Mixed signs (indefinite): level sets are hyperbolas — open, unbounded curves, with the two eigenvector directions as asymptotic directions.
  • Some \lambda_i=0 (semidefinite, not definite): degenerate — the quadratic form doesn't depend on that eigenvector direction at all, giving parallel lines or an unbounded strip instead of a curve.

This is exactly §14.2's second-derivative (Hessian) test for classifying critical points, restated: a critical point is a local min if the Hessian's quadratic form is PD there (an "ellipse" bowl opening upward in every direction), a local max if negative definite, and a saddle if indefinite (a "hyperbola" shape, curving up in one principal direction and down in another) — the geometric picture in this lesson is the geometric picture behind that calculus test.

Doing it in Python

import numpy as np

def Q(A, x):
    return x @ A @ x

A_ellipse = np.array([[3., 1.], [1., 2.]])     # both eigenvalues positive
A_hyperbola = np.array([[1., 2.], [2., 1.]])   # eigenvalues 3, -1

for name, A in [("A_ellipse", A_ellipse), ("A_hyperbola", A_hyperbola)]:
    eigvals = np.linalg.eigvalsh(A)
    print(f"{name}: eigenvalues={[round(v,4) for v in eigvals]}")
    print(f"  Q(1,0)={Q(A,np.array([1.,0.]))}, Q(0,1)={Q(A,np.array([0.,1.]))}, Q(1,1)={Q(A,np.array([1.,1.]))}")

A_ellipse: eigenvalues=[1.382, 3.618]
  Q(1,0)=3.0, Q(0,1)=2.0, Q(1,1)=7.0
A_hyperbola: eigenvalues=[-1.0, 3.0]
  Q(1,0)=1.0, Q(0,1)=1.0, Q(1,1)=6.0

Confirming diagonalization eliminates the cross term, in the rotated coordinates:

import numpy as np

A = np.array([[3., 2.], [2., 1.]])   # Q(x,y) = 3x^2 + 4xy + y^2
eigvals, Q_mat = np.linalg.eigh(A)

x = np.array([2., -1.])
y = Q_mat.T @ x   # coordinates in the principal-axis basis

original = x @ A @ x
diagonalized = eigvals[0]*y[0]**2 + eigvals[1]*y[1]**2

print("Q(x) via original A:      ", round(float(original), 6))
print("Q(x) via diagonal lambda*y^2:", round(float(diagonalized), 6))
print("match:", np.isclose(original, diagonalized))

Q(x) via original A:       5.0
Q(x) via diagonal lambda*y^2: 5.0
match: True

Worked example

Classify the quadratic form Q(x,y)=2x^2-4xy+2y^2.

A=\begin{pmatrix}2&-2\\-2&2\end{pmatrix} (cross-term -4 splits into two -2s). \operatorname{tr}A=4, \det A=4-4=0. \lambda^2-4\lambda=0\Rightarrow\lambda(\lambda-4)=0\Rightarrow\lambda=0,4.

\boxed{\text{Positive semidefinite (degenerate)}: \text{one eigenvalue is }0}

Sanity check. Direct factoring confirms the degeneracy: Q(x,y)=2x^2-4xy+2y^2=2(x-y)^2\ge0 always, and exactly 0 whenever x=y — an entire line of inputs giving Q=0, not just the origin, matching "some \lambda_i=0" precisely: the eigenvector for \lambda=0 turns out to be (1,1), the direction along which Q doesn't grow at all.

Your turn

1. Write the matrix A for Q(x,y)=5x^2+6xy-y^2.

2. Classify Q(x,y)=x^2+y^2 using eigenvalues (should be immediate).

3. True or false: a quadratic form with \det A<0 (for 2\times2 symmetric A) is always indefinite.

Solutions

1. A=\begin{pmatrix}5&3\\3&-1\end{pmatrix} (cross-term 6 splits into two 3s).

2. A=I, eigenvalues 1,1 — both positive. Positive definite (the level sets are literal circles, the simplest possible ellipse).

3. True. For 2\times2, \det A=\lambda_1\lambda_2 (§19.3's trace/determinant-from-eigenvalues identity). \det A<0 forces \lambda_1,\lambda_2 to have opposite signs (their product is negative only when exactly one is negative) — which is precisely the "mixed signs" indefinite case. This gives a one-line determinant check for indefiniteness in 2\times2, no eigenvalue computation needed at all.

Check yourself in code

For Q(x,y)=x^2+4xy+4y^2, i.e. A=\begin{pmatrix}1&2\\2&4\end{pmatrix}, find the eigenvalues and classify.

Print exactly this:

eigenvalues: [0.0, 5.0]
classification: positive semidefinite (degenerate)

import numpy as np

A = np.array([[1., 2.], [2., 4.]])
eigvals = np.linalg.eigvalsh(A)
print("eigenvalues:", [round(v, 4) or 0.0 for v in eigvals])
# print the classification: check if any eigenvalue is (near) zero and
# the rest are >= 0 -> "positive semidefinite (degenerate)"

import numpy as np

A = np.array([[1., 2.], [2., 4.]])
eigvals = np.linalg.eigvalsh(A)
print("eigenvalues:", [round(v, 4) or 0.0 for v in eigvals])

has_zero = np.any(np.isclose(eigvals, 0))
all_nonneg = np.all(eigvals >= -1e-9)
if has_zero and all_nonneg:
    print("classification: positive semidefinite (degenerate)")
else:
    print("classification: other")

A quadratic form Q(\vec x)=\vec x^TA\vec x diagonalizes via the Spectral Theorem into \sum\lambda_iy_i^2 in eigenvector (principal-axis) coordinates, eliminating every cross term. The eigenvalue signs classify its level sets completely — ellipses (definite), hyperbolas (indefinite), or degenerate strips (semidefinite) — which is exactly the geometry underlying the calculus Hessian test for local extrema.

Next: the decomposition that extends everything in this module to matrices that aren't even square — the Singular Value Decomposition.

← Previous Next →