29. Orthogonal and orthonormal bases

§20.3's Gram-Schmidt output is orthogonal but not yet normalized. This short lesson adds the final step, and cashes in the payoff: an orthonormal basis turns §17.3's coordinate-vector formula — which needed a full matrix inverse — into a single dot product.

Definition

A basis \{\vec q_1,\dots,\vec q_n\} is orthonormal if it's orthogonal (§20.2) and every vector is a unit vector: \vec q_i\cdot\vec q_j=0 for i\neq j, and \|\vec q_i\|=1 for every i. Compactly: \vec q_i\cdot\vec q_j=\delta_{ij} (the Kronecker delta, 1 if i=j and 0 otherwise).

Building one: run Gram-Schmidt (§20.3), then normalize each result: \vec q_i=\vec u_i/\|\vec u_i\|. Normalizing doesn't change any direction, so orthogonality survives untouched, and every \vec q_i now has length exactly 1.

The standard basis is already orthonormal — \vec e_i\cdot\vec e_j=\delta_{ij} directly — which is exactly why §16.0 never needed to worry about this issue; every formula in Modules 16–19 that looked simple in standard coordinates was secretly relying on orthonormality all along, without naming it.

Coordinates become dot products

For an orthonormal basis \{\vec q_1,\dots,\vec q_n\}, any vector \vec v decomposes as

\vec v=(\vec v\cdot\vec q_1)\vec q_1+(\vec v\cdot\vec q_2)\vec q_2+\cdots+(\vec v\cdot\vec q_n)\vec q_n

Derivation: write \vec v=c_1\vec q_1+\cdots+c_n\vec q_n (possible since it's a basis, §17.2), then dot both sides with \vec q_i:

\vec v\cdot\vec q_i=c_1(\vec q_1\cdot\vec q_i)+\cdots+c_i(\vec q_i\cdot\vec q_i)+\cdots=c_i(1)=c_i

— every other term vanishes by orthogonality, and the surviving term simplifies because \vec q_i\cdot\vec q_i=1. This is §17.3's [\vec v]_B=P_B^{-1}\vec v formula, with the inverse computed for free: comparing the two, P^{-1}=P^T when P's columns are orthonormal — no elimination, no cofactors, just a transpose. (§21.1 names such P orthogonal matrices and studies this identity directly.)

Orthogonal projection onto a subspace

For a subspace W with orthonormal basis \{\vec q_1,\dots,\vec q_k\}, the projection of any \vec v onto W is

\operatorname{proj}_W\vec v=(\vec v\cdot\vec q_1)\vec q_1+\cdots+(\vec v\cdot\vec q_k)\vec q_k

— the same coordinate formula, just stopped at k<n terms instead of all n. This is the closest point in W to \vec v (not proved here, but the geometric picture — dropping a perpendicular onto W — matches §20.0's single-vector projection exactly one dimension up), and it's precisely the tool §20.5's least squares needs.

Doing it in Python

import numpy as np

def normalize(v):
    return v / np.linalg.norm(v)

u1 = np.array([1., 1., 0.])
u2 = np.array([0.5, -0.5, 1.])   # already orthogonal, from Gram-Schmidt

q1, q2 = normalize(u1), normalize(u2)
print("q1 =", [round(x, 4) for x in q1.tolist()], " ||q1|| =", round(float(np.linalg.norm(q1)), 4))
print("q2 =", [round(x, 4) for x in q2.tolist()], " ||q2|| =", round(float(np.linalg.norm(q2)), 4))
print("q1 . q2 =", round(float(q1 @ q2), 10))

q1 = [0.7071, 0.7071, 0.0]  ||q1|| = 1.0
q2 = [0.4082, -0.4082, 0.8165]  ||q2|| = 1.0
q1 . q2 = 0.0

Coordinates via dot product, and projection onto a plane:

import numpy as np

q1 = np.array([1., 1., 0.]) / np.sqrt(2)
q2 = np.array([1., -1., 0.]) / np.sqrt(2)
q3 = np.array([0., 0., 1.])   # {q1, q2, q3} is orthonormal for R^3

v = np.array([3., 5., 7.])
coords = [v @ q1, v @ q2, v @ q3]
print("coordinates:", [round(c, 4) for c in coords])

reconstructed = coords[0]*q1 + coords[1]*q2 + coords[2]*q3
print("reconstructed v:", [round(x, 4) for x in reconstructed.tolist()])

# Projection onto W = span{q1, q2} (drop the q3 term)
proj = coords[0]*q1 + coords[1]*q2
print("proj_W(v):", [round(x, 4) for x in proj.tolist()])

coordinates: [5.6569, -1.4142, 7.0]
reconstructed v: [3.0, 5.0, 7.0]
proj_W(v): [3.0, 5.0, 0.0]

Worked example

Given the orthonormal basis \vec q_1=(\tfrac35,\tfrac45), \vec q_2=(-\tfrac45,\tfrac35) of \mathbb{R}^2, find the coordinates of \vec v=(7,1).

c_1=\vec v\cdot\vec q_1=7\left(\tfrac35\right)+1\left(\tfrac45\right)=\tfrac{21+4}5=5 c_2=\vec v\cdot\vec q_2=7\left(-\tfrac45\right)+1\left(\tfrac35\right)=\tfrac{-28+3}5=-5

\boxed{[\vec v]_Q=(5,-5)}

Sanity check. Reconstruct: 5\vec q_1-5\vec q_2=5(\tfrac35,\tfrac45)-5(-\tfrac45,\tfrac35)=(3,4)-(-4,3)=(3-(-4),\,4-3)=(7,1) ✓ — matches \vec v exactly, confirming both coordinates without ever computing P^{-1} directly (only two dot products were needed).

Your turn

1. Verify \vec q_1=(\tfrac35,\tfrac45), \vec q_2=(-\tfrac45,\tfrac35) from the worked example really is orthonormal (check norms and the dot product).

2. For the standard basis \{\vec e_1,\vec e_2,\vec e_3\}, what are the "coordinates via dot product" of \vec v=(a,b,c)? Does this match what you'd expect?

3. True or false: for an orthonormal basis, P^{-1}=P^T where P's columns are the basis vectors.

Solutions

1. \|\vec q_1\|=\sqrt{(3/5)^2+(4/5)^2}=\sqrt{9/25+16/25}=\sqrt1=1 ✓. \|\vec q_2\|=\sqrt{16/25+9/25}=1 ✓. \vec q_1\cdot\vec q_2=\tfrac35\left(-\tfrac45\right)+\tfrac45\left(\tfrac35\right)=-\tfrac{12}{25}+\tfrac{12}{25}=0 ✓ — genuinely orthonormal.

2. c_i=\vec v\cdot\vec e_i just picks off the i-th component: c_1=a,c_2=b,c_3=c — the coordinates are (a,b,c), i.e. \vec v itself. This matches §17.3's earlier remark that [\vec v]_E=\vec v exactly — now understood as the special case of this lesson's formula where the orthonormal basis happens to be standard.

3. True. This is exactly the identity stated in the "Coordinates become dot products" section: for orthonormal P, [\vec v]_B=P^T\vec v computes the same thing P^{-1}\vec v would, so P^{-1}=P^T — a genuinely useful shortcut, since transposing costs nothing while inverting generally requires elimination (§16.3).

Check yourself in code

For the orthonormal basis \vec q_1=(0.6,0.8), \vec q_2=(-0.8,0.6), find the coordinates of \vec v=(10,5) and reconstruct \vec v from them.

Print exactly this:

coordinates: [10.0, -5.0]
reconstructed: [10.0, 5.0]

import numpy as np

q1 = np.array([0.6, 0.8])
q2 = np.array([-0.8, 0.6])
v = np.array([10., 5.])

c1, c2 = v @ q1, v @ q2
print("coordinates:", [round(c1, 4), round(c2, 4)])
# reconstruct v from c1*q1 + c2*q2 and print it

import numpy as np

q1 = np.array([0.6, 0.8])
q2 = np.array([-0.8, 0.6])
v = np.array([10., 5.])

c1, c2 = v @ q1, v @ q2
print("coordinates:", [round(c1, 4), round(c2, 4)])

reconstructed = c1*q1 + c2*q2
print("reconstructed:", [round(x, 4) for x in reconstructed.tolist()])

An orthonormal basis is orthogonal and unit-length, and it collapses §17.3's coordinate machinery into single dot products: \vec v=\sum(\vec v\cdot\vec q_i)\vec q_i, with P^{-1}=P^T requiring no elimination at all. Stopping the same sum early gives \operatorname{proj}_W\vec v, the closest point in a subspace W.

Next: the case where the target vector isn't reachable at all — an inconsistent system — and how orthogonal projection turns that failure into the best possible approximate answer: least squares.