BrevFeed
Sign in

27. Orthogonality, orthogonal complements

📖 Reading · 9 min
💡 Every code box below is live — edit it and hit Run.

§20.0 defined orthogonality for a pair of vectors. This lesson scales that idea up to whole subspaces, and introduces the orthogonal complement — every vector orthogonal to an entire subspace at once — which turns out to connect directly back to §17.4's null space and row space, tying this module to Module 17's machinery.

Orthogonal sets

A set of vectors \{\vec v_1,\dots,\vec v_k\} is orthogonal if every pair is orthogonal: \vec v_i\cdot\vec v_j=0 whenever i\neq j.

An orthogonal set of nonzero vectors is automatically linearly independent (§17.1). Suppose c_1\vec v_1+\cdots+c_k\vec v_k=\vec0. Dot both sides with \vec v_i:

c_1(\vec v_1\cdot\vec v_i)+\cdots+c_i(\vec v_i\cdot\vec v_i)+\cdots+c_k(\vec v_k\cdot\vec v_i)=0

Every term with j\neq i vanishes (orthogonality), leaving c_i\|\vec v_i\|^2=0. Since \vec v_i\neq\vec0, \|\vec v_i\|^2\neq0, forcing c_i=0 — for every i. This is a genuinely useful shortcut: checking independence for an orthogonal set never requires §17.1's row-reduction at all, just this one-line dot-product argument.

Orthogonal complement

For a subspace W\subseteq\mathbb{R}^n, the orthogonal complement is

W^\perp=\{\vec v\in\mathbb{R}^n:\vec v\cdot\vec w=0\text{ for every }\vec w\in W\}

— every vector orthogonal to all of W at once, not just to some vector in it. W^\perp is always a subspace (the same closure argument as §17.0, using bilinearity of the dot product in place of linear-combination arithmetic).

Key facts (stated here, and directly usable):

  • \dim W+\dim W^\perp=n — the two pieces exactly account for every dimension, echoing §17.5's rank-nullity in shape though it's a separate theorem.
  • (W^\perp)^\perp=W.
  • W\cap W^\perp=\{\vec0\} — a nonzero vector in both would need \vec v\cdot\vec v=0, forcing \vec v=\vec0 by positive definiteness (§20.1).

The connection to null space and row space

For any matrix A, \operatorname{Row}(A)^\perp=\operatorname{Null}(A). This is nearly definitional once stated correctly: \vec x\in\operatorname{Null}(A) means A\vec x=\vec0, and each entry of A\vec x is (row i of A)\cdot\vec x (§16.2's row-times-column rule) — so A\vec x=\vec0 says precisely that \vec x is orthogonal to every row of A, which (since the rows span \operatorname{Row}(A)) means orthogonal to all of \operatorname{Row}(A).

Combined with §17.4's rank-nullity numbers, this instantly reproves "\dim W+\dim W^\perp=n" for the case W=\operatorname{Row}(A): \dim\operatorname{Row}(A)+\dim\operatorname{Null}(A)=\operatorname{rank}(A)+\operatorname{nullity}(A)=n — §17.5's Rank-Nullity Theorem and this lesson's dimension formula are, in this case, literally the same equation viewed two ways. This connection is also exactly what makes §20.5's least squares work: the residual of a least-squares fit lives in \operatorname{Col}(A)^\perp, computed via \operatorname{Null}(A^T).

Doing it in Python

import sympy as sp

# W = row space of A (a plane through the origin in R^3)
A = sp.Matrix([[1, 2, -1]])   # a single row spans W

W_perp_basis = A.nullspace()   # Null(A) = Row(A)^perp
print(f"dim(W) = {A.rank()}, dim(W_perp) = {len(W_perp_basis)}")
for v in W_perp_basis:
    print(" ", list(v))

# Confirm every basis vector of W_perp really is orthogonal to A's row
row = A.row(0)
for v in W_perp_basis:
    print("row . v =", (row * v)[0])

dim(W) = 1, dim(W_perp) = 2
  [-2, 1, 0]
  [1, 0, 1]
row . v = 0
row . v = 0

Verifying (W^\perp)^\perp=W and orthogonal independence directly:

import numpy as np

v1 = np.array([1., 1., 0.])
v2 = np.array([1., -1., 2.])
v3 = np.array([-1., 1., 1.])   # chosen to be orthogonal to both

print("v1.v2 =", v1 @ v2)
print("v1.v3 =", v1 @ v3)
print("v2.v3 =", v2 @ v3)

A = np.column_stack([v1, v2, v3])
print("orthogonal set independent (full rank)?", np.linalg.matrix_rank(A) == 3)

v1.v2 = 0.0
v1.v3 = 0.0
v2.v3 = 0.0
orthogonal set independent (full rank)? True

Worked example

Find a basis for W^\perp where W=\operatorname{span}\{(1,1,1),(1,-1,0)\}.

\vec v=(x,y,z)\in W^\perp needs \vec v\cdot(1,1,1)=0 and \vec v\cdot(1,-1,0)=0: x+y+z=0 and x-y=0. From the second, x=y; substituting: 2x+z=0\Rightarrow z=-2x. Set x=1: \vec v=(1,1,-2).

\boxed{W^\perp=\operatorname{span}\{(1,1,-2)\}}

Sanity check. \dim W=2 (the two spanning vectors are independent — neither is a multiple of the other) and \dim W^\perp=1: 2+1=3=n ✓. Direct dot products: (1,1,-2)\cdot(1,1,1)=1+1-2=0 ✓ and (1,1,-2)\cdot(1,-1,0)=1-1+0=0 ✓ — orthogonal to both spanning vectors, hence (by bilinearity) to every linear combination of them, confirming it's orthogonal to all of W, not just the two vectors checked.

Your turn

1. Is \{(1,0,1),(0,1,0),(-1,0,1)\} an orthogonal set? If so, is it automatically independent?

2. For W=\operatorname{span}\{(1,0,0)\} (the x-axis) in \mathbb{R}^3, describe W^\perp geometrically.

3. True or false: W\cap W^\perp can contain a nonzero vector if W is large enough.

Solutions

1. Check pairwise: (1,0,1)\cdot(0,1,0)=0 ✓, (1,0,1)\cdot(-1,0,1)=-1+0+1=0 ✓, (0,1,0)\cdot(-1,0,1)=0 ✓ — yes, orthogonal. By the automatic-independence fact, it's independent without any further row-reduction (and since there are 3 vectors in \mathbb{R}^3, it's in fact a basis).

2. W^\perp=\{(x,y,z):x=0\} — the yz-plane. Geometrically, every vector orthogonal to the x-axis lies in the plane perpendicular to it, matching the everyday meaning of "perpendicular" exactly, and \dim W+\dim W^\perp=1+2=3 ✓.

3. False, always. W\cap W^\perp=\{\vec0\} unconditionally, for any subspace W — proved above using only positive definiteness, with no dependence on W's size. A vector can't be simultaneously "in W" and "orthogonal to everything in W, including itself" unless it's \vec0: the self-orthogonality \vec v\cdot\vec v=0 alone already forces this, regardless of \dim W.

Check yourself in code

For A=\begin{pmatrix}1&2&1\end{pmatrix}, find a basis for \operatorname{Row}(A)^\perp=\operatorname{Null}(A) and confirm each basis vector is orthogonal to A's row.

Print exactly this:

basis: [[-2, 1, 0], [-1, 0, 1]]
orthogonal to row: [True, True]

import sympy as sp

A = sp.Matrix([[1, 2, 1]])
basis = A.nullspace()
print("basis:", [list(v) for v in basis])
# print a list of booleans: whether each basis vector dotted with A's row gives 0

import sympy as sp

A = sp.Matrix([[1, 2, 1]])
basis = A.nullspace()
print("basis:", [list(v) for v in basis])

row = A.row(0)
checks = [(row * v)[0] == 0 for v in basis]
print("orthogonal to row:", checks)

An orthogonal set of nonzero vectors is automatically independent, and every subspace W has an orthogonal complement W^\perp with \dim W+\dim W^\perp=n and W\cap W^\perp=\{\vec0\} always. \operatorname{Row}(A)^\perp=\operatorname{Null}(A) ties this directly to Module 17's machinery — the same computation, read through a new lens.

Next: turning any independent set into an orthogonal one, mechanically — the Gram-Schmidt process.

← Previous Next →