BrevFeed
Sign in

26. General inner products (abstract inner product spaces)

📖 Reading · 10 min
💡 Every code box below is live — edit it and hit Run.

§17.0 generalized \mathbb{R}^n's addition and scaling into the vector space axioms. This lesson runs the identical playbook on §20.0's dot product: extract the properties that made it useful, and let anything with those properties be called an inner product — valid on polynomials, matrices, and functions exactly as §17.0 promised those spaces would eventually get every tool built for \mathbb{R}^n.

Definition

An inner product on a vector space V is a function \langle\cdot,\cdot\rangle:V\times V\to\mathbb{R} satisfying, for all \vec u,\vec v,\vec w\in V and scalar c:

  1. Symmetry: \langle\vec u,\vec v\rangle=\langle\vec v,\vec u\rangle
  2. Linearity in the first argument: \langle\vec u+\vec v,\vec w\rangle=\langle\vec u,\vec w\rangle+\langle\vec v,\vec w\rangle and \langle c\vec u,\vec v\rangle=c\langle\vec u,\vec v\rangle (combined with symmetry, this gives linearity in the second argument too — the pair of properties together is called bilinearity)
  3. Positive definiteness: \langle\vec v,\vec v\rangle\ge0, with equality only when \vec v=\vec0

A vector space equipped with an inner product is an inner product space. The dot product on \mathbb{R}^n (§20.0) is the motivating example — all three axioms were already checked there — but nothing in the definition requires "vectors that are lists of numbers."

Every general result from §20.0 transfers immediately: norm \|\vec v\|=\sqrt{\langle\vec v,\vec v\rangle}, Cauchy-Schwarz |\langle\vec u,\vec v\rangle|\le\|\vec u\|\|\vec v\|, angle via \cos\theta=\langle\vec u,\vec v\rangle/\|\vec u\|\|\vec v\|, and orthogonality \langle\vec u,\vec v\rangle=0 — none of the proofs from §20.0 used anything beyond the three axioms above, so they all carry over to any inner product space without modification, exactly the payoff §18.4's isomorphism theorem previewed for vector-space theorems in general.

Examples beyond \mathbb{R}^n

  • Polynomials, \langle p,q\rangle=\int_a^bp(x)q(x)\,dx — check positive definiteness: \langle p,p\rangle=\int_a^bp(x)^2\,dx\ge0, and it's 0 only if p\equiv0 on [a,b] (a nonzero continuous function has p(x)^2>0 somewhere on an interval, making the integral strictly positive). This inner product underlies Legendre polynomials and orthogonal polynomial theory generally, via §20.3's Gram-Schmidt applied to \{1,x,x^2,\dots\}.
  • Matrices, \langle A,B\rangle=\operatorname{tr}(A^TB) — equivalent to flattening both matrices into vectors and taking their dot product (§18.4's coordinate isomorphism M_{m\times n}\cong \mathbb{R}^{mn} makes this exact), so it inherits every inner-product property for free.
  • A weighted dot product, \langle\vec u,\vec v\rangle=\sum_iw_iu_iv_i for fixed positive weights w_i — still bilinear and symmetric trivially, and positive definite exactly because every w_i>0 (if some w_i\le0, positive definiteness would fail, and it would not be a valid inner product) — used constantly in statistics when some coordinates matter more than others.

The general Cauchy-Schwarz proof

Worth seeing once in full generality, since it's genuinely surprising that a positivity condition alone forces an inequality on inner products: for any t\in\mathbb{R},

0\le\langle\vec u-t\vec v,\vec u-t\vec v\rangle=\|\vec u\|^2-2t\langle\vec u,\vec v\rangle+t^2\|\vec v\|^2

using axiom 3 for the first step and bilinearity to expand. This is a quadratic in t that's never negative, so its discriminant can't be positive: 4\langle\vec u,\vec v\rangle^2-4\|\vec u\|^2\|\vec v\|^2\le0, which rearranges directly to Cauchy-Schwarz. Every step used only axioms 1–3 — nothing about \mathbb{R}^n specifically — confirming the inequality really does hold in any inner product space, polynomials and matrices included.

Doing it in Python

import numpy as np
from scipy.integrate import trapezoid

def poly_inner(p, q, a=-1, b=1, n=2000):
    """<p, q> = integral of p(x)q(x) dx via a fine numerical quadrature."""
    xs = np.linspace(a, b, n)
    return trapezoid(p(xs) * q(xs), xs)

p = lambda x: x            # p(x) = x
q = lambda x: x**2          # q(x) = x^2  (odd * even -> integrand is odd)
r = lambda x: 3*x**2 - 1    # a Legendre-like polynomial

print("<p, q> =", round(float(poly_inner(p, q)), 6), " (p is odd, q is even, on [-1,1])")
print("<p, r> =", round(float(poly_inner(p, r)), 6))
print("||p||  =", round(float(np.sqrt(poly_inner(p, p))), 4))

<p, q> = 0.0  (p is odd, q is even, on [-1,1])
<p, r> = 0.0
||p||  = 0.8165

The matrix (trace) inner product, checked against flattening:

import numpy as np

A = np.array([[1., 2.], [3., 4.]])
B = np.array([[0., 1.], [-1., 2.]])

trace_inner = np.trace(A.T @ B)
flat_dot = A.flatten() @ B.flatten()

print("<A, B> via trace(A^T B) =", trace_inner)
print("<A, B> via flatten+dot  =", flat_dot)
print("match:", np.isclose(trace_inner, flat_dot))

<A, B> via trace(A^T B) = 7.0
<A, B> via flatten+dot  = 7.0
match: True

Worked example

Using \langle p,q\rangle=\int_0^1p(x)q(x)\,dx, find \langle p,q\rangle for p(x)=1, q(x)=x, and determine whether they're orthogonal on [0,1].

\langle p,q\rangle=\int_0^1(1)(x)\,dx=\left[\frac{x^2}2\right]_0^1=\frac12

\boxed{\langle p,q\rangle=\tfrac12\neq0\ \Longrightarrow\ \text{not orthogonal on }[0,1]}

Sanity check. Contrast with the earlier "Doing it in Python" example, which used [-1,1] and found p(x)=x,\,q(x)=x^2 orthogonal there — orthogonality genuinely depends on the inner product chosen, not just on the functions themselves: the same two functions can be orthogonal under one inner product (or one interval) and not another, exactly because \langle\cdot,\cdot\rangle is extra structure layered onto V, not a property of V's vectors alone.

Your turn

1. Verify \langle p,p\rangle=\int_0^1p(x)^2\,dx\ge0 conceptually — why can this integral never be negative, for any continuous p?

2. Using the matrix inner product \langle A,B\rangle=\operatorname{tr}(A^TB), compute \langle A,A\rangle for A=\begin{pmatrix}1&2\\0&3\end{pmatrix} and confirm it equals \|A\|^2 under the flattened-vector dot product.

3. True or false: \langle\vec u,\vec v\rangle=u_1v_1-u_2v_2 on \mathbb{R}^2 is a valid inner product.

Solutions

1. p(x)^2\ge0 for every real x (a square is never negative), so integrating a nonnegative function over [0,1] can never produce a negative number — the integral of a nonnegative function is itself nonnegative, term by term in any Riemann-sum sense.

2. \langle A,A\rangle=\operatorname{tr}(A^TA). A^TA=\begin{pmatrix}1&0\\2&3\end{pmatrix}\begin{pmatrix}1&2\\0&3\end{pmatrix}=\begin{pmatrix}1&2\\2&13\end{pmatrix} (only the diagonal is needed for the trace): \operatorname{tr}=1+13=14. Flattened: A as (1,2,0,3), dot with itself =1+4+0+9=14 ✓ — matches exactly, confirming the trace formula is just the flattened dot product in disguise.

3. False. It fails positive definiteness: \vec v=(0,1) gives \langle\vec v,\vec v\rangle=0(0)-1(1)=-1<0 — negative, violating axiom 3 directly. (This particular bilinear form is still useful — it's the Minkowski/Lorentzian form from special relativity — but it is not an inner product in this course's sense; §23.1 studies such non-positive-definite bilinear forms in their own right.)

Check yourself in code

Using the weighted inner product \langle\vec u,\vec v\rangle=2u_1v_1+u_2v_2+3u_3v_3 on \mathbb{R}^3, compute \langle\vec u,\vec v\rangle and \|\vec u\| for \vec u=(1,2,1), \vec v=(2,-1,1).

Print exactly this:

<u, v> = 5
||u|| = 3.0

import numpy as np

def weighted_inner(u, v):
    return 2*u[0]*v[0] + u[1]*v[1] + 3*u[2]*v[2]

u = np.array([1, 2, 1])
v = np.array([2, -1, 1])

print("<u, v> =", weighted_inner(u, v))
# print ||u|| = sqrt(<u, u>), rounded to 4 decimal places

import numpy as np

def weighted_inner(u, v):
    return 2*u[0]*v[0] + u[1]*v[1] + 3*u[2]*v[2]

u = np.array([1, 2, 1])
v = np.array([2, -1, 1])

print("<u, v> =", weighted_inner(u, v))
print("||u|| =", round(float(np.sqrt(weighted_inner(u, u))), 4))

An inner product is any symmetric, bilinear, positive-definite pairing — the dot product is the motivating example, but \int pq\,dx on polynomials and \operatorname{tr}(A^TB) on matrices qualify equally, and every result from §20.0 (norm, Cauchy-Schwarz, angle, orthogonality) transfers automatically to any of them.

Next: orthogonality taken further — whole subspaces that are orthogonal to each other, and the orthogonal complement of a subspace.

← Previous Next →