Dot product, norms, angles between vectors · Linear Algebra

Everything since §16.0 has used addition and scalar multiplication — Modules 17–19 never once asked how "long" a vector is or how "far apart" two vectors point. This module adds that missing geometry, starting with the single operation everything else in it builds from: the dot product.

The dot product

For \vec u,\vec v\in\mathbb{R}^n,

$\vec u\cdot\vec v=u_1v_1+u_2v_2+\cdots+u_nv_n$

Equivalently, in matrix form, \vec u\cdot\vec v=\vec u^T\vec v (a 1\times n times an n\times1, producing a 1\times1 — a scalar, §16.2). The dot product is symmetric (\vec u\cdot\vec v=\vec v\cdot\vec u, obviously, since ordinary multiplication commutes) and bilinear — linear in each argument separately: \vec u\cdot(\vec v+\vec w)=\vec u\cdot\vec v+\vec u\cdot\vec w and \vec u\cdot(c\vec v)=c(\vec u\cdot\vec v), both immediate from distributing the sum defining it.

Norm (length)

$\|\vec v\|=\sqrt{\vec v\cdot\vec v}=\sqrt{v_1^2+\cdots+v_n^2}$

— §16.0's length formula, now expressed through the dot product (the Pythagorean theorem applied n-1 times, extending the familiar \sqrt{x^2+y^2} to any dimension). Properties: \|\vec v\|\ge0, with equality only for \vec v=\vec0; \|c\vec v\|=|c|\,\|\vec v\| (scaling a vector by c scales its length by |c|, matching §16.0's geometric picture of scalar multiplication exactly); and the triangle inequality \|\vec u+\vec v\|\le\|\vec u\|+\|\vec v\| (the straight path is never longer than a detour — not derived here, but used freely).

A unit vector has \|\vec v\|=1; normalizing means replacing \vec v with \vec v/\|\vec v\|, a unit vector in the same direction (§16.0 previewed both).

The Cauchy-Schwarz inequality and angle

$|\vec u\cdot\vec v|\le\|\vec u\|\,\|\vec v\|$

for all \vec u,\vec v — Cauchy-Schwarz, arguably the single most load-bearing inequality in this course (§20.1 restates it for general inner products; §21.2's positive-definiteness and §22.2's matrix norms both lean on the same idea). It guarantees -1\le\frac{\vec u\cdot\vec v}{\|\vec u\|\|\vec v\|}\le1, so this ratio is always a valid cosine, defining the angle \theta between two nonzero vectors:

$\vec u\cdot\vec v=\|\vec u\|\,\|\vec v\|\cos\theta$

This is the Law of Cosines in disguise, and it recovers the familiar 2D dot-product-from-angle formula while extending "angle between two vectors" to any dimension, even where no picture is available.

Orthogonality

\vec u and \vec v are orthogonal (\vec u\perp\vec v) if \vec u\cdot\vec v=0 — from the formula above, exactly \theta=90° (or one of the vectors is \vec0, trivially orthogonal to everything). Orthogonality is this module's central organizing idea: §20.2 studies whole orthogonal subspaces, §20.3–20.4 build orthogonal bases, and §20.5's least squares is, at its core, a single orthogonality condition.

Doing it in Python

import numpy as np

u = np.array([3., -1., 2.])
v = np.array([1., 4., -2.])

dot = u @ v   # @ computes the dot product for 1-D arrays
print("u . v =", dot)
print("||u|| =", round(float(np.linalg.norm(u)), 4))
print("||v|| =", round(float(np.linalg.norm(v)), 4))

cos_theta = dot / (np.linalg.norm(u) * np.linalg.norm(v))
theta = np.arccos(cos_theta)
print("cos(theta) =", round(float(cos_theta), 4))
print("theta (degrees) =", round(float(np.degrees(theta)), 2))

u . v = -5.0
||u|| = 3.7417
||v|| = 4.5826
cos(theta) = -0.2916
theta (degrees) = 106.95

Verifying Cauchy-Schwarz and orthogonality:

import numpy as np

u = np.array([2., 1.])
v = np.array([-1., 2.])   # chosen to be perpendicular to u

print("u . v =", u @ v, " (orthogonal if 0)")

a, b = np.array([3., 4.]), np.array([1., 2.])
print("\n|a.b| =", abs(a @ b))
print("||a||*||b|| =", round(float(np.linalg.norm(a) * np.linalg.norm(b)), 4))
print("Cauchy-Schwarz holds:", abs(a @ b) <= np.linalg.norm(a) * np.linalg.norm(b))

u . v = 0.0  (orthogonal if 0)

|a.b| = 11.0
||a||*||b|| = 11.1803
Cauchy-Schwarz holds: True

Worked example

Find the angle between \vec u=(1,1,0) and \vec v=(1,0,1).

\vec u\cdot\vec v=1(1)+1(0)+0(1)=1. \|\vec u\|=\sqrt{1+1+0}=\sqrt2. \|\vec v\|=\sqrt{1+0+1}=\sqrt2.

$\cos\theta=\frac1{\sqrt2\cdot\sqrt2}=\frac12\ \Longrightarrow\ \theta=60°$

$\boxed{\theta=60°}$

Sanity check. \vec u and \vec v are two edges of a cube meeting at a shared vertex (both length \sqrt2, the face diagonals) — 60° matches the well-known fact that a cube's face diagonals from a shared vertex meet at exactly this angle, a useful independent geometric confirmation beyond just re-checking the arithmetic.

Your turn

1. Compute \vec u\cdot\vec v for \vec u=(2,-3,1), \vec v=(4,1,-2), and state whether they're orthogonal.

2. Find a unit vector in the direction of \vec w=(6,-8).

Solutions

1. 2(4)+(-3)(1)+1(-2)=8-3-2=3\neq0 — not orthogonal.

2. \|\vec w\|=\sqrt{36+64}=\sqrt{100}=10. Unit vector: \vec w/10=(0.6,-0.8) — check: \sqrt{0.36+0.64}=\sqrt1=1 ✓.

3. True. This is the equality case of the triangle inequality: it holds exactly when \vec u,\vec v are nonnegative multiples of each other (both zero, or both pointing the same way) — any deviation in direction makes the "detour" through separate vectors strictly longer than a straight combined path, which is the geometric content of the inequality being strict except in this one case.

Check yourself in code

For \vec u=(4,0,3) and \vec v=(0,5,0), compute the dot product, both norms, and the angle between them in degrees (rounded to 2 dp).

Print exactly this:

u . v = 0.0
||u|| = 5.0
||v|| = 5.0
angle (deg) = 90.0

import numpy as np

u = np.array([4., 0., 3.])
v = np.array([0., 5., 0.])

print("u . v =", u @ v)
# print ||u||, ||v||, and the angle between them in degrees, all rounded appropriately

import numpy as np

u = np.array([4., 0., 3.])
v = np.array([0., 5., 0.])

dot = u @ v
print("u . v =", dot)
print("||u|| =", round(float(np.linalg.norm(u)), 4))
print("||v|| =", round(float(np.linalg.norm(v)), 4))

theta = np.degrees(np.arccos(dot / (np.linalg.norm(u) * np.linalg.norm(v))))
print("angle (deg) =", round(float(theta), 2))

The dot product \vec u\cdot\vec v=\sum u_iv_i gives length (\|\vec v\|=\sqrt{\vec v\cdot\vec v}) and angle (\cos\theta=\vec u\cdot\vec v/\|\vec u\|\|\vec v\|), guaranteed to be a valid cosine by Cauchy-Schwarz. Orthogonality — \vec u\cdot\vec v=0 — is the special case \theta=90°, and this module's central idea from here on.

Next: stripping the dot product down to its essential properties and generalizing — an inner product on any vector space, not just \mathbb{R}^n.

25. Dot product, norms, angles between vectors