BrevFeed
Sign in

3. Matrices and matrix operations (addition, multiplication, transpose)

📖 Reading · 11 min
💡 Every code box below is live — edit it and hit Run.

§16.1's augmented matrix was a bookkeeping trick — a grid of numbers standing in for a system. This lesson promotes the grid itself to a first-class object: a matrix, with its own arithmetic. Matrix addition and scaling turn out to be nothing new (componentwise, exactly like vectors), but matrix multiplication is genuinely different — and understanding what it really means is the single most important idea in this module.

What a matrix is

An m\times n matrix is a rectangular array of numbers with m rows and n columns:

A=\begin{pmatrix}a_{11}&a_{12}&\cdots&a_{1n}\\a_{21}&a_{22}&\cdots&a_{2n}\\\vdots&&\ddots&\vdots\\a_{m1}&a_{m2}&\cdots&a_{mn}\end{pmatrix}

Entry a_{ij} sits in row i, column j — row index always first. A vector in \mathbb{R}^n (§16.0) is just an n\times1 matrix, a single column; this lesson's operations on matrices specialize to §16.0's vector operations exactly when one dimension is 1.

Addition and scalar multiplication

Exactly as with vectors: matrices of the same shape add componentwise, and scale componentwise.

A+B=\begin{pmatrix}a_{11}+b_{11}&\cdots\\ \vdots&\ddots\end{pmatrix},\qquad cA=\begin{pmatrix}ca_{11}&\cdots\\ \vdots&\ddots\end{pmatrix}

Matrices of different shapes simply cannot be added — there's no sensible entry to pair an a_{ij} with if B doesn't have one.

Matrix-vector multiplication

Before general matrix products, the special case that gives them meaning. For an m\times n matrix A and a vector \vec x\in\mathbb{R}^n, A\vec x is defined as the linear combination of A's columns, weighted by \vec x's entries:

A\vec x=x_1\vec a_1+x_2\vec a_2+\cdots+x_n\vec a_n

where \vec a_1,\dots,\vec a_n are A's columns. Written out with A=\begin{pmatrix}1&2\\3&4\end{pmatrix} and \vec x=(5,6):

A\vec x=5\begin{pmatrix}1\\3\end{pmatrix}+6\begin{pmatrix}2\\4\end{pmatrix}=\begin{pmatrix}5+12\\15+24\end{pmatrix}=\begin{pmatrix}17\\39\end{pmatrix}

This is the exact same computation as the more familiar row-times-column rule (entry i of A\vec x is row i of A dotted with \vec x) — the two descriptions always agree, they just group the arithmetic differently. The column picture is the one worth internalizing: it says a matrix, applied to a vector, does nothing but take a linear combination of a fixed set of columns — which is exactly why §16.1's "does a target vector lie in the span of these vectors" question and "is A\vec x=\vec b solvable" are the same question, phrased two ways.

Matrix-matrix multiplication

AB is defined only when A's number of columns matches B's number of rows — an m\times n times an n\times p gives an m\times p result — and column j of AB is exactly A times column j of B:

AB=A\begin{pmatrix}|&&|\\\vec b_1&\cdots&\vec b_p\\|&&|\end{pmatrix}=\begin{pmatrix}|&&|\\A\vec b_1&\cdots&A\vec b_p\\|&&|\end{pmatrix}

So matrix multiplication is nothing but matrix-vector multiplication, repeated once per column of B. Equivalently, entry (i,j) of AB is row i of A dotted with column j of B — the row/column rule usually taught first, and useful for hand computation, but the column-by-column view above is what explains why the shapes must match the way they do, and why multiplication behaves the way the next section describes.

Matrix multiplication is not commutative: AB\neq BA in general — not even the shapes need to match both ways (a 2\times3 times a 3\times4 works one way, not the other at all). It is associative, (AB)C=A(BC), and distributes over addition, A(B+C)=AB+AC — both inherited from the same properties of ordinary numbers underneath, and both used constantly without comment for the rest of this course.

The identity matrix I_n (ones on the diagonal, zeros elsewhere) is the multiplicative identity: IA=AI=A for compatible shapes.

Transpose

The transpose A^T flips a matrix over its main diagonal — row i of A becomes column i of A^T. An m\times n matrix has an n\times m transpose. Key properties:

(A^T)^T=A,\qquad(A+B)^T=A^T+B^T,\qquad(AB)^T=B^TA^T

The last one — transpose of a product reverses the order — is the one worth memorizing deliberately; it's easy to guess wrong, and it reappears constantly (§20.0's dot product is literally \vec u^T\vec v, and the reversal rule is exactly why (A\vec x)\cdot\vec y=\vec x\cdot(A^T\vec y), a fact §20.2 and §21.0 both lean on).

A matrix with A^T=A is called symmetric — Module 21 is largely about how unusually well-behaved these are.

Doing it in Python

import numpy as np

A = np.array([[1, 2], [3, 4]])
B = np.array([[0, 1], [1, 0]])
x = np.array([5, 6])

print("A + B =")
for row in (A + B).tolist():
    print(row)

print("\nA @ x =", list(A @ x))
print("A @ B =")
for row in (A @ B).tolist():
    print(row)
print("B @ A =")
for row in (B @ A).tolist():
    print(row)
print("\nA.T =")
for row in A.T.tolist():
    print(row)

A + B =
[1, 3]
[4, 4]

A @ x = [17, 39]
A @ B =
[2, 1]
[4, 3]
B @ A =
[3, 4]
[1, 2]

A.T =
[1, 3]
[2, 4]

Confirming A@x really is the column-combination in §16.0's sense, and that (AB)^T=B^TA^T:

import numpy as np

A = np.array([[1, 2], [3, 4]])
B = np.array([[0, 1], [1, 0]])

col1, col2 = A[:, 0], A[:, 1]
x = np.array([5, 6])
combo = x[0] * col1 + x[1] * col2
print("column combination:", list(combo), " matches A@x:", list(A @ x))

lhs = (A @ B).T
rhs = B.T @ A.T
print("(AB).T == B.T @ A.T :", np.array_equal(lhs, rhs))

column combination: [17, 39]  matches A@x: [17, 39]
(AB).T == B.T @ A.T : True

Worked example

Compute AB for A=\begin{pmatrix}2&0&1\\-1&3&2\end{pmatrix}, B=\begin{pmatrix}1&1\\0&2\\3&-1\end{pmatrix}.

A is 2\times3, B is 3\times2, so AB is defined and is 2\times2. Entry (1,1): row 1 of A dot column 1 of B: 2(1)+0(0)+1(3)=5. Entry (1,2): 2(1)+0(2)+1(-1)=1. Entry (2,1): -1(1)+3(0)+2(3)=5. Entry (2,2): -1(1)+3(2)+2(-1)=3.

\boxed{AB=\begin{pmatrix}5&1\\5&3\end{pmatrix}}

Sanity check via the column view. Column 1 of AB should equal A times column 1 of B, i.e. A(1,0,3)^T=1\begin{pmatrix}2\\-1\end{pmatrix}+0\begin{pmatrix}0\\3\end{pmatrix}+3\begin{pmatrix}1\\2\end{pmatrix}=\begin{pmatrix}2+0+3\\-1+0+6\end{pmatrix}=\begin{pmatrix}5\\5\end{pmatrix} ✓ — matches column 1 above exactly, confirming both descriptions of matrix multiplication agree.

Your turn

1. For A=\begin{pmatrix}1&-1\\2&0\end{pmatrix} and \vec x=(3,4), compute A\vec x two ways: row-dot-column, and as a column combination.

2. Give a concrete 2\times2 example showing AB\neq BA.

3. True or false: if A is 3\times2 and B is 3\times2, then AB is defined.

Solutions

1. Row-dot-column: entry 1 is 1(3)+(-1)(4)=-1; entry 2 is 2(3)+0(4)=6. So A\vec x=(-1,6). Column combination: 3\begin{pmatrix}1\\2\end{pmatrix}+4\begin{pmatrix}-1\\0\end{pmatrix}=\begin{pmatrix}3-4\\6+0\end{pmatrix}=\begin{pmatrix}-1\\6\end{pmatrix} — same answer, as it must be.

2. A=\begin{pmatrix}1&1\\0&1\end{pmatrix}, B=\begin{pmatrix}1&0\\1&1\end{pmatrix}. AB=\begin{pmatrix}2&1\\1&1\end{pmatrix}, but BA=\begin{pmatrix}1&1\\1&2\end{pmatrix} — different matrices, so AB\neq BA. (Any two matrices where this fails are called commuting; most pairs don't.)

3. False. A is 3\times2: 2 columns. B is 3\times2: 3 rows. AB requires A's column count to match B's row count — 2\neq3, so AB is not defined. (BA isn't either, for the same reason applied the other way: B has 2 columns, A has 3 rows.) A^TB, however, would be defined (2\times3 times 3\times2) — shape-checking is a habit worth building early, since a shape mismatch is the single most common matrix-algebra bug.

Check yourself in code

For A=\begin{pmatrix}2&-1\\0&3\end{pmatrix} and B=\begin{pmatrix}1&4\\-2&1\end{pmatrix}, print AB, BA, and whether they're equal.

Print exactly this:

AB =
[4, 7]
[-6, 3]
BA =
[2, 11]
[-4, 5]
AB == BA: False

import numpy as np

A = np.array([[2, -1], [0, 3]])
B = np.array([[1, 4], [-2, 1]])

print("AB =")
for row in (A @ B).tolist():
    print(row)
# print BA the same way, then whether AB equals BA

import numpy as np

A = np.array([[2, -1], [0, 3]])
B = np.array([[1, 4], [-2, 1]])

print("AB =")
for row in (A @ B).tolist():
    print(row)
print("BA =")
for row in (B @ A).tolist():
    print(row)
print("AB == BA:", np.array_equal(A @ B, B @ A))

A matrix adds and scales componentwise, just like a vector, but multiplies by a genuinely new rule: A\vec x is a linear combination of A's columns, and AB applies A that way to every column of B at once. Multiplication is associative and distributive but not commutative, and the transpose reverses products: (AB)^T=B^TA^T.

Next: which matrices can be "undone" — the matrix inverse, and the conditions that decide whether one exists.

← Previous Next →