🗃️Matrix Operations

Millwright Orin runs the Array Mill on rows and columns, and so does the whole of linear algebra. A matrix is a rectangle of numbers written inside brackets:

A = [[1, 2, 3], [4, 5, 6]]

That matrix has two rows and three columns, so we call it 2 by 3. The size is always said rows first, and it is worth saying out loud, because almost every rule in this craft is a rule about sizes.

Each number inside is an entry, and each entry has an address: its row, counted from the top, and its column, counted from the left — in that order, every time. In the matrix above, the entry in row 2, column 1 is 4, and the entry in row 1, column 2 is 2. They are different entries, and swapping the two numbers in an address is the single most common slip in this whole kingdom.

There is a name for reading every address backwards on purpose. The transpose of A, written A with a small T after it, turns each row of A into a column:

A = [[1, 2, 3], [4, 5, 6]] has transpose [[1, 4], [2, 5], [3, 6]]

A 2 by 3 matrix transposes into a 3 by 2 one. Nothing is added, taken away or changed in sign; the entries only move, and the entry at row i, column j lands at row j, column i.

A matrix with the same number of rows as columns is square, and a square matrix that equals its own transpose is symmetric — every entry matches its mirror across the diagonal that runs from the top left to the bottom right. The entries on that diagonal are free to be anything at all; it is the pairs on either side of it that have to agree.

Two matrices can be added only when they are the same size, and then the rule is as gentle as it looks: add the entries that share an address.

[[1, 2], [3, 4]] + [[5, 6], [7, 8]] = [[6, 8], [10, 12]]

Nothing crosses between positions. The entry in row 1, column 2 of the answer is built from the entries in row 1, column 2 of each matrix, and from nothing else. Subtracting works the same way, and the order matters exactly as it does for numbers: A - B and B - A have opposite signs everywhere.

Scaling is just as direct. Multiplying a matrix by a number — a scalar — multiplies every entry:

3[[1, -2], [0, 4]] = [[3, -6], [0, 12]]

Two traps live here, and they are worth naming before you meet them. First, a scalar written in front of a bracket reaches everything inside that bracket: in 3(A + B) it reaches both matrices, while in 3A + B it reaches only A. Second, a minus sign in front of a scaled matrix belongs to the whole of it, so in 4A - 3B every entry of 3B is taken away, sign included.

Because adding and scaling behave, matrix equations behave too. A + X = B is solved exactly the way 3 + x = 7 is solved: X = B - A.

Multiplication is where matrices stop looking like tidy lists of numbers and start doing something. It is not entry by entry.

When is AB defined? Write the two sizes side by side:

A is 2 by 3 B is 3 by 4

The two inner numbers are 3 and 3. When they agree, the product is defined, and its size is the two outer numbers: 2 by 4. When they disagree, AB simply does not exist — there is nothing to work out and nothing to write down.

How is one entry built? The entry of AB in row i, column j comes from row i of A and column j of B. Pair them off in order, multiply each pair, and add:

A = [[1, 2], [3, 4]] B = [[5, 6], [7, 8]] row 1 of A is [1, 2], column 1 of B is [5, 7] (1)(5) + (2)(7) = 5 + 14 = 19

so the entry in row 1, column 1 of AB is 19. Do the other three pairings and AB = [[19, 22], [43, 50]].

The order matters. For ordinary numbers 3 times 4 and 4 times 3 agree; for matrices they usually do not. With the A and B above, BA = [[23, 34], [31, 46]], which is nowhere near AB. Some special pairs do agree — any matrix agrees with a scalar copy of the identity — but that is a coincidence to be checked, never a rule to lean on. Computing BA and calling it AB is the signature mistake of this kingdom, and it hides very well, because the answer looks perfectly reasonable.

The transpose knows about the order too: the transpose of AB is the transpose of B times the transpose of A, in that order. Transposing turns the product around.

A vector is a matrix with one column, and the product rule already covers it. If A is 3 by 3 and x has three entries, then Ax has three entries: one per row of A.

A = [[2, 0, 1], [1, 3, -1], [0, 4, 2]] x = [1, -2, 3] row 1: (2)(1) + (0)(-2) + (1)(3) = 5 row 2: (1)(1) + (3)(-2) + (-1)(3) = -8 row 3: (0)(1) + (4)(-2) + (2)(3) = -2 so Ax = [5, -8, -2]

Each row of A meets the whole vector, and gives back one number. That is why Ax has as many entries as A has rows, and why the vector needs as many entries as A has columns — the inner numbers again.

Reading Ax this way is the reason matrices are worth having at all: A takes a vector in and hands a vector back, so a matrix is a machine that moves vectors around. Every later craft in this kingdom — spans, systems, transformations, eigenvectors — is a question about what that machine does.

Watch one thing here. Pairing the columns of A with x instead of its rows quietly computes the transpose of A times x, and the answer will look sensible while being about a different machine entirely.

The identity matrix I is square, with 1 down the main diagonal and 0 everywhere else:

I = [[1, 0], [0, 1]]

It does for matrices what the number one does for ordinary arithmetic: AI = A and IA = A, for every A of the right size. Work out one entry and you can see why — each column of I is all zeros but for a single 1, and that 1 picks exactly one entry out of the row it meets.

Because a square matrix can be multiplied by itself, powers make sense. A squared means AA, worked out with the row-meets-column rule and not by squaring each entry:

A = [[1, 2], [0, 3]] A squared = [[1, 8], [0, 9]] (squaring the entries would give [[1, 4], [0, 9]], which is a different matrix)

Brackets expand as usual, as long as you keep the order of every factor. Since A and I agree in either order, (A + I) squared really is A squared + 2A + I — but for two general matrices, (A + B) squared is A squared + AB + BA + B squared, and the two middle terms cannot be joined into 2AB.