๐Ÿงฎ Linear Algebra Kingdom ยท Linear Algebra

Vectors in n Dimensions

A vector stops being an arrow on a page and becomes a list of numbers of any length: add and scale vectors in three, four or n dimensions, measure their length, find the unit vector along one, take a dot product to test whether two are perpendicular, and write a line in space as a point plus a direction.

In short

  • A vector in Rn is an ordered list of n numbers; two vectors are equal only when they have the same length of list and agree in every slot.
  • Addition, subtraction and scaling all happen one slot at a time, and a scalar multiplies every entry of its vector โ€” the sign included.
  • |u| = sqrt(u1^2 + ... + un2), the distance between two points is |Q - P|, and the unit vector along u is u divided by |u|.
  • The dot product u . v = u1v1 + ... + unvn is a single number, and u . v = 0 is what "at right angles" means in every dimension.
  • A line in space is r(t) = p + t*d โ€” a point to start from and a direction to travel โ€” where the parameter scales the direction and never the starting point.

A vector is a list, and the list can be as long as you like

In the Trig Kingdom a vector was an arrow on a page: two numbers, how far across and how far up. On this isle it is something plainer and far more useful. A vector is an ordered list of numbers, and nothing in the arithmetic ever cared how many numbers were in it.

u = [3, -1, 4] a vector in R3 v = [2, 0, -5, 1] a vector in R4

Rn is the name for the set of all lists of n numbers, so R2 is the plane you already know, R3 is space, and R7 is a perfectly ordinary place to work that simply cannot be drawn. Each number in the list is an entry or a component, and its position matters: [3, -1, 4] and [4, -1, 3] are different vectors, exactly as (3, -1) and (-1, 3) were different points.

Two vectors are equal when they have the same length of list and agree in every slot. A vector in R3 is never equal to a vector in R4, however similar the numbers look โ€” there is no rule for comparing lists of different lengths, and no rule for adding them either.

Why bother? Because lists of numbers are everywhere once you look. A shop's stock is a list of counts. A student's term is a list of scores. A colour is a list of three brightnesses. A recipe is a list of quantities. Every one of them can be added, scaled, measured and compared with the rules below, and the rules are the same for all of them.

Gatekeeper Nore's habit is worth copying. He never says where a place is; he says which way and how far along each ruled line. A vector is that instruction, written down.

Adding and scaling: the slots never mix

Two operations do almost all the work, and both of them are done one slot at a time.

Addition and subtraction pair the entries off by position:

[3, -1, 4] + [2, 5, -1] = [3 + 2, -1 + 5, 4 + (-1)] = [5, 4, 3] [3, -1, 4] - [2, 5, -1] = [1, -6, 5]

The first entries only ever meet other first entries. Nothing crosses between the slots, ever, and that single fact is what makes vectors in seventeen dimensions no harder than vectors in two.

Scalar multiplication stretches the whole list by one number:

3[2, -1, 5] = [6, -3, 15] -2[2, -1, 5] = [-4, 2, -10]

A scalar is just an ordinary number, and it multiplies every entry. The commonest error in this whole craft is letting a scalar reach only the first slot, which quietly bends the direction into something else entirely. A negative scalar turns the vector round to point the opposite way, and it does that by changing the sign of every entry, not one of them.

Put the two together and you get a linear combination such as 3u - 2v + w: scale each vector first, then add what is left. Work down the slots and write one line per slot; it is slower to read and much faster to get right.

The laws you would expect all hold, and each is really a rule about one slot repeated down the list:

  • u + v = v + u โ€” addition is commutative.
  • (u + v) + w = u + (v + w) โ€” addition is associative, so brackets do not matter.
  • u + 0 = u, where 0 is the vector with every entry zero.
  • u + (-u) = 0 โ€” every vector has an opposite, and it is the whole vector negated.
  • c(u + v) = cu + cv and (c + d)u = cu + du โ€” a scalar in front of a bracket reaches everything inside it.

Length, distance and the unit vector

Pythagoras does not stop at two legs. The magnitude or length of a vector is

|u| = sqrt(u1^2 + u22 + ... + un2)

Square every entry, add all the squares, take one square root at the end. For u = [1, 2, 2]: 1 + 4 + 4 = 9, so |u| = 3.

Three things fall out of the formula. Squaring removes every minus sign, so a length is never negative and a vector is exactly as long as its opposite. The squares are added before the root, never after โ€” sqrt(9) + sqrt(16) is 7, but sqrt(9 + 16) is 5, and only the second is a length. And the length is not the sum of the entries: walking along each ruled line in turn is always at least as far as going straight there.

The distance between two points P and Q in Rn is the length of the vector between them:

distance = |Q - P|

Subtract slot by slot first, then measure. The order of the subtraction cannot change the answer, because squaring wipes out the difference.

A unit vector has length exactly 1. It carries a direction and nothing else, and you build one by dividing a vector by its own length:

u / |u|

For u = [1, 2, 2] with |u| = 3, the unit vector is [1/3, 2/3, 2/3]. Check it: 1/9 + 4/9 + 4/9 = 1. Every entry is divided, and it is divided by the length โ€” not by one of the entries, and not by the length squared. A quick sanity check: every entry of a unit vector sits between -1 and 1.

That gives a recipe worth remembering. To build a vector of length L in the direction of u: shrink to 1, then stretch to L, which is the same as multiplying u by L / |u| in one go.

The dot product, and what a right angle looks like in R^n

The dot product of two vectors of the same length is a single number:

u . v = u1v1 + u2v2 + ... + unvn

Pair the entries off, multiply each pair, add all the products. For u = [3, -1, 2] and v = [1, 4, 2]: (3)(1) + (-1)(4) + (2)(2) = 3 - 4 + 4 = 3.

Notice what it is not. It is not a vector โ€” the slots are gone by the time you finish. It is not the product of the lengths. And it is not always positive: a negative product from one slot pulls the whole total down, and that is exactly how a dot product reaches zero.

Which brings us to the reason the dot product exists. Two vectors are orthogonal โ€” at right angles โ€” exactly when

u . v = 0

In R2 you can see it. In R7 you cannot, and this equation is what "at right angles" now means. It is a definition that agrees with the picture wherever a picture exists, and carries on working where one does not.

Two useful consequences. First, u . u = |u|2, so the dot product of a vector with itself is its length squared โ€” the magnitude formula in disguise. Second, a vector is never orthogonal to itself unless every entry is zero, and never orthogonal to its own opposite.

Orthogonality is often asked backwards: which value of k makes these two perpendicular? Write the dot product down with k still in it, and it comes out as one equation in one unknown.

u = [3, -2, 5], v = [4, 1, k] u . v = 12 - 2 + 5k = 10 + 5k Set that to 0: 5k = -10, so k = -2.

A line in space: a place to start and a way to go

A line in the plane has an equation y = mx + c, which is no use at all in R3 โ€” there is no single slope. Vectors give a description that works in every dimension, and it is Nore's description exactly: where you start, and which way you go.

r(t) = p + t*d

Here p is a point on the line, d is the direction vector telling you which way it runs, and t is the parameter, a number saying how far along you have travelled. Every value of t gives one point of the line, and every point of the line comes from one value of t.

r(t) = [1, 3, 4] + t*[1, 2, 4] at t = 0: [1, 3, 4] the starting point itself at t = 2: [3, 7, 12] at t = -1: [0, 1, 0] the other side of the start

The parameter multiplies only the direction. The starting point is where the line already is and is never scaled. A negative t simply walks the other way down the same line.

Through two points. If a line passes through P and Q, its direction is the vector from one to the other:

d = Q - P, so r(t) = P + t*(Q - P)

Check it by putting t = 1 back in: you should land exactly on Q. Subtracting the wrong way round, P - Q, gives a direction pointing back the way it came, and every point you compute after that sits on the wrong side of P.

Going backwards. Given a point that lies on the line, the parameter is found by subtracting the starting point and seeing how many copies of the direction are left. One slot is enough to find t; the other slots are the check that the point really is on the line, because the same t has to work in all of them.

Worked examples

Example 1

u = [4, -1, 2, 3] and v = [1, 5, -2, 0]. Find 3u - 2v, and then find |u - v|.

  1. Scale first: 3u = [12, -3, 6, 9].
  2. And 2v = [2, 10, -4, 0].
  3. Subtract slot by slot, remembering the minus reaches all four entries of 2v: 3u - 2v = [12 - 2, -3 - 10, 6 - (-4), 9 - 0] = [10, -13, 10, 9].
  4. Now the second part. u - v = [4 - 1, -1 - 5, 2 - (-2), 3 - 0] = [3, -6, 4, 3].
  5. Square each entry: 9, 36, 16, 9.
  6. Add them: 9 + 36 + 16 + 9 = 70, so |u - v| = sqrt(70), which is about 8.4.
  7. A check on the shape of the answer: 3u - 2v is a vector with four entries, while |u - v| is a single number. Those are always the two kinds of answer in this craft.

Example 2

u = [2, 3, 6]. Find |u|, find the unit vector along u, and find the vector of length 21 pointing the same way.

  1. Length: 22 + 32 + 62 = 4 + 9 + 36 = 49, so |u| = sqrt(49) = 7.
  2. Unit vector: divide every entry by 7, giving [2/7, 3/7, 6/7].
  3. Check it: 4/49 + 9/49 + 36/49 = 49/49 = 1, so that really does have length 1.
  4. For length 21, stretch the unit vector by 21: 21 * [2/7, 3/7, 6/7] = [6, 9, 18].
  5. Or in one step: multiply u by 21/7 = 3, giving 3 * [2, 3, 6] = [6, 9, 18]. Same answer, less arithmetic.
  6. Check: 62 + 92 + 182 = 36 + 81 + 324 = 441, and sqrt(441) = 21.

Example 3

u = [5, -2, 1] and v = [2, 4, k]. Find the k that makes u and v orthogonal, then find the distance from P(1, -1, 3) to Q(3, 2, 9).

  1. Orthogonal means the dot product is zero, so write it down with k still in it.
  2. u . v = (5)(2) + (-2)(4) + (1)(k) = 10 - 8 + k = 2 + k.
  3. Set that to 0: k = -2. Check: (5)(2) + (-2)(4) + (1)(-2) = 10 - 8 - 2 = 0.
  4. For the distance, subtract the points first: Q - P = [3 - 1, 2 - (-1), 9 - 3] = [2, 3, 6].
  5. Square and add: 4 + 9 + 36 = 49.
  6. Distance = sqrt(49) = 7. (Doing P - Q instead would give [-2, -3, -6], and the squares are identical, so the distance is the same.)

Practice problems, with solutions

Three problems of increasing difficulty, each with the full working. In the game these are generated fresh every time; these three are fixed so this page always shows the same ones.

Problem 1

Difficulty 1 of 5

Gatekeeper Nore rules the lines of the isle and marks a direction along each. u = [5, 2, 5] v = [3, 6, 7] Work out u + v. Write the vector as its entries separated by commas, like 1,-2,3.

Answer: [8,8,12]

  1. 1st entry: 5 + 3 = 8.
  2. 2nd entry: 2 + 6 = 8.
  3. 3rd entry: 5 + 7 = 12.
  4. u + v = [8, 8, 12].

Problem 2

Difficulty 3 of 5

u = [5, 7, -5, 1] How long is u? Every length on this isle comes out whole, so give |u| exactly.

Answer: 10

  1. Square each entry: 52 = 25, 72 = 49, (-5)2 = 25, 12 = 1.
  2. Add them: 25 + 49 + 25 + 1 = 100.
  3. |u| = sqrt(100) = 10.

Problem 3

Difficulty 4 of 5

u = [5, -4, -2] v = [-2, -1, k] For which value of k are u and v orthogonal?

Answer: -3

  1. u . v = (5)(-2) + (-4)(-1) + (-2)k.
  2. The part you already know adds to -6, so the equation is -6 - 2k = 0.
  3. Solving: (-2)k = 6, so k = 6/-2 = -3.

Common mistakes

  • Letting a scalar reach only the first entry, so 3[2, 1, 5] comes out as [6, 1, 5] instead of [6, 3, 15].
  • Dropping a minus sign in front of a vector, so only its first entry changes sign when the whole list should.
  • Taking the square roots before adding them, or forgetting the square root entirely and reporting the length squared as the length.
  • Adding the entries of a vector instead of squaring them when finding a length, which gives a total rather than a distance and can even come out negative.
  • Dividing by the wrong thing when building a unit vector: it is the vector over its magnitude, never over one entry and never over the magnitude squared.
  • Treating the dot product as a vector, or expecting it to be positive; a negative product from one slot is exactly what lets the total reach zero.
  • Scaling the starting point of a line as well as the direction, so r(2) comes out as 2(p + d) rather than p + 2d.

What you should be able to do

  • Add, subtract and scale vectors in three or more dimensions, component by component.
  • Find the magnitude of a vector in n dimensions, the distance between two points, and the unit vector in a given direction.
  • Compute the dot product in n dimensions, decide whether two vectors are orthogonal, and find the value that makes them so.
  • Write the vector equation of a line in space and find the point at a given parameter.

Where this fits in the curriculum

Common Core

  • HSN-VM.A.1

    High school โ€” Recognise vector quantities as having both magnitude and direction; represent vector quantities by directed line segments, and use appropriate symbols for vectors and their magnitudes.

    HSN-VM.A.1 is a (+) standard โ€” beyond the college- and career-ready threshold, i.e. precalculus rather than Algebra II. The Common Core writes its vector standards for the plane and for space. A list of n components, the length of one, the dot product that tests for a right angle and the vector equation of a line in space are the same arithmetic carried past where the Common Core goes.

  • HSN-VM.A.2

    High school โ€” Find the components of a vector by subtracting the coordinates of an initial point from the coordinates of a terminal point.

    HSN-VM.A.2 is a (+) standard โ€” beyond the college- and career-ready threshold, i.e. precalculus rather than Algebra II.

  • HSN-VM.B.4

    High school โ€” Add and subtract vectors.

    HSN-VM.B.4 is a (+) standard โ€” beyond the college- and career-ready threshold, i.e. precalculus rather than Algebra II.

  • HSN-VM.B.5

    High school โ€” Multiply a vector by a scalar.

    HSN-VM.B.5 is a (+) standard โ€” beyond the college- and career-ready threshold, i.e. precalculus rather than Algebra II.

Learn these first

This leads on to