Mathematics · Intermediate · 40 min · Chapter 8 of 28
Linear algebra
Vectors, matrices, and the one question they all answer: when does a linear equation have a solution, and what does the solution space look like?
The idea
A vector in Rⁿ is an ordered list of n real numbers, pictured as an arrow when n is 2 or 3. Vector addition and scalar multiplication are componentwise. A set of vectors spans a subspace if every vector in it is a linear combination of them. They are linearly independent if the only combination that gives the zero vector has all coefficients zero. A basis is a linearly independent spanning set, and every basis of a given subspace has the same number of vectors: the dimension.
An m × n matrix A represents a linear map from Rⁿ to Rᵐ. The equation Ax = b asks for the preimage of b. It has a solution precisely when b is in the column space. When there is one solution and the null space is only the zero vector, the solution is unique. For square matrices that is equivalent to det A ≠ 0, and to the existence of an inverse matrix A⁻¹ with AA⁻¹ = I. The 2 × 2 case is small enough to write by hand: the determinant is ad − bc, and the inverse is (1/(ad − bc)) times the matrix of swapped diagonals and negated off-diagonals.
Eigenvalues answer a geometric question. Av = λv means the map sends the line through v back onto itself, stretched by λ. You find λ from the characteristic polynomial det(A − λI) = 0, then find v by solving (A − λI)v = 0. Symmetric real matrices have real eigenvalues and an orthonormal basis of eigenvectors, which is why they diagonalize cleanly and dominate statistics, mechanics, and quantum observables.
Hold onto this
- A linear map sends sums to sums and respects scaling. Matrices are linear maps written in coordinates.
- Solving Ax = b is asking which vectors b lie in the column space of A.
- The determinant is zero exactly when a square matrix collapses volume, i.e., is not invertible.
- Eigenvectors are directions a matrix merely stretches, by the factor λ.
Formulas
Dot product
Zero when the vectors are perpendicular. Also equal to the sum of componentwise products.
- θ
- Angle between the vectors
Determinant, 2 × 2
Signed area of the parallelogram spanned by the columns. Zero means the columns are parallel.
- ad − bc
- Difference of the diagonal products
Inverse, 2 × 2
Exists exactly when ad − bc ≠ 0.
- ad − bc
- The determinant
Eigenvalue equation
v is a direction A only stretches. The second equation finds the possible stretches λ.
- λ
- Eigenvalue
- v
- A nonzero eigenvector
Projection onto a vector
The shadow of v on the line through u. Subtract it from v to get the perpendicular piece.
- u
- The direction you project onto, not necessarily a unit vector
Worked examples
1. A 2 × 2 system
Solve 2x + y = 5 and x − y = 1.
- In matrix form, columns (2, 1) and (1, −1), right-hand side (5, 1).
- Determinant: 2(−1) − (1)(1) = −3, nonzero, so there is a unique solution.
- Add the equations: 3x = 6, so x = 2. Then 2 − y = 1, so y = 1.
x = 2, y = 1. Check: 4 + 1 = 5 and 2 − 1 = 1.
2. Eigenvalues of a stretch
Find the eigenvalues of the matrix with rows (3, 0) and (0, −1).
- This matrix stretches the x-axis by 3 and the y-axis by −1.
- det(A − λI) = (3 − λ)(−1 − λ) = 0.
- So λ = 3 or λ = −1.
Eigenvalues 3 and −1, with eigenvectors along (1, 0) and (0, 1).
Try these
Are the vectors (1, 2) and (2, 4) linearly independent?
Compute the determinant of the matrix with rows (1, 4) and (2, 8).
Common mistakes
- Matrix multiplication is not commutative. AB and BA are different maps, when both even exist.
- A row of zeros, or two proportional rows, means the determinant is zero. Do not keep row-reducing past an obvious dependence and then invent an inverse.
- Eigenvectors are defined only up to scale, and the zero vector is never an eigenvector.