Exam Review

Linear algebra · {course} · {exam date}

Each result stated once, then a worked instance underneath it. The instance is the part worth re-reading.

1. Diagonalisability

A∈ℂ^(n×n) is diagonalisable iff for every eigenvalue the geometric multiplicity equals the algebraic multiplicity. Equivalently, A has n linearly independent eigenvectors.

Worked.

A=([4,1],[0,4])

det(A−λ*I)=(4−λ)^2⟹λ=4 with algebraic multiplicity 2

A−4*I=([0,1],[0,0]),rank(A−4*I)=1⟹dim ker*(A−4*I)=2−1=1

Geometric multiplicity 1 < algebraic multiplicity 2, so A is not diagonalisable. Its Jordan form is A itself.

Distinct eigenvalues always give diagonalisability, but the converse fails — I has one eigenvalue and is already diagonal. Repeated eigenvalues are a warning, not a verdict.

2. Spectral theorem

If A is real symmetric then A=Q∧Q^T with Q orthogonal and ∧ real diagonal. Eigenvalues are real, and eigenvectors for distinct eigenvalues are orthogonal.

Worked.

A=([3,1],[1,3]),det(A−λ*I)=λ^2−6*λ+8=(λ−4)*(λ−2)

(λ_1)=4:(v_1)=1/√(,2)*([1],[1]),(λ_2)=2:(v_2)=1/√(,2)*([1],[−1])

(v_1)^T*(v_2)=1/2*(1−1)=0

Orthogonal without being asked to be. Both eigenvalues positive, so A is positive definite and x^T*A*x>0 for every x≠0.

3. Least squares

A*x=b with A∈ℝ^(m×n), m>n, generally has no solution. The minimiser of ‖A*x−b‖_2 satisfies the normal equations

A^T*A*x=A^T*b

Worked. Fit y=(β_0)+(β_1)*x to (0,1), (1,2), (2,4).

A=([1,0],[1,1],[1,2]),b=([1],[2],[4])

A^T*A=([3,3],[3,5]),A^T*b=([7],[10])

3*(β_0)+3*(β_1)=7,3*(β_0)+5*(β_1)=10⟹(β_1)=3/2,(β_0)=5/6

y=5/6+3/2*x

Geometrically A*x is the orthogonal projection of b onto the column space, which is why the residual b−A*x is orthogonal to every column of A. That orthogonality is the normal equations.

4. Conditioning

For the 2-norm,

(κ_2)(A)=(σ_max)(A)/(σ_min)(A)

and it bounds how much a perturbation in b can be amplified in the solution:

‖δ*x‖/‖x‖≤(κ_2)(A)‖δ*b‖/‖b‖

Worked. For symmetric positive definite A=([3,1],[1,3]) the singular values are the eigenvalues, so

(κ_2)(A)=4/2=2

Well conditioned. A 1% error in b costs at most 2% in x.

Note that (κ_2) is about sensitivity, not invertibility. A matrix can be invertible on paper and useless in floating point — (κ_2)=10^16 means the answer is noise, even though det(A)≠0.