Eigenvalues and Eigenvectors

Subtopic: Eigentheory
Topic: Linear Algebra

Introduction

When we apply a linear transformation to a vector, the result is typically a new vector pointing in a different direction and with a different magnitude. However, certain special vectors behave remarkably simply: they emerge from the transformation pointing in exactly the same direction (or the exact opposite), merely stretched or compressed by some factor. These special vectors are called eigenvectors, and the scaling factors are called eigenvalues. Together, they reveal the intrinsic geometry of a linear transformation.

Consider spinning a globe on its axis. Every point on the globe moves in a circle—except for points on the axis itself. These axial points stay fixed, or at most get reflected through the center. The axis represents an eigenvector of the rotation: a direction that the transformation preserves. This example illustrates a profound principle: even complex transformations often have simple underlying structure, and eigenvectors expose that structure.

Eigenvalues and eigenvectors are not merely computational tools. They are the key to understanding the behavior of dynamical systems, the stability of equilibria, the principal axes of stress in materials, the modes of vibration in mechanical systems, and the fundamental frequencies of quantum states. In data science, they underlie principal component analysis, Google's PageRank algorithm, and spectral clustering. Mastering eigenvalues means gaining insight into the deepest structure of linear maps.

Definition

Let A A be an n×n n times n matrix. A nonzero vector v v is called an eigenvector of A A if applying A A to v v produces a scalar multiple of v v:

A⁢v=λ⁢v A v equals lamda v.

The scalar λ lamda is called the eigenvalue corresponding to the eigenvector v v. The requirement that v≠0 v is not equal to 0 is essential: the zero vector trivially satisfies A⁢0=λ⁢0 A 0 equals lamda 0 for any λ lamda, so we exclude it to make the concept meaningful.

The equation A⁢v=λ⁢v A v equals lamda v says that v v and A⁢v A v are parallel. The transformation A A does not rotate v v; it only scales it. When λ>0 lamda is greater than 0, the vector is stretched (if λ>1 lamda is greater than 1) or compressed (if 0<λ<1 0 is less than lamda is less than 1) while pointing in the same direction. When λ<0 lamda is less than 0, the vector is also reflected through the origin, ending up pointing in the opposite direction. When λ=0 lamda equals 0, the eigenvector is sent to the zero vector — it lies in the null space of A A.

Geometric Interpretation

Geometrically, eigenvectors are the directions along which the linear transformation acts by pure scaling. Think of a matrix A A as a machine that takes in vectors and outputs transformed vectors. Most input vectors come out pointing in different directions. Eigenvectors are the special inputs that come out pointing in the same direction — just longer, shorter, or flipped.

For a 2×2 2 times 2 matrix, the transformation can be visualized as deforming the plane. Consider a shearing transformation that tilts the plane. Most vectors get tilted, but certain directions are preserved. These preserved directions are the eigenvectors, and how much they are stretched gives the eigenvalues.

When all eigenvalues are positive, the transformation preserves orientation — it stretches or compresses without flipping. When an eigenvalue is negative, vectors along that eigenvector are reflected. When eigenvalues are complex, the transformation involves rotation, and there are no real directions that remain fixed (except in trivial cases).

The eigenvalues encode the transformation’s behavior along its principal axes, while the eigenvectors identify those axes. Eigenanalysis decomposes a potentially complicated transformation into simple scalings along well-chosen directions.

Finding Eigenvalues

To find the eigenvalues of a matrix A A, rewrite the eigenvalue equation A⁢v=λ⁢v A v equals lamda v in a form that reveals when nontrivial solutions exist. Subtract λ⁢v lamda v from both sides:

A⁢v−λ⁢v=0 A v minus lamda v equals 0.

Factor out v v:

(A−λ⁢I)⁢v=0 open paren A minus lamda I close paren times v equals 0.

This is a homogeneous linear system. For a nonzero solution v v to exist, the matrix (A−λ⁢I) open paren A minus lamda I close paren must be singular, meaning it has zero determinant. This gives the characteristic equation:

det⁡(A−λ⁢I)=0. the determinant of open paren A minus lamda I close paren equals 0 period

The left-hand side expands to a polynomial in λ lamda, called the characteristic polynomial. For an n×n n times n matrix, this polynomial has degree n n. The eigenvalues are exactly the roots of the characteristic polynomial.

The Characteristic Polynomial

For an n×n n times n matrix A A the characteristic polynomial is:

p⁡(λ)=det⁡(A−λ⁢I)=(−1)n⁢λn+(−1)n−1⁢(tr ⁢A)⁢λn−1+⋯+det⁡(A) p of lamda equals the determinant of open paren A minus lamda I close paren equals open paren negative 1 close paren to the n-th power times lamda to the n-th power plus open paren negative 1 close paren raised to the n minus 1 power times open paren trace A close paren times lamda raised to the n minus 1 power plus dot dot dot plus determinant A

The coefficient of λn−1 lamda raised to the n minus 1 power involves the trace of A A (the sum of diagonal entries), and the constant term is det⁡(A) determinant A By Vieta's formulas, the sum of all eigenvalues equals the trace, and the product of all eigenvalues equals the determinant:

λ1+λ2+⋯+λn=tr⁡(A) lamda sub 1 plus lamda sub 2 plus dot dot dot plus lamda sub n equals trace A

λ1⋅λ2⋯λn=det⁡(A) lamda sub 1 times lamda sub 2 dot dot dot lamda sub n equals determinant A

These relationships provide useful checks on eigenvalue computations and connect eigenvalues to other fundamental matrix properties.

Finding Eigenvectors

Once an eigenvalue λ lamda is known, the corresponding eigenvectors are found by solving the homogeneous system:

(A−λ⁢I)⁢v=0 open paren A minus lamda I close paren times v equals 0.

This amounts to finding the null space of the matrix (A−λ⁢I) open paren A minus lamda I close paren. The null space is nontrivial (contains more than just 0 0) because λ lamda was chosen to make the matrix singular. Any nonzero vector in this null space is an eigenvector corresponding to λ lamda.

The set of all eigenvectors corresponding to a single eigenvalue λ lamda, together with the zero vector, forms a subspace called the eigenspace of λ lamda:

Eλ=Null⁢(A−λ⁢I)={v:A⁢v=λ⁢v} E sub lamda equals Null times open paren A minus lamda I close paren equals the set of all v such that A v equals lamda v.

The dimension of this eigenspace is called the geometric multiplicity of λ lamda. It tells how many linearly independent eigenvectors share that eigenvalue. The geometric multiplicity is at least 1 1 and at most the algebraic multiplicity (the number of times λ lamda appears as a root of the characteristic polynomial).

Worked Example

Consider the matrix:

A=(4123) A equals the 2 by 2 matrix Row 1: 4 1 Row 2: 2 3

We seek all eigenvalues and eigenvectors of A A

Step 1: Find the Characteristic Polynomial

Compute A−λ⁢I A minus lamda I

A−λ⁢I=(4−λ123−λ) A minus lamda I equals the 2 by 2 matrix Row 1: Column 1, 4 minus lamda Column 2, 1 Row 2: Column 1, 2 Column 2, 3 minus lamda

The determinant is:

det⁡(A−λ⁢I)=(4−λ)⁢(3−λ)−(1)⁢(2)=λ2−7⁢λ+10 the determinant of open paren A minus lamda I close paren equals open paren 4 minus lamda close paren times open paren 3 minus lamda close paren minus 1 times 2 equals lamda squared minus 7 lamda plus 10

Step 2: Find the Eigenvalues

Set the characteristic polynomial equal to zero and solve:

λ2−7⁢λ+10=0. lamda squared minus 7 lamda plus 10 equals 0 period

Factoring:

(λ−5)⁢(λ−2)=0. open paren lamda minus 5 close paren times open paren lamda minus 2 close paren equals 0 period

The eigenvalues are λ1=5 lamda sub 1 equals 5 and λ2=2 lamda sub 2 equals 2.

We can verify: the trace of A A is 4+3=7=5+2 4 plus 3 equals 7 equals 5 plus 2, and the determinant is
4⋅3−1⋅2=10=5⋅2 4 times 3 minus 1 times 2 equals 10 equals 5 times 2. Both checks confirm the eigenvalues.

Step 3: Find the Eigenvectors

For λ1=5 lamda sub 1 equals 5, solve (A−5⁢I)⁢v=0 open paren A minus 5 I close paren times v equals 0:

(−112−2)⁢(v1v2)=(00). the 2 by 2 matrix Row 1: negative 1 1 Row 2: 2 negative 2 times the 2 by 1 column matrix v sub 1 v sub 2 equals the 2 by 1 column matrix 0 0 period

Both equations reduce to −v1+v2=0 negative v sub 1 plus v sub 2 equals 0, or v2=v1 v sub 2 equals v sub 1. Taking v1=1 v sub 1 equals 1, we obtain the eigenvector v1=(11) v sub 1 equals the 2 by 1 column matrix 1 1.

For λ2=2 lamda sub 2 equals 2, solve (A−2⁢I)⁢v=0 open paren A minus 2 I close paren times v equals 0:

(2121)⁢(v1v2)=(00) the 2 by 2 matrix Row 1: 2 1 Row 2: 2 1 times the 2 by 1 column matrix v sub 1 v sub 2 equals the 2 by 1 column matrix 0 0.

Both equations give 2⁢v1+v2=0 2 v sub 1 plus v sub 2 equals 0, or v2=−2⁢v1 v sub 2 equals negative 2 v sub 1. Taking v1=1 v sub 1 equals 1, we obtain v2=(1−2) v sub 2 equals the 2 by 1 column matrix 1 negative 2.

Verification: A⁢v1=(4+12+3)=(55)=5⁢v1 A v sub 1 equals the 2 by 1 column matrix Row 1: 4 plus 1 Row 2: 2 plus 3 equals the 2 by 1 column matrix 5 5 equals 5 v sub 1

A⁢v2=(4−22−6)=(2−4)=2⁢v2 A v sub 2 equals the 2 by 1 column matrix Row 1: 4 minus 2 Row 2: 2 minus 6 equals the 2 by 1 column matrix 2 negative 4 equals 2 v sub 2

Fundamental Properties

Eigenvectors of Distinct Eigenvalues Are Independent

If λ1,λ2,…,λk lamda sub 1 comma lamda sub 2 comma dot dot dot comma lamda sub k are distinct eigenvalues of A A with corresponding eigenvectors v1,v2,…,vk v sub 1 comma v sub 2 comma dot dot dot comma v sub k then these eigenvectors are linearly independent. This is a powerful result: it means that an n×n n times n matrix with n n distinct eigenvalues automatically has n n linearly independent eigenvectors.

The proof proceeds by contradiction. Suppose the eigenvectors were dependent. Then we could write one as a linear combination of others. Applying A A to this relation and using the eigenvalue property leads to a contradiction because the different eigenvalues force incompatible scalings.

Eigenvalues of Special Matrices

For a triangular matrix (upper or lower), the eigenvalues are exactly the diagonal entries. This follows immediately from the fact that det⁡(A−λ⁢I) the determinant of open paren A minus lamda I close paren for a triangular matrix is the product of its diagonal entries (a11−λ)⁢(a22−λ)⋯(an⁢n−λ) open paren a sub 11 minus lamda close paren times open paren a sub 22 minus lamda close paren dot dot dot open paren a sub n n minus lamda close paren

For a symmetric matrix (A=AT A equals A to the T power, all eigenvalues are real, even if we allow complex arithmetic. Moreover, eigenvectors corresponding to different eigenvalues are orthogonal. These properties make symmetric matrices especially well-behaved.

For an orthogonal matrix (AT⁢A=I A to the T power A equals I, all eigenvalues have absolute value 1: they lie on the unit circle in the complex plane. This reflects the fact that orthogonal transformations preserve lengths.

Eigenvalues Under Matrix Operations

If λ lamda is an eigenvalue of A A with eigenvector v v, then:

  • λk lamda to the k-th power is an eigenvalue of Ak A to the k-th power (for any positive integer k k), with the same eigenvector v v.

  • If A A is invertible, then 1/λ 1 divided by lamda is an eigenvalue of A−1 A to the negative 1 power, with eigenvector v v.

  • λ+c lamda plus c is an eigenvalue of A+c⁢I A plus c I, with eigenvector v v.

These properties follow directly from the eigenvalue equation. For example, if
A⁢v=λ⁢v A v equals lamda v, then A2⁢v=A⁢(A⁢v)=A⁢(λ⁢v)=λ⁢(A⁢v)=λ2⁢v A squared v equals A times A v equals A times lamda v equals lamda times A v equals lamda squared v.

Complex Eigenvalues

Even for a matrix with all real entries, the characteristic polynomial may have complex roots. When this happens, the complex eigenvalues occur in conjugate pairs: if λ=a+b⁢i lamda equals a plus b i is an eigenvalue, then so is λ¯=a−b⁢i lamda bar equals a minus b i

Complex eigenvalues indicate that the transformation involves rotation. There is no real direction that the transformation preserves (other than by accident). The real and imaginary parts of the complex eigenvector can be combined to describe the rotational behavior on a real invariant plane.

For example, the rotation matrix:

Rθ=(cos⁡θ−sin⁡θsin⁡θcos⁡θ) R sub theta equals the 2 by 2 matrix Row 1: Column 1, cosine theta Column 2, negative sine theta Row 2: Column 1, sine theta Column 2, cosine theta

has eigenvalues λ=cos⁡θ±i⁢sin⁡θ=e±i⁢θ lamda equals cosine theta plus or minus i sine theta equals e raised to the plus or minus i theta power These complex eigenvalues on the unit circle reflect the fact that rotation preserves lengths and involves genuine rotation (no real eigenvectors unless θ theta is a multiple of π pi.

Applications and Connections

Eigenvalues and eigenvectors appear throughout mathematics and its applications. In differential equations, solutions to x′⁢(t)=A⁢x⁢(t) x prime times t equals A x times t have the form x⁢(t)=eλ⁢t⁢v x times t equals e raised to the lamda t power v,
where λ lamda and v v are eigenvalue–eigenvector pairs. The eigenvalues determine stability: negative real parts imply decay, positive real parts imply growth.

In principal component analysis, eigenvectors of the covariance matrix give the principal directions of variation in the data, and eigenvalues measure the variance along each direction. This enables dimensionality reduction while preserving the dominant structure.

In quantum mechanics, physical observables are represented by operators, and measured values are eigenvalues. Eigenvectors represent states with definite values of the observable.

In Google’s PageRank algorithm, web pages are ranked according to the dominant eigenvector of a matrix representing link structure. The eigenvalue equation A⁢v=v A v equals v (with λ=1 lamda equals 1) finds the stationary distribution of a random walk on the web.

Summary

An eigenvector of a matrix A A is a nonzero vector v v satisfying
A⁢v=λ⁢v A v equals lamda v for some scalar λ lamda (the eigenvalue). Eigenvectors represent directions preserved by the transformation; eigenvalues measure the scaling along these directions.

Eigenvalues are found as roots of the characteristic polynomial det⁡(A−λ⁢I)=0 the determinant of open paren A minus lamda I close paren equals 0. Once an eigenvalue is known, eigenvectors are found by solving the homogeneous system (A−λ⁢I)⁢v=0 open paren A minus lamda I close paren times v equals 0.

Key properties include: eigenvectors corresponding to distinct eigenvalues are linearly independent; the sum of eigenvalues equals the trace; the product equals the determinant; symmetric matrices have real eigenvalues and orthogonal eigenvectors. Complex eigenvalues indicate rotational behavior.

Eigenanalysis is one of the most powerful tools in applied mathematics, revealing the intrinsic structure of linear transformations and enabling applications from stability analysis to data science to quantum physics.