Eigenvalues and Eigenvectors

Subtopic: Eigentheory
Topic: Linear Algebra

Introduction

When we apply a linear transformation to a vector, the result is typically a new vector pointing in a different direction and with a different magnitude. However, certain special vectors behave remarkably simply: they emerge from the transformation pointing in exactly the same direction (or the exact opposite), merely stretched or compressed by some factor. These special vectors are called eigenvectors, and the scaling factors are called eigenvalues. Together, they reveal the intrinsic geometry of a linear transformation.

Consider spinning a globe on its axis. Every point on the globe moves in a circle—except for points on the axis itself. These axial points stay fixed, or at most get reflected through the center. The axis represents an eigenvector of the rotation: a direction that the transformation preserves. This example illustrates a profound principle: even complex transformations often have simple underlying structure, and eigenvectors expose that structure.

Eigenvalues and eigenvectors are not merely computational tools. They are the key to understanding the behavior of dynamical systems, the stability of equilibria, the principal axes of stress in materials, the modes of vibration in mechanical systems, and the fundamental frequencies of quantum states. In data science, they underlie principal component analysis, Google's PageRank algorithm, and spectral clustering. Mastering eigenvalues means gaining insight into the deepest structure of linear maps.

Definition

Let A be an n×n matrix. A nonzero vector v is called an eigenvector of A if applying A to v produces a scalar multiple of v:

A*v=λ*v.

The scalar λ is called the eigenvalue corresponding to the eigenvector v. The requirement that v≠0 is essential: the zero vector trivially satisfies A*0=λ*0 for any λ, so we exclude it to make the concept meaningful.

The equation A*v=λ*v says that v and A*v are parallel. The transformation A does not rotate v; it only scales it. When λ>0, the vector is stretched (if λ>1) or compressed (if 0<λ<1) while pointing in the same direction. When λ<0, the vector is also reflected through the origin, ending up pointing in the opposite direction. When λ=0, the eigenvector is sent to the zero vector — it lies in the null space of A.

Geometric Interpretation

Geometrically, eigenvectors are the directions along which the linear transformation acts by pure scaling. Think of a matrix A as a machine that takes in vectors and outputs transformed vectors. Most input vectors come out pointing in different directions. Eigenvectors are the special inputs that come out pointing in the same direction — just longer, shorter, or flipped.

For a 2×2 matrix, the transformation can be visualized as deforming the plane. Consider a shearing transformation that tilts the plane. Most vectors get tilted, but certain directions are preserved. These preserved directions are the eigenvectors, and how much they are stretched gives the eigenvalues.

When all eigenvalues are positive, the transformation preserves orientation — it stretches or compresses without flipping. When an eigenvalue is negative, vectors along that eigenvector are reflected. When eigenvalues are complex, the transformation involves rotation, and there are no real directions that remain fixed (except in trivial cases).

The eigenvalues encode the transformation’s behavior along its principal axes, while the eigenvectors identify those axes. Eigenanalysis decomposes a potentially complicated transformation into simple scalings along well-chosen directions.

Finding Eigenvalues

To find the eigenvalues of a matrix A, rewrite the eigenvalue equation A*v=λ*v in a form that reveals when nontrivial solutions exist. Subtract λ*v from both sides:

A*v−λ*v=0.

Factor out v:

(A−λ*I)*v=0.

This is a homogeneous linear system. For a nonzero solution v to exist, the matrix (A−λ*I) must be singular, meaning it has zero determinant. This gives the characteristic equation:

det(A−λ*I)=0.

The left-hand side expands to a polynomial in λ, called the characteristic polynomial. For an n×n matrix, this polynomial has degree n. The eigenvalues are exactly the roots of the characteristic polynomial.

The Characteristic Polynomial

For an n×n matrix A the characteristic polynomial is:

p(λ)=det(A−λ*I)=(−1)n*λn+(−1)(n−1)*(tr *A)*λ(n−1)+⋯+det(A)

The coefficient of λ(n−1) involves the trace of A (the sum of diagonal entries), and the constant term is det(A) By Vieta's formulas, the sum of all eigenvalues equals the trace, and the product of all eigenvalues equals the determinant:

(λ_1)+(λ_2)+⋯+(λ_n)=tr(A)

(λ_1)⋅(λ_2)⋯(λ_n)=det(A)

These relationships provide useful checks on eigenvalue computations and connect eigenvalues to other fundamental matrix properties.

Finding Eigenvectors

Once an eigenvalue λ is known, the corresponding eigenvectors are found by solving the homogeneous system:

(A−λ*I)*v=0.

This amounts to finding the null space of the matrix (A−λ*I). The null space is nontrivial (contains more than just 0) because λ was chosen to make the matrix singular. Any nonzero vector in this null space is an eigenvector corresponding to λ.

The set of all eigenvectors corresponding to a single eigenvalue λ, together with the zero vector, forms a subspace called the eigenspace of λ:

(E_λ)=Null*(A−λ*I)={v:A*v=λ*v}.

The dimension of this eigenspace is called the geometric multiplicity of λ. It tells how many linearly independent eigenvectors share that eigenvalue. The geometric multiplicity is at least 1 and at most the algebraic multiplicity (the number of times λ appears as a root of the characteristic polynomial).

Worked Example

Consider the matrix:

A=([4,1],[2,3])

We seek all eigenvalues and eigenvectors of A

Step 1: Find the Characteristic Polynomial

Compute A−λ*I

A−λ*I=([4−λ,1],[2,3−λ])

The determinant is:

det(A−λ*I)=(4−λ)*(3−λ)−(1)*(2)=λ2−7*λ+10

Step 2: Find the Eigenvalues

Set the characteristic polynomial equal to zero and solve:

λ2−7*λ+10=0.

Factoring:

(λ−5)*(λ−2)=0.

The eigenvalues are (λ_1)=5 and (λ_2)=2.

We can verify: the trace of A is 4+3=7=5+2, and the determinant is
4⋅3−1⋅2=10=5⋅2. Both checks confirm the eigenvalues.

Step 3: Find the Eigenvectors

For (λ_1)=5, solve (A−5*I)*v=0:

([−1,1],[2,−2])*([(v_1)],[(v_2)])=([0],[0]).

Both equations reduce to −(v_1)+(v_2)=0, or (v_2)=(v_1). Taking (v_1)=1, we obtain the eigenvector (v_1)=([1],[1]).

For (λ_2)=2, solve (A−2*I)*v=0:

([2,1],[2,1])*([(v_1)],[(v_2)])=([0],[0]).

Both equations give 2*(v_1)+(v_2)=0, or (v_2)=−2*(v_1). Taking (v_1)=1, we obtain (v_2)=([1],[−2]).

Verification: A*(v_1)=([4+1],[2+3])=([5],[5])=5*(v_1)

A*(v_2)=([4−2],[2−6])=([2],[−4])=2*(v_2)

Fundamental Properties

Eigenvectors of Distinct Eigenvalues Are Independent

If (λ_1),(λ_2),…,(λ_k) are distinct eigenvalues of A with corresponding eigenvectors (v_1),(v_2),…,(v_k) then these eigenvectors are linearly independent. This is a powerful result: it means that an n×n matrix with n distinct eigenvalues automatically has n linearly independent eigenvectors.

The proof proceeds by contradiction. Suppose the eigenvectors were dependent. Then we could write one as a linear combination of others. Applying A to this relation and using the eigenvalue property leads to a contradiction because the different eigenvalues force incompatible scalings.

Eigenvalues of Special Matrices

For a triangular matrix (upper or lower), the eigenvalues are exactly the diagonal entries. This follows immediately from the fact that det(A−λ*I) for a triangular matrix is the product of its diagonal entries ((a_11)−λ)*((a_22)−λ)⋯((a_n*n)−λ)

For a symmetric matrix (A=AT, all eigenvalues are real, even if we allow complex arithmetic. Moreover, eigenvectors corresponding to different eigenvalues are orthogonal. These properties make symmetric matrices especially well-behaved.

For an orthogonal matrix (AT*A=I, all eigenvalues have absolute value 1: they lie on the unit circle in the complex plane. This reflects the fact that orthogonal transformations preserve lengths.

Eigenvalues Under Matrix Operations

If λ is an eigenvalue of A with eigenvector v, then:

• λk is an eigenvalue of Ak (for any positive integer k), with the same eigenvector v.

• If A is invertible, then 1/λ is an eigenvalue of A(−1), with eigenvector v.

• λ+c is an eigenvalue of A+c*I, with eigenvector v.

These properties follow directly from the eigenvalue equation. For example, if
A*v=λ*v, then A2*v=A*(A*v)=A*(λ*v)=λ*(A*v)=λ2*v.

Complex Eigenvalues

Even for a matrix with all real entries, the characteristic polynomial may have complex roots. When this happens, the complex eigenvalues occur in conjugate pairs: if λ=a+b*i is an eigenvalue, then so is λ=a−b*i

Complex eigenvalues indicate that the transformation involves rotation. There is no real direction that the transformation preserves (other than by accident). The real and imaginary parts of the complex eigenvector can be combined to describe the rotational behavior on a real invariant plane.

For example, the rotation matrix:

(R_θ)=([cos(θ),−sin(θ)],[sin(θ),cos(θ)])

has eigenvalues λ=cos(θ)±i*sin(θ)=e(±i*θ) These complex eigenvalues on the unit circle reflect the fact that rotation preserves lengths and involves genuine rotation (no real eigenvectors unless θ is a multiple of π.

Applications and Connections

Eigenvalues and eigenvectors appear throughout mathematics and its applications. In differential equations, solutions to x′*(t)=A*x*(t) have the form x*(t)=e(λ*t)*v,
where λ and v are eigenvalue–eigenvector pairs. The eigenvalues determine stability: negative real parts imply decay, positive real parts imply growth.

In principal component analysis, eigenvectors of the covariance matrix give the principal directions of variation in the data, and eigenvalues measure the variance along each direction. This enables dimensionality reduction while preserving the dominant structure.

In quantum mechanics, physical observables are represented by operators, and measured values are eigenvalues. Eigenvectors represent states with definite values of the observable.

In Google’s PageRank algorithm, web pages are ranked according to the dominant eigenvector of a matrix representing link structure. The eigenvalue equation A*v=v (with λ=1) finds the stationary distribution of a random walk on the web.

Summary

An eigenvector of a matrix A is a nonzero vector v satisfying
A*v=λ*v for some scalar λ (the eigenvalue). Eigenvectors represent directions preserved by the transformation; eigenvalues measure the scaling along these directions.

Eigenvalues are found as roots of the characteristic polynomial det(A−λ*I)=0. Once an eigenvalue is known, eigenvectors are found by solving the homogeneous system (A−λ*I)*v=0.

Key properties include: eigenvectors corresponding to distinct eigenvalues are linearly independent; the sum of eigenvalues equals the trace; the product equals the determinant; symmetric matrices have real eigenvalues and orthogonal eigenvectors. Complex eigenvalues indicate rotational behavior.

Eigenanalysis is one of the most powerful tools in applied mathematics, revealing the intrinsic structure of linear transformations and enabling applications from stability analysis to data science to quantum physics.