Eigenvalues and Eigenvectors
Subtopic: Eigentheory
Topic: Linear Algebra
Introduction
When we apply a linear transformation to a vector, the result is typically a new vector pointing in a different direction and with a different magnitude. However, certain special vectors behave remarkably simply: they emerge from the transformation pointing in exactly the same direction (or the exact opposite), merely stretched or compressed by some factor. These special vectors are called eigenvectors, and the scaling factors are called eigenvalues. Together, they reveal the intrinsic geometry of a linear transformation.
Consider spinning a globe on its axis. Every point on the globe moves in a circle—except for points on the axis itself. These axial points stay fixed, or at most get reflected through the center. The axis represents an eigenvector of the rotation: a direction that the transformation preserves. This example illustrates a profound principle: even complex transformations often have simple underlying structure, and eigenvectors expose that structure.
Eigenvalues and eigenvectors are not merely computational tools. They are the key to understanding the behavior of dynamical systems, the stability of equilibria, the principal axes of stress in materials, the modes of vibration in mechanical systems, and the fundamental frequencies of quantum states. In data science, they underlie principal component analysis, Google's PageRank algorithm, and spectral clustering. Mastering eigenvalues means gaining insight into the deepest structure of linear maps.
Definition
Let be an matrix. A nonzero vector is called an eigenvector of if applying to produces a scalar multiple of :
.
The scalar is called the eigenvalue corresponding to the eigenvector . The requirement that is essential: the zero vector trivially satisfies for any , so we exclude it to make the concept meaningful.
The equation says that and are parallel. The transformation does not rotate ; it only scales it. When , the vector is stretched (if ) or compressed (if ) while pointing in the same direction. When , the vector is also reflected through the origin, ending up pointing in the opposite direction. When , the eigenvector is sent to the zero vector — it lies in the null space of .
Geometric Interpretation
Geometrically, eigenvectors are the directions along which the linear transformation acts by pure scaling. Think of a matrix as a machine that takes in vectors and outputs transformed vectors. Most input vectors come out pointing in different directions. Eigenvectors are the special inputs that come out pointing in the same direction — just longer, shorter, or flipped.
For a matrix, the transformation can be visualized as deforming the plane. Consider a shearing transformation that tilts the plane. Most vectors get tilted, but certain directions are preserved. These preserved directions are the eigenvectors, and how much they are stretched gives the eigenvalues.
When all eigenvalues are positive, the transformation preserves orientation — it stretches or compresses without flipping. When an eigenvalue is negative, vectors along that eigenvector are reflected. When eigenvalues are complex, the transformation involves rotation, and there are no real directions that remain fixed (except in trivial cases).
The eigenvalues encode the transformation’s behavior along its principal axes, while the eigenvectors identify those axes. Eigenanalysis decomposes a potentially complicated transformation into simple scalings along well-chosen directions.
Finding Eigenvalues
To find the eigenvalues of a matrix , rewrite the eigenvalue equation in a form that reveals when nontrivial solutions exist. Subtract from both sides:
.
Factor out :
.
This is a homogeneous linear system. For a nonzero solution to exist, the matrix must be singular, meaning it has zero determinant. This gives the characteristic equation:
The left-hand side expands to a polynomial in , called the characteristic polynomial. For an matrix, this polynomial has degree . The eigenvalues are exactly the roots of the characteristic polynomial.
The Characteristic Polynomial
For an matrix the characteristic polynomial is:
The coefficient of involves the trace of (the sum of diagonal entries), and the constant term is By Vieta's formulas, the sum of all eigenvalues equals the trace, and the product of all eigenvalues equals the determinant:
These relationships provide useful checks on eigenvalue computations and connect eigenvalues to other fundamental matrix properties.
Finding Eigenvectors
Once an eigenvalue is known, the corresponding eigenvectors are found by solving the homogeneous system:
.
This amounts to finding the null space of the matrix . The null space is nontrivial (contains more than just ) because was chosen to make the matrix singular. Any nonzero vector in this null space is an eigenvector corresponding to .
The set of all eigenvectors corresponding to a single eigenvalue , together with the zero vector, forms a subspace called the eigenspace of :
.
The dimension of this eigenspace is called the geometric multiplicity of . It tells how many linearly independent eigenvectors share that eigenvalue. The geometric multiplicity is at least and at most the algebraic multiplicity (the number of times appears as a root of the characteristic polynomial).
Worked Example
Consider the matrix:
We seek all eigenvalues and eigenvectors of
Step 1: Find the Characteristic Polynomial
Compute
The determinant is:
Step 2: Find the Eigenvalues
Set the characteristic polynomial equal to zero and solve:
Factoring:
The eigenvalues are and .
We can verify: the trace of is , and the determinant is
. Both checks confirm the eigenvalues.
Step 3: Find the Eigenvectors
For , solve :
Both equations reduce to , or . Taking , we obtain the eigenvector .
For , solve :
.
Both equations give , or . Taking , we obtain .
Verification:
Fundamental Properties
Eigenvectors of Distinct Eigenvalues Are Independent
If are distinct eigenvalues of with corresponding eigenvectors then these eigenvectors are linearly independent. This is a powerful result: it means that an matrix with distinct eigenvalues automatically has linearly independent eigenvectors.
The proof proceeds by contradiction. Suppose the eigenvectors were dependent. Then we could write one as a linear combination of others. Applying to this relation and using the eigenvalue property leads to a contradiction because the different eigenvalues force incompatible scalings.
Eigenvalues of Special Matrices
For a triangular matrix (upper or lower), the eigenvalues are exactly the diagonal entries. This follows immediately from the fact that for a triangular matrix is the product of its diagonal entries
For a symmetric matrix (, all eigenvalues are real, even if we allow complex arithmetic. Moreover, eigenvectors corresponding to different eigenvalues are orthogonal. These properties make symmetric matrices especially well-behaved.
For an orthogonal matrix (, all eigenvalues have absolute value 1: they lie on the unit circle in the complex plane. This reflects the fact that orthogonal transformations preserve lengths.
Eigenvalues Under Matrix Operations
If is an eigenvalue of with eigenvector , then:
is an eigenvalue of (for any positive integer ), with the same eigenvector .
If is invertible, then is an eigenvalue of , with eigenvector .
is an eigenvalue of , with eigenvector .
These properties follow directly from the eigenvalue equation. For example, if
, then .
Complex Eigenvalues
Even for a matrix with all real entries, the characteristic polynomial may have complex roots. When this happens, the complex eigenvalues occur in conjugate pairs: if is an eigenvalue, then so is
Complex eigenvalues indicate that the transformation involves rotation. There is no real direction that the transformation preserves (other than by accident). The real and imaginary parts of the complex eigenvector can be combined to describe the rotational behavior on a real invariant plane.
For example, the rotation matrix:
has eigenvalues These complex eigenvalues on the unit circle reflect the fact that rotation preserves lengths and involves genuine rotation (no real eigenvectors unless is a multiple of .
Applications and Connections
Eigenvalues and eigenvectors appear throughout mathematics and its applications. In differential equations, solutions to have the form ,
where and are eigenvalue–eigenvector pairs. The eigenvalues determine stability: negative real parts imply decay, positive real parts imply growth.
In principal component analysis, eigenvectors of the covariance matrix give the principal directions of variation in the data, and eigenvalues measure the variance along each direction. This enables dimensionality reduction while preserving the dominant structure.
In quantum mechanics, physical observables are represented by operators, and measured values are eigenvalues. Eigenvectors represent states with definite values of the observable.
In Google’s PageRank algorithm, web pages are ranked according to the dominant eigenvector of a matrix representing link structure. The eigenvalue equation (with ) finds the stationary distribution of a random walk on the web.
Summary
An eigenvector of a matrix is a nonzero vector satisfying
for some scalar (the eigenvalue). Eigenvectors represent directions preserved by the transformation; eigenvalues measure the scaling along these directions.
Eigenvalues are found as roots of the characteristic polynomial . Once an eigenvalue is known, eigenvectors are found by solving the homogeneous system .
Key properties include: eigenvectors corresponding to distinct eigenvalues are linearly independent; the sum of eigenvalues equals the trace; the product equals the determinant; symmetric matrices have real eigenvalues and orthogonal eigenvectors. Complex eigenvalues indicate rotational behavior.
Eigenanalysis is one of the most powerful tools in applied mathematics, revealing the intrinsic structure of linear transformations and enabling applications from stability analysis to data science to quantum physics.