Linear Independence

Subtopic: Vector Spaces

Vectors are linearly independent if none of them is redundant—no vector can be written as a linear combination of the others. Linear independence means each vector contributes something new to the span. This concept is crucial for defining bases and understanding the dimension of vector spaces.

Introduction

Imagine you're giving directions: "Go 3 blocks east, then 2 blocks north." Those two directions (east, north) are independent—neither can replace the other. But if someone adds "and go 1 block northeast," that's redundant because northeast is already a combination of east and north.

Linear independence captures this idea mathematically. A set of vectors is independent if no vector is a "freeloader"—each one earns its place by adding a genuinely new direction.

Definition

Vectors (v_1),(v_2),…,(v_k) are linearly independent if the only solution to:

(c_1)*(v_1)+(c_2)*(v_2)+⋯+(c_k)*(v_k)=0

is the trivial solution (c_1)=(c_2)=…=(c_k)=0.

If there exists a nontrivial solution (some (c_i)≠0), the vectors are linearly dependent.

Equivalently: vectors are dependent if and only if at least one can be written as a linear combination of the others.

Geometric Interpretation

In (R^2):

• Two vectors are dependent if they're parallel (one is a scalar multiple of the other)

• Two non-parallel vectors are independent

• Three or more vectors in (R^2) are always dependent (the plane is 2-dimensional)

In (R^3):

• Three vectors are dependent if they lie in a common plane

• Three vectors pointing in truly different directions (not coplanar) are independent

The geometric test: vectors are independent if they span a space of dimension equal to their count. If k vectors only span a space of dimension less than k, some are redundant.

The Dependence Test

To check if vectors v₁, ..., vₖ are linearly independent:

1. Set up the equation c₁v₁ + c₂v₂ + ... + cₖvₖ = 0

2. Write as a homogeneous system Ax = 0 where A = [v₁ | v₂ | ... | vₖ]

3. Row reduce to find solutions

4. If only x = 0 is a solution → independent. If free variables exist → dependent.

To check whether vectors (v_1),…,(v_k) are linearly independent:

1. Set up the equation (c_1)*(v_1)+(c_2)*(v_2)+⋯+(c_k)*(v_k)=0.

2. Write this as a homogeneous system A*x=0, where A=[[(v_1),(v_2),…,(v_k)]].

3. Row reduce to find the solutions.

4. If the only solution is x=0, the vectors are independent. If free variables exist, the vectors are dependent.

Worked Example 1

Are (v_1)=(1,2,3), (v_2)=(4,5,6), (v_3)=(7,8,9) linearly independent?

Solve (c_1)*(v_1)+(c_2)*(v_2)+(c_3)*(v_3)=0. Form the matrix and row reduce:

([1,4,7],[2,5,8],[3,6,9])→([1,4,7],[0,−3,−6],[0,−6,−12])→([1,4,7],[0,1,2],[0,0,0])

The last row is all zeros, so (c_3) is a free variable. There are nontrivial solutions.

One solution: (c_3)=1, (c_2)=−2, (c_1)=1. Check:
(v_1)−2*(v_2)+(v_3)=(1,2,3)−2*(4,5,6)+(7,8,9)=(0,0,0) ✓

The vectors are linearly DEPENDENT. (They all lie in a plane.)

Worked Example 2

Are (v_1)=(1,0,0), (v_2)=(1,1,0), (v_3)=(1,1,1) linearly independent?

([1,1,1],[0,1,1],[0,0,1])

This is already upper triangular with all pivots nonzero. The only solution to A*x=0 is x=0.

The vectors are linearly INDEPENDENT.

Key Properties

• Any set containing the zero vector is dependent (0=1⋅0 is a nontrivial combination)

• A single nonzero vector is independent

• Two vectors are dependent if and only if one is a scalar multiple of the other

• In (R^n), any set of more than n vectors must be dependent

• If {(v_1),…,(v_k)} is independent and (v_k+1)∉span {(v_1),…,(v_k)}, then {(v_1),…,(v_k),(v_k+1)} is independent

• Any subset of an independent set is independent

Independence and Matrices

The columns of a matrix A are linearly independent if and only if A*x=0 has only the trivial solution, which happens if and only if A has a pivot in every column.

The rows of A are linearly independent if and only if A has a pivot in every row.

For a square n×n matrix A:

Columns independent ⟺ Rows independent ⟺det(A)≠0⟺A is invertible.

Finding a Dependence Relation

If vectors are dependent, you can express one as a combination of the others. To find the relation:

1. Solve (c_1)*(v_1)+⋯+(c_k)*(v_k)=0 (find the null space).

2. Pick any nontrivial solution.

3. Isolate the vector with nonzero coefficient:
   (v_j)=−(c_1)/(c_j)*(v_1)−… (skipping the j-th term).

Applications

In data analysis, linearly dependent features carry redundant information. Removing them reduces dimensionality without losing information.

In structural engineering, independent force vectors determine stability. Dependent forces mean some are redundant or the structure is over-constrained.

In differential equations, independent solutions form a fundamental set—you need exactly n independent solutions for an (n^th)-order equation.

Summary

Vectors are linearly independent if no nontrivial combination equals zero—equivalently, none is redundant. To test independence, solve the homogeneous system with the vectors as columns: only the trivial solution means independent. Key facts: sets with zero are dependent; in (R^n), at most n vectors can be independent; independence of columns relates to matrix invertibility.