Linear Independence

Subtopic: Vector Spaces

Vectors are linearly independent if none of them is redundant—no vector can be written as a linear combination of the others. Linear independence means each vector contributes something new to the span. This concept is crucial for defining bases and understanding the dimension of vector spaces.

Introduction

Imagine you're giving directions: "Go 3 3 blocks east, then 2 2 blocks north." Those two directions (east, north) are independent—neither can replace the other. But if someone adds "and go 1 1 block northeast," that's redundant because northeast is already a combination of east and north.

Linear independence captures this idea mathematically. A set of vectors is independent if no vector is a "freeloader"—each one earns its place by adding a genuinely new direction.

Definition

Vectors v1,v2,…,vk v sub 1 comma v sub 2 comma dot dot dot comma v sub k are linearly independent if the only solution to:

c1⁢v1+c2⁢v2+⋯+ck⁢vk=0 c sub 1 v sub 1 plus c sub 2 v sub 2 plus dot dot dot plus c sub k v sub k equals 0

is the trivial solution c1=c2=…=ck=0. c sub 1 equals c sub 2 equals dot dot dot equals c sub k equals 0 period

If there exists a nontrivial solution (some ci≠0 c sub i is not equal to 0), the vectors are linearly dependent.

Equivalently: vectors are dependent if and only if at least one can be written as a linear combination of the others.

Geometric Interpretation

In R2 R squared:

  • Two vectors are dependent if they're parallel (one is a scalar multiple of the other)

  • Two non-parallel vectors are independent

  • Three or more vectors in R2 R squared are always dependent (the plane is 2 2-dimensional)

In R3 R cubed:

  • Three vectors are dependent if they lie in a common plane

  • Three vectors pointing in truly different directions (not coplanar) are independent

The geometric test: vectors are independent if they span a space of dimension equal to their count. If k k vectors only span a space of dimension less than k k, some are redundant.

The Dependence Test

To check if vectors v₁, ..., vₖ are linearly independent:

  1. Set up the equation c₁v₁ + c₂v₂ + ... + cₖvₖ = 0

  2. Write as a homogeneous system Ax = 0 where A = [v₁ | v₂ | ... | vₖ]

  3. Row reduce to find solutions

  4. If only x = 0 is a solution → independent. If free variables exist → dependent.

To check whether vectors v1,…,vk v sub 1 comma dot dot dot comma v sub k are linearly independent:

  1. Set up the equation c1⁢v1+c2⁢v2+⋯+ck⁢vk=0 c sub 1 v sub 1 plus c sub 2 v sub 2 plus dot dot dot plus c sub k v sub k equals 0.

  2. Write this as a homogeneous system A⁢x=0 A x equals 0, where A=[v1v2…vk] A equals the 1 by 4 row matrix Column 1: v sub 1 Column 2: v sub 2 Column 3: dot dot dot Column 4: v sub k.

  3. Row reduce to find the solutions.

  4. If the only solution is x=0 x equals 0, the vectors are independent. If free variables exist, the vectors are dependent.

Worked Example 1

Are v1=(1,2,3) v sub 1 equals open paren 1 comma 2 comma 3 close paren, v2=(4,5,6) v sub 2 equals open paren 4 comma 5 comma 6 close paren, v3=(7,8,9) v sub 3 equals open paren 7 comma 8 comma 9 close paren linearly independent?

Solve c1⁢v1+c2⁢v2+c3⁢v3=0 c sub 1 v sub 1 plus c sub 2 v sub 2 plus c sub 3 v sub 3 equals 0. Form the matrix and row reduce:

(147258369)→(1470−3−60−6−12)→(147012000) the 3 by 3 matrix Row 1: 1 4 7 Row 2: 2 5 8 Row 3: 3 6 9 right arrow the 3 by 3 matrix Row 1: 1 4 7 Row 2: 0 negative 3 negative 6 Row 3: 0 negative 6 negative 12 right arrow the 3 by 3 matrix Row 1: 1 4 7 Row 2: 0 1 2 Row 3: 0 0 0

The last row is all zeros, so c3 c sub 3 is a free variable. There are nontrivial solutions.

One solution: c3=1 c sub 3 equals 1, c2=−2 c sub 2 equals negative 2, c1=1 c sub 1 equals 1. Check:
v1−2⁢v2+v3=(1,2,3)−2⁢(4,5,6)+(7,8,9)=(0,0,0) v sub 1 minus 2 v sub 2 plus v sub 3 equals open paren 1 comma 2 comma 3 close paren minus 2 times open paren 4 comma 5 comma 6 close paren plus open paren 7 comma 8 comma 9 close paren equals open paren 0 comma 0 comma 0 close paren ✓

The vectors are linearly DEPENDENT. (They all lie in a plane.)

Worked Example 2

Are v1=(1,0,0) v sub 1 equals open paren 1 comma 0 comma 0 close paren, v2=(1,1,0) v sub 2 equals open paren 1 comma 1 comma 0 close paren, v3=(1,1,1) v sub 3 equals open paren 1 comma 1 comma 1 close paren linearly independent?

(111011001) the 3 by 3 matrix Row 1: 1 1 1 Row 2: 0 1 1 Row 3: 0 0 1

This is already upper triangular with all pivots nonzero. The only solution to A⁢x=0 A x equals 0 is x=0 x equals 0.

The vectors are linearly INDEPENDENT.

Key Properties

  • Any set containing the zero vector is dependent (0=1⋅0 0 equals 1 times 0 is a nontrivial combination)

  • A single nonzero vector is independent

  • Two vectors are dependent if and only if one is a scalar multiple of the other

  • In Rn R to the n-th power, any set of more than n n vectors must be dependent

  • If {v1,…,vk} the set v sub 1 comma dot dot dot comma v sub k is independent and vk+1∉ v sub k plus 1 is not a member ofspan {v1,…,vk} the set v sub 1 comma dot dot dot comma v sub k, then {v1,…,vk,vk+1} the set v sub 1 comma dot dot dot comma v sub k comma v sub k plus 1 is independent

  • Any subset of an independent set is independent

Independence and Matrices

The columns of a matrix A A are linearly independent if and only if A⁢x=0 A x equals 0 has only the trivial solution, which happens if and only if A A has a pivot in every column.

The rows of A A are linearly independent if and only if A A has a pivot in every row.

For a square n×n n times n matrix A A:

Columns independent ⟺ long left right double arrow Rows independent ⟺det⁡(A)≠0⟺A long left right double arrow determinant A is not equal to 0 long left right double arrow A is invertible.

Finding a Dependence Relation

If vectors are dependent, you can express one as a combination of the others. To find the relation:

  1. Solve c1⁢v1+⋯+ck⁢vk=0 c sub 1 v sub 1 plus dot dot dot plus c sub k v sub k equals 0 (find the null space).

  2. Pick any nontrivial solution.

  3. Isolate the vector with nonzero coefficient:
    vj=−c1cj⁢v1−… v sub j equals negative the fraction with numerator c sub 1 and denominator c sub j v sub 1 minus dot dot dot (skipping the j j-th term).

Applications

In data analysis, linearly dependent features carry redundant information. Removing them reduces dimensionality without losing information.

In structural engineering, independent force vectors determine stability. Dependent forces mean some are redundant or the structure is over-constrained.

In differential equations, independent solutions form a fundamental set—you need exactly n n independent solutions for an nth n raised to the th power-order equation.

Summary

Vectors are linearly independent if no nontrivial combination equals zero—equivalently, none is redundant. To test independence, solve the homogeneous system with the vectors as columns: only the trivial solution means independent. Key facts: sets with zero are dependent; in Rn R to the n-th power, at most n n vectors can be independent; independence of columns relates to matrix invertibility.