3.1 Matrix-Vector Product

The matrix-vector product is an important special case of the matrix-matrix product. The product of a 3×2 matrix A and the 2×1 column vector x results in a 3×1 vector y given by:

y=A*x=[[(y_1)],[(y_2)],[(y_3)]]=[[(a_11),(a_12)],[(a_21),(a_22)],[(a_31),(a_32)]]*[[(x_1)],[(x_2)]]=[[(a_11)*(x_1)+(a_12)*(x_2)],[(a_21)*(x_1)+(a_22)*(x_2)],[(a_31)*(x_1)+(a_32)*(x_2)]]

There are two fundamentally different yet equivalent ways to interpret the matrix-vector product. In the column picture, (C), the multiplication of the matrix A by the vector x produces a linear combination of the columns of the matrix: y=A*x=(x_1)*(A_:,1)+(x_2)*(A_:,2), where (A_:,1) and (A_:,2) are the first and second columns of the matrix A.

In the row picture, (R), multiplication of the matrix A by the vector x produces a column vector with coefficients equal to the dot products of rows of the matrix with the vector x.

Linear transformations

The matrix-vector product is used to define the notion of a linear transformation, which is one of the key notions in the study of linear algebra. Multiplication by a matrix A∈ℝ(m×n) can be thought of as computing a linear transformation (T_A) that takes n-vectors as inputs and produces m-vectors as outputs:

(T_A):ℝn→ℝm

Instead of writing y=(T_A)*x for the linear transformation (T_A) applied to the vector x, we simply write y=A*x. Applying the linear transformation (T_A) to the vector x corresponds to the product of the matrix A and the column vector x. We say (T_A) is represented by the matrix A.

You can think of linear transformations as "vector functions" and describe their properties in analogy with the regular functions you are familiar with:

function ƒ:ℝ→ℝ ⇔ linear transformation (T_A):ℝn→ℝm

input x∈ℝ⇔ input x∈ℝn

output ƒ(x)⇔ output (T_A)(x)=A*x∈ℝm

g∘ƒ=g(ƒ(x))⇔ (T_B)((T_A)(x))=B*A*x

function inverse ƒ(-1)⇔ matrix inverse A(-1)

zeros of ƒ⇔ 𝘕(A)= null space of A

range of ƒ⇔ 𝘊(A)= column space of A= range of (T_A)

Note that the combined effect of applying the transformation (T_A) followed by (T_B) on the input vector x is equivalent to the matrix product B*A*x.

Fundamental vector spaces

A vector space consists of a set of vectors and all linear combinations of these vectors. For example the vector space 𝘚=span{(v_1),(v_2)} consists of all vectors of the form v=α

v+β*v, where α and β are real numbers. We now define three fundamental vector spaces associated with a matrix A.

The column space of a matrix A is the set of vectors that can be produced as linear combinations of the columns of the matrix A:

𝘊(A)≡{y∈ℝm|y=A*x)for some (x∈ℝn}

The column space is the range of the linear transformation (T_A) the set of possible outputs. You can convince yourself of this fact by reviewing the definition of the matrix-vector product in the column picture C. The vector A*x contains (x_1) times the 1st column of A, (x_2) times the 2nd column of A, etc. Varying over all possible inputs x, we obtain all possible linear combinations of the columns of A, hence the name "column space."

The null space 𝘕(A) of a matrix A∈ℝ(m×n) consists of all the vectors that the matrix A sends to the zero vector:

𝘕(A)≡{x∈ℝn|A*x=O}

The vectors in the null space are orthogonal to all the rows of the matrix. We can see this from the row picture (R); the output vectors is O if and only if the input vector x is orthogonal to all the rows of A.

The row space of a matrix A, denoted R(A), is the set of linear combinations of the rows of A. The row space R(A) is the orthogonal complement of the null space 𝘕(A). This means that for all vectors v∈R(A) and all vectors w∈𝘕(A), we have v⋅w=0. Together, the null space and the row space form the domain of the transformation (T_A), ℝn=𝘕(A)⨁R(A), where ⨁ stands for orthogonal direct sum.

More Cheat Sheets ->