2. Matrix-Vector Product

The matrix-vector product is an important special case of the matrix-matrix product. The product of a matrix and the column vector results in a vector given by:

There are two fundamentally different yet equivalent ways to interpret the matrix-vector product. In the column picture, , the multiplication of the matrix by the vector produces a linear combination of the columns of the matrix: , where and are the first and second columns of the matrix .

In the row picture, , multiplication of the matrix by the vector produces a column vector with coefficients equal to the dot products of rows of the matrix with the vector .

Linear transformations

The matrix-vector product is used to define the notion of a linear transformation, which is one of the key notions in the study of linear algebra. Multiplication by a matrix can be thought of as computing a linear transformation that takes -vectors as inputs and produces -vectors as outputs:

Instead of writing for the linear transformation applied to the vector , we simply write . Applying the linear transformation to the vector corresponds to the product of the matrix and the column vector . We say is represented by the matrix .

You can think of linear transformations as "vector functions" and describe their properties in analogy with the regular functions you are familiar with:

function linear transformation

input input

output output

function inverse matrix inverse

zeros of null space of

range of column space of range of

Note that the combined effect of applying the transformation followed by on the input vector is equivalent to the matrix product .

Fundamental vector spaces

A vector space consists of a set of vectors and all linear combinations of these vectors. For example the vector space span consists of all vectors of the form

, where and are real numbers. We now define three fundamental vector spaces associated with a matrix .

The column space of a matrix is the set of vectors that can be produced as linear combinations of the columns of the matrix :

for some

The column space is the range of the linear transformation the set of possible outputs. You can convince yourself of this fact by reviewing the definition of the matrix-vector product in the column picture C. The vector contains times the column of , times the column of , etc. Varying over all possible inputs , we obtain all possible linear combinations of the columns of , hence the name "column space."

The null space of a matrix consists of all the vectors that the matrix sends to the zero vector:

The vectors in the null space are orthogonal to all the rows of the matrix. We can see this from the row picture ; the output vectors is if and only if the input vector is orthogonal to all the rows of .

The row space of a matrix , denoted , is the set of linear combinations of the rows of . The row space is the orthogonal complement of the null space . This means that for all vectors and all vectors , we have . Together, the null space and the row space form the domain of the transformation , , where stands for orthogonal direct sum.

More Cheat Sheets ->