3.1 Matrix-Vector Product

The matrix-vector product is an important special case of the matrix-matrix product. The product of a 3 times 2 matrix A and the 2 times 1 column vector x right arrow results in a 3 times 1 vector y right arrow given by:

y right arrow equals A x right arrow equals the 3 by 1 column matrix y sub 1 y sub 2 y sub 3 equals the 3 by 2 matrix Row 1: a sub 11 a sub 12 Row 2: a sub 21 a sub 22 Row 3: a sub 31 a sub 32 times the 2 by 1 column matrix x sub 1 x sub 2 equals the 3 by 1 column matrix Row 1: a sub 11 x sub 1 plus a sub 12 x sub 2 Row 2: a sub 21 x sub 1 plus a sub 22 x sub 2 Row 3: a sub 31 x sub 1 plus a sub 32 x sub 2

There are two fundamentally different yet equivalent ways to interpret the matrix-vector product. In the column picture, C, the multiplication of the matrix A by the vector x right arrow produces a linear combination of the columns of the matrix: y right arrow equals A x right arrow equals x sub 1 A sub colon comma 1 plus x sub 2 A sub colon comma 2, where A sub colon comma 1 and A sub colon comma 2 are the first and second columns of the matrix A.

In the row picture, R, multiplication of the matrix A by the vector x right arrow produces a column vector with coefficients equal to the dot products of rows of the matrix with the vector x right arrow.

Linear transformations

The matrix-vector product is used to define the notion of a linear transformation, which is one of the key notions in the study of linear algebra. Multiplication by a matrix A is a member of r-m times n can be thought of as computing a linear transformation T sub A that takes n-vectors as inputs and produces m-vectors as outputs:

T sub A colon r-n right arrow r-m

Instead of writing y right arrow equals T sub A x right arrow for the linear transformation T sub A applied to the vector x right arrow, we simply write y right arrow equals A x right arrow. Applying the linear transformation T sub A to the vector x right arrow corresponds to the product of the matrix A and the column vector x right arrow. We say T sub A is represented by the matrix A.

You can think of linear transformations as "vector functions" and describe their properties in analogy with the regular functions you are familiar with:

function f with hook colon the real numbers right arrow the real numbers left right double arrow linear transformation T sub A colon r-n right arrow r-m

input x is a member of the real numbers left right double arrow input x right arrow is a member of r-n

output f with hook of x left right double arrow output T sub A of open paren x right arrow close paren equals A x right arrow is a member of r-m

g composed with f with hook equals g of f with hook of x left right double arrow T sub B of open paren T sub A of open paren x right arrow close paren close paren equals B A x right arrow

function inverse f with hook to the negative 1 power left right double arrow matrix inverse A to the negative 1 power

zeros of f with hook left right double arrow sans serif italic N of A equals null space of A

range of f with hook left right double arrow sans serif italic C of A equals column space of A equals range of T sub A

Note that the combined effect of applying the transformation T sub A followed by T sub B on the input vector x right arrow is equivalent to the matrix product B A x right arrow.

Fundamental vector spaces

A vector space consists of a set of vectors and all linear combinations of these vectors. For example the vector space sans serif italic S equalsspanthe set v right arrow sub 1 comma v right arrow sub 2 consists of all vectors of the form v right arrow equals alpha

v right arrow plus beta v right arrow, where alpha and beta are real numbers. We now define three fundamental vector spaces associated with a matrix A.

The column space of a matrix A is the set of vectors that can be produced as linear combinations of the columns of the matrix A:

sans serif italic C of A is identical to open brace y right arrow is a member of r-m divides y right arrow equals A x right arrowfor some x right arrow is a member of r-n close brace

The column space is the range of the linear transformation T sub A the set of possible outputs. You can convince yourself of this fact by reviewing the definition of the matrix-vector product in the column picture C. The vector A x right arrow contains x sub 1 times the 1 raised to the stones power column of A, x sub 2 times the 2 raised to the nd power column of A, etc. Varying over all possible inputs x right arrow, we obtain all possible linear combinations of the columns of A, hence the name "column space."

The null space sans serif italic N of A of a matrix A is a member of r-m times n consists of all the vectors that the matrix A sends to the zero vector:

sans serif italic N of A is identical to the set of all x right arrow in r-n such that A x right arrow equals O right arrow

The vectors in the null space are orthogonal to all the rows of the matrix. We can see this from the row picture R; the output vectors is O right arrow if and only if the input vector x right arrow is orthogonal to all the rows of A.

The row space of a matrix A, denoted R of A, is the set of linear combinations of the rows of A. The row space R of A is the orthogonal complement of the null space sans serif italic N of A. This means that for all vectors v right arrow is a member of R of A and all vectors w right arrow is a member of sans serif italic N of A, we have v right arrow times w right arrow equals 0. Together, the null space and the row space form the domain of the transformation T sub A, r-n equals sans serif italic N of A the n ary circled plus of R of A, where n ary circled plus stands for orthogonal direct sum.

More Cheat Sheets ->