3.1 Matrix-Vector Product

The matrix-vector product is an important special case of the matrix-matrix product. The product of a $3\times 2$ matrix $A$ and the $2\times 1$ column vector $\overrightarrow{x}$ results in a $3\times 1$ vector $\overrightarrow{y}$ given by:

$\overrightarrow{y}=A\overrightarrow{x}=\left[\begin{array}{c}{y}_1\\ y_2\\ y_3\end{array}\right]=\left[\begin{array}{cc}{a}_{11}& a_{12}\\ a_{21}& a_{22}\\ a_{31}& a_{32}\end{array}\right]\left[\begin{array}{c}{x}_1\\ x_2\end{array}\right]=\left[\begin{array}{c}{a}_{11}{x}_1+a_{12}{x}_2\\ a_{21}{x}_1+a_{22}{x}_2\\ a_{31}{x}_1+a_{32}{x}_2\end{array}\right]$

There are two fundamentally different yet equivalent ways to interpret the matrix-vector product. In the column picture, $\left(C\right)$, the multiplication of the matrix $A$ by the vector $\overrightarrow{x}$ produces a linear combination of the columns of the matrix: $\overrightarrow{y}=A\overrightarrow{x}=x_1{A}_{:,1}+x_2{A}_{:,2}$, where $A_{:,1}$ and $A_{:,2}$ are the first and second columns of the matrix $A$.

In the row picture, $\left(R\right)$, multiplication of the matrix $A$ by the vector $\overrightarrow{x}$ produces a column vector with coefficients equal to the dot products of rows of the matrix with the vector $\overrightarrow{x}$.

Linear transformations

The matrix-vector product is used to define the notion of a linear transformation, which is one of the key notions in the study of linear algebra. Multiplication by a matrix $A\in {ℝ}^{m\times n}$ can be thought of as computing a linear transformation $T_A$ that takes $n$-vectors as inputs and produces $m$-vectors as outputs:

$T_A:{ℝ}^n\to {ℝ}^m$

Instead of writing $\overrightarrow{y}=T_A\overrightarrow{x}$ for the linear transformation $T_A$ applied to the vector $\overrightarrow{x}$, we simply write $\overrightarrow{y}=A\overrightarrow{x}$. Applying the linear transformation $T_A$ to the vector $\overrightarrow{x}$ corresponds to the product of the matrix $A$ and the column vector $\overrightarrow{x}$. We say $T_A$ is represented by the matrix $A$.

You can think of linear transformations as "vector functions" and describe their properties in analogy with the regular functions you are familiar with:

function $ƒ:ℝ\to ℝ$ $\iff$ linear transformation $T_A:{ℝ}^n\to {ℝ}^m$

input $x\in ℝ\iff$ input $\overrightarrow{x}\in {ℝ}^n$

output $ƒ\left(x\right)\iff$ output $T_A\left(\overrightarrow{x}\right)=A\overrightarrow{x}\in {ℝ}^m$

$g\circ ƒ=g(ƒ\left(x\right))\iff$ $T_B(T_A\left(\overrightarrow{x}\right))=BA\overrightarrow{x}$

function inverse ${ƒ}^{-1}\iff$ matrix inverse $A^{-1}$

zeros of $ƒ\iff$ $𝘕\left(A\right)=$ null space of $A$

range of $ƒ\iff$ $𝘊\left(A\right)=$ column space of $A=$ range of $T_A$

Note that the combined effect of applying the transformation $T_A$ followed by $T_B$ on the input vector $\overrightarrow{x}$ is equivalent to the matrix product $BA\overrightarrow{x}$.

Fundamental vector spaces

A vector space consists of a set of vectors and all linear combinations of these vectors. For example the vector space $𝘚=$span$\{{\overrightarrow{v}}_1,{\overrightarrow{v}}_2\}$ consists of all vectors of the form $\overrightarrow{v}=\alpha$

$\overrightarrow{v}+\beta \overrightarrow{v}$, where $\alpha$ and $\beta$ are real numbers. We now define three fundamental vector spaces associated with a matrix $A$.

The column space of a matrix $A$ is the set of vectors that can be produced as linear combinations of the columns of the matrix $A$:

$𝘊\left(A\right)\equiv \{\overrightarrow{y}\in {ℝ}^m|\overrightarrow{y}=A\overrightarrow{x}$for some $\overrightarrow{x}\in {ℝ}^n\}$

The column space is the range of the linear transformation $T_A$ the set of possible outputs. You can convince yourself of this fact by reviewing the definition of the matrix-vector product in the column picture C. The vector $A\overrightarrow{x}$ contains $x_1$ times the $1^{\mathrm{st}}$ column of $A$, $x_2$ times the $2^{\mathrm{nd}}$ column of $A$, etc. Varying over all possible inputs $\overrightarrow{x}$, we obtain all possible linear combinations of the columns of $A$, hence the name "column space."

The null space $𝘕\left(A\right)$ of a matrix $A\in {ℝ}^{m\times n}$ consists of all the vectors that the matrix $A$ sends to the zero vector:

$𝘕\left(A\right)\equiv \{\overrightarrow{x}\in {ℝ}^n|A\overrightarrow{x}=\overrightarrow{O}\}$

The vectors in the null space are orthogonal to all the rows of the matrix. We can see this from the row picture $\left(R\right)$; the output vectors is $\overrightarrow{O}$ if and only if the input vector $\overrightarrow{x}$ is orthogonal to all the rows of $A$.

The row space of a matrix $A$, denoted $R\left(A\right)$, is the set of linear combinations of the rows of $A$. The row space $R\left(A\right)$ is the orthogonal complement of the null space $𝘕\left(A\right)$. This means that for all vectors $\overrightarrow{v}\in R\left(A\right)$ and all vectors $\overrightarrow{w}\in 𝘕\left(A\right)$, we have $\overrightarrow{v}\cdot \overrightarrow{w}=0$. Together, the null space and the row space form the domain of the transformation $T_A$, ${ℝ}^n=𝘕\left(A\right)⨁R\left(A\right)$, where $⨁$ stands for orthogonal direct sum.

More Cheat Sheets ->