5. *extra*

Basis

Intuitively, a basis is any set of vectors that can be used as a coordinate system for a vector space. You are certainly familiar with the standard basis for the -plane that is made up of two orthogonal axes: the -axis and the -axis. A vector can be described as a coordinate pair with respect to these axes, or equivalently as , where and are unit vectors that point along the -axis and -axis respectively. However, other coordinate systems are also possible.

A basis for a -dimensional vector space is any set of linearly independent vectors that are part of .

Any set of two linearly independent vectors can serve as a basis for . We can write any vector as a linear combination of these basis vectors .

Note the same vector corresponds to different coordinate pairs depending on the basis used: in the standard basis , and in the basis . Therefore, it is important to keep in mind the basis with respect to which the coefficients are taken, and if necessary specify the basis as a subscript, e.g., or .

Converting a coordinate vector from the basis to the basis is performed as a multiplication by a change of basis matrix:

Note the change of basis matrix is actually an identity transformation. The vector remains unchanged--it is simply expressed with respect to a new coordinate system. The change of basis from the -basis to the -basis is accomplished using the inverse matrix: .

Matrix representations of linear transformations

Bases play an important role in the representation of linear transformations . To fully describe the matrix that corresponds to some linear transformation , it is sufficient to know the effects of to the vectors of the standard basis for the input space. For a linear transformation , the matrix representation corresponds to

As a first example, consider the transformation which projects vectors onto the -axis. For any vector , we have . The matrix representation of is

As a second example, let's find the matrix representation of , the counterclockwise rotation by the angle :

The first column of shows that maps the vector to the vector . The second column shows that maps the vector to the vector .

Dimension and bases for vector spaces

The dimension of a vector space is defined as the number of vectors in a basis for that vector space. Consider the following vector space

span.

Seeing that the space is described by three vectors, we might think that is -dimensional. This is not the case, however, since the three vectors are not linearly independent so they don't form a basis for . Two vectors are sufficient to describe any vector in ; we can write

span, and we see these two vectors are linearly independent so they form a basis and dim .

There is a general procedure for finding a basis for a vector space. Suppose you are given a description of a vector space in terms of vectors

span and you are asked to find a basis for and the dimension of . To find a basis for , you must find a set of linearly independent vectors that span . We can use the Gauss-Jordan elimination procedure to accomplish this task. Write the vectors as the rows of a matrix . The vector space corresponds to the row space of the matrix . Next, use row operations to find the reduced row echelon form RREF of the matrix . Since row operations do not change the row space of the matrix, the row space of reduced row echelon form of the matrix is the same as the row space of the original set of vectors. The nonzero rows in the RREF of the matrix form a basis for vector space and the numbers of nonzero rows is the dimension of .

Row space, columns space, and rank of a matrix

Recall the fundamental vector spaces for matrices that we defined in Section II-E: the column space , the null space , and the row space . A standard linear algebra exam question is to give you a certain matrix and ask you to find the dimension and a basis for each of its fundamental spaces.

In the previous section we described a procedure based on Gauss-Jordan elimination which can be used "distill" a set of linearly independent vectors which form a basis for the row space . We will now illustrate this procedure with an example, and also show how to use the RREF of the matrix to find bases for and . Consider the following matrix and its reduced row echelon form:

The reduced row echelon form of the matrix contains three pivots. The locations of the pivots will play an important role in the following steps.

The vectors form a basis for .

To find a basis for the column space of the matrix we need to find which of the columns of are linearly independent. We can do this by identifying the columns which contain the leading ones in . The corresponding columns in the original matrix form a basis for the column space of . Looking at we see the first, third, and fourth columns of the matrix are linearly independent so the vectors form a basis for .

Now let's find a basis for the null space, . The second column does not contain a pivot, therefore it corresponds to a free variable, which we will denote . We are looking for a vector with three unknowns and one free variable that obeys the conditions:

11

Let's express the unknowns , , and in terms of the free variable . We immediately see that and , and we can write . Therefore, any vector of the form , for any , is in the null space of . We write .

Observe that the dim dim , this is known as the rank of the matrix . Also, dim dim , which is the dimension of the input space of the linear transformation .

Invertible matrix theorem

There is an important distinction between matrices that are invertible and those that are not as formalized by the following theorem.

For an matrix , the following statements are equivalent:

1. is invertible

2. The RREF of is the identity matrix

3. The rank of a matrix is

4. The row space of is

5. The column space of is

6. doesn't have a null space (only the zero vector )

7. The determinant of is a nonzero

For a given matrix , the above statements are either all true or all false. An invertible matrix corresponds to a linear transformation which maps the -dimensional input vector space to the -dimensional output vector space such that there exists an inverse transformation that can faithfully undo the effects of .

On the other hand, an matrix that is not invertible maps the input vector space to a subspace and has a nonempty null space. Once sends a vector to the zero vector, there is no that can undo this operation.

Eigenvalues and eigenvectors

The set of eigenvectors of a matrix is a special set of input vectors for which the action of the matrix is described as a simple scaling. When a matrix is multiplied by one of its eigenvectors the output is the same eigenvector multiplied by a constant . The constant is called an eigenvalue of .

To find the eigenvalues of a matrix we start from the eigenvalue equation , insert the identity , and rewrite it as a null-space problem:

This equation will have a solution whenever . The eigenvalues of , denoted , are the roots of the . The eigenvectors associated with the eigenvalue are the vectors in the null space of the matrix .

Certain matrices can be written entirely in terms of their eigenvectors and their eigenvalues. Consider the matrix that has the eigenvalues of the matrix on the diagonal, and the matrix constructed from the eigenvectors of as columns:

, , then

Matrices that can be written this way are called diagonalizable.

More Cheat Sheets ->