Math 511: Linear Algebra

4.3 Subspaces

---

4.3.1 What is a Subspace?

$$ \require{color} \definecolor{brightblue}{rgb}{.267, .298, .812} \definecolor{darkblue}{rgb}{0.0, 0.0, 1.0} \definecolor{palepink}{rgb}{1, .73, .8} \definecolor{softmagenta}{rgb}{.99,.34,.86} \definecolor{blueviolet}{rgb}{.537,.192,.937} \definecolor{jonquil}{rgb}{.949,.792,.098} \definecolor{shockingpink}{rgb}{1, 0, .741} \definecolor{royalblue}{rgb}{0, .341, .914} \definecolor{alien}{rgb}{.529,.914,.067} \definecolor{crimson}{rgb}{1, .094, .271} \def\ihat{\mathbf{\hat{\unicode{x0131}}}} \def\jhat{\mathbf{\hat{\unicode{x0237}}}} \def\khat{\mathrm{\hat{k}}} \def\tombstone{\unicode{x220E}} \def\contradiction{\unicode{x2A33}} $$

Having established the definition of a vector space last section, we can talk about subspaces of a vector space.

A subspace is a vector space that is formed by taking a select smaller part of an existing vector space which is itself a vector space

Given a vector space $V$, suppose $S$ is a subset of $V$. This subset will consist of vectors that have the same definitions for vector addition and scalar multiplication defined on $V$. Any set of vectors inherits all of the eight properties of vectors A2-A5 and A7-A10 because the are elements of a vector space. However the closure properties A1 and A6 are special.

If in addition, the sum of any two vectors in the subset $S$ is another vector in $S$, and any scalar multiple of a vector in $S$ is another vector in $S$, then we cay that $S$ is closed under vector addition ans scalar multiplication. We are saying that any linear combination of vectors in set $S$ is another vector in set $S$. We can also say that set $S$ is closed with respect to linear combinations. A subset of $S$ of a vector space $V$ that satisfies the two closure properties is called a subspace.

Definition of a Subspace

If $S$ is a nonempty subset of a vector space $V$, and $S$ satisfies the closure properties

(i) $\ \alpha \mathbf{x}\in S$ for any $\ \mathbf{x}\in S$ and scalar $\alpha$

(ii) $\ \mathbf{x} + \mathbf{y} \in S$ for any $\ \mathbf{x},\mathbf{y}\in S$

then $S$ is subspace of $V$.

---

4.3.2 A Line in 2D Space

Let us start with a simple example:

Example 1

Suppose that we have the vector space $\mathbb{R}^2$, and consider a subset $S$ written using set builder notation:

$$ S = \left\{\begin{bmatrix}\ x_1\ \\ \ x_2\ \end{bmatrix} : x_2 = -x_1 \right\} $$

Show that set $S$ is a subspace of $\mathbb{R}^2$.

Solution

First we must show that the set is nonempty. Consider the zero vector $\mathbf{u} = \begin{bmatrix}\ 0\ \\ \ 0\ \end{bmatrix}$. Since $u_2 = 0 = -0 = u_1$ we have that $\mathbf{u}\in S$, and $S\neq\emptyset$.

We need a way to represent elements of our nonempty set $S$. Since the second coordinate is always the negative of the first coordinate, we can use the notation

$$ \mathbf{a} = \begin{bmatrix}\ \ a\ \\ -a\ \end{bmatrix}\in S $$

for some number $a$, and multiply this by a scalar $\alpha$. This results in another vector in $S$,

$$ \alpha\mathbf{a} = \alpha\begin{bmatrix}\ \ a\ \\ -a\ \end{bmatrix} = \begin{bmatrix}\ \ \alpha a\ \\ -\alpha a\ \end{bmatrix}\in S $$
This shows that $S$ is closed under scalar multiplication. For the closure of vector addition, we consider two elements of $S$

$$ \begin{bmatrix}\ \ a\ \\ -a\ \end{bmatrix} \qquad\text{and}\qquad \begin{bmatrix}\ \ b\ \\ -b\ \end{bmatrix} $$
whose sum

$$ \begin{bmatrix}\ \ a\ \\ -a\ \end{bmatrix} + \begin{bmatrix}\ \ b\ \\ -b\ \end{bmatrix} = \begin{bmatrix}\ \ a + b\ \\ -a + (-b)\ \end{bmatrix} = \begin{bmatrix}\ \ a + b\ \\ -(a + b)\ \end{bmatrix}\in S $$
Since we have satisfied the closure properties, we say that $S$ is a subspace of $\mathbb{R}^2$, and that $S$ is itself a vector space.

---

4.3.3 Properties of Subspaces

One very significant claim that we have made about subspaces $S\subset V$ is that they are vector spaces. . Axioms A2-A5 and A7-A10 are guaranteed by Theorem 4.2.1. They are inherited by $V$. Therefore, since any subspace $S$ is a vector space all its own, we need only check the two closure properties to determine that $S$ is a subspace.

Lemma 4.3.1

Any subspace $S$ of vector space $V$ contains the origin of $V$.

Proof

Subspaces, by definition, are nonempty subsets $S$ of $V$ for which the closure properties hold. Since $S$ is nonempty let vector $\mathbf{x}\in S$. Since $S$ is closed under scalar multiplication, $-1\cdot\mathbf{x} = -\mathbf{x}\in S$. Since $S$ is closed under vector addition, $-\mathbf{x}+\mathbf{x} = \mathbf{0}\in S$. $\unicode{x220E}$

There are two examples of uninteresting subspaces of any vector space,

• $V$ (since every set is technically a subset of itself).
• the set containing only the zero vector $\left\{\mathbf{0}\right\}$ is called the trivial subspace of vector space $V$.

Every other subspace of $V$ is a called a proper subspace of $V$.

---

4.3.4 Examples of Subsets and Subspaces

Example 2

Let the set $S$ be defined by

$$ S = \left\{ \begin{bmatrix}\ x_1\ \\ \ x_2\ \\ \ x_3\ \end{bmatrix} \in\mathbb{R}^3 : x_1 = x_2 \right\} $$
To verify that this is subspace, we check that it contains $\mathbf{0}$, which it does since

$$ \mathbf{0} = \begin{bmatrix}\ 0\ \\ \ 0\ \\ \ 0\ \end{bmatrix} \in S $$
Next, we need to see if it obeys the closure properties.
(i) If $(a,a,b)^T\in S$, then for any scalar $\alpha$ we have

$$ \alpha (a,a,b)^T = (\alpha a,\alpha a,\alpha b)^T \in S $$
(ii) If $(a,a,b)^T,(c,c,d)^T\in S$, then

$$ (a,a,b)^T + (c,c,d)^T = (a+c,a+c,b+d)^T \in S $$
$S$ is a subspace of $\mathbb{R}^3$. It is a plane through the origin.

---

Example 3

Let $S = \left\{ (1,x)^T : x\in\mathbb{R}\right\} $.

This is not a subspace of $\mathbb{R}^2$ for multiple reasons:

1. It does not contain $\mathbf{0}$, since there is no choice of $x$ where $(1,x)^T$ will be $(0,0)^T$. This will ultimately prevent the subset from satisfying axiom A3.
   
   
   
2. This subset is not closed under scalar multiplication
   
   $$ \alpha \begin{bmatrix}\ \ 1\ \\ \ \ x\ \end{bmatrix} = \begin{bmatrix}\ \ \alpha\ \\ \ \ \alpha x\ \end{bmatrix} \notin S \text{ unless } \alpha = 1 $$
   
   
3. This subset is not closed under vector addition
   
   $$ \begin{bmatrix}\ \ 1\ \\ \ \ x\ \end{bmatrix} + \begin{bmatrix}\ \ 1\ \\ \ \ y\ \end{bmatrix} = \begin{bmatrix}\ \ 2\ \\ x+y \end{bmatrix} \notin S $$
   

One need only show one of these properties is false for our set $S$ to show that it cannot be a subspace of $\mathbb{R}^2$.

---

4.3.5 The Span of a Set of Vectors

We learned in the second 3Blue1Brown video Span that the set of all possible linear combinations of a collection of vectors is called the span of those vectors.

Definition of Span

Let $\mathbf{v}_1,\mathbf{v}_2,\ldots,\mathbf{v}_n$ be vectors in a vector space $V$. The set of all possible linear combinations $\ \alpha_1 \mathbf{v}_1 + \alpha_2 \mathbf{v}_2 + \ldots + \alpha_n \mathbf{v}_n$ for scalars $\alpha_j$ is called the span of $\mathbf{v}_1,\mathbf{v}_2,\ldots,\mathbf{v}_n$ and is denoted $\ \text{Span}\left(\mathbf{v}_1,\mathbf{v}_2,\ldots,\mathbf{v}_n\right)$.

The span of a set of vectors is the set of all vectors in a space that may be "reached" by taking a linear combination of the vectors in the set. We see this in the Example 4 solution reiterated above.

We can also easily write down a set that spans the subspace from Example 2

$$ \begin{bmatrix}\ a\ \\ \ a\ \\ \ b\ \end{bmatrix} = a\begin{bmatrix}\ 1\ \\ \ 1\ \\ \ 0\ \end{bmatrix} + b\begin{bmatrix}\ 0\ \\ \ 0\ \\ \ 1\ \end{bmatrix} $$

A linear combination of these two vectors will generate a vector that reaches each point on the plane $S$ in the figure from that example.

$$ S = \text{Span}\left\{ \begin{bmatrix}\ 1\ \\ \ 1\ \\ \ 0\ \end{bmatrix},\ \begin{bmatrix}\ 0\ \\ \ 0\ \\ \ 1\ \end{bmatrix} \right\} $$

Since this subset $S$ is the span of two linearly independent vectors, it is a plane. Since it is a span of vectors it is a subspace.

Theorem 4.3.2

If $\mathbf{v}_1,\mathbf{v}_2,\ldots,\mathbf{v}_n$ are vectors in a vector space $V$, then $\ \text{Span}\left(\mathbf{v}_1,\mathbf{v}_2,\ldots,\mathbf{v}_n\right)$ is a subspace of $V$.

Proof:

The $\text{Span}\left(\mathbf{v}_1,\mathbf{v}_2,\ldots,\mathbf{v}_n\right)$ is nonempty, since we can take each $\alpha_j = 0$, so $\mathbf{0}\in\text{Span}\left(\mathbf{v}_1,\mathbf{v}_2,\ldots,\mathbf{v}_n\right)$.

Let $\beta$ be a scalar and let $\mathbf{v} = \alpha_1 \mathbf{v}_1 + \alpha_2 \mathbf{v}_2 + \ldots + \alpha_n \mathbf{v}_n$ be any vector in $\text{Span}\left(\mathbf{v}_1,\mathbf{v}_2,\ldots,\mathbf{v}_n\right)$. Then we have

$$ \begin{align*} \beta\mathbf{v} &= \beta\left(\alpha_1 \mathbf{v}_1 + \alpha_2 \mathbf{v}_2 + \ldots + \alpha_n \mathbf{v}_n\right) \\ \\ &= (\beta\alpha_1)\mathbf{v}_1 + (\beta\alpha_2) \mathbf{v}_2 + \ldots + (\beta\alpha_n) \mathbf{v}_n \end{align*} $$
which is linear combination of the set of $\mathbf{v}_j$'s, so it is in $\text{Span}\left(\mathbf{v}_1,\mathbf{v}_2,\ldots,\mathbf{v}_n\right)$.

If we let $\mathbf{v} = \mathbf{v} = \alpha_1 \mathbf{v}_1 + \alpha_2 \mathbf{v}_2 + \ldots + \alpha_n \mathbf{v}_n$ and $\mathbf{w} = \beta_1 \mathbf{v}_1 + \beta_2 \mathbf{v}_2 + \ldots + \beta_n \mathbf{v}_n$ be any two vectors in the span, then

$$ \mathbf{v} + \mathbf{w} = (\alpha_1 + \beta_1)\mathbf{v}_1 + (\alpha_2 + \beta_2)\mathbf{v}_2 + \ldots + (\alpha_n + \beta_n)\mathbf{v}_n $$
Again, this is a linear combination of the $\mathbf{v}_j$'s, so spans are closed under addition. Hence, $\text{Span}\left(\mathbf{v}_1,\mathbf{v}_2,\ldots,\mathbf{v}_n\right)$ is a subspace of $V$.
$\unicode{x220E}$

4.3.6 Spanning Set

We call sets of vectors like the ones above associated with Example 2 spanning sets.

Definition of Spanning Set

The set $\left\{\mathbf{v}_1,\mathbf{v}_2,\ldots,\mathbf{v}_n\right\}$ is a spanning set for $V$ if and only if every vector $\mathbf{v}\in V$ may be written as a linear combination of $\mathbf{v}_1,\mathbf{v}_2,\ldots,\mathbf{v}_n$.

Spanning sets are not unique.

Example 4

Both the sets

$$ \mathbf{e}_1 = \begin{bmatrix}\ 1\ \\ \ 0\ \end{bmatrix}\qquad \mathbf{e}_2 = \begin{bmatrix}\ 0\ \\ \ 1\ \end{bmatrix} $$
and
$$ \mathbf{v}_1 = \begin{bmatrix}\ 1\ \\ \ 1\ \end{bmatrix}\qquad \mathbf{v}_2 = \begin{bmatrix}\ 0\ \\ \ 1\ \end{bmatrix} $$
span $\mathbb{R}^2$ since it is always possible to write any vector in $\mathbb{R}^2$ as a linear combination of either $\left\{\mathbf{e}_1,\mathbf{e}_2\right\}$ or $\left\{\mathbf{v}_1,\mathbf{v}_2\right\}$.

The way we show this is to let $(a,b)^T$ be any point in $\mathbb{R}^2$ and demonstrate that it is possible to always write this vector as a linear combination of the proposed spanning set. For the set $\left\{\mathbf{e}_1,\mathbf{e}_2\right\}$, this is easy.

$$ \begin{bmatrix}\ a\ \\ \ b\ \end{bmatrix} = a \begin{bmatrix}\ 1\ \\ \ 0\ \end{bmatrix} + b \begin{bmatrix}\ 0\ \\ \ 1\ \end{bmatrix} = \begin{bmatrix}\ 1\ &\ 0\ \\ \ 0\ &\ 1\ \end{bmatrix}\begin{bmatrix}\ a\ \\ \ b\ \end{bmatrix} $$

For $\left\{\mathbf{v}_1,\mathbf{v}_2\right\}$, we have

$$ \begin{bmatrix}\ a\ \\ \ b\ \end{bmatrix} = x_1 \begin{bmatrix}\ 1\ \\ \ 1\ \end{bmatrix} + x_2 \begin{bmatrix}\ 0\ \\ \ 1\ \end{bmatrix} = \begin{bmatrix}\ 1\ &\ 0\ \\ \ 1\ &\ 1\ \end{bmatrix}\begin{bmatrix}\ x_1\ \\ \ x_2\ \end{bmatrix} $$

and we need to show that the associated linear system

$$ \begin{align*} x_1 &= a \\ x_1 + x_2 &= b \end{align*} $$

is consistent.

$$ \begin{bmatrix}\ 1\ &\ 0 & | & a\ \\ \ 1\ &\ 1 & | & b\ \end{bmatrix}\begin{array}{l} \\ R_2-R_1 \end{array} \rightarrow \begin{bmatrix}\ 1\ &\ 0 & | & a\ \\ \ 0\ &\ 1 & | & b-a \end{bmatrix} $$

Both columns are pivot columns so the matrix $\begin{bmatrix}\ 1\ &\ 0\ \\ \ 1\ &\ 1 \ \end{bmatrix}$ is nonsingular. The solution to the linear system is the vector $\begin{bmatrix}\ a\ \\ b-a \end{bmatrix}$. We can conclude that the solution is unique because the matrix is nonsingular.

We have shown that every vector $\begin{bmatrix}\ a\ \\ \ b\ \end{bmatrix}\in\mathbb{R}^2$ can be written as a linear combination of $\mathbf{v}_1$ and $\mathbf{v}_2$. Hence we say that the set $\left\{\mathbf{v}_1,\mathbf{v}_2\right\}$ spans $\mathbb{R}^2$.

---

Exercise 1

Determine if

$$ \left\{\ \begin{bmatrix} -1\ \\ \ 2\ \\ \ 1\ \end{bmatrix},\ \begin{bmatrix}\ 0\ \\ \ 3\ \\ -1\ \end{bmatrix},\ \begin{bmatrix}\ 2\ \\ -1\ \\ -3\ \end{bmatrix}\ \right\}$$
spans $\mathbb{R}^3$.

Follow Along

For our set to span $\mathbb{R}^3$, we need to be able to express an arbitrary point $(a,b,c)^T$ in $\mathbb{R}^3$ as a linear combination of the vectors in the set.
$$ \begin{bmatrix}\ a\ \\ \ b\ \\ \ c\ \end{bmatrix} = \alpha_1 \begin{bmatrix} -1\ \\ \ 2\ \\ \ 1\ \end{bmatrix} + \alpha_2 \begin{bmatrix}\ 0\ \\ \ 3\ \\ -1\ \end{bmatrix} + \alpha_3 \begin{bmatrix}\ 2\ \\ -1\ \\ -3\ \end{bmatrix} $$

This corresponds to the linear system

$$ \begin{array}{rcr} -\alpha_1 \, \ \ \ \ \ \ \ \ \ \ + 2\alpha_3 & = & a \\ 2\alpha_1 + 3\alpha_2 - 1\alpha_3 & = & b \\ \ \ \alpha_1 - 1\alpha_2 - 3\alpha_3 & = & c \end{array} $$

whose coeffcient matrix

$$ \begin{bmatrix} -1\ &\ 0\ &\ 2\ \\ \ 2\ &\ 3\ & -1\ \\ \ 1\ & -1\ & -3\ \end{bmatrix} $$

has a reduced row echelon form of

$$ \left[\begin{array}{rrr}\ 1\ &\ 0\ & -2\ \\ \ 0\ &\ 1\ &\ 1\ \\ \ 0\ &\ 0\ &\ 0\ \end{array}\right] $$

This implies that the system is singular, since the coefficient matrix is a $3\times 3$ matrix with only two pivot columns. Because the system is singular, it cannot be solved for all vectors $(a,b,c)^T\in\mathbb{R}^3$. Our set does not span $\mathbb{R}^3$.

In fact the span of these three vectors is a two-dimensional subspace of $\mathbb{R}^3$ since the matrix has only two pivot columns.

This problem could have been solved using inspection by seeing how to write the third column vector in our set as a linear combination of the first two. The coefficients for this are revealed by the above reduced row echelon form.

$$ \begin{bmatrix}\ 1\ &\ 0\ & \color{#9900FF}{-2}\ \\ \ 0\ &\ 1\ &\ \color{#CC0099}{1}\ \\ \ 0\ &\ 0\ &\ 0\ \end{bmatrix} $$

$$ \begin{bmatrix}\ 2\ \\ -1\ \\ -3\ \end{bmatrix} = \color{#9900FF}{-2} \begin{bmatrix} -1\ \\ \ 2\ \\ \ 1\ \end{bmatrix} + \color{#CC0099}{1}\begin{bmatrix}\ 0\ \\ \ 3\ \\ -1\ \end{bmatrix} $$

This means that the span of these three vectors

$$ \left\{\,\begin{bmatrix}\ 2\ \\ -1\ \\ \ 0\ \end{bmatrix},\ \begin{bmatrix} -1\ \\ \ 2\ \\ \ 3\ \end{bmatrix},\ \begin{bmatrix} -3\ \\ \ 1\ \\ -1\ \end{bmatrix}\,\right\} $$

is only a plane because it only has two linearly independent vectors and one dependent vector.

---

4.3.7 Column Space and Null Space

Recall that the columns of any $m\times n$ matrix $A$ are the locations where the basis vectors of $\mathbb{R}^n$ are transformed into vectors in $\mathbb{R}^m$. In standard coordinates we have the standard vectors $\mathbf{e}_1$, $\mathbf{e}_2$, $\dots$, $\mathbf{e}_n$. These are also special, and they have a familiar pattern to them in our normal coordinate system.

In $\mathbb{R}^2$, we have

$$ \mathbf{e}_1 = \ihat = \begin{bmatrix}\ 1\ \\ \ 0\ \end{bmatrix},\ \text{ and }\ \mathbf{e}_2 = \jhat = \begin{bmatrix}\ 0\ \\ \ 1\ \end{bmatrix} $$

In $\mathbb{R}^3$, we have

$$ \mathbf{e}_1 = \ihat = \begin{bmatrix}\ 1\ \\ \ 0\ \\ \ 0\ \end{bmatrix},\ \mathbf{e}_2 = \jhat = \begin{bmatrix}\ 0\ \\ \ 1\ \\ \ 0\ \end{bmatrix},\ \text{and}\ \mathbf{e}_3 = \khat = \begin{bmatrix}\ 0\ \\ \ 0\ \\ \ 1\ \end{bmatrix} $$

In $\mathbb{R}^n$, we have

$$ \mathbf{e}_1 = \begin{bmatrix}\ 1\ \\ \ 0\ \\ \ \vdots\ \\ \ 0\ \end{bmatrix},\ \mathbf{e}_2 = \begin{bmatrix}\ 0\ \\ \ 1\ \\ \ \vdots\ \\ \ 0\ \end{bmatrix},\ \dots,\ \text{and}\ \mathbf{e}_n = \begin{bmatrix}\ 0\ \\ \ \vdots\ \\ \ 0\ \\ \ 1\ \end{bmatrix} $$

Recall that $A\mathbf{e}_k$ is the $k^{\text{th}}$ column of matrix $A$, and this column of matrix $A$ contain the coordinates of $L(\mathbf{e}_k) = A\mathbf{e}_k = \mathbf{a}_k$ in $\mathbb{R}^m$. This is the image of vector $\mathbf{e}_k$ under the linear transformation, as well as the column $\mathbf{a}_k$ of matrix $A$.

Any vector in $\mathbf{x}\in\mathbb{R}^n$ can be written as a linear combination of our standard vectors,

$$ \mathbf{x} = \begin{bmatrix}\ x_1\ \\ \ x_2\ \\ \vdots\ \\ \ x_n\ \end{bmatrix} = x_1\mathbf{e}_1 + x_2\mathbf{e}_2 + \dots + x_n\mathbf{e}_n $$

Thus we have the image of every vector $\mathbf{x}\in\mathbb{R}^n$ is the same linear combination of the columns of matrix $A$.

$$ \begin{align*} A\mathbf{x} &= A\left(x_1\mathbf{e}_1 + x_2\mathbf{e}_2 + \dots + x_n\mathbf{e}_n\right) \\ \\ &= x_1A\mathbf{e}_1 + x_2A\mathbf{e}_2 + \dots + x_nA\mathbf{e}_n \\ \\ &= x_1\mathbf{a}_1 + x_2\mathbf{a}_2 + \dots + x_n\mathbf{a}_n \end{align*} $$

Notice that the elements of vector $\mathbf{x}$ are the coefficients of the standard vectors, and the coefficients of the columns of matrix $A$. Now we can ask ourselves an interesting question,

$$ \text{What is the set of all image vectors in $\mathbb{R}^m$?} $$

This would be the set called the column space of matrix $A$,

$$ \begin{align*} C(A) &:= \left\{ A\mathbf{x}:\,\mathbf{x}\in\mathbb{R}^n \right\}\subset\mathbb{R}^m \\ \\ &= \left\{ x_1\mathbf{a}_1 + x_2\mathbf{a}_2 + \dots + x_n\mathbf{a}_n\,:\,(x_1,x_2,\dots,x_n)^T\in\mathbb{R}^n\right\} \end{align*} $$

Definition

The column space $C(A)$ of an $m\times n$ matrix $A$ is the span of the column vectors in matrix $A$. $C(A)$ is a subspace of the $\mathbb{R}^m$.

$$ C(A) := \left\{ A\mathbf{x}:\,\mathbf{x}\in\mathbb{R}^n \right\}\subset\mathbb{R}^m $$

In Column Space and Null Space Dr. Strang explains how to compute the column space of a matrix as the span of the pivot columns of the matrix. He goes on to show how to use the solutions to the homogeneous problem

$$ A\mathbf{x} = \mathbf{0} $$

to find a set of vectors whose span is the null space of a matrix.

---

4.3.8 The Null Space of a Matrix

Let $A$ be an $m\times n$ matrix, and let $N(A)$ be set of all solutions to the homogeneous system $A\mathbf{x} = \mathbf{0}$,

Definition

The null space $N(A)$ of an $m\times n$ matrix $A$ is the set of vectors in $\mathbb{R}^n$ defined by

$$ N(A) = \left\{ \mathbf{x}\in\mathbb{R}^n\,:\,A\mathbf{x}=\mathbf{0}\right\} $$

The column space of a matrix $C(A)$ is defined to be the span of the column vectors of the matrix and we already know that it is a subspace of $\mathbb{R}^m$ because it is the span or all possible linear combinations of the columns of matrix $A$. However, the null space is to be the set of all possible solutions to $A\mathbf{x}=\mathbf{0}$. This is not immediately a subspace. We are obligated to show that $N(A)$ is indeed a subspace of $\mathbb{R}^n$.

Lemma 4.3.3

The null space $N(A)$ of $m\times n$ matrix $A$ is a subspace of $\mathbb{R}^n$.

Proof

To show that $N(A)$ is a subspace of $\mathbb{R}^n$, we need to show it is nonempty and satisfies the closure properties. The set is nonempty, since $A\mathbf{0} = \mathbf{0}$. For scalar multiplication, we have for each $\mathbf{x}\in N(A)$ and all scalars $\alpha\in\mathbb{R}$

$$ A\left(\alpha \mathbf{x}\right) = \alpha\,A\mathbf{x} = \alpha\,\mathbf{0} = \mathbf{0} $$

It is closed under vector addition since for each $\mathbf{x},\mathbf{y}\in N(A)$

$$ A\left(\mathbf{x} + \mathbf{y}\right) = A\mathbf{x} + A\mathbf{y} = \mathbf{0} + \mathbf{0} = \mathbf{0} $$

Since $N(A)$ is a nonempty subset of $\mathbb{R}^n$ that is closed under vector addition and scalar multiplication, $N(A)$ is a subspace of $\mathbb{R}^n$. $\unicode{x220E}$

Example 5

Find $N(A)$ if


$$ A = \begin{bmatrix}\ 1\ &\ 2\ & -3\ & -1\ \\ -2\ & -4\ &\ 6\ &\ 3\ \end{bmatrix} $$

Solution

We begin by writing the augmented matrix that represents the homogeneous equation $A\mathbf{x} = \mathbf{0}$


$$ \begin{bmatrix}\ 1\ &\ 2\ & -3\ & -1 & | & 0\ \\ -2\ & -4\ &\ 6\ &\ 3 & | & 0\ \end{bmatrix} $$
and performing the necessary elementary row operations to place it into reduced row echelon form

$$ \begin{align*} \begin{bmatrix}\ 1\ &\ 2\ & -3\ & -1 & | & 0\ \\ -2\ & -4\ &\ 6\ &\ 3 & | & 0\ \end{bmatrix}\begin{array}{c} \ \\ R_2 + 2R_1 \end{array} & = \begin{bmatrix}\ 1\ &\ 2\ & -3\ & -1 & | & 0\ \\ \ 0\ &\ 0\ &\ 0\ &\ 1 & | & 0\ \end{bmatrix}\begin{array}{c} \ R_1+R_2 \\ \ \end{array} \\ \\ & = \left[\begin{array}{rrrr|r}\ 1\ &\ 2\ & -3\ &\ 0 & 0\ \\ \ 0\ &\ 0\ &\ 0\ &\ 1 & 0\ \end{array}\right] \end{align*} $$
We see from reduced row echelon form that we have two free variables $x_2$ and $x_3$


$$ \begin{align*} x_2 &= \alpha\in\mathbb{R} \\ x_3 &= \beta\in\mathbb{R} \\ x_4 &= 0 \\ x_1 + 2\alpha - 3\beta &= 0 \\ x_1 &= -2\alpha + 3\beta \end{align*} $$

$$ \mathbf{x} = \begin{bmatrix} -2\alpha + 3\beta \\ \alpha \\ \beta \\ 0 \end{bmatrix} = \alpha \begin{bmatrix} -2\ \\ \ \ 1\ \\ \ \ 0\ \\ \ \ 0\ \end{bmatrix} + \beta \begin{bmatrix}\ \ 3\ \\ \ \ 0\ \\ \ \ 1\ \\ \ \ 0\ \end{bmatrix} $$
are solutions of $A\mathbf{x} = \mathbf{0}$. Hence, $N(A)$ is composed of all vectors of the form


$$ \alpha \begin{bmatrix} -2\ \\ \ \ 1\ \\ \ \ 0\ \\ \ \ 0\ \end{bmatrix} + \beta \begin{bmatrix}\ 3\ \\ \ 0\ \\ \ 1\ \\ \ 0\ \end{bmatrix} $$

We can say that the null space of this matrix is a span of vectors

$$ N(A) = \text{Span}\left\{ \begin{bmatrix} -2\ \\ \ \ 1\ \\ \ \ 0\ \\ \ \ 0\ \end{bmatrix},\ \begin{bmatrix}\ 3\ \\ \ 0\ \\ \ 1\ \\ \ 0\ \end{bmatrix} \right\} $$

---

4.3.9 Properties of the Null Space

The most important property of a null space is its relationship with nonhomogenous systems:

Theorem 4.3.4

The solution of a nonhomogenous system

$$ A\mathbf{x} = \mathbf{b} $$

is unique if and only if the null space of $A$ is trivial, that is it is the subspace of $\mathbb{R}^n$ containing exactly the zero vector

$$ N(A) = \left\{\mathbf{0}\right\} $$

Proof:

To prove this, we need to show both

• a trivial null space $\implies$ that a solution to $A\mathbf{x}=\mathbf{b}$ is unique
• $A\mathbf{x}=\mathbf{b}$ having a unique solution $\implies$ that $N(A)$ is trivial.

$\left(\Rightarrow\right)$ First, suppose that $N(A)$ is trivial and the system $A\mathbf{x} = \mathbf{b}$ has a second solution $A\mathbf{y} = \mathbf{b}$. Then

$$ \begin{align*} A\mathbf{x} - A\mathbf{y} &= \mathbf{b} - \mathbf{b} \\ \\ A(\mathbf{x} - \mathbf{y}) &= \mathbf{0} \end{align*} $$
Since $N(A)$ is trivial, it must be the case that

$$ \begin{align*} \mathbf{x} - \mathbf{y} &= \mathbf{0} \\ \\ \mathbf{x} &= \mathbf{y} \end{align*} $$
The solution to the system is unique.

$\left(\Leftarrow\right)$ Now, suppose the solution to $A\mathbf{x} = \mathbf{b}$ is unique. Let $\mathbf{y}$ be an element of the null space $N(A)$, so $A\mathbf{y} = \mathbf{0}$. Then we have

$$ \begin{align*} A\mathbf{x} + A\mathbf{y} &= \mathbf{b} + \mathbf{0} \\ \\ A(\mathbf{x} + \mathbf{y}) &= \mathbf{b} \end{align*} $$

This says that $\mathbf{x} + \mathbf{y}$ is a solution to the system $A\mathbf{x} = \mathbf{b}$, but we assumed that $\mathbf{x}$ is unique. The only way that

$$ \begin{align*} \mathbf{x} + \mathbf{y} &= \mathbf{x} \\ \mathbf{y} &= \mathbf{0} \end{align*} $$

Since every element of the null space $\mathbf{y} = \mathbf{0}$, it must be that $\mathbf{0}$ is the only element of the null space. $N(A)$ is trivial. $\tombstone$

Now in Solving Ax = 0 we can follow along with Dr. Strang and use Gaussian Elimination to find the null space of a matrix. This new understanding of the special meaning of the free columns with their relation to the null space will help you apply linear algebra to other fields of study.

---

4.3.10 Solving $A\mathbf{x} = \mathbf{b}$

Finally we are in a position to review Solving Ax = b using the column space and the null space. This helps us understand when a solution exists, and when there are infinitely many solutions.

Theorem 4.3.5

The linear system $A\mathbf{x}=\mathbf{b}$ is consistent if and only if $\mathbf{b}\in C(A)$.

Proof:

As with our previous if and only if theorems, we must show both that

• The linear system $A\mathbf{x}=\mathbf{b}$ is consistent $\implies$ $\mathbf{b}\in C(A)$
• $\mathbf{b}\in C(A)\ \implies$ the linear system $A\mathbf{x}=\mathbf{b}$ is consistent.

$\left(\Rightarrow\right)$ If the linear system $A\mathbf{x}=\mathbf{b}$ is consistent, then there is at least one vector $\mathbf{x}_0\in\mathbb{R}^n$ so that $A\mathbf{x}_0=\mathbf{b}$. As $C(A) = \left\{ A\mathbf{x}\,:\,\mathbf{x}\in\mathbb{R}^n\right\}$, we have that $\mathbf{b}=A\mathbf{x}_0\in C(A)$.

$\left(\Leftarrow\right)$ If $\mathbf{b}\in C(A)$, then $\mathbf{b}\in\left\{ A\mathbf{x}\,:\,\mathbf{x}\in\mathbb{R}^n\right\}$. So there must be at least one vector $\mathbf{x}_0$ so that $\mathbf{b}=A\mathbf{x}_0$. Thus the linear system $A\mathbf{x}=\mathbf{b}$ has at least one solution and it is consistent. $\tombstone$

---

General Vector Spaces

In our previous discussions and proofs we limited ourselves to finite dimensional vector space, and more specifically to finite dimensional Euclidean vector spaces $\mathbb{R}^n$. This is unnecessarily restrictive. We need to understand that the methods we use are equally valid for any vector spaces. Some linear transformations cannot be represented by matrices. For example

$$ D\,:\,C^1[a,b]\rightarrow C[a,b] $$

defined by $D(f) = f'(x)$ is an example of a linear transformation from an infinite dimensional vector space $C^1[a,b]$ to another infinite dimensional vector space $C[a,b]$. It is linear because if $g,h\in C^1[a,b]$, then they are continuously differentiable on the real interval $[a,b]$. We can compute the derivative of $(g+h)(x)$ using the linearity properties of the derivative

$$ D(g + h) = (g + h)'(x) = g'(x) + h'(x) = D(g) + D(h) $$

Furthermore, if $\alpha\in\mathbb{R}$ is a scalar, then

$$ D(\alpha g) = (\alpha g)'(x) = \alpha\,g'(x) = \alpha\,D(g) $$

I hope you see the linear combination of vectors in $C^1[a,b]$ in our example. We will need to define and study linear transformations more carefully so that we can extend the ideas of null space and column space to these very important linear transformations.

---

4.3.11 More Exercises

Exercise 2

Let $A\in\mathbb{R}^{2\times 2}$ be a particular  vector in the vector space of real $2\times 2$ matrices $\mathbb{R}^{2\times 2}$. Determine whether the following are subspaces of $\mathbb{R}^{2\times 2}$. (When we say particular  we mean a fixed but arbitrary vector; that is, pick any vector and then keep it a constant one for the duration of this discussion. Remember that for this exercise a $2\times 2$ matrix is a vector in the vector space $\mathbb{R}^{2\times 2}$.)

1. $S_1 = \left\{\,B\in\mathbb{R}^{2\times 2}\,|\,BA=O\,\right\}$

2. $S_2 = \left\{\,B\in\mathbb{R}^{2\times 2}\,|\,AB\neq BA\,\right\}$

3. $S_3 = \left\{\,B\in\mathbb{R}^{2\times 2}\,|\,BA + B = O\,\right\}$
   
   

   View Solution

   
   
   To answer these questions one must either

   • show that the subset is a subspace, or
   • provide a counter example of a property of a subspace that does not hold for the subset

   
   

   1. $S_1 = \left\{\,B\in\mathbb{R}^{2\times 2}\,|\,BA=O\,\right\}$

   • $S_1$ is nonempty because the $2\times 2$ zero matrix is in $S_1$ because $OA = O$.
   • If $\alpha\in\mathbb{R}$ is a scalar and $B\in S_1$, then $\left(\alpha B\right)A = \alpha(BA) = \alpha(O) = O$. So $\alpha B\in S_1$.
   • If $B$, $C$ are both in $S_1$, then $(B + C)A = BA + CA = O + O = O$. So $B+C\in S_1$.

   Hence $S_1$ is a subspace of $\mathbb{R}^{2\times 2}$.

   
   

   2. $S_2 = \left\{\,B\in\mathbb{R}^{2\times 2}\,|\,AB\neq BA\,\right\}$

   $S_2$ is not a subspace of $\mathbb{R}^{2\times 2}$ because $AO = O = OA$. The $2\times 2$ zero matrix $O\notin S_2$.

   
   

   3. $S_3 = \left\{\,B\in\mathbb{R}^{2\times 2}\,|\,BA + B = O\,\right\}$

   • $S_3$ is nonempty because the $2\times 2$ zero matrix is in $S_3$ as $OA + O = O$.
   • If $\alpha\in\mathbb{R}$ is a scalar and $B\in S_3$, then $\left(\alpha B\right)A + \left(\alpha B\right) = \alpha(BA) + \alpha B = \alpha\left(BA + B\right) = \alpha(O) = O$. So $\alpha B\in S_3$.
   • If $B$, $C$ are both in $S_3$, then $(B + C)A + (B + C) = BA + B + CA + C = O + O = O$. So $B+C\in S_3$.

   Hence $S_3$ is a subspace of $\mathbb{R}^{2\times 2}$.

Exercise 3

In $\mathbb{R}^{2\times 2}$, let

$$ E_{11} = \begin{bmatrix} 1 & 0 \\ 0 & 0 \end{bmatrix},\ E_{12} = \begin{bmatrix} 0 & 1 \\ 0 & 0 \end{bmatrix},\ E_{21} = \begin{bmatrix} 0 & 0 \\ 1 & 0 \end{bmatrix},\ E_{22} = \begin{bmatrix} 0 & 0 \\ 0 & 1 \end{bmatrix} $$
Show that $E_{11}$, $E_{12}$, $E_{21}$, and $E_{22}$, span $\mathbb{R}^{2\times 2}$.

View Solution

To answer this questions one must show that ever vector, or $2\times 2$ matrix in $\mathbb{R}^{2\times 2}$ is a linear combination of $E_{11}$, $E_{12}$, $E_{21}$, and $E_{22}$.
For every vector $B = \begin{bmatrix} a & b \\ c & d \end{bmatrix}\in\mathbb{R}^{2\times 2}$ we have
$$ B = \begin{bmatrix} a & b \\ c & d \end{bmatrix} = aE_{11} + bE_{12} + cE_{21} + dE_{22} $$
Since every vector in $\mathbb{R}^{2\times 2}$ is a linear combination of $E_{11}$, $E_{12}$, $E_{21}$, and $E_{22}$. The set $\left\{E_{11},E_{12},E_{21},E_{22}\right\}$ spans $\mathbb{R}^{2\times 2}$, or
$$ \mathbb{R}^{2\times 2} = \text{Span}\left\{E_{11},E_{12},E_{21},E_{22}\right\} $$

Exercise 4

Which of the following are spanning sets for $P_3$?
(a) $\left\{\,1,\ x^2,\ x^2-2\,\right\}$

(b) $\left\{\,2,\ x^2,\ x,\ 2x+3\,\right\}$

(c) $\left\{\,x+2,\ x+1,\ x^2-1\,\right\}$

(d) $\left\{\,x+2,\ x^2-1\,\right\}$

View Solution

There is a clear relationship between the vector space $\mathbb{R}^3$ and the vector space $P_3$. In both vector spaces we can represent any vector as a list of numbers. When we see a list of numbers, or column vector
$$ \mathbf{x} = \begin{bmatrix}\ \ 1\ \\ \ \ 2\ \\ -1\ \end{bmatrix}, $$
does this vector represent a

• arrow in $\mathbb{R}^3$?
• a point in space?
• a polynomial in $P_3$
• the intensities of a triadic color scheme

From the mathematician's perspective, the linear algebra is indifferent to the physical interpretation. One needs to step back and only consider

Do these vectors span our vector space?
Since $\mathbb{R}^3$, $P_3$, Three Dimensional Space, and a triadic color scheme can all be represented as a list of three numbers or 3-tuple, we call these vector spaces isomorphic.

(a) $\left\{\,1,\ x^2,\ x^2-2\,\right\}$

Let us complete this first one using polynomials. The polynomial $q(x) = x$ is a polynomial in $P_3$; that is $q\in P_3$, but every linear combination of our set
$$ a\,1 + bx^2 + c\left(x^2 - 2\right) = (a-2c)1 + (b + c)x^2 $$
There is no $x$ term so $q\notin\text{Span}\left\{\,1,\ x^2,\ x^2-2\,\right\}$. This set does not span $P_3$.

(b) $\left\{\,2,\ x^2,\ x,\ 2x+3\,\right\}$

Let us solve this question using the list of numbers representation of vectors in $P_3$.
$$ \left\{\,2,\ x^2,\ x,\ 2x+3\,\right\}\longleftrightarrow\left\{\,\begin{bmatrix} 2 \\ 0 \\ 0 \end{bmatrix},\ \begin{bmatrix} 0 \\ 0 \\ 1 \end{bmatrix}, \begin{bmatrix} 0 \\ 1 \\ 0 \end{bmatrix},\ \begin{bmatrix} 3 \\ 2 \\ 0 \end{bmatrix}\,\right\} $$
We are asking can any vector $\mathbf{q}\in P_3$ be written as a linear combination of the vectors in our set?
$$ \begin{align*} q &= q_0 + q_1x + q_2x^2 = \begin{bmatrix} q_0 \\ q_1 \\ q_2 \end{bmatrix} \\ \\ q &= a\begin{bmatrix} 2 \\ 0 \\ 0 \end{bmatrix} + b\begin{bmatrix} 0 \\ 0 \\ 1 \end{bmatrix} + c\begin{bmatrix} 0 \\ 1 \\ 0 \end{bmatrix} + d\begin{bmatrix} 3 \\ 2 \\ 0 \end{bmatrix} \end{align*} $$
We have written our problem using the language of a linear system!
$$ \begin{bmatrix}\ 2\ &\ 0\ &\ 0\ &\ 3\ \\ \ 0\ &\ 0\ &\ 1\ &\ 2\ \\ \ 0\ &\ 1\ &\ 0\ &\ 0\ \end{bmatrix}\begin{bmatrix}\ a\ \\ \ b\ \\ \ c\ \\ \ d\ \end{bmatrix} = \mathbf{q} $$
We also have a way of determining which of these vectors are linearly independent, and which of these vectors are linearly dependent. If we have a matrix with these four vectors as its columns, then the pivot columns will be the linearly independent vectors, and the free columns will be the linearly dependent vectors.
$$ \begin{align*} \begin{bmatrix}\ 2\ &\ 0\ &\ 0\ &\ 3 & | & q_0\ \\ \ 0\ &\ 0\ &\ 1\ &\ 2 & | & q_1\ \\ \ 0\ &\ 1\ &\ 0\ &\ 0\ & | & q_2\ \end{bmatrix}\begin{array}{c} \frac{1}{2}R_1 \\ R_3 \\ R_2 \\ \end{array} &\rightarrow \begin{bmatrix}\ 1\ &\ 0\ &\ 0\ &\ \frac{3}{2} & | & \frac{1}{2}q_0\ \\ \ 0\ &\ 1\ &\ 0\ &\ 0 & | & q_2 \\ \ 0\ &\ 0\ &\ 1\ &\ 2 & | & q_1\ \end{bmatrix} \end{align*} $$
The first three columns are pivot columns, hence the system
$$ \begin{bmatrix} 2 & 0 & 0 & 3 \\ 0 & 0 & 1 & 2 \\ 0 & 1 & 0 & 0 \end{bmatrix}\begin{bmatrix} a \\ b \\ c \\ d \end{bmatrix} = \begin{bmatrix} q_0 \\ q_1 \\ q_2 \end{bmatrix} = \mathbf{q} $$
is consistent for any polynomial $\mathbf{q}$. So this set of polynomial spans $P_3$.

(c) $\left\{\,x+2,\ x+1,\ x^2-1\,\right\}$

This gives us the polynomials, or vectors
$$ \left\{\,x+2,\ x+1,\ x^2-1\,\right\}\longleftrightarrow\left\{\,\begin{bmatrix} 2 \\ 1 \\ 0 \end{bmatrix},\ \begin{bmatrix} 1 \\ 1 \\ 0 \end{bmatrix},\ \begin{bmatrix} -1 \\ 0 \\ 1 \end{bmatrix}\,\right\} $$
For any polynomial $\mathbf{q}\in P_3$ we obtain the linear system
$$ \begin{bmatrix} 2 & 1 & -1 \\ 1 & 1 & 0 \\ 0 & 0 & 1 \end{bmatrix}\mathbf{x} = \mathbf{q} $$
The matrix can be reduced
$$ \begin{bmatrix} 2 & 1 & -1 \\ 1 & 1 & 0 \\ 0 & 0 & 1 \end{bmatrix}\begin{array}{c} R_1-R_2 \\ \\ \\ \end{array} \rightarrow \begin{bmatrix} 1 & 0 & -1 \\ 1 & 1 & 0 \\ 0 & 0 & 1 \end{bmatrix}\begin{array}{c} \\ R_2-R_1 \\ \\ \end{array} \rightarrow \begin{bmatrix} 1 & 0 & -1 \\ 0 & 1 & 1 \\ 0 & 0 & 1 \end{bmatrix}\begin{array}{c} R_1+R_3 \\ R_2-R_3 \\ \\ \end{array} \rightarrow \begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{bmatrix} $$
All three columns are pivot columns. All three polynomials are linearly independent. The linear system is consistent for every $\mathbf{q}\in P_3$. This set of polynomials spans $P_3$.

(d) $\left\{\,x+2,\ x^2-1\,\right\}$

$$ \left\{\,x+2,\ x^2-1\,\right\}\longleftrightarrow \left\{\,\begin{bmatrix} 2 \\ 1 \\ 0 \end{bmatrix}, \begin{bmatrix} -1 \\ 0 \\ 1 \end{bmatrix}\,\right\} $$

For any polynomial $\mathbf{q}\in P_3$ we obtain the linear system
$$ \begin{bmatrix} 2 & -1 \\ 1 & 0 \\ 0 & 1 \end{bmatrix}\mathbf{x} = \mathbf{q} $$
The matrix can be reduced
$$ \begin{bmatrix} 2 & -1 & | & q_1 \\ 1 & 0 & | & q_2 \\ 0 & 1 & | & q_3 \end{bmatrix}\begin{array}{c} R_1-2R_2 \\ \\ \\ \end{array} \rightarrow \begin{bmatrix} 0 & -1 & | & q_1-2q_2 \\ 1 & 0 & | & q_2 \\ 0 & 1 & | & q_3 \end{bmatrix}\begin{array}{c} R_1+R_3 \\ \\ \\ \end{array} \rightarrow \begin{bmatrix} 0 & 0 & | & q_1-2q_2+q_3 \\ 1 & 0 & | & q_2 \\ 0 & 1 & | & q_3 \end{bmatrix} $$

This linear system is consistent if and only if $q_1-2q_2+q_3 = 0$. So for example the system is inconsistent if $\mathbf{q} = \begin{bmatrix} 1 \\ 0 \\ 0 \end{bmatrix}$ or $q(x) = 1$. Hence this set does not span $P_3$ because there are vectors in $P_3$ that are not in the span.

---

Your use of this self-initiated mediated course material is subject to our Creative Commons License 4.0

---