Math 511: Linear Algebra

4.6 The Four Fundamental Subspaces

---

4.6.1 Commutative Diagrams

$$ \require{color} \definecolor{brightblue}{rgb}{.267, .298, .812} \definecolor{darkblue}{rgb}{0.0, 0.0, 1.0} \definecolor{palepink}{rgb}{1, .73, .8} \definecolor{softmagenta}{rgb}{.99,.34,.86} \definecolor{blueviolet}{rgb}{.537,.192,.937} \definecolor{jonquil}{rgb}{.949,.792,.098} \definecolor{shockingpink}{rgb}{1, 0, .741} \definecolor{royalblue}{rgb}{0, .341, .914} \definecolor{alien}{rgb}{.529,.914,.067} \definecolor{crimson}{rgb}{1, .094, .271} \def\ihat{\mathbf{\hat{\unicode{x0131}}}} \def\jhat{\mathbf{\hat{\unicode{x0237}}}} \def\khat{\mathrm{\hat{k}}} \def\tombstone{\unicode{x220E}} \def\contradiction{\unicode{x2A33}} $$

Every $m\times n$ matrix represents a linear transformation from a a finite dimensional vector space $\mathbb{R}^n$ called the domain to a finite dimensional vector space $\mathbb{R}^m$ called the codomain; $A\,:\,\mathbb{R}^n\rightarrow\mathbb{R}^m$ defined by $A(\mathbf{x})=A\mathbf{x}$ for all $\mathbf{x}\in\mathbb{R}^2$.

Example 1

$$ A\in\mathbb{R}^{2\times 5} = \begin{bmatrix}\ \ 5 & -1 \\ \ \ 5 &\ \ 5 \\ \ \ 0 &\ \ 3 \\ \ \ 3 &\ \ 5 \\ -4 &\ \ 2 \end{bmatrix} $$

This matrix represents a linear transformation from $A\,:\,\mathbb{R}^2\rightarrow\mathbb{R}^5$ defined by

$$ A(\mathbf{x})=A\mathbf{x} = \begin{bmatrix} 5x_1-x_2 \\ 5x_1+5x_2 \\ 3x_2 \\ 3x_1+5x_2 \\ 2x_2-4x_1 \end{bmatrix}\in\mathbb{R}^5 $$

for all $\mathbf{x} = \begin{bmatrix} x_1 \\ x_2 \end{bmatrix}\in\mathbb{R}^2$.

1. The domain of $A$ is $\mathbb{R}^2$

2. The codomain of $A$ is $\mathbb{R}^5$.

We can represent this relationship between matrix $A$, and its domain and codomain using a commutative diagram

Figure 1

Definition

A commutative diagram in linear algebra is a diagram of vector spaces and directed arrows that represent linear transformations so that any path along the arrows (composition of linear transformations) with the same starting vector space and ending vector space are equivalent.

Notice that in the commutative diagram we also depict the relationship between matrix $A^T$. The domain of $A^T$ is the codomain of $A$, and the codomain of $A^T$ is the domain of $A$.

Example 2

Chen, Rui & Wang, Meiling & Lai, Yi. (2020). Analysis of the role and robustness of artificial intelligence in commodity image recognition under deep learning neural network. PLOS ONE. 15. e0235783. 10.1371/journal.pone.0235783. Figure 2

In the RGB color model, a 3-dimensional vector of integer values between 0 and 255 are modeled to be intensities of red, green, and blue. This is an additive color model so the absence of light, $\mathbf{0} = \langle 0,0,0 \rangle$ is black; the origin of the color space. The sum of all three colors at maximum intensity $\langle 255, 255, 255 \rangle$ is white; the far corner in the figure.

Hence in the infinite dimensional vector space of possible hues of color, only $256^3 = 16,777,216$ colors can be represented in our computer model, and displayed by electronic devices such as televisions, computers and watches.

The value of 255 comes from the fact that in 8 bits (1 word) the binary value $11111111 = 2^8-1 = 256-1 = 255$. The 256 values are used to store each color creating a color palette that contains $\left(2^8\right)^3 = 256^3 = 16,777,216$ colors. The image in figure 2 illustrates the linear transformation from $\mathbb{R}^3$ to the continuum of visible colors our eyes can detect, a one dimensional vector space of frequencies. However we limit ourselves the the subset of values from 0 to 255 in each of the coordinates in $\mathbb{R}^3$ to be the domain of our map. Similarly the image of our linear transformation contains only those frequencies of visible light corresponding in the RGB color palette. In the default color palette we have for vector $\langle R, G, B \rangle$,

$$ \begin{align*} \text{Intensity} &\qquad &I &= \dfrac{R + G + B}{3} \\ \\ \text{Saturation} &\qquad &S &= 1 - I\,\min(R,G,B) \\ \\ \text{Hue} &\qquad &H &= \left\{\begin{array}{ccc} \cos^{-1}\left(\dfrac{(R-G) + (R-B)}{2\sqrt{(R-G)^2 + (R-B)(G-B)}}\right), & \ & \text{if } G > B \\ \\ 360 - H & \ & \text{if } B > G \end{array} \right. \end{align*} $$

4.6.2 The Right Null Space and the Left Null Space of a Matrix

Definition:

The null space of the transpose of matrix $A\in\mathbb{R}^{m\times n}$ (or $\mathbb{C}^{m\times n}$) or left null space of matrix $A$ is defined

$$ N(A^T) = \left\{\,\mathbf{y}\in\mathbb{R}^m\,:\,A^T\mathbf{y}=\mathbf{0}\,\right\}$ $$

The null space of matrix $A$, $N(A)$ can also be referred to as the right null space of matrix $A$. For any $m\times n$ matrix, the (right) null space is a subspace of $\mathbb{R}^n$ (or $\mathbb{C}^n$), and the left null space is a subspace of $\mathbb{R}^m$ (or $\mathbb{C}^m$).

Example 3

Find the right null space and the left null space of the follow matrix:

$$ A = \begin{bmatrix}\ \ 1 &\ \ 3 & -1 & -4 \\ \ \ 2 &\ \ 3 &\ \ 4 &\ \ 4 \end{bmatrix} $$

Solution:

$$ \begin{align*} A = \begin{bmatrix}\ \ 1 &\ \ 3 & -1 & -4 \\ \ \ 2 &\ \ 3 &\ \ 4 &\ \ 4 \end{bmatrix} &\rightarrow \begin{bmatrix}\ \ 1 &\ \ 3 & -1 & -4 \\ \ \ 0 & -3 &\ \ 6 &\ 12 \end{bmatrix} \\ \\ &\rightarrow \begin{bmatrix}\ \ 1 &\ \ 3 & -1 & -4 \\ \ \ 0 &\ \ 1 & -2 & -4 \end{bmatrix} \\ \\ &\rightarrow \begin{bmatrix}\ \ 1 &\ \ 0 &\ \ 5 &\ \ 8 \\ \ \ 0 &\ \ 1 & -2 & -4 \end{bmatrix} \\ \\ x_3 &= \alpha\in\mathbb{R} \\ x_4 &= \beta\in\mathbb{R} \\ \\ x_2 - 2\alpha - 4\beta &= 0 \\ x_2 &= 2\alpha + 4\beta \\ \\ x_1 + 5\alpha + 8\beta &= 0 \\ x_1 &= -5\alpha - 8\beta \\ \\ \mathbf{x} &= \begin{bmatrix} -5\alpha - 8\beta \\ 2\alpha + 4\beta \\ \alpha \\ \beta \end{bmatrix} = \alpha\begin{bmatrix} -5 \\ \ \ 2 \\ \ \ 1 \\ \ \ 0 \end{bmatrix} + \beta\begin{bmatrix} -8 \\ \ \ 4 \\ \ \ 0 \\ \ \ 1 \end{bmatrix} \\ \\ N(A) &= \text{Span}\left\{\,\begin{bmatrix} -5 \\ \ \ 2 \\ \ \ 1 \\ \ \ 0 \end{bmatrix},\ \begin{bmatrix} -8 \\ \ \ 4 \\ \ \ 0 \\ \ \ 1 \end{bmatrix}\,\right\} \\ \\ A^T = \begin{bmatrix}\ \ 1 &\ \ 2 \\ \ \ 3 &\ \ 3 \\ -1 &\ \ 4 \\ -4 &\ \ 4 \end{bmatrix} &\rightarrow \begin{bmatrix}\ \ 1 &\ \ 2 \\ \ \ 0 & -3 \\ \ \ 0 &\ \ 6 \\ \ \ 0 &\ 12 \end{bmatrix} \rightarrow \begin{bmatrix}\ \ 1 &\ \ 0 \\ \ \ 0 &\ \ 1 \\ \ \ 0 &\ \ 0 \\ \ \ 0 &\ \ 0 \end{bmatrix} \\ \\ N\left(A^T\right) &= \left\{\,\mathbf{0}\,\right\} \end{align*} $$
All of the columns of matrix $A^T$ are pivot columns, so the left null space $N(A^T)$ is trivial.

---

4.6.3 The Column Space and the Row Space

Definition:

For matrix $A\in\mathbb{R}^{m\times n}$ (or $\mathbb{C}^{m\times n}$), the row space of matrix $A$, is the column space $C(A^T)$.

One must be aware that there exist conflicts of notation concerning the row space of a matrix.

(1) Some authors denote the column space using $R(A)$ for range of matrix $A$. This seems unfortunate because Row also starts with an $R$.

(2) Some authors distinguish between the row space and the column space of $A^T$. They denote the row space $R(A)$ as a subspace of $\mathbb{R}^{1\times n}$ of row vectors that is isomorphic to $C(A^T)$ which is a subspace of $\mathbb{R}^n$.

(3) One of the purposes of mathematics is clear communication. One must use the definitions and notation in these notes to get a passing score for any written exercise in written assignments, tests or exams.

In these notes, we always denote the row space $C(A)$ of matrix $A\in\mathbb{R}^{m\times n}$ as a subspace of $\mathbb{R}^n$. Moreover we will always denote the column space of a matrix using the first letter in column, $C(A)$. This means that when one is asked for the row space of a matrix, one must provide a list $\left\{ \mathbf{v}_1, \dots, \mathbf{v}_r \right\}$ of column vectors to get any credit for the exercise on a written homework or exam.

Using Corollary 4.5.8 we can determine the row space and the column space of a matrix by reducing the matrix into upper triangular, row echelon or reduced row echelon form.

Example 4

Find the column space and the row space of the matrix

$$ A = \begin{bmatrix}\ \ 1 &\ \ 3 & -1 & -4 \\ \ \ 2 &\ \ 3 &\ \ 4 &\ \ 4 \end{bmatrix} $$

Solution

$$ \begin{align*} A = \begin{bmatrix}\ \ 1 &\ \ 3 & -1 & -4 \\ \ \ 2 &\ \ 3 &\ \ 4 &\ \ 4 \end{bmatrix} &\rightarrow \begin{bmatrix}\ \ 1 &\ \ 3 & -1 & -4 \\ \ \ 0 & -3 &\ \ 6 &\ 12 \end{bmatrix}\rightarrow \begin{bmatrix}\ \ 1 &\ \ 3 & -1 & -4 \\ \ \ 0 &\ \ 1 & -2 & -4 \end{bmatrix}\rightarrow \begin{bmatrix}\ \ 1 &\ \ 0 &\ \ 5 &\ \ 8 \\ \ \ 0 &\ \ 1 & -2 & -4 \end{bmatrix} \\ \\ C(A) &= \text{Span}\left\{\, \begin{bmatrix} 1 \\ 2 \end{bmatrix},\ \begin{bmatrix} 3 \\ 3 \end{bmatrix}\,\right\} \\ \\ C\left(A^T\right) &= \text{Span}\left\{\, \begin{bmatrix} 1 \\ 0 \\ 5 \\ 8 \end{bmatrix},\ \begin{bmatrix}\ \ 0 \\ \ \ 1 \\ -2 \\ -4 \end{bmatrix}\,\right\} = \text{Span}\left\{\,\begin{bmatrix}\ \ 1 \\ \ \ 3 \\ -1 \\ -4 \end{bmatrix},\ \begin{bmatrix} 2 \\ 3 \\ 4 \\ 4 \end{bmatrix}\,\right\} \end{align*} $$

Corollary 4.5.8 tells us that performing row operations on a matrix preserves the row space. However this is not true for the columns of the matrix.

1. Always use the original columns of the matrix as a basis for the column space when using row operations to determine pivot columns.

2. Use the most convenient rows of the matrix to determine a basis for the row space.

---

4.6.4 The Four Fundamental Subspaces of a Matrix

Definition: The Four Fundamental Subspaces of a Matrix

For any matrix $A\in\mathbb{R}^{m\times n}$ (or $\mathbb{C}^{m\times n}$), the four main subspaces are

$$ \begin{align*} 1.\ &\text{the row space } C(A^T)\subset\mathbb{R}^n \\ 2.\ &\text{the null space } N(A)\subset\mathbb{R}^n \\ 3.\ &\text{the column space } C(A)\subset\mathbb{R}^m \\ 4.\ &\text{the left null space } N(A^T)\subset\mathbb{R}^m &\end{align*} $$

One can determine bases for the column space, row space and null space by reducing the matrix into reduced row echelon form and performing backward substitution. However this must be performed again on the transpose of the matrix to determine the left null space. There are other ways to determine the left null space if we know that it is a trivial subspace.

Dr. Strang demonstrates determining The Four Fundamental Subspaces in this video.

Example 5

Determine bases for the 4 main subspaces of the following matrix,

$$ A = \begin{bmatrix}\ \ 2\ & -1\ &\ \ 0\ &\ \ 2\ &\ \ 2\ \\ \ \ 8\ & -3\ &\ \ 3\ &\ \ 7\ &\ 4\ \\ \ \ 10\ & -7\ & -4\ &\ \ 6\ &\ \ 6\ \end{bmatrix} $$

Solution:

$$ \begin{align*} A = \begin{bmatrix}\ \ 2\ & -1\ &\ \ 0\ &\ \ 2\ &\ \ 2\ \\ \ \ 8\ & -3\ &\ \ 3\ &\ \ 7\ &\ 4\ \\ \ \ 10\ & -7\ & -4\ &\ \ 6\ &\ \ 6\ \end{bmatrix} &\rightarrow\begin{bmatrix}\ \ 2\ & -1\ &\ \ 0\ &\ \ 2\ &\ \ 2\ \\ \ \ 0\ &\ \ 1\ &\ \ 3\ & -1\ & -4\ \\ \ \ 0\ & -2\ & -4\ & -4\ & -4\ \end{bmatrix} \\ \\ \rightarrow\begin{bmatrix}\ \ 2\ & -1\ &\ \ 0\ &\ \ 2\ &\ \ 2\ \\ \ \ 0\ &\ \ 1\ &\ \ 3\ & -1\ & -4\ \\ \ \ 0\ &\ \ 0\ &\ \ 2\ & -6\ & -12\ \end{bmatrix} &\rightarrow\begin{bmatrix}\ \ 2\ & -1\ &\ \ 0\ &\ \ 2\ &\ \ 2\ \\ \ \ 0\ &\ \ 1\ &\ \ 3\ & -1\ & -4\ \\ \ \ 0\ &\ \ 0\ &\ \ 1\ & -3\ & -6\ \end{bmatrix} \\ \\ \rightarrow\begin{bmatrix}\ \ 2\ & -1\ &\ \ 0\ &\ \ 2\ &\ \ 2\ \\ \ \ 0\ &\ \ 1\ &\ \ 0\ &\ \ 8\ &\ \ 14\ \\ \ \ 0\ &\ \ 0\ &\ \ 1\ & -3\ & -6\ \end{bmatrix} &\rightarrow\begin{bmatrix}\ \ 2\ &\ \ 0\ &\ \ 0\ &\ \ 10\ &\ \ 16\ \\ \ \ 0\ &\ \ 1\ &\ \ 0\ &\ \ 8\ &\ \ 14\ \\ \ \ 0\ &\ \ 0\ &\ \ 1\ & -3\ & -6\ \end{bmatrix} \\ \\ \rightarrow\begin{bmatrix}\ \ 1\ &\ \ 0\ &\ \ 0\ &\ \ 5\ &\ \ 8\ \\ \ \ 0\ &\ \ 1\ &\ \ 0\ &\ \ 8\ &\ \ 14\ \\ \ \ 0\ &\ \ 0\ &\ \ 1\ & -3\ & -6\ \end{bmatrix} \end{align*} $$

The pivot columns are the first, second and third columns. The pivot rows are the first, second and third rows. Performing backward substitution one obtains

$$ \begin{align*} x_4 &= s\in\mathbb{R} \\ x_5 &= t\in\mathbb{R} \\ x_3 &= 3s + 6t \\ x_2 &= -8s - 14t \\ x_1 &= -5s - 8t \\ \end{align*} $$

A basis for the column space has 3 vectors (pivot columns). A basis for the row space has 3 vector (pivot rows). A basis for the null space has 2 vectors (free columns). Matrix $A^T$ has only 3 pivot columns (pivot rows) and no free columns so the left null space is trivial.

$$ \begin{align*} C(A) &= \text{Span}\left\{\,\begin{bmatrix}\ 2\ \\ \ 8\ \\ \ 10\ \end{bmatrix},\ \begin{bmatrix} -1\ \\ -3\ \\ -7\ \end{bmatrix},\ \begin{bmatrix}\ \ 0\ \\ \ \ 3\ \\ -4\ \end{bmatrix}\,\right\} \\ \\ C(A^T) &= \text{Span}\left\{\,\begin{bmatrix}\ 1\ \\ \ 0\ \\ \ 0\ \\ \ 5\ \\ \ 8\ \end{bmatrix},\ \begin{bmatrix}\ 0\ \\ \ 1\ \\ \ 0\ \\ \ 8\ \\ \ 14\ \end{bmatrix},\ \begin{bmatrix}\ \ 0\ \\ \ \ 0\ \\ \ \ 1\ \\ -3\ \\ -6\ \end{bmatrix}\,\right\} = \text{Span}\left\{\,\begin{bmatrix}\ \ 2\ \\ -1\ \\ \ \ 0\ \\ \ \ 2\ \\ \ \ 2\ \end{bmatrix},\ \begin{bmatrix}\ \ 8\ \\ -3\ \\ \ \ 3\ \\ \ \ 7\ \\ \ \ 4\ \end{bmatrix},\ \begin{bmatrix}\ \ 10\ \\ -7\ \\ -4\ \\ \ \ 6\ \\ \ \ 6\ \end{bmatrix}\,\right\} \\ \\ N(A) &= \text{Span}\left\{\,\begin{bmatrix} -5\ \\ -8\ \\ \ \ 3\ \\ \ \ 1\ \\ \ \ 0\ \end{bmatrix},\ \begin{bmatrix} -8\ \\ -14\ \\ \ \ 6\ \\ \ \ 0\ \\ \ \ 1\ \end{bmatrix}\,\right\} \\ \\ N(A^T) &= \left\{\,\begin{bmatrix}\ 0\ \\ \ 0\ \\ \ 0\ \end{bmatrix}\,\right\} \end{align*} $$

---

4.6.5 The Rank and Nullity of a Matrix

Notice that in an $m\times n$ matrix, a column is a pivot column if and only if the pivot is also a pivot for a pivot row. The means that the dimension of the column space always equals the dimension of the row space.

Definition:

For any $m\times n$ matrix $A$, the dimension of the row space $C\left(A^T\right)$ is called the rank of matrix $A$.

It is curious that the rank is defined by the dimension of the row space. This is due to Corollary 4.5.7. Since we use elementary row operations to reduce a matrix, the linear independence of the rows is preserved as a result of Corollary 4.5.7. Notice that for any $m\times n$ matrix, the the number of pivots $r$ must be less than or equal to the number of rows, $r \le m$. However it is also true that the number of pivots must also be less than or equal to the number of columns, $r \le n$. Thus if $r$ is the number of pivots in our reduced matrix,

$$ r \le \min\left\{m, n \right\} $$

Theorem 4.6.1

For any $m\times n$ matrix $A$, $\dim\left(C(A^T)\right) = \dim\left(C(A)\right)$.

Proof:

Let $A$ be an $m\times n$ matrix with $r$ pivots and thus $r$ pivot columns, $\left\{\,\mathbf{a}_{i_1},\ \dots,\ \mathbf{a}_{i_r}\,\right\}$ where the columns are in the same order as in matrix $A$, $i_1\lt\dots\lt i_r$. We can create and $m\times r$ matrix

$$ C = \begin{bmatrix} \mathbf{a}_{i_1} & \dots & \mathbf{a}_{i_r} \end{bmatrix} $$

This is the matrix $C$ from the $CR$ factorization of matrix $A$. Every column of matrix $A$ is a linear combination of the columns in matrix $C$. As in the $CR$ factorization, we construct $r\times n$ matrix $R$ whose $i^{\text{th}}$ column is the linear combination that forms the $i^{\text{th}}$ column of matrix $A$.

$$ \mathbf{a}_i = C\mathbf{r}_i $$

This is just the reduced row echelon form of matrix $A$ with any rows of zeros removed to indicate that the corresponding column if $A$ is a free column and must be a linear combination of the pivot columns in $C$. Now we have the $CR$ factorization of matrix $A$

$$ A = CR $$

Notice that the columns of matrix $A$ are linear combinations of the linearly independent columns of matrix $C$ so the dimension of the column space of matrix $A$ = dimension of the column space of matrix $C$ = $r$.

Likewise the rows of matrix $A$ are linear combinations of the rows of matrix $R$. This means that the transposes of the rows of matrix $R$ spans the row space of $A$. Hence $r$ is greater than or equal to the dimension of $C(A^T)$.

We can perform the same construction with $A^T$ and get that the dimension of $C(A^T)$ is greater than or equal to the dimension of $C\left((A^T)^T\right) = C(A)$.

Therefore the dimension of the row space $C(A^T)$ equals the dimension of the column space $C(A)$. $\unicode{x220E}$

If all of the columns of a square matrix are pivot columns, we call the matrix nonsingular. If an $m\times n$ matrix of over determined ($m\gt n$), then it is possible for all of the columns to be pivot columns.

Definition:

If every column of matrix $A$ is a pivot column, we say that matrix $A$ has full rank.

In this case the rank of matrix $m\times n$ matrix $A$ is equal to the number of columns, $n$.

Since the $\dim\left(C(A)\right) = \dim\left(C(A^T)\right)$, then what does this mean for the right and left null space? If we look at the $CR$ factorization as in the proof of theorem 4.6.2, we see that the number of free columns for an $m\times n$ matrix is equal to $n-r$ since every column must be either a free column or a pivot column, but not both. Hence if $m\neq n$, then $A$ and $A^T$ must have a different number of free columns. We see that $A^T$ has $m-r$ free columns.

Definition:

The dimension of the (right) null space of an $m\times n$ matrix $A$ is called the nullity of the matrix.

---

4.6.6 The Rank-Nullity Theorem

Theorem 4.6.2 The Rank-Nullity Theorem

For any $m\times n$ matrix $A$ then the rank of $A$ plus the nullity of $A$ equals $n$, the number of columns.

$$ \text{rank}(A) + \text{nullity}(A) = n $$

You should convince yourself of the truth of this theorem using the theorems and definitions in 4.6.6.

If $A$ is an $m\times n$ matrix with rank $r$, then we can make several observations

• $\text{nullity}(A) = n - r$
• $\text{nullity}(A^T) = m-r$
• the number of pivot columns of $A = r$
• the number of pivot columns of $A^T = r$
• matrix $A$ has full rank if and only if $\text{nullity}(A)=0$
• a consistent linear system $A\mathbf{x}=\mathbf{b}$ has a unique solution if and only if $\text{nullity}(A)=0$

---

4.6.7 Vector Sum

The concept of the vector sum of two subspaces of a vector space follows directly from the overlap (intersection) and accumulation (union) of sets.

One may form a new set from two sets that contains all of the elements in both sets. This is the intersection of sets. The intersection of two sets $E$ and $F$ is the set that contains all of the elements in both sets. This new set is denoted using the $\cap$ symbol

$$ E\cap F = \left\{\,x\,:\,x\in E\ \text{and}\ x\in F\,\right\} $$

Example 6

Suppose $E = \left\{ 1, 2, 3 \right\}$ and $F=\left\{3, 4, 5\right\}$, then

$$ E\cap F = \left\{\,3\,\right\} $$

Consider the following properties of subsets

$$ \begin{align*} E\cap \left\{\ \right\} &= E\cap\emptyset = \emptyset \\ \\ E\cap E &= E \\ \\ E\cap F &= F\cap E \end{align*} $$

If $E\subset F$, then $E\cap F = E$.

If $E\cap F = \emptyset$, then we say that the two sets are disjoint.

Subspaces of a vector space are subsets of the vector space. Remember that every subspace of a vector space must contain the zero vector, so subspaces are nonempty. Two subspaces always have a nonempty intersection since the zero vector is in both subspaces. Subspaces of a vector space are never disjoint. However their bases may not have any vectors in common.

Theorem 4.6.3

The intersection of two subspaces in a vector space is a subspace of the vector space.

If $U$ and $W$ are subspaces of vector space $V$, then a basis for $U\cap W$ can be extended to a basis for $U$ and extended to a basis for $W$. Hence

$$ \dim(U\cap W) \le \min\left\{\,\dim(U), \dim(W)\,\right\} $$

Proof:

Suppose that $U$ and $W$ are subspaces of vector space $V$. Then $U\cap W\neq\emptyset$ as the zero vector is in both $U$ and $W$.

Suppose further that $\mathbf{x},\ \mathbf{y}\in U\cap W$, and $s,\ t\in\mathbb{R}$. Then the linear combination $s\mathbf{x} + t\mathbf{y}\in U$ since $U$ is a subspace of $V$. Also $s\mathbf{x} + t\mathbf{y}\in W$ as $W$ is also a subspace of $V$. Since this vector is in both subspaces,

$$ s\mathbf{x} + t\mathbf{y}\in U\cap W $$

Since $U\cap W$ is closed under linear combinations, $U\cap W$ is a subspace of vector space $V$.

If $\mathfrak{B}$ is a basis for $U\cap W$, then $\mathfrak{B}$ can be extended to a basis for $U$ since $U\cap W$ is a subspace of $U$. Thus $\dim(U\cap W)\le\dim(U)$. Likewise $\mathfrak{B}$ can be extended to a basis for $W$ since $U\cap W$ is a subspace of $W$. Thus $\dim(U\cap W)\le\dim(W)$. Hence

$$ \dim(U\cap W)\le\min\left\{\,\dim(U),\ \dim(W)\,\right\}\qquad\tombstone $$

One may also accumulate two subsets together into a larger set of the elements of either set. This is the union of the sets. This new set is denoted using the $\cup$ symbol

$$ E\cup F = \left\{\,x\,:\,x\in E,\ \text{or}\ x\in F\,\right\} $$

Example 7

Suppose $E = \left\{ 1, 2, 3 \right\}$ and $F=\left\{3, 4, 5\right\}$, then

$$ E\cup F = \left\{ 1, 2, 3, 4, 5 \right\} $$

Notice that 3 does not appear twice in $E\cup F$. Consider the following properties of subsets

$$ \begin{align*} E\cup \left\{\ \right\} &= E\cup\emptyset = E \\ \\ E\cup E &= E \\ \\ E\cup F &= F \cup E \end{align*} $$

If $E\subset F$, then $E\cup F = F$.

If $U$ and $W$ are subspaces of vector space $V$, then we can form the union of these two sets, $U\cup W$, however the union is usually not a subspace.

Example 8

Suppose that $U = \text{Span}\left\{\ihat\right\}$ and $W = \text{Span}\left\{\jhat\right\}$ in vector space $\mathbb{R}^2$.

$$ \begin{align*} U\cup W &= \left\{\,c\ihat\,:\,c\in\mathbb{R}\,\right\}\cup\left\{\,c\jhat\,:\,c\in\mathbb{R}\,\right\} \\ \\ &= \left\{\,c\ihat\ \text{or}\ c\jhat\,:\,c\in\mathbb{R}\,\right\} \end{align*} $$

This is not a vector space because it does not contain $\ihat + \jhat$. $U\cup W$ is an infintely large plus sign. However it is not closed under vector addition.

Definition

The vector sum of two subspaces $U$ and $W$ of a vector space $V$ is the span of both vector spaces.

$$ U + W := \text{Span}\left\{ U\cup W \right\} $$

A vector sum yields a new subspace with all of the vectors in both subspaces, and all of their linear combinations. Now $U$ has a basis $\left\{\,\mathbf{u}_1,\ \dots,\ \mathbf{u}_r\,\right\}$, and $W$ has a basis $\left\{\,\mathbf{w}_1,\ \dots,\ \mathbf{w}_s\,\right\}$, and all of the vectors in either subspace is a linear combination of the basis vectors. If one notes that a linear combination of linear combinations is a linear combination we have the following theorem.

Theorem 4.6.4

If $U$ and $W$ are subspaces of vector space $V$, and the subspaces have bases

$$ \begin{align*} U &= \text{Span}\left\{\,\mathbf{u}_1,\ \dots,\ \mathbf{u}_r\,\right\} \\ W &= \text{Span}\left\{\,\mathbf{w}_1,\ \dots,\ \mathbf{w}_s\,\right\} \\ \end{align*} $$

then

$$ U + W = \text{Span}\left\{\,\mathbf{u}_1,\ \dots,\ \mathbf{u}_r,\mathbf{w}_1,\ \dots,\ \mathbf{w}_w\,\right\} $$

Since subspaces $U$ and $W$ may have share basis vectors, or a basis vector for one subspace may be a vector in the other subspace,

$$ \dim(U + W) \le \dim(U) + \dim(W) $$

If the intersection of two subspaces is the trivial subspace $U\cap W = \left\{\mathbf{0}\right\}$, then clearly any bases of $U$ and $W$ will be disjoint.

Definition:

If $U$ and $W$ are subspaces of vector space $V$, and $U\cap W=\left\{\mathbf{0}\right\}$, then the vector sum of $U$ and $W$ is called a direct sum and denoted

$$ U\oplus W $$

A basis for $U\oplus W$ is the union of any two bases for $U$ and $W$, and

$$ \dim(U\oplus W) = \dim(U) + \dim(W) $$

---

4.6.8 Consequences of the Rank-Nullity Theorem

Theorem 4.6.5

For any $m\times n$ matrix $A$, $C(A^T)\cap N(A) = \left\{\mathbf{0}\right\}$.

Proof:

Let $A$ be an $m\times n$ matrix. If $\mathbf{x}\in\mathbb{R}^n$ is in both the null space $N(A)$ and the row space $C\left(A^T\right)$, then $A\mathbf{x} = \mathbf{0}$ and $\mathbf{x} = A^T\mathbf{y}$ for some $\mathbf{y}\in\mathbb{R}^m$. This gives us

$$ x_1^2 + x_2^2 + \cdots + x_n^2 = \mathbf{x}\cdot\mathbf{x} = \mathbf{x}^T\mathbf{x} = \left(\mathbf{y}^TA\right)\mathbf{x} = \mathbf{y}^T\left(A\mathbf{x}\right) = \mathbf{y}^T\mathbf{0} = 0. $$

A sum of nonnegative scalars is zero only when they are all zero, so $\mathbf{x}=\mathbf{0}$. Thus the row space and the null space have only the zero vector in common. $\unicode{x220E}$

Replacing matrix $A$ with $A^T$ one obtains

$$ C(A)\cap N\left(A^T\right) = \left\{\mathbf{0}\right\} $$

Now the row space of $m\times n$ matrix $A$ has dimension $r =$ the rank$(A)$, the null space of matrix $A$ as dimension $n-r=$ the nullity$(A)$, and they have only the zero vector in common. Hence the vector sum of the row space $C(A^T)$ and the null space $N(A)$ has dimension $n$.

Theorem 4.6.6

If $A$ is an $m\times n$ matrix, then

$$ \begin{align*} C(A)\oplus N(A^T) &= \mathbb{R}^m \\ \\ C(A^T)\oplus N(A) &= \mathbb{R}^n \end{align*} $$

The rank-nullity theorem and theorem 4.6.6 have other interesting implications for a linear system of equations.

Theorem 4.6.7

A linear system of equations $A\mathbf{x} = \mathbf{b}$ has at most one solution if and only if $N(A)=\left\{\mathbf{0}\right\}$.

$\qquad$ This means that the rank$(A) = n - \text{nullity}(A) = n - 0 = n$.

A linear system of equations $A\mathbf{x} = \mathbf{b}$ has at least one solution if and only if $\mathbf{b}\in C(A)$.

A linear system of equations $A\mathbf{x} = \mathbf{b}$ has at most one solution if and only if $A$ has full rank.

If $n\times n$ linear system of equations $A\mathbf{x} = \mathbf{b}$ is consistent for every $\mathbf{b}\in\mathbb{R}^n$, then $N\left(A^T\right) = \left\{\mathbf{0}\right\}$ and $A$ must be a nonsingular matrix.

An $n\times n$ matrix $A$ is nonsingular if and only if the column vectors of $A$ form a basis for $\mathbb{R}^n$.

There are many similar conclusions one can make from the theorems of chapter 4. A student must become familiar enough with the important definitions and theorems to be able to understand when statements about linear systems are true and false. This is especially true for modern students. The linear systems of equations of interest are very large with possibly billions of unknowns and billions of equations. Only familiarity with the properties and theorems will help one understand these linear models.

---

4.6.9 Exercises

Exercise 1

Let

$$ A = \begin{bmatrix}\ \ 1 &\ \ 2 &\ \ 8 &\ \ 0 &\ \ 4 \\ -2 &\ \ 2 &\ \ 2 & -4 &\ \ 2 \\ \ \ 2 & -3 & -5 &\ \ 2 & -1 \\ \ \ 1 & -2 & -4 & -1 &\ \ 1 \end{bmatrix}. $$
Determine the four main subspaces of matrix $A$.

View Solution

We need to reduce matrix $A$ to upper triangular form.

$$ \begin{align*} \begin{bmatrix}\ \ {\color{shockingpink} 1} &\ \ 2 &\ \ 8 &\ \ 0 &\ \ 4 \\ -2 &\ \ 2 &\ \ 2 & -4 &\ \ 2 \\ \ \ 2 & -3 & -5 &\ \ 2 & -1 \\ \ \ 1 & -2 & -4 & -1 &\ \ 1 \end{bmatrix}\begin{array}{c} \\ R_2 + 2R_1 \\ R_3 - 2R_1 \\ R_4 - R_1 \end{array} &\rightarrow \begin{bmatrix}\ \ {\color{shockingpink} 1} &\ \ 2 &\ \ 8 &\ \ 0 &\ \ 4 \\ \ \ 0 &\ \ {\color{shockingpink} 6} &\ 18 & -4 &\ 10 \\ \ \ 0 & -7 & -21 &\ \ 2 & -9 \\ \ \ 0 & -4 & -12 & -1 & -3 \end{bmatrix}\begin{array}{c} \\ R_2 + R_3 \\ \\ \\ \end{array} \\ \\ &\rightarrow\begin{bmatrix}\ \ {\color{shockingpink} 1} &\ \ 2 &\ \ 8 &\ \ 0 &\ \ 4 \\ \ \ 0 & {\color{shockingpink} -1} & -3 & -2 &\ \ 1 \\ \ \ 0 & -7 & -21 &\ \ 2 & -9 \\ \ \ 0 & -4 & -12 & -1 & -3 \end{bmatrix}\begin{array}{c} \\ \\ R_3 - 7R_2 \\ R_4 - 4R_2 \\ \end{array} \\ \\ &\rightarrow\begin{bmatrix}\ \ {\color{shockingpink} 1} &\ \ 2 &\ \ 8 &\ \ 0 &\ \ 4 \\ \ \ 0 & {\color{shockingpink} -1} & -3 & -2 &\ \ 1 \\ \ \ 0 &\ \ 0 &\ \ 0 & {\color{shockingpink} 16} & -16 \\ \ \ 0 &\ \ 0 &\ \ 0 & 7 & -7 \end{bmatrix}\begin{array}{c} \\ -R_2 \\ \frac{1}{16}R_3 \\ R_4 - 7R_3 \\ \end{array} \\ \\ &\rightarrow\begin{bmatrix}\ \ {\color{shockingpink} 1} &\ \ 2 &\ \ 8 &\ \ 0 &\ \ 4 \\ \ \ 0 &\ \ {\color{shockingpink} 1} &\ \ 3 &\ \ 2 & -1 \\ \ \ 0 &\ \ 0 &\ \ 0 &\ \ {\color{shockingpink} 1} & -1 \\ \ \ 0 &\ \ 0 &\ \ 0 &\ \ 0 &\ \ 0 \end{bmatrix} = U \\ \end{align*} $$
Matrix $U$ has $3$ pivot columns, so matrix $A$ has three pivot columns. The rank$(A) = 3$.
The column space of matrix $A$ consists of the span of the $3$ pivot columns of matrix $A$. We must use the original three pivot columns of matrix $A$ because we performed row operations and row operations destroy the column space.
$$ C(A) = \text{Span} \left\{\,\mathbf{a}_1, \mathbf{a}_2, \mathbf{a}_4\,\right\} = \text{Span}\left\{\,\begin{bmatrix}\ \ 1 \\ -2 \\ \ \ 2 \\ \ \ 1 \end{bmatrix},\ \begin{bmatrix}\ \ 2 \\ \ \ 2 \\ -3 \\ -2 \end{bmatrix},\ \begin{bmatrix}\ \ 0 \\ -4 \\ \ \ 2 \\ -1 \end{bmatrix}\,\right\} $$
The row space of matrix $A$ consists of the span of the $3$ pivot columns of $A^T$. Since row operations are linear combinations of the rows of matrix $A$, row operations preserve the row space. We may use the rows of $U$ or the rows of row equivalent matrix $A$.
$$ \begin{align*} C\left(A^T\right) &= \text{Span} \left\{\,\begin{bmatrix} 1 \\ 2 \\ 8 \\ 0 \\ 4 \end{bmatrix},\ \begin{bmatrix} -2 \\ \ \ 2 \\ \ \ 2 \\ -4 \\ \ \ 2 \end{bmatrix},\ \begin{bmatrix}\ \ 2 \\ -3 \\ -5 \\ \ \ 2 \\ -1 \end{bmatrix}\,\right\} \\ \\ &= \text{Span} \left\{\,\begin{bmatrix} 1 \\ 2 \\ 8 \\ 0 \\ 4 \end{bmatrix},\ \begin{bmatrix}\ \ 0 \\ \ \ 1 \\ \ \ 3 \\ \ \ 2 \\ -1 \end{bmatrix},\ \begin{bmatrix}\ \ 0 \\ \ \ 0 \\ \ \ 0 \\ \ \ 1 \\ -1 \end{bmatrix}\,\right\} \\ \end{align*} $$
We must complete backward substitution for $A\mathbf{x} = \mathbf{0}$ to obtain the null space. Let us start by reducing $U$ to reduced row echelon form $R$.
$$ \begin{align*} \begin{bmatrix}\ \ 1 &\ \ 2 &\ \ 8 &\ \ 0 &\ \ 4 \\ \ \ 0 &\ \ 1 &\ \ 3 &\ \ 2 & -1 \\ \ \ 0 &\ \ 0 &\ \ 0 &\ \ 1 & -1 \\ \ \ 0 &\ \ 0 &\ \ 0 &\ \ 0 &\ \ 0 \end{bmatrix}\begin{array}{c} \\ R_2 - 2R_3 \\ \\ \\ \end{array} &\rightarrow\begin{bmatrix}\ \ 1 &\ \ 2 &\ \ 8 &\ \ 0 &\ \ 4 \\ \ \ 0 &\ \ 1 &\ \ 3 &\ \ 0 &\ \ 1 \\ \ \ 0 &\ \ 0 &\ \ 0 &\ \ 1 & -1 \\ \ \ 0 &\ \ 0 &\ \ 0 &\ \ 0 &\ \ 0 \end{bmatrix}\begin{array}{c} R_1 - 2R_2 \\ \\ \\ \\ \end{array} \\ \\ &\rightarrow\begin{bmatrix}\ \ 1 &\ \ 0 &\ \ 2 &\ \ 0 &\ \ 2 \\ \ \ 0 &\ \ 1 &\ \ 3 &\ \ 0 &\ \ 1 \\ \ \ 0 &\ \ 0 &\ \ 0 &\ \ 1 & -1 \\ \ \ 0 &\ \ 0 &\ \ 0 &\ \ 0 &\ \ 0 \end{bmatrix} \end{align*} $$
Now for backward substitution
$$ \begin{align*} x_3 &= \alpha\in\mathbb{R} \\ x_5 &= \beta\in\mathbb{R} \\ \\ x_4 - \beta &= 0 \\ x_4 &= \beta \\ \\ x_2 + 3\alpha + \beta &= 0 \\ x_2 &= -3\alpha - \beta \\ \\ x_1 + 2\alpha + 2\beta &= 0 \\ x_1 &= -2\alpha -2\beta \\ \\ N(A) &= \left\{\,\begin{bmatrix} -2\alpha - 2\beta \\ -3\alpha - \beta \\ \alpha \\ \beta \\ \beta \end{bmatrix}\,:\,\alpha,\beta\in\mathbb{R}\,\right\} \\ \\ &= \left\{\,\alpha\begin{bmatrix} -2 \\ -3 \\ 1 \\ 0 \\ 0 \end{bmatrix} + \beta\begin{bmatrix} -2 \\ -1 \\ 0 \\ 1 \\ 1 \end{bmatrix}\,:\,\alpha\beta\in\mathbb{R}\,\right\} \\ \\ &= \text{Span}\left\{\,\begin{bmatrix} -2 \\ -3 \\ 1 \\ 0 \\ 0 \end{bmatrix},\ \begin{bmatrix} -2 \\ -1 \\ 0 \\ 1 \\ 1 \end{bmatrix}\,\right\} \end{align*} $$
To determine the null space of $A^T$ we must reduce $A^T$ enough to use backward substitution.
$$ \begin{align*} \begin{bmatrix}\ \ 1 & -2 &\ \ 2 &\ \ 1 \\ \ \ 2 &\ \ 2 & -3 & -2 \\ \ \ 8 &\ \ 2 & -5 & -4 \\ \ \ 0 & -4 &\ \ 2 & -1 \\ \ \ 4 &\ \ 2 & -1 &\ \ 1 \end{bmatrix}\begin{array}{c} \\ R_2-2R_1 \\ R_3-8R_1 \\ \\ R_5-4R_1 \end{array} &\rightarrow\begin{bmatrix}\ \ 1 & -2 &\ \ 2 &\ \ 1 \\ \ \ 0 &\ \ 6 & -7 & -4 \\ \ \ 0 &\ 18 & -21 & -12 \\ \ \ 0 & -4 &\ \ 2 & -1 \\ \ \ 0 &\ 10 & -9 & -3 \end{bmatrix}\begin{array}{c} \\ R_2+R_4 \\ \\ \\ \\ \end{array} \\ \\ &\rightarrow\begin{bmatrix}\ \ 1 & -2 &\ \ 2 &\ \ 1 \\ \ \ 0 &\ \ 2 & -5 & -5 \\ \ \ 0 &\ 18 & -21 & -12 \\ \ \ 0 & -4 &\ \ 2 & -1 \\ \ \ 0 &\ 10 & -9 & -3 \end{bmatrix}\begin{array}{c} \\ \\ R_3-9R_2 \\ R_4+2R_2 \\ R_5-5R_2 \end{array} \\ \\ &\rightarrow\begin{bmatrix}\ \ 1 & -2 &\ \ 2 &\ \ 1 \\ \ \ 0 &\ \ 2 & -5 & -5 \\ \ \ 0 &\ \ 0 & 24 & 33 \\ \ \ 0 &\ \ 0 & -8 & -11 \\ \ \ 0 &\ \ 0 & 16 & 22 \end{bmatrix}\begin{array}{c} \\ \\ R_3-R_5 \\ \\ \\ \end{array} \\ \\ &\rightarrow\begin{bmatrix}\ \ 1 & -2 &\ \ 2 &\ \ 1 \\ \ \ 0 &\ \ 2 & -5 & -5 \\ \ \ 0 &\ \ 0 &\ \ 8 & 11 \\ \ \ 0 &\ \ 0 & -8 & -11 \\ \ \ 0 &\ \ 0 & 16 & 22 \end{bmatrix}\begin{array}{c} \\ \\ \\ R_4+R_3 \\ R_5-2R_3 \end{array} \\ \\ &\rightarrow\begin{bmatrix}\ \ 1 & -2 &\ \ 2 &\ \ 1 \\ \ \ 0 &\ \ 2 & -5 & -5 \\ \ \ 0 &\ \ 0 &\ \ 8 & 11 \\ \ \ 0 &\ \ 0 &\ \ 0 &\ \ 0 \\ \ \ 0 &\ \ 0 &\ \ 0 &\ \ 0 \end{bmatrix}\begin{array}{c} R_1+R_2 \\ \\ \\ \\ \\ \end{array} \\ \\ &\rightarrow\begin{bmatrix}\ \ 1 &\ \ 0 & -3 & -4 \\ \ \ 0 &\ \ 2 & -5 & -5 \\ \ \ 0 &\ \ 0 &\ \ 8 & 11 \\ \ \ 0 &\ \ 0 &\ \ 0 &\ \ 0 \\ \ \ 0 &\ \ 0 &\ \ 0 &\ \ 0 \end{bmatrix}\begin{array}{c} 8R_1 \\ 8R_2 \\ \\ \\ \\ \end{array} \\ \\ &\rightarrow\begin{bmatrix}\ \ 8 &\ \ 0 & -24 & -32 \\ \ \ 0 & 16 & -40 & -40 \\ \ \ 0 &\ \ 0 &\ \ 8 & 11 \\ \ \ 0 &\ \ 0 &\ \ 0 &\ \ 0 \\ \ \ 0 &\ \ 0 &\ \ 0 &\ \ 0 \end{bmatrix}\begin{array}{c} R_1+3R_3 \\ R_2+5R_3 \\ \\ \\ \\ \end{array} \\ \\ &\rightarrow\begin{bmatrix}\ \ 8 &\ \ 0 &\ \ 0 &\ \ 1 \\ \ \ 0 & 16 &\ \ 0 & 15 \\ \ \ 0 &\ \ 0 &\ \ 8 & 11 \\ \ \ 0 &\ \ 0 &\ \ 0 &\ \ 0 \\ \ \ 0 &\ \ 0 &\ \ 0 &\ \ 0 \end{bmatrix} \\ \\ x_4 &= 16\alpha\in\mathbb{R} \\ \\ 8x_3 + 11\cdot16\alpha &= 0 \\ x_3 &= -22\alpha \\ \\ 16x_2 + 15\cdot16\alpha &= 0 \\ x_2 &= -15\alpha \\ \\ x_1 + 16\alpha &= 0 \\ x_1 &= -16\alpha \\ N\left(A^T\right) &= \left\{\,\begin{bmatrix} -16\alpha \\ -15\alpha \\ -22\alpha \\ 16\alpha \end{bmatrix}\,:\,\alpha\in\mathbb{R}\,\right\} \\ \\ &= \text{Span}\left\{\,\begin{bmatrix} -16 \\ -15 \\ -22 \\ 16 \end{bmatrix}\,\right\} \end{align*} $$

Exercise 2

Determine the rank factorization of the matrix $A$ in Exercise 1

View Solution

We have
$$ A = \begin{bmatrix}\ \ 1 &\ \ 2 &\ \ 8 &\ \ 0 &\ \ 4 \\ -2 &\ \ 2 &\ \ 2 & -4 &\ \ 2 \\ \ \ 2 & -3 & -5 &\ \ 2 & -1 \\ \ \ 1 & -2 & -4 & -1 &\ \ 1 \end{bmatrix}\qquad\qquad \text{rref}(A) = \begin{bmatrix}\ \ 1 &\ \ 0 &\ \ 2 &\ \ 0 &\ \ 2 \\ \ \ 0 &\ \ 1 &\ \ 3 &\ \ 0 &\ \ 1 \\ \ \ 0 &\ \ 0 &\ \ 0 &\ \ 1 & -1 \\ \ \ 0 &\ \ 0 &\ \ 0 &\ \ 0 &\ \ 0 \end{bmatrix} $$
So
$$ C = \begin{bmatrix} \mathbf{a}_1 & \mathbf{a}_2 & \mathbf{a}_4 \end{bmatrix} = \begin{bmatrix}\ \ 1 &\ \ 2 &\ \ 0 \\ -2 &\ \ 2 & -4 \\ \ \ 2 & -3 &\ \ 2 \\ \ \ 1 & -2 & -1 \end{bmatrix} $$
and matrix $R$ is the reduced row echelon form of $A$ with the rows of zeros removed...

$$ R = \begin{bmatrix}\ \ 1 &\ \ 0 &\ \ 2 &\ \ 0 &\ \ 2 \\ \ \ 0 &\ \ 1 &\ \ 3 &\ \ 0 &\ \ 1 \\ \ \ 0 &\ \ 0 &\ \ 0 &\ \ 1 & -1 \end{bmatrix} $$

Exercise 3

Using matrix $A$ from exercise 1, if $\mathbf{b}\in C(A)$, then how many solutions has the linear system $A\mathbf{x} = \mathbf{b}$?

View Solution

The nullity$(A) = 2 \gt 0$, so the null space is nontrivial. As $\mathbf{b}\in C(A)$, the system is consistent. Since the null space is nontrivial, there are infinitely many solutions.

Your use of this self-initiated mediated course material is subject to our Creative Commons License 4.0

---