Example 1 - One Dimensional Real Vector Space¶
While we are used to drawing vectors in the plane $\mathbb{R}^2$ and in a three-dimensional grid $\mathbb{R}^3$, it is likely new to think of the real line as a vector space.
One may easily verify the set of scalars with scalar addition and multiplication satisfies all eight axioms of a vector space.
Vectors in $\mathbb{R}^1$ have their base at zero, and tip pointing to the right or left and located at the value of the scalar.
For a mathematician, the field of real scalars $\mathbb{R}$ and the one dimensional vector space $\mathbb{R}^1$ are in fact two different mathematical objects. However they are so similar to one another that often one does not distinguish between them.
The set of points on the coordinate plane $\mathcal{D} = \left\{\,(x,y)\,\big|\, x,y\in\mathbb{R}\,\right\}$ and the vector space $\mathbb{R}^2$ are distinct mathematical objects as well. The former is a set of points and the latter a set of vectors. Yet their similarity tempts us to ignore their differences when it suits us. The field of complex numbers $\mathbb{C} = \left\{\,a + bi\,\big|\, a,b\in\mathbb{R}\,\right\}$ yields a third interpretation of the coordinates on the plane.
A mathematician considers them to be different because
The set of points in the coordinate plane, the
Cartesian Plane do not have any algebraic operation associated with them such as addition and scalar multiplication.
The vector space $\mathbb{R}^2$ satisfies the axioms for a vector space but is not a field and there is no concept of multiplication or division for vectors.
The
All three share a common method of locating a particular element in their space using a list of two real numbers or a tuple. An ordered list of two numbers can also be called a 2-tuple.
Example 2 - Two Dimensional Real Vector Space¶
The set of points in the Cartesian Plane $\mathcal{D}$ utilize an
The vector space $\mathbb{R}^2$ uses an ordered list of two numbers to indicate the scalar coefficients for two basis vectors $\ihat$ and $\jhat$. In standard coordinates each vector in $\mathbb{R}^2$ is described as a linear combination of our basis vectors
$$ \begin{bmatrix}\ x\ \\ \ y\ \end{bmatrix} = x\ihat + y\jhat $$
- Every element of the field $\mathbb{C}$ represents each complex number as a sum of the real part and the imaginary part, $z = a + bi$. Here $a = \mathfrak{Re}(z)$ is the real part and $b = \mathfrak{Im}(z)$ is the imaginary part.
They share the length of the list of numbers necessary to describe every point in their space, two. Notice that each of these spaces utilize a unit length in the horizontal and vertical direction to geometrically locate each element in the space.
Example 3 - Finite Dimensional Real Vector Space¶
Finite dimensional real vector spaces are usually denoted with an ordered $n$-tuple of real elements. The common element to finite dimensional vector spaces is that they have an ordered list of $n$ special vectors, and every vector in the vector space is interpreted as a linear combination of these special vectors $\left\{\, \mathbf{b}_1,\dots,\mathbf{b}_n \,\right\}$. If $\mathbf{v}$ is a vector in $n$-dimensional real vector space $V$, then
$$\mathbf{v} = v_1\mathbf{b}_1 + \cdots + v_n\mathbf{b}_n$$
We encode each vector as an $n$-tuple of coefficients $\langle v_1,\ \dots,\ v_n \rangle$.
The real vector space $\mathbb{R}^n$ utilizes the list of coefficients for the elementary basis vectors. For each $1\le k\le n$, column vector $\mathbf{e}_k$ has coordinates $$\left(\mathbf{e}_k\right)_j = \left\{ \begin{array}{rcl} 0 & \text{ if } & j\neq k \\ 1 & \text{ if } & j=k \end{array} \right. = \left[\, \delta_{jk} \,\right].$$
Polynomial vector space $P_{n-1}$ is the vector space of real univariate polynomials with degree at most $(n-1)$. This is an $n$-dimensional vector space. Univariate polynomials are linear combinations of nonnegative integer powers of a fixed symbol called an indeterminate or variable. $$p(x) = p_0 + p_1x + \dots + p_{n-1}x^{n-1}$$
The atmosphere of the globe is divided into a three-dimensional grid. Horizontal spacing ranges from 5 km to 50 km in the $x$ and $y$ directions, and vertical spacing is measured from zero at mean sea level (the average distance from the center of the earth to sea level). This three-dimensional vector space is used for Numerical Weather Prediction.
There is a natural bijection from any finite dimensional vector space over the real numbers to $\mathbb{R}^n$. One must choose an ordering of the basis vectors of the finite dimensional real vector space $V = \mathrm{Span}\left\{ \mathbf{b}_1,\ \cdots,\ \mathbf{b}_n \right\}$ before the bijection can be defined. The natural bijection from $V$ to $\mathbb{R}^n$ is the bijection between the ordered lists of coefficients of each basis. For any vector $\mathbf{v}\in V$,
$$\mathbf{v} = \sum_{k=1}^n c_k\mathbf{b}_k \leftrightarrow \sum_{k=1}^n c_k\mathbf{e}_k = \langle c_1,\dots,c_n \rangle$$
The bijection from $P_{n-1}$ to $\mathbb{R}^n$ takes the form
$$\mathbf{p} = \sum_{j=0}^{n-1} p_jx^j = \sum_{k=1}^n p_{k-1}x^{k-1} \leftrightarrow \sum_{k=1}^n p_{k-1}\mathbf{e}_k = \langle p_0,\dots,p_{n-1} \rangle$$
This natural bijection from a real finite dimensional vector space $V$ to $\mathbb{R}^n$ allows us to represent any element of an $n$-dimensional vector space by a column vector in $\mathbb{R}^n$ that is derived from the linear combination $$\mathbf{v} = c_1\mathbf{b}_1 + \cdots + c_n\mathbf{b}_n$$
Given a set of vectors in a vector space, answering a linear algebra question often requires determining what vectors are in their span.
Example 4 - Determine the Linear Combination¶
Determine whether each of the vectors are in the span of $U = \left\{\, \begin{bmatrix}\ \ 3\ \\ \ \ 4\ \\ -3\ \\ \ \ 4\ \end{bmatrix},\ \begin{bmatrix}\ \ 1\ \\ -4\ \\ -2\ \\ \ \ 0\ \end{bmatrix},\ \begin{bmatrix}\ \ 4\ \\ \ \ 4\ \\ -3\ \\ \ \ 4\ \end{bmatrix} \,\right\}$
$\mathbf{v} = \langle 0,\ 0,\ 0,\ 1 \rangle$
$\mathbf{w} = \langle 1,\ 0,\ 0,\ 0 \rangle$
$\mathbf{z} = \langle 1,\ 0,\ 0,\ 1 \rangle$
View Solution - 1.
To determine whether $\mathbf{v}\in\mathrm{Span}(U)$, one must decide if $\mathbf{v}$ is a linear combination of the vectors in $U$. $$\mathbf{v} = c_1\mathbf{u}_1 + c_2\mathbf{u}_2 + c_3\mathbf{u}_3$$ We learned in chapter 1 to recognize the right-hand side of this equation as matrix-vector multiplication $$\begin{bmatrix}\ 0\ \\ \ 0\ \\ \ 0\ \\ \ 1\ \end{bmatrix} = \begin{bmatrix}\ \ 3\ &\ \ 1\ &\ \ 4\ \\ \ \ 4\ & -4\ &\ \ 4\ \\ -3\ & -2\ & -3\ \\ \ \ 4\ &\ \ 0\ &\ \ 4\ \end{bmatrix}\,\begin{bmatrix} c_1 \\ c_2 \\ c_3 \end{bmatrix}$$ We can solve this problem with an augmented matrix. $$\begin{align*} \left[ \begin{array}{ccc|c}\ \ 3\ &\ \ 1\ &\ \ 4\ &\ \ 0 \\ \ \ 4\ & -4\ &\ \ 4\ &\ \ 0\ \\ -3\ & -2\ & -3\ &\ \ 0\ \\ \ \ 4\ &\ \ 0\ &\ \ 4\ &\ \ 1\ \end{array} \right]\begin{array}{r} \\ R_2-R_1 \\ \\ R_4+R_3 \end{array} &\rightarrow \left[ \begin{array}{ccc|c}\ \ 3\ &\ \ 1\ &\ \ 4\ &\ \ 0 \\ \ \ 1\ & -5\ &\ \ 0\ &\ \ 0\ \\ -3\ & -2\ & -3\ &\ \ 0\ \\ \ \ 1\ & -2\ &\ \ 1\ &\ \ 1\ \end{array} \right]\begin{array}{r} R_4 \\ R_2-R_4 \\ R_3+3R_4 \\ R_1-3R_4 \end{array} \rightarrow \left[ \begin{array}{ccc|c}\ \ 1\ & -2\ &\ \ 1\ &\ \ 1\ \\ \ \ 0\ & -3\ & -1\ & -1\ \\ \ \ 0\ & -8\ &\ \ 0\ &\ \ 3\ \\ \ \ 0\ &\ \ 7\ &\ \ 1\ & -3\ \end{array} \right]\begin{array}{r} \\ \\ R_3+R_4 \\ \\ \end{array} \rightarrow \\ \\ \left[ \begin{array}{ccc|c}\ \ 1\ & -2\ &\ \ 1\ &\ \ 1\ \\ \ \ 0\ & -3\ & -1\ & -1\ \\ \ \ 0\ & -1\ &\ \ 1\ &\ \ 0\ \\ \ \ 0\ &\ \ 7\ &\ \ 1\ & -3\ \end{array} \right]\begin{array}{r} \\ -R_3 \\ R_2-3R_3 \\ R_4+7R_3 \end{array} &\rightarrow \left[ \begin{array}{ccc|c}\ \ 1\ & -2\ &\ \ 1\ &\ \ 1\ \\ \ \ 0\ &\ \ 1\ & -1\ &\ \ 0\ \\ \ \ 0\ &\ \ 0\ & -4\ & -1\ \\ \ \ 0\ &\ \ 0\ &\ \ 8\ & -3\ \end{array} \right]\begin{array}{r} \\ \\ \\ R_4+2R_3 \end{array} \rightarrow \left[ \begin{array}{ccc|c}\ \ 1\ & -2\ &\ \ 1\ &\ \ 1\ \\ \ \ 0\ &\ \ 1\ & -1\ &\ \ 0\ \\ \ \ 0\ &\ \ 0\ & -4\ & -1\ \\ \ \ 0\ &\ \ 0\ &\ \ 0\ & -5\ \end{array} \right] \end{align*}$$ The last equation of our linear system states $0c_1 + 0c_2 + 0c_3 = -5$, so the linear system is inconsistent. Thus $\mathbf{v}$ is not a linear combination of the vectors in $U$. Therefore $\mathbf{v}\notin\mathrm{Span}(U)$.Also notice that set $U$ has three vectors in $\mathbb{R}^4$. Adding a fourth vector yields a set of four vectors in $\mathbb{R}^4$. Asking,
We use the algebra we learned in the first three chapters. What the linear system represents, and how we interpret the result has changed.
View Solution - 2.
To determine whether $\mathbf{w}\in\mathrm{Span}(U)$, one must decide if $\mathbf{w}$ is a linear combination of the vectors in $U$. $$\mathbf{w} = c_1\mathbf{u}_1 + c_2\mathbf{u}_2 + c_3\mathbf{u}_3$$ We recognize the right-hand side of this equation as matrix-vector multiplication $$\begin{bmatrix}\ 1\ \\ \ 0\ \\ \ 0\ \\ \ 0\ \end{bmatrix} = \begin{bmatrix}\ \ 3\ &\ \ 1\ &\ \ 4\ \\ \ \ 4\ & -4\ &\ \ 4\ \\ -3\ & -2\ & -3\ \\ \ \ 4\ &\ \ 0\ &\ \ 4\ \end{bmatrix}\,\begin{bmatrix} c_1 \\ c_2 \\ c_3 \end{bmatrix}$$ We can solve this problem with an augmented matrix. $$\begin{align*} \left[ \begin{array}{ccc|c}\ \ 3\ &\ \ 1\ &\ \ 4\ &\ \ 1\ \\ \ \ 4\ & -4\ &\ \ 4\ &\ \ 0\ \\ -3\ & -2\ & -3\ &\ \ 0\ \\ \ \ 4\ &\ \ 0\ &\ \ 4\ &\ \ 0\ \end{array} \right]\begin{array}{r} \\ R_2-R_1 \\ \\ R_4+R_3 \end{array} &\rightarrow \left[ \begin{array}{ccc|c}\ \ 3\ &\ \ 1\ &\ \ 4\ &\ \ 1\ \\ \ \ 1\ & -5\ &\ \ 0\ & -1\ \\ -3\ & -2\ & -3\ &\ \ 0\ \\ \ \ 1\ & -2\ &\ \ 1\ &\ \ 0\ \end{array} \right]\begin{array}{r} R_4 \\ R_2-R_4 \\ R_3+3R_4 \\ R_1-3R_4 \end{array} \rightarrow \left[ \begin{array}{ccc|c}\ \ 1\ & -2\ &\ \ 1\ &\ \ 0\ \\ \ \ 0\ & -3\ & -1\ & -1\ \\ \ \ 0\ & -8\ &\ \ 0\ &\ \ 0\ \\ \ \ 0\ &\ \ 7\ &\ \ 1\ &\ \ 1\ \end{array} \right]\begin{array}{r} \\ \\ R_3+R_4 \\ \\ \end{array} \rightarrow \\ \\ \left[ \begin{array}{ccc|c}\ \ 1\ & -2\ &\ \ 1\ &\ \ 0\ \\ \ \ 0\ & -3\ & -1\ & -1\ \\ \ \ 0\ & -1\ &\ \ 1\ &\ \ 1\ \\ \ \ 0\ &\ \ 7\ &\ \ 1\ &\ \ 1\ \end{array} \right]\begin{array}{r} \\ -R_3 \\ R_2-3R_3 \\ R_4+7R_3 \end{array} &\rightarrow \left[ \begin{array}{ccc|c}\ \ 1\ & -2\ &\ \ 1\ &\ \ 0\ \\ \ \ 0\ &\ \ 1\ & -1\ & -1\ \\ \ \ 0\ &\ \ 0\ & -4\ & -4\ \\ \ \ 0\ &\ \ 0\ &\ \ 8\ &\ \ 8\ \end{array} \right]\begin{array}{r} \\ \\ \\ R_4+2R_3 \end{array} \rightarrow \left[ \begin{array}{ccc|c}\ \ 1\ & -2\ &\ \ 1\ &\ \ 0\ \\ \ \ 0\ &\ \ 1\ & -1\ & -1\ \\ \ \ 0\ &\ \ 0\ & -4\ & -4\ \\ \ \ 0\ &\ \ 0\ &\ \ 0\ &\ \ 0\ \end{array} \right] \end{align*}$$ The bottom row is $0c_1 + 0c_2 + 0c_3 = 0$, so the linear system is consistent. Back-substituting, $-4c_3 = -4$ gives $c_3 = 1$; then $c_2 - c_3 = -1$ gives $c_2 = 0$; and $c_1 - 2c_2 + c_3 = 0$ gives $c_1 = -1$. Thus $\mathbf{w} = -\mathbf{u}_1 + \mathbf{u}_3$, so $\mathbf{w}\in\mathrm{Span}(U)$.We can also ask whether $A = \left[ \mathbf{u}_1\,\mathbf{u}_2\,\mathbf{u}_3\,\mathbf{w} \right]$ is singular. Expanding along the fourth column, $$\begin{align*} \begin{vmatrix}\ \ 3\ &\ \ 1\ &\ \ 4\ &\ \ {\color{crimson}1}\ \\ \ \ 4\ & -4\ &\ \ 4\ &\ \ {\color{royalblue}0}\ \\ -3\ & -2\ & -3\ &\ \ {\color{crimson}0}\ \\ \ \ 4\ &\ \ 0\ &\ \ 4\ &\ \ {\color{royalblue}0}\ \end{vmatrix} &= {\color{crimson}-\begin{vmatrix}\ \ 4\ & -4\ &\ \ 4\ \\ -3\ & -2\ & -3\ \\ \ \ 4\ &\ \ 0\ &\ \ 4\ \end{vmatrix}}{\color{royalblue}+0}{\color{crimson}-0}{\color{royalblue}+0} = -\,(0) = 0 \end{align*}$$ The remaining $3\times 3$ determinant is zero because its first and third columns are identical. Matrix $A$ is singular, so column $4$, $\mathbf{w}$, is a linear combination of the first three columns. $\mathbf{w}\in\mathrm{Span}(U)$.
View Solution - 3.
To determine whether $\mathbf{z}\in\mathrm{Span}(U)$, one must decide if $\mathbf{z}$ is a linear combination of the vectors in $U$. $$\mathbf{z} = c_1\mathbf{u}_1 + c_2\mathbf{u}_2 + c_3\mathbf{u}_3$$ We recognize the right-hand side of this equation as matrix-vector multiplication $$\begin{bmatrix}\ 1\ \\ \ 0\ \\ \ 0\ \\ \ 1\ \end{bmatrix} = \begin{bmatrix}\ \ 3\ &\ \ 1\ &\ \ 4\ \\ \ \ 4\ & -4\ &\ \ 4\ \\ -3\ & -2\ & -3\ \\ \ \ 4\ &\ \ 0\ &\ \ 4\ \end{bmatrix}\,\begin{bmatrix} c_1 \\ c_2 \\ c_3 \end{bmatrix}$$ We can solve this problem with an augmented matrix. $$\begin{align*} \left[ \begin{array}{ccc|c}\ \ 3\ &\ \ 1\ &\ \ 4\ &\ \ 1\ \\ \ \ 4\ & -4\ &\ \ 4\ &\ \ 0\ \\ -3\ & -2\ & -3\ &\ \ 0\ \\ \ \ 4\ &\ \ 0\ &\ \ 4\ &\ \ 1\ \end{array} \right]\begin{array}{r} \\ R_2-R_1 \\ \\ R_4+R_3 \end{array} &\rightarrow \left[ \begin{array}{ccc|c}\ \ 3\ &\ \ 1\ &\ \ 4\ &\ \ 1\ \\ \ \ 1\ & -5\ &\ \ 0\ & -1\ \\ -3\ & -2\ & -3\ &\ \ 0\ \\ \ \ 1\ & -2\ &\ \ 1\ &\ \ 1\ \end{array} \right]\begin{array}{r} R_4 \\ R_2-R_4 \\ R_3+3R_4 \\ R_1-3R_4 \end{array} \rightarrow \left[ \begin{array}{ccc|c}\ \ 1\ & -2\ &\ \ 1\ &\ \ 1\ \\ \ \ 0\ & -3\ & -1\ & -2\ \\ \ \ 0\ & -8\ &\ \ 0\ &\ \ 3\ \\ \ \ 0\ &\ \ 7\ &\ \ 1\ & -2\ \end{array} \right]\begin{array}{r} \\ \\ R_3+R_4 \\ \\ \end{array} \rightarrow \\ \\ \left[ \begin{array}{ccc|c}\ \ 1\ & -2\ &\ \ 1\ &\ \ 1\ \\ \ \ 0\ & -3\ & -1\ & -2\ \\ \ \ 0\ & -1\ &\ \ 1\ &\ \ 1\ \\ \ \ 0\ &\ \ 7\ &\ \ 1\ & -2\ \end{array} \right]\begin{array}{r} \\ -R_3 \\ R_2-3R_3 \\ R_4+7R_3 \end{array} &\rightarrow \left[ \begin{array}{ccc|c}\ \ 1\ & -2\ &\ \ 1\ &\ \ 1\ \\ \ \ 0\ &\ \ 1\ & -1\ & -1\ \\ \ \ 0\ &\ \ 0\ & -4\ & -5\ \\ \ \ 0\ &\ \ 0\ &\ \ 8\ &\ \ 5\ \end{array} \right]\begin{array}{r} \\ \\ \\ R_4+2R_3 \end{array} \rightarrow \left[ \begin{array}{ccc|c}\ \ 1\ & -2\ &\ \ 1\ &\ \ 1\ \\ \ \ 0\ &\ \ 1\ & -1\ & -1\ \\ \ \ 0\ &\ \ 0\ & -4\ & -5\ \\ \ \ 0\ &\ \ 0\ &\ \ 0\ & -5\ \end{array} \right] \end{align*}$$ The bottom row is $0c_1 + 0c_2 + 0c_3 = -5$, so the linear system is inconsistent. Thus $\mathbf{z}$ is not a linear combination of the vectors in $U$, and $\mathbf{z}\notin\mathrm{Span}(U)$.We can also ask whether $A = \left[ \mathbf{u}_1\,\mathbf{u}_2\,\mathbf{u}_3\,\mathbf{z} \right]$ is singular. Expanding along the fourth column, $$\begin{align*} \begin{vmatrix}\ \ 3\ &\ \ 1\ &\ \ 4\ &\ \ {\color{crimson}1}\ \\ \ \ 4\ & -4\ &\ \ 4\ &\ \ {\color{royalblue}0}\ \\ -3\ & -2\ & -3\ &\ \ {\color{crimson}0}\ \\ \ \ 4\ &\ \ 0\ &\ \ 4\ &\ \ {\color{royalblue}1}\ \end{vmatrix} &= {\color{crimson}-\begin{vmatrix}\ \ 4\ & -4\ &\ \ 4\ \\ -3\ & -2\ & -3\ \\ \ \ 4\ &\ \ 0\ &\ \ 4\ \end{vmatrix}}{\color{royalblue}+0}{\color{crimson}-0}{\color{royalblue}+\begin{vmatrix}\ \ 3\ &\ \ 1\ &\ \ 4\ \\ \ \ 4\ & -4\ &\ \ 4\ \\ -3\ & -2\ & -3\ \end{vmatrix}} \\ \\ &= -\,(0) + (-20) = -20 \neq 0 \end{align*}$$ The first determinant vanishes because its first and third columns are equal; the second is the determinant evaluated in Solution 1, equal to $-20$. Matrix $A$ is non-singular, so column $4$, $\mathbf{z}$, is not a linear combination of the first three columns. $\mathbf{z}\notin\mathrm{Span}(U)$.
Exercise 1 - Determine the Linear Combination¶
Determine whether each of the vectors is in the span of $U = \left\{\, \begin{bmatrix}\ \ 1\ \\ \ \ 2\ \\ -1\ \end{bmatrix},\ \begin{bmatrix}\ \ 2\ \\ -1\ \\ \ \ 3\ \end{bmatrix} \,\right\}$
$\mathbf{a} = \langle 3,\ 1,\ 2 \rangle$
$\mathbf{b} = \langle 1,\ 0,\ 0 \rangle$
$\mathbf{c} = \langle 0,\ 5,\ -5 \rangle$
View Solution - 1.
We ask whether $\mathbf{a} = c_1\mathbf{u}_1 + c_2\mathbf{u}_2$ for some scalars $c_1,c_2$. $$\begin{bmatrix}\ 3\ \\ \ 1\ \\ \ 2\ \end{bmatrix} = \begin{bmatrix}\ \ 1\ &\ \ 2\ \\ \ \ 2\ & -1\ \\ -1\ &\ \ 3\ \end{bmatrix}\begin{bmatrix} c_1 \\ c_2 \end{bmatrix}$$ We solve with an augmented matrix. $$\begin{align*} \left[ \begin{array}{cc|c}\ \ 1\ &\ \ 2\ &\ \ 3\ \\ \ \ 2\ & -1\ &\ \ 1\ \\ -1\ &\ \ 3\ &\ \ 2\ \end{array} \right]\begin{array}{r} \\ R_2-2R_1 \\ R_3+R_1 \end{array} &\rightarrow \left[ \begin{array}{cc|c}\ \ 1\ &\ \ 2\ &\ \ 3\ \\ \ \ 0\ & -5\ & -5\ \\ \ \ 0\ &\ \ 5\ &\ \ 5\ \end{array} \right]\begin{array}{r} \\ \\ R_3+R_2 \end{array} \rightarrow \left[ \begin{array}{cc|c}\ \ 1\ &\ \ 2\ &\ \ 3\ \\ \ \ 0\ & -5\ & -5\ \\ \ \ 0\ &\ \ 0\ &\ \ 0\ \end{array} \right] \end{align*}$$ The system is consistent. Back-substituting, $-5c_2 = -5$ gives $c_2 = 1$, and $c_1 + 2c_2 = 3$ gives $c_1 = 1$. Thus $\mathbf{a} = \mathbf{u}_1 + \mathbf{u}_2$, so $\mathbf{a}\in\mathrm{Span}(U)$.Equivalently, we check whether $\left[ \mathbf{u}_1\,\mathbf{u}_2\,\mathbf{a} \right]$ is singular. Each operation adds a multiple of one row to another, so the determinant is unchanged; then we expand down the first column. $$\begin{align*} \begin{vmatrix}\ \ 1\ &\ \ 2\ &\ \ 3\ \\ \ \ 2\ & -1\ &\ \ 1\ \\ -1\ &\ \ 3\ &\ \ 2\ \end{vmatrix}\begin{array}{r} \\ R_2-2R_1 \\ \\ \end{array} &= \begin{vmatrix}\ \ 1\ &\ \ 2\ &\ \ 3\ \\ \ \ 0\ & -5\ & -5\ \\ -1\ &\ \ 3\ &\ \ 2\ \end{vmatrix}\begin{array}{r} \\ \\ R_3+R_1 \end{array} = \begin{vmatrix}\ \ 1\ &\ \ 2\ &\ \ 3\ \\ \ \ 0\ & -5\ & -5\ \\ \ \ 0\ &\ \ 5\ &\ \ 5\ \end{vmatrix} \\ \\ &= 1\cdot\begin{vmatrix} -5\ & -5\ \\ \ \ 5\ &\ \ 5\ \end{vmatrix} = (-5)(5) - (-5)(5) = -25 + 25 = 0 \end{align*}$$ The matrix is singular, so its third column is a linear combination of the first two — explicitly $\mathbf{a} = \mathbf{u}_1 + \mathbf{u}_2$. $\mathbf{a}\in\mathrm{Span}(U)$.
View Solution - 2.
We ask whether $\mathbf{b} = c_1\mathbf{u}_1 + c_2\mathbf{u}_2$ for some scalars $c_1,c_2$. $$\begin{bmatrix}\ 1\ \\ \ 0\ \\ \ 0\ \end{bmatrix} = \begin{bmatrix}\ \ 1\ &\ \ 2\ \\ \ \ 2\ & -1\ \\ -1\ &\ \ 3\ \end{bmatrix}\begin{bmatrix} c_1 \\ c_2 \end{bmatrix}$$ We solve with an augmented matrix. $$\begin{align*} \left[ \begin{array}{cc|c}\ \ 1\ &\ \ 2\ &\ \ 1\ \\ \ \ 2\ & -1\ &\ \ 0\ \\ -1\ &\ \ 3\ &\ \ 0\ \end{array} \right]\begin{array}{r} \\ R_2-2R_1 \\ R_3+R_1 \end{array} &\rightarrow \left[ \begin{array}{cc|c}\ \ 1\ &\ \ 2\ &\ \ 1\ \\ \ \ 0\ & -5\ & -2\ \\ \ \ 0\ &\ \ 5\ &\ \ 1\ \end{array} \right]\begin{array}{r} \\ \\ R_3+R_2 \end{array} \rightarrow \left[ \begin{array}{cc|c}\ \ 1\ &\ \ 2\ &\ \ 1\ \\ \ \ 0\ & -5\ & -2\ \\ \ \ 0\ &\ \ 0\ & -1\ \end{array} \right] \end{align*}$$ The bottom row is $0c_1 + 0c_2 = -1$, so the system is inconsistent. Thus $\mathbf{b}$ is not a linear combination of $\mathbf{u}_1,\mathbf{u}_2$, and $\mathbf{b}\notin\mathrm{Span}(U)$.Equivalently, we check whether $\left[ \mathbf{u}_1\,\mathbf{u}_2\,\mathbf{b} \right]$ is singular. Each operation adds a multiple of one row to another, so the determinant is unchanged; then we expand down the first column. $$\begin{align*} \begin{vmatrix}\ \ 1\ &\ \ 2\ &\ \ 1\ \\ \ \ 2\ & -1\ &\ \ 0\ \\ -1\ &\ \ 3\ &\ \ 0\ \end{vmatrix}\begin{array}{r} \\ R_2-2R_1 \\ \\ \end{array} &= \begin{vmatrix}\ \ 1\ &\ \ 2\ &\ \ 1\ \\ \ \ 0\ & -5\ & -2\ \\ -1\ &\ \ 3\ &\ \ 0\ \end{vmatrix}\begin{array}{r} \\ \\ R_3+R_1 \end{array} = \begin{vmatrix}\ \ 1\ &\ \ 2\ &\ \ 1\ \\ \ \ 0\ & -5\ & -2\ \\ \ \ 0\ &\ \ 5\ &\ \ 1\ \end{vmatrix} \\ \\ &= 1\cdot\begin{vmatrix} -5\ & -2\ \\ \ \ 5\ &\ \ 1\ \end{vmatrix} = (-5)(1) - (-2)(5) = -5 + 10 = 5 \neq 0 \end{align*}$$ The matrix is non-singular, so its third column is not a linear combination of the first two. $\mathbf{b}\notin\mathrm{Span}(U)$.
View Solution - 3.
We ask whether $\mathbf{c} = c_1\mathbf{u}_1 + c_2\mathbf{u}_2$ for some scalars $c_1,c_2$. $$\begin{bmatrix}\ \ 0\ \\ \ \ 5\ \\ -5\ \end{bmatrix} = \begin{bmatrix}\ \ 1\ &\ \ 2\ \\ \ \ 2\ & -1\ \\ -1\ &\ \ 3\ \end{bmatrix}\begin{bmatrix} c_1 \\ c_2 \end{bmatrix}$$ We solve with an augmented matrix. $$\begin{align*} \left[ \begin{array}{cc|c}\ \ 1\ &\ \ 2\ &\ \ 0\ \\ \ \ 2\ & -1\ &\ \ 5\ \\ -1\ &\ \ 3\ & -5\ \end{array} \right]\begin{array}{r} \\ R_2-2R_1 \\ R_3+R_1 \end{array} &\rightarrow \left[ \begin{array}{cc|c}\ \ 1\ &\ \ 2\ &\ \ 0\ \\ \ \ 0\ & -5\ &\ \ 5\ \\ \ \ 0\ &\ \ 5\ & -5\ \end{array} \right]\begin{array}{r} \\ \\ R_3+R_2 \end{array} \rightarrow \left[ \begin{array}{cc|c}\ \ 1\ &\ \ 2\ &\ \ 0\ \\ \ \ 0\ & -5\ &\ \ 5\ \\ \ \ 0\ &\ \ 0\ &\ \ 0\ \end{array} \right] \end{align*}$$ The system is consistent. Back-substituting, $-5c_2 = 5$ gives $c_2 = -1$, and $c_1 + 2c_2 = 0$ gives $c_1 = 2$. Thus $\mathbf{c} = 2\mathbf{u}_1 - \mathbf{u}_2$, so $\mathbf{c}\in\mathrm{Span}(U)$.Equivalently, we check whether $\left[ \mathbf{u}_1\,\mathbf{u}_2\,\mathbf{c} \right]$ is singular. Each operation adds a multiple of one row to another, so the determinant is unchanged; then we expand down the first column. $$\begin{align*} \begin{vmatrix}\ \ 1\ &\ \ 2\ &\ \ 0\ \\ \ \ 2\ & -1\ &\ \ 5\ \\ -1\ &\ \ 3\ & -5\ \end{vmatrix}\begin{array}{r} \\ R_2-2R_1 \\ \\ \end{array} &= \begin{vmatrix}\ \ 1\ &\ \ 2\ &\ \ 0\ \\ \ \ 0\ & -5\ &\ \ 5\ \\ -1\ &\ \ 3\ & -5\ \end{vmatrix}\begin{array}{r} \\ \\ R_3+R_1 \end{array} = \begin{vmatrix}\ \ 1\ &\ \ 2\ &\ \ 0\ \\ \ \ 0\ & -5\ &\ \ 5\ \\ \ \ 0\ &\ \ 5\ & -5\ \end{vmatrix} \\ \\ &= 1\cdot\begin{vmatrix} -5\ &\ \ 5\ \\ \ \ 5\ & -5\ \end{vmatrix} = (-5)(-5) - (5)(5) = 25 - 25 = 0 \end{align*}$$ The matrix is singular, so its third column is a linear combination of the first two — explicitly $\mathbf{c} = 2\mathbf{u}_1 - \mathbf{u}_2$. $\mathbf{c}\in\mathrm{Span}(U)$.
Video Lecture 1 - Independence, Basis, and Dimension¶
Independence, Basis, and DimensionExample 5 - Determine the Spanning Set¶
If we have a set of vectors
$$U = \left\{\,\ihat,\ \jhat,\ \ihat+2\jhat\,\right\}$$
in which one of the vectors is a linear combination of the other vectors in the set
$$\ihat+2\jhat = 1\cdot\ihat + 2\cdot\jhat,$$
the third vector adds nothing to the span of the vectors.
$$\text{span}\left\{\,\ihat,\ \jhat,\ \ihat+2\jhat\,\right\} = \text{span}\left\{\,\ihat,\ \jhat\,\right\}$$
Definition¶
Linearly Dependent
A finite set of vectors $U = \left\{\,\mathbf{u}_1,\dots,\mathbf{u}_k\,\right\}$ in vector space $V$ whose span is equal to the span of a proper subset of $U$
$$\left\{\, \mathbf{u}_{i_1},\dots,\mathbf{u}_{i_{k-1}} \,\right\} $$ is called a linearly dependent set of vectors in $V$. Here the set of indices $\left\{\,i_1,\dots,i_{k-1} \,\right\}$ is a subset of only $k-1$ of the indices $\left\{\,1,\dots,k\,\right\}$. We write $$\mathrm{Span}(U) = \mathrm{Span}\left\{\,\mathbf{u}_1,\dots,\mathbf{u}_k\,\right\} = \mathrm{Span}\left\{\, \mathbf{u}_{i_1},\dots,\mathbf{u}_{i_{k-1}} \,\right\}$$
An infinite set of vectors $U$ is called linearly dependent if it contains a finite linearly dependent subset.
Example 5 - Determine the Spanning Set - Continued¶
The set of vectors $U = \left\{\,\ihat,\ \jhat,\ \ihat+2\jhat\,\right\}$ is linearly dependent. The vector $\mathbf{u}_3=\ihat+2\jhat$ is already in the span of the other vectors.
In our example we subtract the resultant vector $\mathbf{u}_3 = \ihat+2\jhat$ from both sides and obtain
$$\begin{align*} \ihat + 2\jhat - \left(\ihat+2\jhat\right) &= \mathbf{0} \\ \\ \mathbf{u}_1 + 2\mathbf{u}_2 - \mathbf{u}_3 &= \mathbf{0} \end{align*}$$
In other words, there are scalars, $c_1=1$, $c_2=2$ and $c_3=-1$ so that
$$\begin{align*} c_1\mathbf{u}_1 + c_2\mathbf{u}_2 + c_3\mathbf{u}_3 &= \mathbf{0} \\ \\ \begin{bmatrix} \mathbf{u}_1 & \mathbf{u}_2 & \mathbf{u}_3 \end{bmatrix}\begin{bmatrix} c_1 \\ c_2 \\ c_3 \end{bmatrix} &= \mathbf{0} \\ \\ U\mathbf{c} &= \mathbf{0} \end{align*}$$
Notice that we are blurring the distinction between
$U = \left\{\,\mathbf{u}_1,\dots,\mathbf{u}_k\,\right\}$, the set of vectors in $V$
$U\in\mathbb{R}^{n\times k}$, the $n\times k$ matrix whose $k$ columns are the vectors in finite-dimensional vector space $V$.
We are looking for a nontrivial solution to the homogeneous system $U\mathbf{c}=\mathbf{0}$. In other words we want to know if the null space is nontrivial.
Conversely if we have a finite number of vectors in a finite dimensional vector space and the linear system $U\mathbf{c}=\mathbf{0}$ has a nontrivial solution, then the set of vectors is linearly dependent. In our example, there are real scalars $\mathbf{c} = \langle c_1, c_2, c_3 \rangle$ so that
$$ \begin{align*} U\mathbf{c} &= \mathbf{0} \\ \\ c_1\ihat + c_2\jhat + c_3\left(\ihat+2\jhat\right) &= \mathbf{0} \end{align*} $$
then we can subtract $c_3\left(\ihat+2\jhat\right)$ from both sides. In this case $c_3=-1$ so
$$ \begin{align*} c_1\ihat + c_2\jhat &= -c_3\left(\ihat + 2\jhat\right) \\ \\ -\dfrac{c_1}{c_3}\ihat - \dfrac{c_2}{c_3}\jhat &= \ihat + 2\jhat \\ \\ a\ihat + b\jhat &= \ihat + 2\jhat \\ \end{align*} $$
The vector $\ihat + 2\jhat$ is a linear combination of the other vectors in the set. We conclude that the set of vectors is linearly dependent.
Theorem 1¶
Linear Dependence
Let $U$ be any subset of nonzero vectors in vector space $V$.
One vector in $U$ is a linear combination of the other nonzero vectors in $U$ if and only if the set $U$ is linearly dependent.
The homogeneous linear equation formed from a finite subset of vectors in $U$ has a nontrivial solution if and only if $U$ is linearly dependent.
Proof of Theorem 1¶
Before we start the proof, we need to recall that a linear combination is a finite linear combination of vectors in a vector space. This makes our proof easier because we need only consider a finite subset of vectors whose linear combination is another vector in our vector space. However this also makes our proof trickier because now we need to come up with some way of denoting a finite subset of vectors in our vector space.
Let $U$ be a subset of vector space $V$.
$\left(1 \Rightarrow 2 \right)$ Suppose that one nonzero vector in our set $U$ is a linear combination of the other nonzero vectors in $U$. Say $$\mathbf{0} \neq \mathbf{v} = c_1\mathbf{u}_1 + \dots + c_m\mathbf{u}_m$$ where $\mathbf{v} \neq \mathbf{u}_k$ for any $1\le k\le m$. There must be at least one $c_k\neq 0$ or the linear combination yields $\mathbf{v}=\mathbf{0}$. If this is true then $$c_1\mathbf{u}_1 + \dots + c_m\mathbf{u}_m - \mathbf{v} = \mathbf{0}.$$ Hence there is a homogeneous linear combination of a finite subset of nonzero vectors in $U$ that has a nontrivial solution.
$\left(2 \Rightarrow \text{linearly dependent}\right)$ Suppose that a homogeneous linear equation formed from a finite subset of vectors in $U$ has a nontrivial solution, $$c_1\mathbf{u}_1 + \dots + c_m\mathbf{u}_m = \mathbf{0}.$$ Since the linear equation has a nontrivial solution, and the vectors are all nonzero, there are at least two nonzero coefficients. Let $c_k\neq 0$. Subtracting $c_k\mathbf{u}_k$ from both sides of the equation results in $$c_1\mathbf{u}_1 + \dots + c_{k-1}\mathbf{u}_{k-1} + c_{k+1}\mathbf{u}_{k+1} + \dots + c_m\mathbf{u}_m = -c_k\mathbf{u}_k.$$Thus, $$\mathbf{u}_k\in\mathrm{Span}\left\{\, \mathbf{u}_1,\ \dots,\ \mathbf{u}_{k-1},\ \mathbf{u}_{k+1},\ \dots,\ \mathbf{u}_m \,\right\} = \mathrm{Span}\left\{\, \mathbf{u}_1,\ \dots,\ \mathbf{u}_m \,\right\}.$$ Therefore $U$ has a finite linearly dependent subset, and $U$ is linearly dependent.
$\left( \text{linearly dependent} \Rightarrow 1 \right)$ Suppose that set $U$ is linearly dependent. Then $U$ contains a finite linearly dependent subset $\left\{\, \mathbf{u}_1,\ \dots,\ \mathbf{u}_m \,\right\}$. The vectors in this set are nonzero, and there is a vector $\mathbf{u}_k$ so that $$\mathrm{Span}\left\{\, \mathbf{u}_1,\ \dots,\ \mathbf{u}_m \,\right\} = \mathrm{Span}\left\{\, \mathbf{u}_1,\ \dots,\ \mathbf{u}_{k-1},\ \mathbf{u}_{k+1},\ \dots,\ \mathbf{u}_m \,\right\}$$ $\mathbf{u}_k\in\mathrm{Span}\left\{\, \mathbf{u}_1,\ \dots,\ \mathbf{u}_{k-1},\ \mathbf{u}_{k+1},\ \dots,\ \mathbf{u}_m \,\right\}$ implies that $\mathbf{u}_k$ is a linear combination of the other vectors, $$\mathbf{u}_k = c_1\mathbf{u}_1 + \cdots + c_{k-1}\mathbf{u}_{k-1} + c_{k+1}\mathbf{u}_{k+1} + \cdots + c_m\mathbf{u}_m.\ \blacksquare$$
Exercise 2 - Determine the Spanning Set¶
The set $U = \left\{\,\mathbf{u}_1,\ \mathbf{u}_2,\ \mathbf{u}_3,\ \mathbf{u}_4\,\right\}$ below is linearly dependent.
$$\mathbf{u}_1 = \begin{bmatrix}\ \ 1\ \\ \ \ 2\ \\ \ \ 1\ \end{bmatrix},\quad \mathbf{u}_2 = \begin{bmatrix}\ \ 2\ \\ \ \ 4\ \\ \ \ 2\ \end{bmatrix},\quad \mathbf{u}_3 = \begin{bmatrix}\ \ 1\ \\ \ \ 1\ \\ \ \ 0\ \end{bmatrix},\quad \mathbf{u}_4 = \begin{bmatrix}\ \ 2\ \\ \ \ 3\ \\ \ \ 1\ \end{bmatrix}$$
Determine which vectors must be removed so that the remaining vectors are linearly independent and span the same subspace, and state the resulting spanning set.
View Solution.
Form the matrix $U$ whose columns are the four vectors and reduce it to reduced row echelon form. The pivot columns are linearly independent; each non-pivot column is a linear combination of the pivot columns to its left, and so may be removed without changing the span. $$\begin{align*} \begin{bmatrix}\ \ 1\ &\ \ 2\ &\ \ 1\ &\ \ 2\ \\ \ \ 2\ &\ \ 4\ &\ \ 1\ &\ \ 3\ \\ \ \ 1\ &\ \ 2\ &\ \ 0\ &\ \ 1\ \end{bmatrix}\begin{array}{r} \\ R_2-2R_1 \\ R_3-R_1 \end{array} &\rightarrow \begin{bmatrix}\ \ 1\ &\ \ 2\ &\ \ 1\ &\ \ 2\ \\ \ \ 0\ &\ \ 0\ & -1\ & -1\ \\ \ \ 0\ &\ \ 0\ & -1\ & -1\ \end{bmatrix}\begin{array}{r} \\ \\ R_3-R_2 \end{array} \rightarrow \begin{bmatrix}\ \ 1\ &\ \ 2\ &\ \ 1\ &\ \ 2\ \\ \ \ 0\ &\ \ 0\ & -1\ & -1\ \\ \ \ 0\ &\ \ 0\ &\ \ 0\ &\ \ 0\ \end{bmatrix}\begin{array}{r} \\ -R_2 \\ \\ \end{array} \rightarrow \\ \\ \begin{bmatrix}\ \ 1\ &\ \ 2\ &\ \ 1\ &\ \ 2\ \\ \ \ 0\ &\ \ 0\ &\ \ 1\ &\ \ 1\ \\ \ \ 0\ &\ \ 0\ &\ \ 0\ &\ \ 0\ \end{bmatrix}\begin{array}{r} R_1-R_2 \\ \\ \\ \end{array} &\rightarrow \begin{bmatrix}\ \ 1\ &\ \ 2\ &\ \ 0\ &\ \ 1\ \\ \ \ 0\ &\ \ 0\ &\ \ 1\ &\ \ 1\ \\ \ \ 0\ &\ \ 0\ &\ \ 0\ &\ \ 0\ \end{bmatrix} \end{align*}$$ The pivots lie in columns $1$ and $3$, so $\mathbf{u}_1$ and $\mathbf{u}_3$ are linearly independent. Columns $2$ and $4$ are free; reading their entries in the reduced row echelon form gives the dependence relations $$\mathbf{u}_2 = 2\mathbf{u}_1, \qquad \mathbf{u}_4 = \mathbf{u}_1 + \mathbf{u}_3.$$ Since $\mathbf{u}_2$ and $\mathbf{u}_4$ are linear combinations of $\mathbf{u}_1$ and $\mathbf{u}_3$, they add nothing to the span. We remove $\mathbf{u}_2$ and $\mathbf{u}_4$, and the spanning set is $$\mathrm{Span}(U) = \mathrm{Span}\left\{\,\mathbf{u}_1,\ \mathbf{u}_2,\ \mathbf{u}_3,\ \mathbf{u}_4\,\right\} = \mathrm{Span}\left\{\,\mathbf{u}_1,\ \mathbf{u}_3\,\right\}.$$Theorem 2¶
Linear Dependence of Zero
Any set of vectors $U$ in vector space $V$ that contains the zero vector is linearly dependent.
Proof of Theorem 2¶
Suppose that $\mathbf{0}\in U$, and $U$ is a set of vectors in vector space $V$. Even if $U$ is the trivial subspace of $V$, we have
$$5280\,\mathbf{0} = \mathbf{0}$$
The linear system $c\mathbf{u} = \mathbf{0}$ has infinitely many solutions when $\mathbf{u}=\mathbf{0}$. If $U$ contains a nonzero vector $\mathbf{u}$, then the linear system
$$c_1\mathbf{u} + c_2\mathbf{0} = \mathbf{0}$$
has infinitely many nontrivial solutions, $c_1=0$, and $c_2\neq0$. $\blacksquare$
Definition¶
Linearly Independent
Any set of vectors $U$ in a vector space $V$ is called linearly independent if it is not linearly dependent.
Theorem 3¶
Let $U$ be a set of nonzero vectors in vector space $V$. The following are equivalent:
$U$ is a linearly independent set.
For every finite subset of vectors $\left\{\, \mathbf{u}_1, \dots, \mathbf{u}_k \,\right\}$ in $U$, $$c_1\mathbf{u}_1 + \dots + c_k\mathbf{u}_k = \mathbf{0}$$ has only the trivial solution $c_1=\dots=c_k=0$.
For every finite subset of vectors $\left\{ \mathbf{u}_1, \dots, \mathbf{u}_k \right\}$ in $U$, the null space of matrix $\left[ \mathbf{u}_1\ \dots\ \mathbf{u}_k \right]$ is trivial.
Exercise 3 - Linear Independence¶
Determine if the set of vectors in $\mathbb{R}^4$ are linearly independent.
$$ \left\{\,\begin{bmatrix}\ \ 1\ \\ -3\ \\ \ \ 2\ \\ \ \ 0\ \end{bmatrix}\,\ \begin{bmatrix}\ \ 2\ \\ \ \ 2\ \\ -4\ \\ -3\ \end{bmatrix}\,\ \begin{bmatrix}\ \ 0\ \\ -5\ \\ \ \ 2\ \\ -1\ \end{bmatrix}\,\ \begin{bmatrix} -7\ \\ \ \ 0\ \\ \ \ 4\ \\ \ \ 5\ \end{bmatrix}\,\right\} $$
Follow Along
We want to determine if the linear system $$ c_1\begin{bmatrix}\ \ 1\ \\ -3\ \\ \ \ 2\ \\ \ \ 0\ \end{bmatrix} + c_2\begin{bmatrix}\ \ 2\ \\ \ \ 2\ \\ -4\ \\ -3\ \end{bmatrix} + c_3\begin{bmatrix}\ \ 0\ \\ -5\ \\ \ \ 2\ \\ -1\ \end{bmatrix} + c_4\begin{bmatrix} -7\ \\ \ \ 0\ \\ \ \ 4\ \\ \ \ 5\ \end{bmatrix}\ = \begin{bmatrix}\ \ 1\ &\ \ 2\ &\ \ 0\ & -7 \\ -3\ &\ \ 2\ & -5\ &\ \ 0\ \\ \ \ 2\ & -4\ &\ \ 2\ &\ \ 4\ \\ \ \ 0\ & -3\ & -1\ &\ \ 5\ \end{bmatrix}\mathbf{c} = \mathbf{0} $$ has any nontrivial solutions. If the system is dependent and has nontrivial solutions, the set of vectors is linearly dependent. If the system is independent and has only the trivial solution, the set of vectors is linearly independent. This is equivalent to finding out whether the null space of the matrix is trivial. $$ \begin{align*} \begin{bmatrix}\ \ 1\ &\ \ 2\ &\ \ 0\ & -7 \\ -3\ &\ \ 2\ & -5\ &\ \ 0\ \\ \ \ 2\ & -4\ &\ \ 2\ &\ \ 4\ \\ \ \ 0\ & -3\ & -1\ &\ \ 5\ \end{bmatrix} \begin{array}{r}\ \\ R_2 + 3R_1 \\ R_3 - 2R_1 \\ \ \end{array} &\rightarrow \begin{bmatrix}\ \ 1\ &\ \ 2\ &\ \ 0\ & -7 \\ \ \ 0\ &\ \ 8\ & -5\ & -21\ \\ \ \ 0\ & -8\ &\ \ 2\ &\ 18\ \\ \ \ 0\ & -3\ & -1\ &\ \ 5\ \end{bmatrix} \begin{array}{r}\ \\ \\ R_3 + R_2 \\ \ \end{array} \rightarrow \begin{bmatrix}\ \ 1\ &\ \ 2\ &\ \ 0\ & -7 \\ \ \ 0\ &\ \ 8\ & -5\ & -21\ \\ \ \ 0\ &\ \ 0\ & -3\ & -3\ \\ \ \ 0\ & -3\ & -1\ &\ \ 5\ \end{bmatrix} \begin{array}{r} \\ R_2+2R_4 \\ \\ \\ \end{array} \rightarrow \\ \\ \begin{bmatrix}\ \ 1\ &\ \ 2\ &\ \ 0\ & -7 \\ \ \ 0\ &\ \ 2\ & -7\ & -11\ \\ \ \ 0\ &\ \ 0\ & -3\ & -3\ \\ \ \ 0\ & -3\ & -1\ &\ \ 5\ \end{bmatrix} \begin{array}{r}\ \\ \ \\ -\frac{1}{3}R_3 \\ R_4 + R_2 \end{array} &\rightarrow \begin{bmatrix}\ \ 1\ &\ \ 2\ &\ \ 0\ & -7 \\ \ \ 0\ &\ \ 2\ & -7\ & -11\ \\ \ \ 0\ &\ \ 0\ &\ \ 1\ &\ \ 1\ \\ \ \ 0\ & -1\ & -8\ & -6\ \end{bmatrix} \begin{array}{r}\ \\ R_2 + 7R_3 \\ \ \\ R_4 + 8R_3 \end{array} \rightarrow \begin{bmatrix}\ \ 1\ &\ \ 2\ &\ \ 0\ & -7 \\ \ \ 0\ &\ \ 2\ &\ \ 0\ & -4\ \\ \ \ 0\ &\ \ 0\ &\ \ 1\ &\ \ 1\ \\ \ \ 0\ & -1\ &\ \ 0\ &\ \ 2\ \end{bmatrix} \begin{array}{r}\ \\ \frac{1}{2}R_2 \\ \ \\ \\ \end{array} \rightarrow \\ \\ \begin{bmatrix}\ \ 1\ &\ \ 2\ &\ \ 0\ & -7 \\ \ \ 0\ &\ \ 1\ &\ \ 0\ & -2\ \\ \ \ 0\ &\ \ 0\ &\ \ 1\ &\ \ 1\ \\ \ \ 0\ & -1\ &\ \ 0\ &\ \ 2\ \end{bmatrix} \begin{array}{r}\ R_1-2R_2 \\ \ \\ \ \\ R_4+R_2 \end{array} &\rightarrow \begin{bmatrix}\ \ 1\ &\ \ 0\ &\ \ 0\ & -3 \\ \ \ 0\ &\ \ 1\ &\ \ 0\ & -2\ \\ \ \ 0\ &\ \ 0\ &\ \ 1\ &\ \ 1\ \\ \ \ 0\ &\ \ 0\ &\ \ 0\ &\ \ 0\ \end{bmatrix} \end{align*} $$ The matrix has a free column so the null space of matrix $U$ is nontrivial, and the set of vectors is linearly dependent. $\mathbf{u}_4 = \mathbf{u}_3 - 2\mathbf{u}_2 - 3\mathbf{u}_1$.Exercise 4 - Prove Theorem 3¶
Prove Theorem 3.
Follow Along
Recall from the definition that $U$ is linearly independent if and only if $U$ is not linearly dependent. We prove the equivalence by establishing $(1)\Leftrightarrow(2)$ and $(2)\Leftrightarrow(3)$.$\left(1 \Leftrightarrow 2\right)$ By Theorem 1, part 2, a homogeneous linear combination of a finite subset of nonzero vectors in $U$ has a nontrivial solution if and only if $U$ is linearly dependent. Negating both sides of this biconditional, every homogeneous linear combination of every finite subset of nonzero vectors in $U$ has only the trivial solution if and only if $U$ is not linearly dependent. By definition, $U$ is not linearly dependent exactly when $U$ is linearly independent. Therefore statement $(1)$ holds if and only if statement $(2)$ holds.
$\left(2 \Leftrightarrow 3\right)$ Let $\left\{\, \mathbf{u}_1, \dots, \mathbf{u}_k \,\right\}$ be any finite subset of $U$, and let $A = \left[\, \mathbf{u}_1\ \dots\ \mathbf{u}_k \,\right]$ be the matrix whose columns are these vectors. For any scalars $\mathbf{c} = \langle c_1, \dots, c_k \rangle$, matrix-vector multiplication gives
$$A\mathbf{c} = c_1\mathbf{u}_1 + \dots + c_k\mathbf{u}_k.$$
Hence $\mathbf{c}$ is a solution of $c_1\mathbf{u}_1 + \dots + c_k\mathbf{u}_k = \mathbf{0}$ if and only if $\mathbf{c}$ is in the null space of $A$. The equation has only the trivial solution $\mathbf{c} = \mathbf{0}$ if and only if the null space of $A$ contains only $\mathbf{0}$, that is, the null space of $A$ is trivial. Since the finite subset was arbitrary, statement $(2)$ holds if and only if statement $(3)$ holds.
Combining the two equivalences, $(1)\Leftrightarrow(2)\Leftrightarrow(3)$, and the three statements are equivalent. $\blacksquare$
How do we determine what functions are linearly independent or linearly dependent?
Example 6 - Determine Linear Independence¶
Consider the set $U = \left\{\, 1, \cos(x), \sin(x) \,\right\}\subseteq C^{\infty}(\mathbb{R})$. How do we determine whether $U$ is a linearly independent set? We need to determine if there are any nontrivial solutions to the equation
$$c_1\cdot1 + c_2\,\cos(x) + c_3\,\sin(x) = 0.$$
Keep in mind that we are performing vector algebra in the vector space $C^{\infty}(\mathbb{R})$. Hence the $0$ on the right-hand side is the zero function, whose graph is a horizontal line with slope zero, and $y$-intercept zero. Therefore, it is not enough to find an isolated point $x_0\in\mathbb{R}$ so that the equation
$$c_1\cdot1 + c_2\cos(x_0) + c_3\sin(x_0) = 0$$
is satisfied. The vector expression $c_1\cdot1 + c_2\cos(x) + c_3\sin(x)$ must be zero for all $x\in\mathbb{R}$.
Józef Maria Hoene-Wroński noted that since the linearity properties of the derivative preserve the coefficients, we can compute the derivative of both sides of the equation and obtain
$$c_1\cdot0 - c_2\sin(x) + c_3\cos(x) = 0$$
Differentiating again yields
$$0 - c_2\cos(x) - c_3\sin(x) = 0$$
We do not normally consider the system of equations
$$\begin{align*} c_1 + c_2\cos(x) + c_3\sin(x) &= 0 \\ -c_2\sin(x) + c_3\cos(x) &= 0 \\ -c_2\cos(x) - c_3\sin(x) &= 0 \end{align*}$$
This system of equations does not look like a linear system due to the terms with $\cos$ and $\sin$, however as vectors in $C^{\infty}(\mathbb{R})$, the left-hand side is a linear combination of vectors. This linear system of equations can be written using matrix-vector multiplication,
$$A\mathbf{c} = \begin{bmatrix}\ 1\ &\ \cos(x)\ &\ \sin(x) \\ \ 0\ & -\sin(x)\ &\ \cos(x)\ \\ \ 0\ & -\cos(x)\ & -\sin(x)\ \end{bmatrix}\,\begin{bmatrix}\ c_1\ \\ \ c_2\ \\ \ c_3\ \end{bmatrix} = \mathbf{0}$$
Definition¶
Function Matrix
A function matrix or operator matrix is an $m\times n$ array where each element of the matrix $a_{ij}(x)$ is a function rather than a scalar. One may denote a function matrix by
$$A = A(x) = \left[ a_{ij}(x) \right]$$
A function matrix has $n$ $m\times 1$ column function vectors $\mathbf{a}_j = \mathbf{a}_j(x) = \begin{bmatrix} a_{1j}(x) \\ \vdots \\ a_{mj}(x) \end{bmatrix}$, $1\le j \le n$.
A function matrix has $m$ $1\times n$ row function vectors $\mathbf{a}^i = \mathbf{a}^i(x) = \begin{bmatrix} a_{i1}(x) & \cdots & a_{in}(x) \end{bmatrix}$, $1\le i\le m$.
In order to avoid confusion we will present a definition that arises in the course vector calculus.
Definition¶
Matrix Function
A scalar function or scalar-valued function is a function whose output is a scalar.
A vector function or vector-valued function is a function whose output is a vector.
A matrix function or matrix-valued function is a function whose output is a matrix.
In all three items of each definition, the first word is an adjective, and the second a noun. So a complex function or complex-valued function is a function whose output is a complex number. But a complex matrix is a matrix whose elements are complex numbers.
In vector calculus, the trajectory of a rigid object in $3$-dimensional space is a vector-valued function $\mathbf{r}\,:\,\mathbb{R}\rightarrow\mathbb{R}^3$ defined by the function vector
$$\mathbf{r}(t) = \langle\, x(t),\ y(t),\ z(t) \,\rangle$$
Each of the components of the function vector is called a component function of vector function $\mathbf{r}$. Describing vector functions using function matrices (operators) is a powerful tool used in modern applied mathematics.
Example 6 - Continued¶
We need to determine if the Wronskian matrix $W\left[1,\cos,\sin\right](x)$, a function matrix, has a nontrivial null space.
$$W\left[1,\cos,\sin\right](x) = \begin{bmatrix}\ 1\ &\ \cos(x)\ &\ \sin(x) \\ \ 0\ & -\sin(x)\ &\ \cos(x)\ \\ \ 0\ & -\cos(x)\ & -\sin(x)\ \end{bmatrix}$$
Of course we computing the determinant and find out if the matrix is non-singular.
$$\begin{align*} \det\left(W\left[1,\cos,\sin\right]\right)(x) &= \begin{vmatrix}\ 1\ &\ \cos(x)\ &\ \sin(x) \\ \ 0\ & -\sin(x)\ &\ \cos(x)\ \\ \ 0\ & -\cos(x)\ & -\sin(x)\ \end{vmatrix} = 1\begin{vmatrix} -\sin(x)\ &\ \cos(x)\ \\ -\cos(x)\ & -\sin(x)\ \end{vmatrix}\ -\ 0\ +\ 0 \\ \\ &= \sin^2(x) + \cos^2(x) = 1 \end{align*}$$
Since the Wronskian is never equal to zero for any input $x$, the Wronskian matrix is non-singular for every input $x$. Hence the null space of the Wronskian matrix is trivial for every input $x$. One concludes that the functions $1$, $\cos(x)$, and $\sin(x)$ are linearly independent in $C^n(\mathbb{R})$ for any integer $n\ge 2$, and linearly independent in $C^{\infty}(\mathbb{R})$. The functions must be in $C^2(\mathbb{R})$ because we computed $2$ derivatives to complete the third row of the Wronskian matrix.
Definition¶
Wronskian
For a set of $n$ functions $f_1$, $\dots$, $f_n\ $ in $\ C^{(n-1)}[a,b]$, the Wronskian matrix of these functions is a function matrix
$$ W\left[f_1,\dots,f_n\right] := \begin{bmatrix} f_1(x)\ &\ \cdots\ &\ f_n(x) \\ f_1'(x)\ &\ \cdots\ & f_n'(x) \\ \vdots\ &\ \ddots\ &\ \vdots\ \\ f_1^{(n-1)}(x)\ &\ \cdots\ & f_n^{(n-1)}(x) \end{bmatrix} $$
The Wronskian function is a matrix-valued function
$$W\left[f_1,\dots,f_n\right]\,:\,[a,b]\rightarrow\mathbb{R}^{n\times n}$$
defined by the Wronskian matrix
$$W\left[f_1,\dots,f_n\right](x) = \begin{bmatrix} f_1(x)\ &\ \cdots\ &\ f_n(x) \\ f_1'(x)\ &\ \cdots\ & f_n'(x) \\ \vdots\ &\ \ddots\ &\ \vdots\ \\ f_1^{(n-1)}(x)\ &\ \cdots\ & f_n^{(n-1)}(x) \end{bmatrix}$$
The Wronskian is the determinant of the Wronskian function. This is a real-valued function
$$\det\left(\, W\left[f_1,\dots,f_n\right] \,\right)\,:\,[a,b]\rightarrow\mathbb{R}$$
defined by
$$\det\left(\, W\left[f_1,\dots,f_n\right] \,\right)(x) = \begin{vmatrix} f_1(x)\ &\ \cdots\ & f_n(x) \\ f_1'(x)\ &\ \cdots\ & f_n'(x) \\ \vdots\ &\ \ddots\ &\ \vdots\ \\ f_1^{(n-1)}(x)\ &\ \cdots\ & f_n^{(n-1)}(x) \end{vmatrix}$$
The set of functions is linearly independent if for even one input $x_0$,
$$ W\left[f_1,\dots,f_n\right](x_0) \neq 0 $$
Theorem 4¶
Wronskian Independence Test
For a set of $n$ functions $f_1$, $\dots$, $f_n$ in $C^{(n-1)}[a,b]$, if the Wronskian does not vanish identically, then the functions are linearly independent in $C^{(n-1)}[a,b]$.
Proof of Theorem 4¶
If the Wronskian is nonzero for even one value of the domain $x_0$, then the Wronskian matrix evaluated at $x_0$ is non-singular. Suppose that there exists a linear combination of the functions $f_1$, $\dots$, $f_n$ equal to zero,
$$c_1f_1(x) + \cdots + c_nf_n(x) = 0.$$
Differentiating both sides of this equation $n-1$ times yields equations for $1\le k\le n-1$,
$$c_1f_1^{(k)}(x) + \cdots + c_nf_n^{(k)}(x) = 0.$$
Therefore the Wronskian matrix satisfies the equation, $W[f_1,\dots,f_n](x)\mathbf{c} = \mathbf{0}$. In particular we have $W[f_1,\dots,f_n](x_0)\mathbf{c} = \mathbf{0}$, which implies that $\mathbf{c}=\mathbf{0}$. Hence the functions $f_1$, $\dots$, $f_n$ are linearly independent. $\blacksquare$
Unfortunately, the converse of Theorem 4 is not true. The simplest example comes from information theory, cryptograph, and error correction.
Example 7 - Cryptography¶
Let $p$ be a prime number and consider polynomials over a field of characteristic $p$. We already determined that the polynomials $x^p$ and $1$ are linearly independent in section 4.2. Working over a field of characteristic $p$, we use the formal derivative of polynomials,
$$\frac{d}{dx} x^p = px^{p-1}$$
However $px^{p-1}$ is a multiple of $p$, hence
$$\frac{d}{dx} x^p \equiv 0\ \mathrm{mod}\ p$$
Therefore,
$$\det\left(W\left[x^p, 1\right]\right)(x) = \begin{vmatrix} x^p & 1 \\ 0 & 0 \end{vmatrix} \equiv 0$$
Exercise 5 - Wronskian¶
For any positive integer $m$, determine if the functions $\left\{\sin(mx),\ \cos(mx)\right\}$ are linearly independent in the vector space $C^1(\mathbb{R})$, the vector space of continuously differentiable functions on the real line.
Follow Along
We want to determine if the linear system has a nontrivial solution for any $x\in\mathbb{R}$.$$ \begin{align*} c_1\sin(mx) + c_2\cos(mx) &= 0 \\ \\ c_1m\cos(mx) - c_2m\sin(mx) &= 0 \end{align*} $$
In matrix form we have
$$\begin{align*} \begin{vmatrix} \sin(mx) & \cos(mx) \\ m\cos(mx) & -m\sin(mx) \end{vmatrix} &= -m\sin^2(mx) - m\cos^2(mx) = -m\left(\sin^2(mx)+\cos^2(mx)\right) \\ \\ &= -m\left( 1 \right) = -m \neq 0. \end{align*}$$
The Wronskian is nonzero for every $x\in\mathbb{R}$. Hence the Wronskian matrix is nonsingular for every $x\in\mathbb{R}$ and the set of functions $\left\{\,\sin(mx),\ \cos(mx)\,\right\}$ is linearly independent in the vector space $C^1(\mathbb{R})$.
