Math 511: Linear Algebra

4.4 Linear Independence

---

4.4.1 Span, Linear Independence, and Basis

$$ \require{color} \definecolor{brightblue}{rgb}{.267, .298, .812} \definecolor{darkblue}{rgb}{0.0, 0.0, 1.0} \definecolor{palepink}{rgb}{1, .73, .8} \definecolor{softmagenta}{rgb}{.99,.34,.86} \definecolor{blueviolet}{rgb}{.537,.192,.937} \definecolor{jonquil}{rgb}{.949,.792,.098} \definecolor{shockingpink}{rgb}{1, 0, .741} \definecolor{royalblue}{rgb}{0, .341, .914} \definecolor{alien}{rgb}{.529,.914,.067} \definecolor{crimson}{rgb}{1, .094, .271} \def\ihat{\mathbf{\hat{\unicode{x0131}}}} \def\jhat{\mathbf{\hat{\unicode{x0237}}}} \def\khat{\mathrm{\hat{k}}} \def\tombstone{\unicode{x220E}} \def\contradiction{\unicode{x2A33}} $$

In the last section we discussed the span of a set of vectors. In this section we will carefully define linear independence so that we can discuss the basis of a vector space or subspace.

---

4.4.2 Finite Dimensional Euclidean Vector Spaces and Span

We previously noted examples of vector spaces, and their well-known names. The cryptic names indicate the dimension or geometric properties of each vector space. In this chapter we will generally limit ourselves to the algebraic properties of vector spaces. However we will discuss the geometry of three familiar vector spaces

$$ \mathbb{R}^1,\ \mathbb{R^2},\text{ and }\mathbb{R}^3, $$

the Euclidean real line, plane and three dimensional space we often think of as world in which we live.

One Dimensional Euclidean Space

While we are used to drawing vectors in the plane $\mathbb{R}^2$ and in a three-dimensional grid $\mathbb{R}^3$, it is likely new to think of the real line, as a vector space. If one reviews the eight rules for a vector space one clearly verifies that the set of scalars with scalar addition and multiplication satisfies all of them. In this case a vector in $\mathbb{R}^1$ has its base at zero and its tip pointing to the right or left at the value of the scalar.

For a mathematician, the field of real scalars $\mathbb{R}$ and the one dimensional vector space $\mathbb{R}^1$ are in fact two different mathematical objects. However they are so similar to one another that often does not distinguish between them.

Two Dimensional Spaces

The set of points on the coordinate plane $\mathcal{D} = \left\{\,(x,y)\,\big|\, x,y\in\mathbb{R}\,\right\}$ and the vector space $\mathbb{R}^2$ are distinct mathematical objects as well. The former is a set of points and the latter a set of vectors. Yet their similarity tempts us to ignore their differences when it suits us. The field of complex numbers $\mathbb{C} = \left\{\,a + bi\,\big|\, a,b\in\mathbb{R}\,\right\}$ yields a third interpretation of the coordinates on the plane.

A mathematician considers them to be different because

1. The set of points in the coordinate plane, the Cartesian Plane, do not have any algebraic operation associated with them such as addition and scalar multiplication.
   
   
2. The vector space $\mathbb{R}^2$ satisfies the axioms for a vector space but are not a field and there is no concept of multiplication or division for vectors.
   
   
   
3. The Complex Plane is a field) of scalars, and does include definitions for multiplication and division (for nonzero complex numbers). We found in the previous section that the complex numbers also from a real vector space.

All three share a common method of locating a particular element in their space using a list of two real numbers or a tuple. A list of two numbers can also be called a 2-tuple.

1. The set of points in the Cartesian Plane $\mathcal{D}$ utilize an ordered pair of real numbers. The first element of the list, the abscissa, indicates the distance of the point to the left or right along the horizontal axis. The second element of the list, the ordinate, indicates how far parallel to the vertical axis to travel above or below the horizontal axis. Distance is determined by marking a unit distance from the origin to $(1,0)$ on the horizontal axis and $(0,1)$ on the vertical axis. All other distances are multiples of these unit distances.
   
   
   
2. The vector space $\mathbb{R}^2$ uses an ordered list of two numbers to indicate the scalar coefficients for two basis vectors $\ihat$ and $\jhat$. In standard coordinates each vector in $\mathbb{R}^2$ is described as a linear combination of our basis vectors
   
   $$ \begin{bmatrix}\ x\ \\ \ y\ \end{bmatrix} = x\ihat + y\jhat $$
   
   
   
   
3. Every element of the field $\mathbb{C}$ represents each complex number as a sum of the real part and the complex part, $z = a + bi$. Here $a = \mathfrak{Re}(z)$ is the real part and $b = \mathfrak{Im}(z)$ is the imaginary part.

They share the length of the list of numbers necessary to describe every point in their space, two. Notice that each of these spaces utilize a unit length in the horizontal and vertical direction to geometrically locate each element in the space.

In $\mathbb{R}^2$, the unit vectors are the vectors

$$ \ihat = \begin{bmatrix}\ 1\ \\ \ 0\ \end{bmatrix}\text{ and }\ \jhat = \begin{bmatrix}\ 0\ \\ \ 1\ \end{bmatrix} $$
Every vector in the vector space $\mathbb{R}^2$ can be identified by the scalar coefficients of $\ihat$ and $\jhat$. Equivalently we say that $\mathbb{R}^2$ is the span of vectors $\ihat$ and $\jhat$ and denoted

$$ \begin{align*} \mathbb{R}^2 &= \text{span}\left\{\ihat,\,\jhat\right\} \\ \\ &= \left\{\,a\ihat + b\jhat\,\big|\,a,b\in\mathbb{R}\,\right\} \\ \\ &= \left\{\,\begin{bmatrix}\ a\ \\ \ b\ \end{bmatrix}\,\Bigg|\,a,b\in\mathbb{R}\,\right\} \end{align*} $$

The coefficients in the list are called the coordinates or components of the vector relative to the basis $\mathbf{\left\{\,\ihat,\ \jhat\,\right\}}$ This basis is referred to as the canonical basis or standard basis.


We must be careful here about our discussion of the standard basis vectors. In two dimensional space $\mathbb{R}^2$

$$ \ihat = \begin{bmatrix}\ 1\ \\ \ 0\ \end{bmatrix}\text{ and }\ \jhat = \begin{bmatrix}\ 0\ \\ \ 1\ \end{bmatrix} $$

Three Dimensional Euclidean Vector Space

In the three dimensional space $\mathbb{R}^3$ we need a list of 3 coordinates or 3-tuple.

$$ \ihat = \begin{bmatrix}\ 1\ \\ \ 0\ \\ \ 0\ \end{bmatrix},\ \ \jhat = \begin{bmatrix}\ 0\ \\ \ 1\ \\ \ 0\ \end{bmatrix},\text{ and }\ \khat = \begin{bmatrix}\ 0\ \\ \ 0\ \\ \ 1\ \end{bmatrix} $$
Notice that the definition of $\ihat$ and $\jhat$ has changed! For higher dimensional vector spaces $\mathbb{R}^n$, we switch to using subscripts instead of unique letters. We denote the canonical or standard basis vectors by

$$ \mathbf{e}_1,\ \mathbf{e}_2,\ \dots\ ,\ \mathbf{e}_n $$
So in $\mathbb{R}^2$ we have

$$ \mathbf{e}_1 = \ihat = \begin{bmatrix}\ 1\ \\ \ 0\ \end{bmatrix}\text{ and }\ \mathbf{e}_2 = \jhat = \begin{bmatrix}\ 0\ \\ \ 1\ \end{bmatrix} $$
While in $\mathbb{R}^3$ we have

$$ \mathbf{e}_1 = \ihat = \begin{bmatrix}\ 1\ \\ \ 0\ \\ \ 0\ \end{bmatrix},\ \ \mathbf{e}_2 = \jhat = \begin{bmatrix}\ 0\ \\ \ 1\ \\ \ 0\ \end{bmatrix},\text{ and }\ \mathbf{e}_3 = \khat = \begin{bmatrix}\ 0\ \\ \ 0\ \\ \ 1\ \end{bmatrix} $$
In $\mathbb{R}^4$

$$ \mathbf{e}_1 = \begin{bmatrix}\ 1\ \\ \ 0\ \\ \ 0\ \\ \ 0\ \end{bmatrix},\ \ \mathbf{e}_2 = \begin{bmatrix}\ 0\ \\ \ 1\ \\ \ 0\ \\ \ 0\ \end{bmatrix},\ \ \mathbf{e}_3 = \begin{bmatrix}\ 0\ \\ \ 0\ \\ \ 1\ \\ \ 0\ \end{bmatrix},\text{ and }\ \mathbf{e}_4 = \begin{bmatrix}\ 0\ \\ \ 0\ \\ \ 0\ \\ \ 1\ \end{bmatrix} $$
and so on...

Notice that for the Euclidean Vector Space $\mathbb{R}^n$, the value of $n$ is the number of coordinates in the list or $n$-tuple.

Example 1

What are the coordinates of vector $\mathbf{v} = \begin{bmatrix}\ 1\ \\ \ 3\ \\ \ 2\ \\ \ 0\ \end{bmatrix}$ in $\:\!\mathbb{R}^4\:\!$? How can we represent the vector as a linear combination of standard basis vectors?

The coordinates are $1$, $3$, $2$, and $0$. The vector can be written as a linear combination of the standard basis vectors using the coordinates

$$ \mathbf{v} = 1\mathbf{e}_1 + 3\mathbf{e}_2 + 2\mathbf{e}_3 + 0\mathbf{e}_4 = \mathbf{e}_1 + 3\mathbf{e}_2 + 2\mathbf{e}_3 $$

$\mathbb{R}^4 = \text{Span}\left\{\,\mathbf{e}_1,\ \mathbf{e}_2,\ \mathbf{e}_3,\ \mathbf{e}_4\,\right\}$

---

4.4.4 Finite Dimensional Vector Spaces

Now let us consider the vector space of polynomials of degree less than $n$, where $n$ is a positive integer.

$$ P_n := \left\{\,a_1x^{n-1} + a_2x^{n-2} + \cdots + a_{n-1}x + a_n\,\big|\,a_1,a_2,\cdots,a_n\in\mathbb{R}\,\right\} $$

This is a finite dimensional vector space with dimension $n$ because every polynomial of degree less than $n$ can be written as an $n$-tuple. Vector $\mathbf{p}\in P_n$, or polynomial function $p(x)\in P_n$ can be written as a linear combination

$$ \begin{align*} p(x) &= a_{n-1}x^{n-1} + a_{n-2}x^{n-2} + \cdots + a_1x + a_0\cdot 1 \\ \\ \mathbf{p} &= a_0\mathbf{e}_1 + \dots + a_{n-1}\mathbf{e}_{n} \\ \\ &= \langle a_0, \dots, a_{n-1} \rangle\qquad\text{our $n$-tuple!} \end{align*} $$

Notice that in this case, the standard basis vectors for vector space $P_n$ are polynomials and

$$ 1 \longleftrightarrow \mathbf{e}_1 = \begin{bmatrix}\ 1\ \\ \ 0\ \\ \ \vdots\ \\ \ 0\ \end{bmatrix},\ x \longleftrightarrow \mathbf{e}_2 = \begin{bmatrix}\ 0\ \\ \ 1\ \\ \ \vdots\ \\ \ 0\ \end{bmatrix},\ \dots\ ,\ x^{n-1} \longleftrightarrow \mathbf{e}_n = \begin{bmatrix}\ 0\ \\ \ 0\ \\ \ \vdots\ \\ \ 1\ \end{bmatrix}\ $$

We write

$$ \begin{align*} P_n &= \text{Span}\,\left\{\,1,\ x,\ \dots,\ x^{n-2},\ x^{n-1}\,\right\} \\ \\ &\cong \text{Span}\,\left\{ \mathbf{e}_1,\ \mathbf{e}_2,\ \dots\ ,\ \mathbf{e}_n\,\right\} \end{align*} $$

We use the symbol for isomorphic ($\cong$) because the vector $\mathbf{e}_k$ in a vector space $\mathbb{R}^n$ and the polynomial $x^{k}$ are different mathematical objects. However there is a one-to-one correspondence between them that preserves the linear algebra of each of them. Elements of both vector spaces can be uniquely determined by an $n$-tuple. The following describes a bijection $\Phi\,:\,P_n\longleftrightarrow\mathbb{R}^n$:

$$ \mathbf{p} = p(x) = a_0 + a_1x + \cdots + a_{n-1}x^{n-1} \longleftrightarrow \begin{bmatrix} a_0\ \\ \vdots\ \\ a_{n-1}\ \end{bmatrix} \in\mathbb{R}^n $$

For every polynomial $p(x)$ there is a list of $n$ scalars $a_0,\ \dots\ ,\ a_{n-1}$ or $n$-tuple that uniquely identifies vector $\mathbf{p}$.

Furthermore this bijection from $\ \Phi\,:\,P_n\longleftrightarrow\mathbb{R}^n\ $ preserves vector addition, scalar multiplication and all eight of the axioms of a vector space. Such a bijection that preserves these properties of the vectors spaces is called an isomorphism from $P_n$ to $\mathbb{R}^n$. For example, if

$$ \mathbf{p} = p(x) = 3x^2 - 5x + 4\qquad\text{ and }\qquad\mathbf{\hat{p}} = \begin{bmatrix}\ \ 3\ \\ -5\ \\ \ \ 4\ \end{bmatrix}, $$

we have

$$ \Phi\left( 3x^2 - 5x + 4 \right) = \begin{bmatrix}\ \ 3\ \\ -5\ \\ \ \ 4\ \end{bmatrix}\qquad\text{ or }\qquad\Phi\left(\mathbf{p}\right) = \Phi\left(p(x)\right) = \mathbf{\hat{p}} $$

Isomorphisms have obvious inverses. For example

$$ \Phi^{-1}\left(\ \begin{bmatrix}\ \ 3\ \\ -5\ \\ \ \ 4\ \end{bmatrix}\ \right) = 3x^2 - 5x + 4\qquad\text{ or }\qquad\Phi^{-1}\left(\mathbf{\hat{p}}\right) = \mathbf{p} = p(x) $$

One can say that the two vectors are equivalent, denoted

$$ 3x^2 - 5x + 4\ \cong\ \begin{bmatrix}\ \ 3\ \\ -5\ \\ \ \ 4\ \end{bmatrix}\qquad\text{ or }\qquad \mathbf{p}\ \cong\ \mathbf{\hat{p}} $$

Example 2

If one reads other textbooks or computer manuals, one finds that there are a couple of ways to define an isomorphism between $P_3$ and $\mathbb{R}^3$. In the programming languages MATLAB and Octave, the isomorphism is given by $\Psi\,:\,P_3\rightarrow\mathbb{R}^3$ defined from leading coefficient to constant term by

$$ \Psi\left(ax^2 + bx + c\right) = \begin{bmatrix}\ a\ \\ \ b\ \\ \ b \end{bmatrix} $$

1. Show that image of the sum of polynomials
   
   $$ p(x) = x^2 - 9x + 4\qquad\text{and}\qquad q(x) = 2x^2 + 7x + 1 $$
   is the sum of their images in $\mathbb{R}^3$.
   
   $$ \begin{align*} \Psi\left(\,p(x) + q(x)\,\right) &= \Psi\left(\,x^2 - 9x + 4 + 2x^2 + 7x + 1\,\right) \\ \\ &= \Psi\left( 3x^2 - 2x + 5\right) \\ \\ &= \begin{bmatrix}\ \ 3\ \\ -2\ \\ \ \ 5 \end{bmatrix} = \begin{bmatrix}\ \ 1 + 2\ \\ -9 + 7\ \\ \ \ 4 + 1\ \end{bmatrix} = \begin{bmatrix}\ \ 1\ \\ -9\ \\ \ \ 4\ \end{bmatrix} + \begin{bmatrix}\ \ 2\ \\ \ \ 7\ \\ \ \ 1\ \end{bmatrix} \\ \\ &= \Psi\left(x^2 - 9x + 4\right) + \Psi\left( 2x^2 + 7x + 1\right) \end{align*} $$
   
   

2. Show that the image $\Phi(3p(x)) = 3\Phi(p(x))$.
   
   $$ \begin{align*} \Psi\left( 3 \cdot (x^2 - 9x + 4) \right) &= \Psi\left( 3x^2 - 27x + 12 \right) \\ \\ &= \begin{bmatrix}\ \ 3\ \\ -27\ \\ \ \ 12\ \end{bmatrix} = 3\cdot\begin{bmatrix}\ \ 1\ \\ -9\ \\ \ \ 4\ \end{bmatrix} \\ \\ &= 3\cdot\Psi\left(x^2 - 9x + 4\right) \end{align*} $$
   

3. Show that $\Psi$ preserves vector addition, that is
   $$ \Psi\left(p(x) + q(x)\right) = \Psi(p(x)) + \Psi(q(x)) $$
   
   Let $p(x) = p_1x^2 + p_2x + p_3$ and $q(x) = q_1x^2 + q_2x + q_3$. Then
   $$ \begin{align*} \Psi\left(p(x) + q(x)\right) &= \Psi\left( p_1x^2 + p_2x + p_3 + q_1x^2 + q_2x + q_3\right) \\ \\ &= \Psi\left( (p_1+q_1)x^2 + (p_2+q_2)x + (p_3+q_3)\right) \\ \\ &= \begin{bmatrix} p_1+q_1\ \\ p_2+q_2\ \\ p_3+q_3\ \end{bmatrix} = \begin{bmatrix} p_1\ \\ p_2\ \\ p_3\ \end{bmatrix} + \begin{bmatrix} q_1\ \\ q_2\ \\ q_3\ \end{bmatrix} \\ \\ &= \Psi\left(p_1x^2 + p_2x + p_3\right) + \Psi\left(q_1x^2 + q_2x + q_3\right) \\ \\ &= \Psi\left(p(x)\right) + \Psi\left(q(x)\right) \end{align*} $$
   

4. Show that $\Psi$ preserves scalar multiplication, that is for any scalar $\alpha\in\mathbb{R}$,
   $$ \Psi(\alpha\,p(x)) = \alpha\,\Phi(p(x)) $$
   $$ \begin{align*} \Psi\left(\alpha\,p(x)\right) &= \Psi\left(\alpha(p_1x^2 + p_2x + p_3)\right) \\ \\ &= \Psi\left(\alpha p_1x^2 + \alpha p_2x + \alpha p_3)\right) \\ \\ &= \begin{bmatrix} \alpha p_1\ \\ \alpha p_2\ \\ \alpha p_3\ \end{bmatrix} = \alpha\,\begin{bmatrix} p_1\ \\ p_2\ \\ p_3\ \end{bmatrix} \\ \\ &= \alpha\,\Psi\left( p_1x^2 + p_2x + p_3 \right) = \alpha\,\Psi(p(x)) \end{align*} $$
   
   This allows us to use the list of numbers perspective of a finite dimensional vector space interchangeably with a more abstract definition of a finite dimensional vector space. It helps us understand that we can describe a finite dimensional vector space as the span of a finite number of vectors in the vector space. If vector space $V$ is a finite dimensional vector space and we know of an isomorphism from our vector space $\Phi\,:\,V\longrightarrow\mathbb{R}^n$, and
   
   $$ V \longleftrightarrow \text{Span}\,\left\{ \mathbf{e}_1, \mathbf{e}_2, \cdots, \mathbf{e}_n \right\} $$
   
   

---

4.4.5 Subspaces and Span

We also describe subspaces of a vector space as the span of some of the vectors in the vector space. For example,

• the $\text{Span}\left\{ \ihat, \jhat \right\}$ in $\mathbb{R}^3$ is the plane $z=0$.

• $\text{Span}\left\{ \ihat, \jhat, \ihat+\jhat \right\}$ is still the plane $z=0$ because $\hat{\imath} + \hat{\jmath}$ is a linear combination of the first two vectors in the list. It doesn't add anything to the span.

• $\text{Span}\left\{ \ihat, \jhat, \khat \right\}$ is all of $\mathbb{R}^3$.

We don't require any special properties of a list of vectors in order to consider their span. If we have vectors

$$ \left\{ \mathbf{u}_1, \mathbf{u}_2, \cdots, \mathbf{u}_k \right\} $$
in an $n$-dimensional vector space $V$, then

$$ \begin{align*} \text{span}\left\{\mathbf{u}_1, \mathbf{u}_2, \cdots, \mathbf{u}_k \right\} &= \left\{ c_1\mathbf{u}_1 + c_2\mathbf{u}_2 + \cdots + c_k\mathbf{u}_k\ \big|\ c_1,c_2,\dots,c_k\in\mathbb{R}\right\} \\ \\ &= \left\{ \displaystyle\sum_{j=1}^k c_j\mathbf{u}_j\ \big|\ c_1,c_2,\dots,c_k\in\mathbb{R}\right\} \\ \\ &= \left\{ \displaystyle\sum_{j=1}^k c_j\mathbf{u}_j\ \big|\ (c_1,c_2,\dots,c_k)^T\in\mathbb{R}^k\right\} \\ \\ &= \left\{\text{ all possible linear combinations of }\mathbf{u}_1, \mathbf{u}_2, \dots, \mathbf{u}_k \right\} \\ \\ &\cong\ \text{$k$-dimensional hyperplane in $\mathbb{R}^n$} \end{align*} $$

Exercise 1

Show that the span of polynomials $\left\{ x^2+1, 2x \right\}$ constitute a "plane" in the finite dimensional vector space $P_3$.

Follow Along

We must first determine if the two polynomials "point in the same direction" in vector space $P_3$. That means we need to find out if one of the polynomials is a scalar multiple of the other. There is however no scalar value $\alpha$ that we can multiply by $2x$ and get $x^2+1$ so they must "point in different directions". We could also determine that they are not co-linear using the isomorphism from $P_3$ to $\mathbb{R}^3$.
$$ \Phi\left(\,x^2 + 1\,\right) = \begin{bmatrix} 1\ \\ 0\ \\ 1\ \end{bmatrix}\qquad\text{and}\qquad\Phi\left(\,2x\,\right) = \begin{bmatrix} 0\ \\ 2\ \\ 0\ \end{bmatrix} $$
Suppose that vector $\begin{bmatrix} 0\ \\ 2\ \\ 0\ \end{bmatrix}$ by a scalar multiple of $\begin{bmatrix} 1\ \\ 0\ \\ 1\ \end{bmatrix}$. This gives us $\begin{bmatrix} 0\ \\ 2\ \\ 0\ \end{bmatrix}\alpha = \begin{bmatrix} 1\ \\ 0\ \\ 1\ \end{bmatrix}$. Or the over-determined linear system
$$ \begin{align*} 0\alpha &= 1 \\ 2\alpha &= 0 \\ 0\alpha &= 1 \end{align*} $$
Equation 2 gives us that $\alpha=0$ however $0\cdot0\neq 1$. This tells us that the two polynomials "point in different directions". This tells us that
$$ \text{span}\left\{\,\begin{bmatrix} 1\ \\ 0\ \\ 1\ \end{bmatrix},\ \begin{bmatrix} 0\ \\ 2\ \\ 0\ \end{bmatrix}\,\right\} $$
is a two dimensional subspace of $P_3$. Thus the span is a "plane" of polynomials in $P_3$.

---

4.4.6 Linear Dependence

If we have a set of vectors such as

$$ \left\{\,\ihat,\ \jhat,\ \ihat+2\jhat\,\right\} $$

in which one of the vectors is a linear combination of the other vectors in the set

$$ \ihat+2\jhat = 1\cdot\ihat + 2\cdot\jhat, $$

the third vector add nothing to the span of the vectors.

$$ \text{span}\left\{\,\ihat,\ \jhat,\ \ihat+2\jhat\,\right\} = \text{span}\left\{\,\ihat,\ \jhat\,\right\} $$

Definition

A set of vectors $U = \left\{\,\mathbf{u}_1,\dots,\mathbf{u}_k\,:\,k\in\mathbb{Z}^+\cup\left\{\infty\right\}\,\right\}$ in vector space $V$ whose span is equal to the span of a proper subset of $U$ is called a linearly dependent set of vectors in $V$.

We call this set of vectors $\left\{\mathbf{u}_1,\mathbf{u}_2,\mathbf{u}_3\right\} = \left\{\,\ihat,\ \jhat,\ \ihat+2\jhat\,\right\}$ linearly dependent. The vector $\mathbf{u}_3=\ihat+2\jhat$ is already in the span of the other vectors.

In our example we subtract the resultant vector $\mathbf{u}_3 = \ihat+2\jhat = \mathbf{u}_1 + 2\mathbf{u}_2$ from both sides to obtain

$$ \begin{align*} \ihat + 2\jhat - \left(\ihat+2\jhat\right) &= \mathbf{0} \\ \\ \mathbf{u}_1 + 2\mathbf{u}_2 - \mathbf{u}_3 &= \mathbf{0} \end{align*} $$

In other words, there are scalars, $c_1=1$, $c_2=2$ and $c_3=-1$ so that

$$ \begin{align*} c_1\mathbf{u}_1 + c_2\mathbf{u}_2 + c_3\mathbf{u}_3 &= \mathbf{0} \\ \\ \begin{bmatrix} \mathbf{u}_1 & \mathbf{u}_2 & \mathbf{u}_3 \end{bmatrix}\begin{bmatrix} c_1 \\ c_2 \\ c_3 \end{bmatrix} &= \mathbf{0} \\ \\ U\mathbf{c} &= \mathbf{0} \end{align*} $$

Notice that we are blurring the distinction between

• $U = \left\{\,\mathbf{u}_1,\dots,\mathbf{u}_k\,:\,k\in\mathbb{Z}^+\cup\left\{\infty\right\}\,\right\}$, the possibly infinite set of vectors in $V$
• $U\in\mathbb{R}^{n\times k}$, the $n\times k$ matrix whose $k$ columns are the vectors in finite-dimensional vector space $V$. This only works if we are working with a finite number of vectors in a finite dimensional vector space.
• We are looking for a nontrivial solution to the homogeneous system $U\mathbf{c}=\mathbf{0}$. In other words we want to know if the null space if nontrivial.

Conversely if we have a finite number of vectors in a finite dimensional vector space and the linear system $U\mathbf{c}=\mathbf{0}$ has a nontrivial solution, then the set of vectors is linearly dependent. In our example, there are real scalars $\mathbf{c} = \langle c_1, c_2, c_3 \rangle$ so that

$$ \begin{align*} U\mathbf{c} &= \mathbf{0} \\ \\ c_1\ihat + c_2\jhat + c_3\left(\ihat+2\jhat\right) &= \mathbf{0} \end{align*} $$

then we can subtract $c_3\left(\ihat+2\jhat\right)$ from both sides. In this case $c_3=-1$ so

$$ \begin{align*} c_1\ihat + c_2\jhat &= -c_3\left(\ihat + 2\jhat\right) \\ \\ -\dfrac{c_1}{c_3}\ihat - \dfrac{c_2}{c_3}\jhat &= \ihat + 2\jhat \\ \\ a\ihat + b\jhat &= \ihat + 2\jhat \\ \end{align*} $$

The vector $\ihat + 2\jhat$ is a linear combination of the other vectors in the set. We conclude that the set of vectors is linearly dependent.

1. We will need other methods to determine if a set of vectors is linearly dependent if we have either an infinite number of vectors, or vectors in an infinite dimensional vector space.
2. Never give a matrix answer when a list of vectors is required in a homework problem or test question!

Theorem 4.4.1

For any set of vectors $U = \left\{\,\mathbf{u}_1,\dots,\mathbf{u}_k\,:\,k\in\mathbb{Z}^+\cup\left\{\infty\right\}\,\right\}$ in vector space $V$, at least one vector is a linear combination of the others in the set if and only if the set is linearly dependent.

Proof:

Before we start the proof, we need to recall that a linear combination is a finite linear combination of vectors in a vector space. We have no notion yet of what $\displaystyle\sum_{j=1}^{\infty} \mathbf{v}_j$ even means. This makes our proof easier because we need only consider a finite subset of vectors whose linear combination is another vector in our vector space. However also makes our proof harder because now we need to come up with some way of denoting a finite subset of vectors in our vector space.

$\left(\Rightarrow\right)$ Suppose that one vector in our set $U$ is a linear combination of the other vectors in $U$. Say

$$ \mathbf{u}_j = c_1\mathbf{u}_{j_1} + \dots + c_l\mathbf{u}_{j_m}\qquad j\neq j_i \text{ for any {i} from 1 to $m$} $$

We are saying that one of the vectors $\mathbf{u}_j$ where $j$ is one of the indexes from 1 to $k$, is equal to a linear combination of $m$ other vectors in $U$ with indexes $j_1$, $j_2$, $\dots$, $j_m$. We must have some way to write down what we mean in vector algebra notation.

If this is true then $\mathbf{u}_j\in\text{Span}\left\{ \mathbf{u}_{j_1}, \dots, \mathbf{u}_{j_m} \right\}$. We required $\mathbf{u}_j$ cannot be any of the $m$ vectors in our linear combination, hence all of these $m$ vectors are still in $ U-\left\{\mathbf{u}_j\right\}$.

Thus $\text{Span}\left\{ \mathbf{u}_{j_1}, \dots, \mathbf{u}_{j_m} \right\}\subset\text{Span}(U-\left\{\mathbf{u}_j\right\})$ and $\mathbf{u}_j\in\text{Span}(U-\left\{\mathbf{u}_j\right\})$.

Since the span of $U-\left\{\mathbf{u}_j\right\}$ includes the vector $\mathbf{u}_j$,

$$ \text{Span}(U-\left\{\mathbf{u}_j\right\}) = \text{Span}(U), $$

and $U-\left\{\mathbf{u}_j\right\}$ is a proper subset of $U$. Therefore set $U$ is a linearly dependent set of vectors.

$\left(\Leftarrow\right)$ Suppose that $U$ is a linearly dependent set of vectors and there is a proper subset of $U$ with the same span. There must be at least one vector $\mathbf{u}_j$ that adds nothing to the span

$$ \text{Span}\left(U-\left\{\mathbf{u}_j\right\}\right) = \text{Span}(U) $$

Since $\mathbf{u}_j\in\text{Span}(U)$, we have that $\mathbf{u}_j\in\text{Span}\left(U-\left\{\mathbf{u}_j\right\}\right)$, and $\mathbf{u}_j$ must be a linear combination of other vectors in $U$

$$ \mathbf{u}_j = c_1\mathbf{u}_{j_1} + \dots + c_l\mathbf{u}_{j_m}\qquad j\neq j_i \text{ for any {i} from 1 to $m$} \qquad\tombstone $$

---

4.4.7 Exercise 2

Determine if the set of vectors in $\mathbb{R}^4$ are linear dependent.


$$ \left\{\,\begin{bmatrix}\ \ 1\ \\ -3\ \\ \ \ 2\ \\ \ \ 0\ \end{bmatrix}\,\ \begin{bmatrix}\ \ 2\ \\ \ \ 2\ \\ -4\ \\ -3\ \end{bmatrix}\,\ \begin{bmatrix}\ \ 0\ \\ -5\ \\ \ \ 2\ \\ -1\ \end{bmatrix}\ \right\} $$

Follow Along

We want to determine of the linear system

$$ c_1\begin{bmatrix}\ \ 1\ \\ -3\ \\ \ \ 2\ \\ \ \ 0\ \end{bmatrix} + c_2\begin{bmatrix}\ \ 2\ \\ \ \ 2\ \\ -4\ \\ -3\ \end{bmatrix} + c_3\begin{bmatrix}\ \ 0\ \\ -5\ \\ \ \ 2\ \\ -1\ \end{bmatrix}\ = \begin{bmatrix}\ \ 1\ &\ \ 2\ &\ \ 0\ \\ -3\ &\ \ 2\ & -5\ \\ \ \ 2\ & -4\ &\ \ 2\ \\ \ \ 0\ & -3\ & -1\ \end{bmatrix}\mathbf{c} = \mathbf{0} $$
is dependent or independent. If the system is dependent and has nontrivial solutions, the set of vectors is linearly dependent. If the system is independent and has only the trivial solution, the set of vectors is linearly dependent

We don't need the augmented matrix because the system is homogeneous. We only need to determine if the columns of our matrix are all pivot columns. We proceed with Gaussian elimination.

$$ \begin{align*} \begin{bmatrix}\ \ 1\ &\ \ 2\ &\ \ 0\ \\ -3\ &\ \ 2\ & -5\ \\ \ \ 2\ & -4\ &\ \ 2\ \\ \ \ 0\ & -3\ & -1\ \end{bmatrix}\begin{array}{l} \ \\ R_2 + 3R_1 \\ R_3 - 2R_1 \\ \ \end{array} &\longrightarrow \begin{bmatrix}\ \ 1\ &\ \ 2\ &\ \ 0\ \\ \ \ 0\ &\ \ 8\ & -5\ \\ \ \ 0\ & -8\ &\ \ 2\ \\ \ \ 0\ & -3\ & -1\ \end{bmatrix}\begin{array}{l} \ \\ \ \\ R_2 + R_3 \\ \ \end{array} \longrightarrow \begin{bmatrix}\ \ 1\ &\ \ 2\ &\ \ 0\ \\ \ \ 0\ &\ \ 8\ & -5\ \\ \ \ 0\ &\ \ 0\ & -3\ \\ \ \ 0\ & -3\ & -1\ \end{bmatrix}\begin{array}{l} \ \\ 2R_4 + R_2 \\ -R_4 + R_3 \\ \ \end{array} \\ \\ &\longrightarrow \begin{bmatrix}\ \ 1\ &\ \ 2\ &\ \ 0\ \\ \ \ 0\ &\ \ 2\ & -7\ \\ \ \ 0\ &\ \ 3\ & -2\ \\ \ \ 0\ & -3\ & -1\ \end{bmatrix}\begin{array}{l} \ \\ \ \\ -R_2 + R_3 \\ R_3 + R_4 \end{array} \longrightarrow \begin{bmatrix}\ \ 1\ &\ \ 2\ &\ \ 0\ \\ \ \ 0\ &\ \ 2\ & -7\ \\ \ \ 0\ &\ \ 1\ &\ \ 5\ \\ \ \ 0\ &\ \ 0\ & -3\ \end{bmatrix}\begin{array}{l} \ \\ \ \\ -\frac{1}{2}R_2 + R_3 \\ -\frac{1}{3}R_4 \end{array} \\ \\ &\longrightarrow \begin{bmatrix}\ \ 1\ &\ \ 2\ &\ \ 0\ \\ \ \ 0\ &\ \ 2\ & -7\ \\ \ \ 0\ &\ \ 0\ & -\frac{17}{2}\ \\ \ \ 0\ &\ \ 0\ &\ \ 1\ \end{bmatrix}\begin{array}{l} \ \\ \ \\ -\frac{2}{17}R_3 \\ \ \end{array} \longrightarrow \begin{bmatrix}\ \ 1\ &\ \ 2\ &\ \ 0\ \\ \ \ 0\ &\ \ 2\ & -7\ \\ \ \ 0\ &\ \ 0\ &\ \ 1\ \\ \ \ 0\ &\ \ 0\ &\ \ 1\ \end{bmatrix}\begin{array}{l} \ \\ -7R_4 + R_2 \\ \ \\ -R_3 + R_4 \end{array} \\ \\ &\longrightarrow \begin{bmatrix}\ \ 1\ &\ \ 2\ &\ \ 0\ \\ \ \ 0\ &\ \ 2\ &\ \ 0\ \\ \ \ 0\ &\ \ 0\ &\ \ 1\ \\ \ \ 0\ &\ \ 0\ &\ \ 0\ \end{bmatrix}\begin{array}{l} -R_2 + R_1 \\ \frac{1}{2}R_2 \\ \ \\ \ \end{array} \longrightarrow \begin{bmatrix}\ \ 1\ &\ \ 0\ &\ \ 0\ \\ \ \ 0\ &\ \ 1\ &\ \ 0\ \\ \ \ 0\ &\ \ 0\ &\ \ 1\ \\ \ \ 0\ &\ \ 0\ &\ \ 0\ \end{bmatrix} \end{align*} $$
All of the columns of our matrix are pivot columns so the set of vectors

$$ \left\{\,\begin{bmatrix}\ \ 1\ \\ -3\ \\ \ \ 2\ \\ \ \ 0\ \end{bmatrix}\,\ \begin{bmatrix}\ \ 2\ \\ \ \ 2\ \\ -4\ \\ -3\ \end{bmatrix}\,\ \begin{bmatrix}\ \ 0\ \\ -5\ \\ \ \ 2\ \\ -1\ \end{bmatrix}\ \right\} $$
is linearly independent.

---

4.4.8 Linear Independence

Definition

Any set of vectors $U = \left\{\,\mathbf{u}_1,\dots,\mathbf{u}_k\,:\,k\in\mathbb{Z}^+\cup\left\{\infty\right\}\,\right\}$ in a vector space $V$ is called linearly independent if it is not linearly dependent.

• For any set of vectors in a vector space $U = \left\{\,\mathbf{u}_1,\ \dots,\ \mathbf{u}_k\,:\,k\in\mathbb{Z}^+\cup\left\{\infty\right\}\,\right\}$, the vector equation
  $$ c_1\mathbf{u}_{j_1} + \cdots + c_m\mathbf{u}_{j_m} = \mathbf{0} $$
  has the unique solution $\langle 0,\dots 0\rangle$ if and only if the set of vectors $\left\{\,\mathbf{u}_{j_1},\ \mathbf{u}_{j_2},\ \dots,\ \mathbf{u}_{j_m}\,\right\}$ is linearly independent.
  
  
  
  
• For any set of vectors $\left\{\,\mathbf{u}_{j_1},\ \mathbf{u}_{j_2},\ \dots,\ \mathbf{u}_{j_m}\,\right\}$ in a vector space, the system of equations
  
  
  $$ \begin{bmatrix}\ \mathbf{u}_{j_1}\ &\ \cdots\ &\ \mathbf{u}_{j_m}\ \end{bmatrix}\mathbf{c} = \mathbf{0} $$
  has only the trivial solution if and only if the set of vectors is linearly independent.
  
  
  
  
  

Theorem 4.4.2

Any finite set of vectors $U = \left\{ \mathbf{u}_1, \dots, \mathbf{u}_k \right\}$ is linearly independent if and only if the linear system

$$ c_1\mathbf{u}_1 + \dots + c_k\mathbf{u}_k = \mathbf{0} $$

has only the solution $c_1=\dots=c_k=0$.

Corollary 4.4.3

Any finite set of vectors $U = \left\{ \mathbf{u}_1, \dots, \mathbf{u}_k \right\}$ is linearly independent if and only if the null space of the linear system

$$ U\mathbf{c} = \begin{bmatrix} \mathbf{u}_1 & \dots & \mathbf{u}_k \end{bmatrix}\mathbf{c} = \mathbf{0} $$

is trivial.

Exercise 3

Determine of the set of vectors in $\mathbb{R}^4$ are linear independent.


$$ \left\{\,\begin{bmatrix}\ \ 1\ \\ -3\ \\ \ \ 2\ \\ \ \ 0\ \end{bmatrix}\,\ \begin{bmatrix}\ \ 2\ \\ \ \ 2\ \\ -4\ \\ -3\ \end{bmatrix}\,\ \begin{bmatrix}\ \ 0\ \\ -5\ \\ \ \ 2\ \\ -1\ \end{bmatrix}\,\ \begin{bmatrix} -7\ \\ \ \ 0\ \\ \ \ 4\ \\ \ \ 5\ \end{bmatrix}\,\right\} $$

Follow Along

We want to determine of the linear system

$$ c_1\begin{bmatrix}\ \ 1\ \\ -3\ \\ \ \ 2\ \\ \ \ 0\ \end{bmatrix} + c_2\begin{bmatrix}\ \ 2\ \\ \ \ 2\ \\ -4\ \\ -3\ \end{bmatrix} + c_3\begin{bmatrix}\ \ 0\ \\ -5\ \\ \ \ 2\ \\ -1\ \end{bmatrix} + c_4\begin{bmatrix} -7\ \\ \ \ 0\ \\ \ \ 4\ \\ \ \ 5\ \end{bmatrix}\ = \begin{bmatrix}\ \ 1\ &\ \ 2\ &\ \ 0\ & -7 \\ -3\ &\ \ 2\ & -5\ &\ \ 0\ \\ \ \ 2\ & -4\ &\ \ 2\ &\ \ 4\ \\ \ \ 0\ & -3\ & -1\ &\ \ 5\ \end{bmatrix}\mathbf{c} = \mathbf{0} $$
is dependent or independent. If the system is dependent and has nontrivial solutions, the set of vectors is linearly dependent. If the system is independent and has only the trivial solution, the set of vectors is linearly independent.

We don't need the augmented matrix because the system is homogeneous. We only need to determine if the columns of our matrix are all pivot columns. We proceed with Gaussian elimination.

$$ \begin{align*} \begin{bmatrix}\ \ 1\ &\ \ 2\ &\ \ 0\ & -7 \\ -3\ &\ \ 2\ & -5\ &\ \ 0\ \\ \ \ 2\ & -4\ &\ \ 2\ &\ \ 4\ \\ \ \ 0\ & -3\ & -1\ &\ \ 5\ \end{bmatrix} \begin{array} \ \\ R_2 + 3R_1 \\ R_3 - 2R_1 \\ \ \end{array} &\longrightarrow \begin{bmatrix}\ \ 1\ &\ \ 2\ &\ \ 0\ & -7 \\ \ \ 0\ &\ \ 8\ & -5\ & -21\ \\ \ \ 0\ & -8\ &\ \ 2\ &\ 18\ \\ \ \ 0\ & -3\ & -1\ &\ \ 5\ \end{bmatrix} \begin{array} \ \\ R_2 + 2R_4 \\ R_3 + R_2 \\ \ \end{array} \longrightarrow \\ \\ \begin{bmatrix}\ \ 1\ &\ \ 2\ &\ \ 0\ & -7 \\ \ \ 0\ &\ \ 2\ & -7\ & -11\ \\ \ \ 0\ &\ \ 0\ & -3\ & -3\ \\ \ \ 0\ & -3\ & -1\ &\ \ 5\ \end{bmatrix} \begin{array} \ \\ \ \\ -\frac{1}{3}R_3 \\ R_4 + R_2 \end{array} &\longrightarrow \begin{bmatrix}\ \ 1\ &\ \ 2\ &\ \ 0\ & -7 \\ \ \ 0\ &\ \ 2\ & -7\ & -11\ \\ \ \ 0\ &\ \ 0\ &\ \ 1\ &\ \ 1\ \\ \ \ 0\ & -1\ & -8\ & -6\ \end{bmatrix} \begin{array} \ \\ R_2 + 7R_3 \\ \ \\ R_4 + 8R_3 \end{array} \longrightarrow \\ \\ \begin{bmatrix}\ \ 1\ &\ \ 2\ &\ \ 0\ & -7 \\ \ \ 0\ &\ \ 2\ &\ \ 0\ & -4\ \\ \ \ 0\ &\ \ 0\ &\ \ 1\ &\ \ 1\ \\ \ \ 0\ & -1\ &\ \ 0\ &\ \ 2\ \end{bmatrix} \begin{array} \ \\ \frac{1}{2}R_2 \\ \ \\ R_4 + \frac{1}{2}R_2 \end{array} &\longrightarrow \begin{bmatrix}\ \ 1\ &\ \ 2\ &\ \ 0\ & -7 \\ \ \ 0\ &\ \ 1\ &\ \ 0\ & -2\ \\ \ \ 0\ &\ \ 0\ &\ \ 1\ &\ \ 1\ \\ \ \ 0\ &\ \ 0\ &\ \ 0\ &\ \ 0\ \end{bmatrix} \begin{array}\ R_1-2R_2 \\ \ \\ \ \\ \ \end{array} \longrightarrow \\ \\ \begin{bmatrix}\ \ 1\ &\ \ 0\ &\ \ 0\ & -3 \\ \ \ 0\ &\ \ 1\ &\ \ 0\ & -2\ \\ \ \ 0\ &\ \ 0\ &\ \ 1\ &\ \ 1\ \\ \ \ 0\ &\ \ 0\ &\ \ 0\ &\ \ 0\ \end{bmatrix} \end{align*} $$
The matrix is singular so the null space of matrix $U$ is nontrivial and the set of vectors is linearly dependent.

4.4.8 The Infinite Dimensional Vector Spaces of Functions

Can we find linearly independent set of vectors in infinite dimensional vector spaces? Consider the infinite dimensional vector space of continuous functions on the real line

$$ C(\mathbb{R}) = C(-\infty,\,\infty), $$

the $n$-times continuously differential functions on the real line

$$ C^{n}(\mathbb{R}) = C^{n}(-\infty,\,\infty), $$

and the smooth functions that have and $n^{\text{th}}$ derivative for every positive integer $n$

$$ C^{\infty}(\mathbb{R}) $$

Examples

1. Polynomials are $n$-times continuously differentiable for every positive integer $n$.

$$ \begin{align*} \dfrac{d^4}{dx^4}( x^3 + 2x^2 - x - 5) &= \dfrac{d^3}{dx^3}(3x^2 + 4x - 1) \\ \\ &= \dfrac{d^2}{dx^2}(6x + 4) \\ \\ &= \dfrac{d}{dx}(6) = 0 \end{align*} $$
The $(n+1)^{\text{st}}$ derivative of a polynomial of degree $n$ is always the zero function and all subsequent derivatives are the zero function.

2. The trigonometric functions $\sin(x)$ and $\cos(x)$ are $n$-times continuously differentiable for every positive integer $n$.

3. The trigonometric functions $\sin(kx)$ and $\cos(kx)$ are $n$-times continuously differentiable for every positive integers $k$ and $n$.

4. The exponential function $e^{kx}$ is $n$-times continuously differentiable for every positive integer $n$.

5. The absolute value function $|x|$ is continuous, but not differentiable on the real line. In particular, this function is not differentiable at $x=0$.

6. The function
   $$ f(x) = \left\{\begin{array}{rcl} \frac{x^2}{2}\ &\ \ &\ x\ge 0 \\ -\frac{x^2}{2}\ &\ \ &\ x\lt 0\ \end{array}\right. $$
   is continuously differentiable but its derivative if $f'(x) = |x|$ so $f$ does not have two derivatives.

It is not necessary to use the entire real line as the domain for our functions to create a vector space of $n$-times continuously differentiable functions.

7. The infinite dimensional vector space of all functions with at least $n$ derivatives and the $n^{\text{th}}$ derivative $f^{(n)}(x)$ is continuous on the interval $[a,b]$,

$$ C^n[a,b], $$

These have become the foundation of the mathematical models of 21$^{\text{st}}$ century technology. Expressing our mathematical models in the language of vector spaces of functions allows us to use computing devices to numerically calculate solutions in seconds that a human would not be able to complete in a lifetime. You are taking this course to learn about these vector spaces; not $\mathbb{R}^2$ and $\mathbb{R}^3$.

---

4.4.9 The Wronskian and Linear Independence

How do we determine what functions are linearly independent or linearly dependent? For example, suppose we have $n$ functions that have at least $n-1$ derivatives,

$$ \left\{\,f_1(x),\ f_2(x),\ f_3(x),\ \dots,\ f_n(x)\,\right\} $$
To determine if they are linearly independent, we need to determine if the linear system

$$ c_1f_1(x) + c_2f_2(x) + c_3f_3(x) +\ \cdots\ + c_nf_n(x) = 0 $$
has any nontrivial solutions.

Computing the derivative of both sides of this equation yields

$$ c_1f_1'(x) + c_2f_2'(x) + c_3f_3'(x) +\ \cdots\ + c_nf_n'(x) = 0 $$
Computing the derivative of both sides $n-2$ more times results is a system of equations.

$$ \begin{align*} c_1f_1(x) +\ \ \ \ \ \ c_2f_2(x) +\ \ \ \ \ \ c_3f_3(x) +\ \cdots\ +\ \ \ \ \ \ c_nf_n(x) &= 0 \\ \\ c_1f_1'(x) +\ \ \ \ \ \ c_2f_2'(x) +\ \ \ \ \ \ c_3f_3'(x) +\ \cdots\ +\ \ \ \ \ \ c_nf_n'(x) &= 0 \\ \\ c_1f_1''(x) +\ \ \ \ \ c_2f_2''(x) +\ \ \ \ \ c_3f_3''(x) +\ \cdots\ +\ \ \ \ \ c_nf_n''(x) &= 0 \\ \\ \ddots &\ \\ \\ c_1f_1^{(n-2)}(x) + c_2f_2^{(n-2)}(x) + c_3f_3^{(n-2)}(x) +\ \cdots\ + c_nf_n^{(n-2)}(x) &= 0 \\ \\ c_1f_1^{(n-1)}(x) + c_2f_2^{(n-1)}(x) + c_3f_3^{(n-1)}(x) +\ \cdots\ + c_nf_n^{(n-1)}(x) &= 0 \\ \end{align*} $$
The right-hand side consists of the zero function for all $n$ equations and so the left-hand side must be also. The constants are the same for all $n$ equations, hence we have a matrix equation

$$ \begin{align} \begin{bmatrix} f_1(x)\ &\ f_2(x)\ &\ f_3(x)\ &\ \cdots\ & f_n(x) \\ f_1'(x)\ &\ f_2'(x)\ &\ f_3'(x)\ &\ \cdots\ & f_n'(x) \\ f_1''(x)\ &\ f_2''(x)\ &\ f_3''(x)\ &\ \cdots\ & f_n''(x) \\ \vdots\ &\ \vdots\ &\ \vdots\ &\ \ddots\ &\ \vdots\ \\ f_1^{(n-1)}(x)\ &\ f_2^{(n-1)}(x)\ &\ f_3^{(n-1)}(x)\ &\ \cdots\ & f_n^{(n-1)}(x) \\ \end{bmatrix} \begin{bmatrix}\ c_1\ \\ \ c_2\ \\ \ c_3\ \\ \ \vdots\ \\ \ c_n\ \end{bmatrix} = \begin{bmatrix}\ 0\ \\ \ 0\ \\ \ 0\ \\ \ \vdots\ \\ \ 0\ \end{bmatrix} \end{align}\tag{1} $$

Definition

The Wronskian Matrix

For a set of $n$ functions $f_1$, $f_2$, $f_3$, $\dots$, $f_n\ $ in $\ C^{(n-1})[a,b]$, the Wronskian matrix of these functions is the matrix

$$ W\left[f_1,f_2,f_3,\dots,f_n\right](x) := \begin{bmatrix} f_1(x)\ &\ f_2(x)\ &\ f_3(x)\ &\ \cdots\ & f_n(x) \\ f_1'(x)\ &\ f_2'(x)\ &\ f_3'(x)\ &\ \cdots\ & f_n'(x) \\ f_1''(x)\ &\ f_2''(x)\ &\ f_3''(x)\ &\ \cdots\ & f_n''(x) \\ \vdots\ &\ \vdots\ &\ \vdots\ &\ \ddots\ &\ \vdots\ \\ f_1^{(n-1)}(x)\ &\ f_2^{(n-1)}(x)\ &\ f_3^{(n-1)}(x)\ &\ \cdots\ & f_n^{(n-1)}(x) \\ \end{bmatrix} $$

Most of the matrices presented thus far have constants for the entries. The Wronskian $W\left[f_1,f_2,f_3,\dots,f_n\right](x)$ is a matrix-valued function with one real input and whose output is a matrix,

$$ W\left[f_1,f_2,f_3,\dots,f_n\right]\,:\,\mathbb{R}\longrightarrow\mathbb{R}^{n\times n} $$

The set of functions is linearly dependent if and only if there is a nontrivial solution $\left[\,c_1,\ c_2,\ c_3,\ \dots,\ c_n\,\right]^T$ to equation (1).

$$ c_1f_1'(x) + c_2f_2'(x) + c_3f_3'(x) +\ \cdots\ + c_nf_n'(x) = 0 $$

That means the set of functions is linearly dependent if and only if the equation

$$ W\left[f_1,f_2,f_3,\dots,f_n\right](x) \equiv 0 $$

Theorem 4.4.4

For a set of $n$ functions $f_1$, $\dots$, $f_n\ $ in $\ C^{(n-1})[a,b]$, the set of functions is linearly dependent if and only if the Wronskian matrix of these functions is singular for every $x\in[a,b]$.

$$ \text{det}\left(W\left[f_1,f_2,f_3,\dots,f_n\right](x)\right) := \begin{vmatrix} f_1(x)\ &\ f_2(x)\ &\ f_3(x)\ &\ \cdots\ & f_n(x) \\ f_1'(x)\ &\ f_2'(x)\ &\ f_3'(x)\ &\ \cdots\ & f_n'(x) \\ f_1''(x)\ &\ f_2''(x)\ &\ f_3''(x)\ &\ \cdots\ & f_n''(x) \\ \vdots\ &\ \vdots\ &\ \vdots\ &\ \ddots\ &\ \vdots\ \\ f_1^{(n-1)}(x)\ &\ f_2^{(n-1)}(x)\ &\ f_3^{(n-1)}(x)\ &\ \cdots\ & f_n^{(n-1)}(x) \\ \end{vmatrix} \equiv 0 $$

Exercise 4

For any positive integer $m$, determine if the functions $\left\{\sin(mx),\ \cos(mx)\right\}$ are linearly independent in the vector space $C(\mathbb{R})$, the vector space of continuous functions on the real line.

Follow Along

We want to determine of the linear system has a nontrivial solution for any $x\in\mathbb{R}$.
$$ \begin{align*} c_1\sin(mx) + c_2\cos(mx) &= 0 \\ \\ c_1m\cos(mx) - c_2m\sin(mx) &= 0 \end{align*} $$
In matrix form we have $$ \begin{bmatrix} \sin(mx) & \cos(mx) & | & 0 \\ m\cos(mx) & -m\sin(mx) & | & 0 \end{bmatrix} $$
Before we start reducing our matrix, we need to ask ourselves whether there are any values of $x$ for which $\sin(mx)=0$ or $\cos(mx)=0$?
$$ \begin{align*} \sin(mx) &= 0 &\text{or} &\qquad &\cos(mx) &= 0 \\ \\ x &= \frac{k\pi}{m} &\ &\qquad &x &= \frac{(2k+1)\pi}{2m} \end{align*} $$
for any $k\in\mathbb{Z}$. If $x = \frac{k\pi}{m}$, then $\sin(mx) = \sin\left(k\pi\right) = 0$, but $\cos\left(k\pi\right)=(-1)^k$. Our Wronski matrix evaluated at $x=\frac{k\pi}{m}$ becomes
$$ W\left(\frac{k\pi}{m}\right) = \begin{bmatrix} 0 & (-1)^k \\ (-1)^km & 0 \end{bmatrix} $$
Both of the columns of $W\left(\frac{k\pi}{m}\right)$ are pivot columns so the matrix is nonsingular. Likewise
$$ W\left(\frac{(2k+1)\pi}{m}\right) = \begin{bmatrix} (-1)^k & 0 \\ 0 & (-1)^{k+1}m \end{bmatrix} $$
Again, both columns of $W\left(\frac{(2k+1)\pi}{m}\right)$ are pivot columns so the matrix is nonsingular. If neither $\cos(mx)=0$ nor $\sin(mx)=0$, then
$$ \begin{align*} \begin{bmatrix} \sin(mx) & \cos(mx) \\ m\cos(mx) & -m\sin(mx) \end{bmatrix}\ \begin{array}{l} m\cos(mx)R_1 \\ -\sin(mx)R_2 \end{array} &\rightarrow \begin{bmatrix} m\sin(mx)\cos(mx) & m\cos^2(mx) \\ -m\sin(mx)\cos(mx) & m\sin^2(mx) \end{bmatrix}\ \begin{array}{l} \\ R_2+R_1 \end{array} \\ \\ &\rightarrow \begin{bmatrix} m\sin(mx)\cos(mx) & m\cos^2(mx) \\ 0 & m \end{bmatrix}\ \begin{array}{l} R_1-\cos^2(mx)R_2 \\ \frac{1}{m}R_2 \end{array} \\ \\ &\rightarrow \begin{bmatrix} m\sin(mx)\cos(mx) & 0 \\ 0 & 1 \end{bmatrix}\ \begin{array}{l} \\ \end{array} \\ \end{align*} $$
The linear system has been reduced to an equivalent linear system
$$ \begin{align*} c_1m\sin(mx)\cos(mx) &= 0 \\ c_2 &= 0 \end{align*} $$
In our reduction we have $\sin(mx)\neq 0$, $\cos(mx)\neq 0$ and $m$ is a positive integer, so $m\neq 0$. Therefore $c_1=c_2=0$. Since the null space for $W(x)$ is trival, $W(x)$ is a nonsingular matrix. Hence the Wronskian matrix is nonsingular for every $x\in\mathbb{R}$ and the set of functions $\left\{\,\sin(mx),\ \cos(mx)\,\right\}$ is linearly independent in the vector space $C(\mathbb{R})$.

---

Your use of this self-initiated mediated course material is subject to our Creative Commons License 4.0

---