Math 511: Linear Algebra

1.1 Systems Of Linear Equations

---

1.1.1 What is Linear Algebra?

$$ \require{color} \definecolor{brightblue}{rgb}{.267, .298, .812} \definecolor{darkblue}{rgb}{0.0, 0.0, 1.0} \definecolor{palepink}{rgb}{1, .73, .8} \definecolor{softmagenta}{rgb}{.99,.34,.86} \definecolor{blueviolet}{rgb}{.537,.192,.937} \definecolor{jonquil}{rgb}{.949,.792,.098} \definecolor{shockingpink}{rgb}{1, 0, .741} \definecolor{royalblue}{rgb}{0, .341, .914} \definecolor{alien}{rgb}{.529,.914,.067} \definecolor{crimson}{rgb}{1, .094, .271} \def\ihat{\mathbf{\hat{\unicode{x0131}}}} \def\jhat{\mathbf{\hat{\unicode{x0237}}}} \def\khat{\mathbf{\hat{\unicode{x1d458}}}} \def\tombstone{\unicode{x220E}} \def\contradiction{\unicode{x2A33}} $$

To begin our study of linear algebra, it is important to have a basic notion our subject before diving in. Linear algebra is a subject with a lot of interconnected ideas that we could define generally as follows.

Linear Algebra is the mathematical study of Linear Equations

$$ a_1 x_1 + a_2 x_2 + \ldots + a_n x_n = b, $$

and Linear Mappings

$$ (x_1,x_2,\ldots,x_n)\mapsto a_1 x_1 + a_2 x_2 + \ldots + a_n x_n, $$

and their associated representations using vectors and matrices.

However, that isn't terribly helpful until we know more about what those boldface terms mean.

Videos

Throughout these online notes we will see videos to study. These videos are placed at the best point to study them. Videos are often decorated with an icon to make them easy to find in the notes. There three sources of video for our class.

3Blue1Brown

1. 3Blue1Brown by Grant Sanderson features a series videos that explore linear algebra at a high level. These videos demonstrate the primary concepts of this subject in a way that will help you develop your intuition without being bogged down by technical details. The 3Blue1Brown video series is a superb way to develop and reinforce your understanding of these interrelated geometrical and algebraic ideas of linear algebra. Give the videos a re-watch prior to the midterm and final. You'll be amazed how much all the pieces fit together on those second and third viewings.

A Vision of Linear Algebra

2. Professor Gilbert Strang's A Vision of Linear Algebra and his course Linear Algebra includes video, downloadable pdf versions of the included slides, lecture notes, sample exams with solutions, recitation videos and problem sets with solutions. You can download the entire course if you want a personal copy. Professor Strang has a review video Five Factorizations of a Matrix will help you review for the final exam.

Instructor Videos

3. There are several short videos created by Dr. Nathan Thompson and myself providing examples, demonstrations and solutions to problems included in these notes.

Vocabulary is Important

Finally be aware that this course contains an abundance of vocabulary that normally lower level mathematics courses understate. This is a graduate/undergraduate level course. Vocabulary and concept questions will be the subject of most questions in proctored exams in this course. Every word in the online notes displayed in bold text indicates a vocabulary term.

---

1.1.2 Systems of Linear Equations

In Chapter 4 we will learn a very formal definition of a linear transformation. For now, we need one that is based on our previous mathematical education to help us get started understanding the structure and importance of linear algebra.

The Essence of Linear Algebra

The Essence of Linear Algebra

This video gives us an introduction into the relationship between the geometrical and computational aspects of linear algebra. We need master these concepts of linear algebra and let computers perform numerical computations.

We will use linear algebra to

1. Create mathematical models that describe STEM phenomena,
2. Specify the necessary computations to computing devices, and
3. Interpret the results.

Linear Expressions

A linear expression is a mathematical expression in which each variable that appears in the expression has only the power one.

Algebraic Expression	Classification
$3x + 2y$	is a linear expression
$5x + 9y^2$	is not a linear expression because the variable $y$ is raised to the power of 2.
$5x + \dfrac{1}{y}$	is not a linear expression because the variable $y$ is raised to the power -1.
$3x + \sin(y)$	is not a linear expression because $\sin(y) = \displaystyle\sum_{k=0}^{\infty} (-1)^k\dfrac{y^{2k+1}}{(2k+1)!}$

Linear Equations

The difference between an expression and an equation is that an equation must have a verb phrase. An equation is a complete mathematical sentence. The phases "is equal to", " is greater than", " is less than or equal to" are common verb phrases represented by specific symbols. A linear equation has a linear expression on the left-hand side of an equality symbol and another linear expression on the right-hand side.

Algebraic Equation	Classification
$3x + 5y = 2$	is a linear equation
$3x + 5y = 0$	is a linear equation
$3x = -5y$	is a linear equation

Generally one wants to arrange linear equations to have a constant on the right-hand side of the equation. One expresses a pattern by using letters to represent our constant real number instead of just examples of specific real numbers. This means that $ax + by = c$ represents a linear equation with variables $x$ and $y$; and constants $a$, $b$ and $c$. We refer to the variables as unknowns and refer to the constants as coefficients. To express three equations with three unknowns, one might write

$$ \begin{array}{rcl} x + y - z & = & 0 \\ -x + y + z & = & 2 \\ 3x - y + 1 & = & 2 \end{array} $$

Linear Systems

A list of several equations is called a system of equations. This course will be focused on finding solutions to systems of linear equations.

We will not just solve linear systems of equations. We will also analyze them and their mathematical structure so that we gain an understanding of how systems of linear equations can be solved, how they can be utilized in our work, and how best to understand their geometric nature.

To avoid running out of letters for coefficients we will employ subscripts such as $a_{21}$. The subscripts indicates that this coefficient is the coefficient in the second equation for variable number one. This works so well that subscripts are utilized for our variables. We will define our three unknowns to be $x_1$, $x_2$ and $x_3$. Finally, we need to represent the constants on the right-hand sides of the three equations $b_1$, $b_2$ and $b_3$.

$$ \begin{array}{rcl} a_{11}x_1 + a_{12}x_2 + a_{13}x_3 & = & b_1 \\ a_{21}x_1 + a_{22}x_2 + a_{23}x_3 & = & b_2 \\ a_{31}x_1 + a_{32}x_2 + a_{33}x_3 & = & b_3 \end{array} $$

This allows us to talk about all linear systems with three equations and three unknowns. We can discover what geometric properties they share. In this course we will develop a vocabulary for describing linear systems such as this one in addition to discovering algebraic methods to solve them.

---

1.1.3 Square Linear Systems

Using subscripts is an especially useful for even larger systems of equations. Consider a system of 8 equations with 8 unknowns

$$\begin{array}{rcl} a_{11}x_1 + a_{12}x_2 + a_{13}x_3 + a_{14}x_4 + a_{15}x_5 + a_{16}x_6 + a_{17}x_7 + a_{18}x_8 & = & b_1 \\ a_{21}x_1 + a_{22}x_2 + a_{23}x_3 + a_{24}x_4 + a_{25}x_5 + a_{26}x_6 + a_{27}x_7 + a_{28}x_8 & = & b_2 \\ a_{31}x_1 + a_{32}x_2 + a_{33}x_3 + a_{34}x_4 + a_{35}x_5 + a_{36}x_6 + a_{37}x_7 + a_{38}x_8 & = & b_3 \\ a_{41}x_1 + a_{42}x_2 + a_{43}x_3 + a_{44}x_4 + a_{45}x_5 + a_{46}x_6 + a_{47}x_7 + a_{48}x_8 & = & b_4 \\ a_{51}x_1 + a_{52}x_2 + a_{53}x_3 + a_{54}x_4 + a_{55}x_5 + a_{56}x_6 + a_{57}x_7 + a_{58}x_8 & = & b_5 \\ a_{61}x_1 + a_{62}x_2 + a_{63}x_3 + a_{64}x_4 + a_{65}x_5 + a_{66}x_6 + a_{67}x_7 + a_{68}x_8 & = & b_6 \\ a_{71}x_1 + a_{72}x_2 + a_{73}x_3 + a_{74}x_4 + a_{75}x_5 + a_{76}x_6 + a_{77}x_7 + a_{78}x_8 & = & b_7 \\ a_{81}x_1 + a_{82}x_2 + a_{83}x_3 + a_{84}x_4 + a_{85}x_5 + a_{86}x_6 + a_{87}x_7 + a_{88}x_8 & = & b_8. \end{array}$$
Mathematicians hate writing out long repetitive expressions like this so let me show some ways we can reduce the about of typing we have to do and still communicate all of the information from the previous example:

$$\begin{array}{rcl} a_{11}x_1 + a_{12}x_2 +\ \dots\ + a_{18}x_8 & = & b_1 \\ a_{21}x_1 + a_{22}x_2 +\ \dots\ + a_{28}x_8 & = & b_2 \\ \\ \vdots\ \ \ \ +\ \ \ \vdots\ \ \ \ +\ \dots\ +\ \ \ \vdots\ \ \ \ & \vdots &\ \vdots \\ \\ a_{81}x_1 + a_{82}x_2 +\ \dots\ + a_{88}x_8 & = & b_8. \end{array}$$
The three horizontal or vertical dots literally mean, "follow the pattern, follow the pattern, yada yada yada, follow the pattern." Mathematicians are lazy. This laziness allows us to express a system of 88 equations with 88 unknowns or 30,600 equations with 30,600 unknowns. We will write our system of $n$ equations and $n$ unknowns as

$$\begin{array}{rcl} a_{11}x_1 + a_{12}x_2 +\ \dots\ + a_{1n}x_n & = & b_1 \\ a_{21}x_1 + a_{22}x_2 +\ \dots\ + a_{2n}x_n & = & b_2 \\ \\ \vdots\ \ \ \ +\ \ \ \vdots\ \ \ \ +\ \dots\ +\ \ \ \vdots\ \ \ \ & \vdots &\ \vdots \\ \\ a_{n1}x_1 + a_{n2}x_2 +\ \dots\ + a_{nn}x_n & = & b_n. \end{array}$$

---

1.1.4 Nonsquare Linear Systems

Linear systems for which there are an equal number of equations and unknowns are called square systems and the name will become obvious later. There is no law that says the number of equations and the number of unknowns has to the same.

The system
$$\begin{array}{rcl} 3x + 2y & = & 1 \\ x - y & = & 2 \\ 2x + 3y & = & -1 \end{array}$$
has 3 equations with 2 unknowns, the system

$$\begin{array}{rcl} 3x + 2y - z & = & 4 \\ x + y + z & = & 3 \end{array}$$
has 2 equations and 3 unknowns, and the system

$$\begin{array}{rcl} a_{11}x_1 + a_{12}x_2 +\ \cdots\ + a_{1n}x_n & = & b_1 \\ a_{21}x_1 + a_{22}x_2 +\ \cdots\ + a_{2n}x_n & = & b_2 \\ \\ \vdots\ \ \ \ +\ \ \ \vdots\ \ \ \ +\ \cdots\ +\ \ \ \vdots\ \ \ \ & = & \vdots \\ \\ a_{m1}x_1 + a_{m2}x_2 +\ \cdots + a_{mn}x_n & = & b_m \end{array}$$
has $m$ equations and $n$ unknowns. We will call a system of $m$ linear equations and $n$ unknowns an

An m x n System of Linear Equations

consists of a list of $m$ equations containing a total of $n$ unknowns.

Overdetermined Linear Systems

We will find that if there are more equations than unknowns then it is harder for the system to have any solution at all; there are just too many equations limiting the number of possible solutions until there are none. We call a system of equations with more equations than unknowns overdetermined.

Undetermined Linear Systems

If there are fewer equations than unknowns, then we do not have enough information about our system to have only one solution and there will usually be infinitely many solutions. We call a system of equations with fewer equations than unknowns underdetermined.

Square Linear Systems

Linear systems with the same number of equations and unknowns are just right.

---

1.1.5 The Row Picture for Two-by-Two Systems of Linear Equations

Fortunately for us there exists a straightforward geometric interpretation of a system of 2 equations and 2 unknowns. We can demonstrate the three (yes only three) types of $2\times 2$ systems and draw their graphs to illustrate what can happen.

$$ \begin{align*} 1.\quad x_1 + x_2 &= 3 &\qquad 2.\quad\ \,x_1 +\ \,x_2 &= 3 &\qquad 3.\quad x_1-x_2 &= 1 \\ x_1 - x_2 &= -1 &\qquad 3x_1 + 3x_2 &= 9 &\qquad x_1-x_2 &= -3 \end{align*} $$

This geometric interpretation is called the row picture and it was introduced to us in regular algebra class. Each equation in the linear system is the equation of a line.

Consistent and Independent

If the two lines have different slopes, so they must intersect at a single point. This is a point on the plane $(x_1,\ x_2)$ that lies on both lines. This gives us a single point that makes both linear equations true. Since it is a solution to the equations of both lines, it is call a solution to the linear system.

• A linear system with any solution at all, no matter how many, is called consistent.

• When the two lines have different slopes, then system is called independent.

$$ \begin{align*} 1.\quad x_1 + x_2 &= 3 \\ x_1 - x_2 &= -1 \end{align*} $$

Consistent and Dependent

If the two lines have the same slope and the same $y$-intercept, the two lines are in fact the same line and every solution for one of the equations is also a solution for both equations. The system has infinitely many solutions.

• A linear system with infinitely many solutions is still consistent.

• When the two lines have the same slope, the system is called dependent.

$$ \begin{align*} 2.\quad\ \,x_1 + \ \,x_2 &= 3 \\ 3x_1 + 3x_2 &= 9 \end{align*} $$

Inconsistent and Dependent

If the two lines have the same slope, but different $y$-intercepts, the two lines are parallel and never intersect. While there are infinitely many solutions to each equation; there is no point that is the solution to both equations. The system has no solutions.

• A linear system with no solutions at all is called inconsistent.

• When the two lines have the same slope, the system is called dependent.

$$ \begin{align*} 3.\quad x_1 - x_2 &= 1 \\ x_1 - x_2 &= -3 \end{align*} $$

---

1.1.6 Vectors

Before we go on we need to understand one of the new basic elements of linear algebra. We need to understand

Vectors

Vectors, what even are they?

When both scalar variables and constants appear in equations with vectors and matrices, one requires notation that distinguishes them from each other in the writing. Scalars are real or complex numbers that we already understand from our previous mathematics courses. Vectors are elements of a Vector Space. We will learn that there are many types of vector spaces. All have uses in mathematical models.

In order to denote a vector-valued variable, one needs to decorate the letter or name to distinguish it from a scalar variable or matrix.

Typical Vector Notations

Decoration	Representation
1. Arrow over	$\overrightarrow{v_1},\ \ \overrightarrow{v_2}$
2. Overline	$\overline{v_1},\ \ \overline{v_2}$
3. Underline	$\underline{x},\ \ \underline{y_2}$
4. Bold face	$\mathbf{v}_1,\ \ \mathbf{v}_2$
5. Hat	$\ihat,\ \ \hat{\mathbf{x}}$
6. Tilde	$\tilde{v}_1,\ \ \tilde{x}$

Bold face will be preferred in these online notes. Arrows, overline or underline should be used for handwritten work. A vector decorated with the "hat" or "tilde" are typically reserved for special cases, such as the canonical basis vectors $\ihat$, $\jhat$, and $\khat$ or a "changed" or shifted vector $\mathbf{x}\mapsto\tilde{\mathbf{x}}$. These will be discussed in more detail when they come up.

You must use one of the notations above to distinguish scalars from vectors in your handwritten computations. You need to learn to communicate clearly and accurately in a STEM field. All too often the future engineer that needs to accurately read your notes is you.

---

1.1.7 The Column Picture

The problem with the row picture of a $2\times 2$ systems of linear equations is that it does not extend very well to larger $m\times n$ systems of linear equations. To extend our ideas of consistent and independent to larger $m\times n$ systems of linear equations, we need to view the same three systems using the column picture. Recall our three $2\times 2$ systems of linear equations

$$ \begin{align*} 1.\quad x_1 + x_2 &= 3 &\qquad 2.\quad\ \,x_1 +\ \,x_2 &= 3 &\qquad 3.\quad x_1-x_2 &= 1 \\ x_1 - x_2 &= -1 &\qquad 3x_1 + 3x_2 &= 9 &\qquad x_1-x_2 &= -3 \end{align*} $$

Consistent and Independent

Let us re-write the first linear system in vector form.

$$1.\ \ \begin{bmatrix} x_1 + x_2 \\ x_1 - x_2 \end{bmatrix} = \begin{bmatrix} x_1 \\ x_1 \end{bmatrix} + \begin{bmatrix}\ \ x_2 \\ -x_2 \end{bmatrix} = x_1\begin{bmatrix} 1 \\ 1 \end{bmatrix} + x_2\begin{bmatrix} \ \ 1 \\ -1 \end{bmatrix} = \begin{bmatrix}\ \ 3 \\ -1 \end{bmatrix}$$
Notice that we start out with a column vector on the left side of the equation

$$\begin{bmatrix} x_1 + x_2 \\ x_1 - x_2 \end{bmatrix}$$
This vector equals the vector on the right side of the equation

$$\begin{bmatrix}\ \ 3 \\ -1 \end{bmatrix}$$
However, we use vector arithmetic, vector addition and scalar multiplication, to write the algebraic expression on the left-hand side of the equation as a linear combination.

$$x_1\begin{bmatrix} 1 \\ 1 \end{bmatrix} + x_2\begin{bmatrix} \ \ 1 \\ -1 \end{bmatrix}$$

Multiplying vectors by scalars and adding them together results in a linear combination of vectors.

This is one of the fundamental ideas of this course. You should take the time to recognize a linear combination each time it appears in our notes, the videos, the lectures or the textbook.

This gives us the vector equation

$$x_1\begin{bmatrix} 1 \\ 1 \end{bmatrix} + x_2\begin{bmatrix} \ \ 1 \\ -1 \end{bmatrix} = \begin{bmatrix}\ \ 3 \\ -1 \end{bmatrix}$$

If we name our vectors $\mathbf{v}_1$ and $\mathbf{v}_2$, then we have

$$ \mathbf{v}_1 = \begin{bmatrix} 1 \\ 1 \end{bmatrix},\qquad \mathbf{v}_2 = \begin{bmatrix}\ \ 1 \\ -1 \end{bmatrix}$$
We typically name the vector on the right-hand side of the equation $\mathbf{b}$. Our system of equations can now be written

$$ x_1\mathbf{v}_1 + x_2\mathbf{v}_2 = \mathbf{b}. $$

Since $1\,\mathbf{v}_1 + 2\,\mathbf{v}_2 = \mathbf{b}$, we have $x_1=1$ and $x_2=2$, so our unique solution is the vector

$$\begin{bmatrix} x_1 \\ x_2 \end{bmatrix} = \begin{bmatrix} 1 \\ 2 \end{bmatrix}$$

• When there is any linear combination of the two vectors $\mathbf{v}_1$ and $\mathbf{v}_2$ that equals $\mathbf{b}$, the linear system is called consistent.

• When the vectors $\mathbf{v}_1$ and $\mathbf{v}_2$ point in different directions or have different slopes, the linear system is called linearly independent.

Consistent and Dependent

Let us write the last linear system in vector form.

$$\begin{bmatrix} x_1 + x_2 \\ 3x_1 + 3x_2 \end{bmatrix} = x_1\begin{bmatrix} 1 \\ 3 \end{bmatrix} + x_2\begin{bmatrix} 1 \\ 3 \end{bmatrix} = \begin{bmatrix} 3 \\ 9 \end{bmatrix} = 3\begin{bmatrix} 1 \\ 3 \end{bmatrix} $$
Notice that $\mathbf{v}_1 = \mathbf{v}_2 = \begin{bmatrix} 1 \\ 3 \end{bmatrix}$, so they are certainly linearly dependent. Notice also that $\mathbf{b} = 3\begin{bmatrix} 1 \\ 3 \end{bmatrix}$. Our vector equation becomes

$$ \left(x_1 + x_2\right)\begin{bmatrix} 1 \\ 3 \end{bmatrix} = 3\begin{bmatrix} 1 \\ 3 \end{bmatrix}$$
Thus we have

$$x_1 + x_2 = 3.$$

We obtain an equation for $x_1$ and $x_2$ that has infinitely many solutions, $x_1 + x_2 = 3$. We could plot this equation on a Cartesian plane, and the set of solutions is a line with slope $-1$ and $y$-intercept $3$. All of the points $(x_1, x_2)$ on this line gives us a solution to the linear system. This linear system has infinitely many solutions.

• When there are infinitely many solutions to a system of linear equations, the system is still called consistent.

• When the vectors $\mathbf{v}_1$ and $\mathbf{v}_1$ belong to the same span, the linear system is called linearly dependent.

Inconsistent and Dependent

Let us write the second system of linear equations in vector form.

$$\begin{bmatrix} x_1-x_2 \\ x_1-x_2 \end{bmatrix} = \begin{bmatrix} x_1 \\ x_1 \end{bmatrix} + \begin{bmatrix} -x_2 \\ -x_2 \end{bmatrix} = x_1\begin{bmatrix} 1 \\ 1 \end{bmatrix} + x_2\begin{bmatrix} -1 \\ -1 \end{bmatrix} = \begin{bmatrix} \ \ 1 \\ -1 \end{bmatrix}$$
In this case, $\mathbf{v}_1 = \begin{bmatrix} 1 \\ 1 \end{bmatrix}$ and $\mathbf{v}_2 = \begin{bmatrix} -1 \\ -1 \end{bmatrix} = -\mathbf{v}_1$. In the language of our linear algebra videos, $\mathbf{v}_1$ and $\mathbf{v}_2$ lie on the same span. They point opposite directions but they lie on the same line so that any linear combination of vectors $\mathbf{v}_1$ and $\mathbf{v}_2$ will be on that line with slope $1$ and $y$-intercept $0$.

These vectors are linearly dependent because they are colinear. Since $\mathbf{b}$ is not a vector in their span, there is no linear combination of vectors $\mathbf{v}_1$ and $\mathbf{v}_2$ that will equal $\mathbf{b}$. This system of linear equations has no solution.

• When there is no linear combination of our vectors $\mathbf{v}_1$ and $\mathbf{v}_2$ that equals $\mathbf{b}$, the linear system is called inconsistent.

• When the vectors $\mathbf{v}_1$ and $\mathbf{v}_2$ belong to the same span, the linear system is called linearly dependent.

The column picture is more useful to use because we can apply the same ideas to larger systems of $m$ equations and $n$ unknowns. Keep in mind that every linear system can have only three types of solutions. This remains true regardless of the number of equations and the number of unknowns.

Three Possible Outcomes for a Linear System

There are only three possible outcomes for the number of solutions to a system of linear equations:

1. A linear system that is consistent and independent has exactly one solution.

2. A linear system is inconsistent and dependent has no solutions.

3. A linear system is consistent and dependent has infinitely many solutions.

---

1.1.8 Linear Combination and Span

Span

Watch this video to gain a geometrical understanding of linear combinations and the span of a set of vectors.

Span

The span of a set of vectors $S = \left\{ \mathbf{v}_1,\ \mathbf{v}_2,\ \dots,\ \mathbf{v}_k \right\}$ is the set of all possible linear combinations of the vectors in the set $S$.

Geometrically the span of one vector is a line.

The span of two vectors is more complicated. If two vectors are colinear, they belong to the same span and they are constant multiples of each other.

1. The span of two linearly independent vectors is a plane.
2. The span of two linearly dependent or colinear vectors is still just a line.

The span of three vectors gets even more complicated because we have several possible outcomes.

1. The span of three linearly independent vectors is three dimensional space because linearly independent vectors all point in different directions.
2. The span of three colinear vectors is still a line.
3. The span to two linearly independent vectors and one dependent vector is a plane.

We will need to build more experience with vectors in order to study the geometrical relationships between larger numbers of vectors in very large dimensional problems.

---

1.1.9 How Do We Solve Linear Systems of Equations?

In our previous algebra classes, we solved a system of two equations and two unknowns using either substitution or elimination. Using the method of substitution becomes horrible when trying to solve larger systems of linear equations. Fortunately, the method of elimination works very well for even very large systems of linear equations.

Let us consider a system of equations and use the elimination method to solve it.

$$\begin{align*} 2x_1 + 3x_2 + \ \ x_3 &= \ \ 4 \\ 3x_1 - \ \ x_2 - 3x_3 &= -1 \\ x_1 + 2x_2 + \ \ x_3 &= \ \ 3 \end{align*}$$
We want to determine whether there is one solution, no solutions, or infinitely many solutions. We will not know the number of solutions until we solve the linear system. For this reason, we will call our unknown number of solutions a set of solutions. A set of solutions can have one solution in the set, no solutions in the set, or infinitely many solutions in the set.

If we are going to use elimination, the arithmetic will work out better if the coefficient of $x_1$ is a $1$ in the first equation. It does not matter what order the equations appear in the system. Let us swap the first and the last equation

$$\begin{align*} x_1 + 2x_2 + \ \ x_3 &= \ \ 3 \\ 3x_1 - \ \ x_2 - 3x_3 &= -1 \\ 2x_1 + 3x_2 + \ \ x_3 &= \ \ 4 \end{align*}$$
If we change the order of the equations then technically we have a different system of equations, but they will have the same set of solutions. We call these two systems of linear equations equivalent when they have the same set of solutions.

Next, we must eliminate the first variable $x_1$ from the second two equations. To accomplish this task we will multiply the first equation on both sides by the nonzero number $3$. This gives us a new system of equations

$$\begin{align*} 3x_1 + 6x_2 + 3x_3 &= \ \ 9 \\ 3x_1 - \ \ x_2 - 3x_3 &= -1 \\ 2x_1 + 3x_2 + \ \ x_3 &= \ \ 4 \end{align*}$$

The graph of the first equation is still the same plane in three-dimensional space, so the system of equations has the same set of solutions. Thus, this new system is also an equivalent system of equations. We only point this out currently to illustrate that multiplying one of the equations on both sides by the same nonzero number does not change the set of solutions of the system of equations. We really want to add $-3$ times the first equation to the second equation. This results in

$$\begin{align*} -3x_1 - 6x_2 - \ 3x_3 &= -9 \\ \ 3x_1 - \ \ x_2 - 3x_3 &= -1 \\ \end{align*}$$

Adding these two equation yields the equation

$$-7x_2 - 6x_3 = -10.$$

Now we have the equivalent linear system of equations

$$\begin{align*} x_1 + 2x_2 + \ \ x_3 &= \ \ 3 \\ -7x_2 - 6x_3 &= -10 \\ 2x_1 + 3x_2 + \ \ x_3 &= \ \ 4 \end{align*}$$

It may not be apparent at this time but adding a nonzero multiple of one equation to another equation results in an equivalent linear system of equations; that is the two systems of equations have the same set of solutions.

We can also add $-2$ times the first equation to the third equation to obtain
$$\begin{align*} x_1 + 2x_2 + \ \ x_3 &= \ \ 3 \\ -7x_2 - 6x_3 &= -10 \\ -x_2 - \ \ x_3 &= -2 \end{align*}$$

At this point we want to make our arithmetic a simple as possible so we will multiply both sides of the second equation by $-1$, and multiply both sides of the third equation by $-1$ to get the equivalent linear system

$$\begin{align*} x_1 + 2x_2 + \ \ x_3 &= 3 \\ 7x_2 + 6x_3 &= 10 \\ x_2 + \ \ x_3 &= 2 \end{align*}$$

As in the first step, it would help to swap the second equation and the third equation. This will give us the equivalent linear system

$$\begin{align*} x_1 + 2x_2 + \ \ x_3 &= 3 \\ x_2 + \ \ x_3 &= 2 \\ 7x_2 + 6x_3 &= 10 \end{align*}$$

Now we need to eliminate the variable $x_2$ from the third equation. Adding $-7$ times the second equation to the third equation yields

$$\begin{align*} x_1 + 2x_2 + \ \ x_3 &= 3 \\ x_2 + \ \ x_3 &= 2 \\ -x_3 &= -4 \end{align*}$$

Multiplying both sides of the third equation by $-1$ gives us the equivalent linear system

$$\begin{align*} x_1 + 2x_2 + \ \ x_3 &= 3 \\ x_2 + \ \ x_3 &= 2 \\ x_3 &= 4 \end{align*}$$

Now we have an equivalent linear system of equations with a special structure to the equations. This is called strict triangular form. I hope you can see the triangle on the left side of the equation. It will become more obvious when we write the linear system using matrices. The advantage of strict triangular form is that we can readily solve the system of equations using backward substitution. We start at the last equation and determine that

$$x_3 = 4.$$

Using the value of $x_3$ we can substitute the value $4$ in the second equation for $x_3$ to obtain

$$x_2 + 4 = 2.$$

Subtracting $4$ from both sides of the second equation yields

$$x_2 = -2.$$

Substituting the values of $x_3$ and $x_2$ into the first equation gives us

$$x_1 + 2(-2) + 4 = 3.$$

Using this equation we determine that

$$x_1 = 3.$$

Thus our set of solutions has only one vector in the set,

$$ \begin{bmatrix} x_1 \\ x_2 \\ x_3 \end{bmatrix} = \begin{bmatrix}\ \ 3 \\ -2 \\ \ \ 4 \end{bmatrix} $$

Notice that we found the solution using just three basic operations, and that each basic operation results in an equivalent linear system of equations.

Elementary Operations

1. Interchange two equations
2. Multiply an equation on both sides by a nonzero number
3. Add a multiple of one equation to another equation

---

1.1.10 The Geometry of Linear Equations

If you are taking an HYB section or need additional help with the geometry of linear systems study The Geometry of Linear Systems by Professor Strang.

In the next section we will replace the list of equations with matrices and vectors. This will allow us to solve larger linear systems and create computer algorithms for solving these linear systems.

---

Your use of this self-initiated mediated course material is subject to our Creative Commons License 4.0

---