Mathematics, Statistics & Physics

Math 511: Linear Algebra

5.3b Gram-Schmidt Orthogonalization

---

5.3b.1 Invariance of Dot Product

$$ \require{color} \definecolor{brightblue}{rgb}{.267, .298, .812} \definecolor{darkblue}{rgb}{0.0, 0.0, 1.0} \definecolor{palepink}{rgb}{1, .73, .8} \definecolor{softmagenta}{rgb}{.99,.34,.86} \definecolor{blueviolet}{rgb}{.537,.192,.937} \definecolor{jonquil}{rgb}{.949,.792,.098} \definecolor{shockingpink}{rgb}{1, 0, .741} \definecolor{royalblue}{rgb}{0, .341, .914} \definecolor{alien}{rgb}{.529,.914,.067} \definecolor{crimson}{rgb}{1, .094, .271} \definecolor{indigo}{rgb}{.8, 0, .6} \def\ihat{\hat{\mmlToken{mi}[mathvariant="bold"]{ı}}} \def\jhat{\hat{\mmlToken{mi}[mathvariant="bold"]{ȷ}}} \def\khat{\hat{\mathsf{k}}} \def\tombstone{\unicode{x220E}} \def\contradiction{\unicode{x2A33}} \def\textregistered{^{\unicode{xAE}}} \def\trademark{^{\unicode{x54}\unicode{x4D}}} $$

We have seen that the best choice of a basis for an inner product space is an orthonormal basis; that is a basis for which all of the basis vectors are unit vectors and mutually orthogonal to each other.

The list of numbers used to represent a vector in a vector space depends on the choice of basis, (reference frame, orientation, etc.). However the geometry defined by the inner product does not change. Once you have a definition of this geometry, then lengths, and angles are fixed. However computing the correct value for

• the inner product $\langle \mathbf{x},\mathbf{y} \rangle$
• length $\left\|\mathbf{x}\right\|$, and
• angle between vectors $\alpha = \dfrac{\langle\mathbf{x},\mathbf{y}\rangle}{\left\|\mathbf{x}\right\|\,\left\|\mathbf{y}\right\|}$

can be very difficult unless one chooses and orthonormal basis.

Example 1

Consider basis $B = \left\{\, \begin{bmatrix}\ 1\ \\ \ 2\ \end{bmatrix},\ \begin{bmatrix}\ 2\ \\ \ 1\ \end{bmatrix} \,\right\}$ for vector space $\mathbb{R}^2$. Each of these basis vectors has a length of $\sqrt{5}$. We know this because we understand the simple geometry of $\mathbb{R}^2$ and that the vector forms the hypotenuse of a right triangle with sides 2 and 1 in the Cartesian Plane. Now let us pick another vector with familiar geometry, vector $\mathbf{x}=\begin{bmatrix}\ 1\ \\ \ 1\ \end{bmatrix}$ in standard coordinates. There are several ways to determine the $B$-coordinates of vector $\mathbf{x}$.

Use the transition matrix

$$ \begin{align*} B &= \begin{bmatrix}\ 1\ &\ 2\ \\ \ 2\ &\ 1\ \end{bmatrix} \\ \\ B^{-1} &= \frac{1}{|B|}\textrm{adj}(B) = \frac{1}{1 - 4}\begin{bmatrix}\ \ 1\ & -2\ \\ -2\ &\ \ 1\ \end{bmatrix} = \begin{bmatrix} -\frac{1}{3}\ &\ \ \frac{2}{3}\ \\ \ \ \frac{2}{3}\ & -\frac{1}{3}\ \end{bmatrix} \\ \\ \mathbf{x}_B &= B^{-1}\mathbf{x} = \begin{bmatrix} -\frac{1}{3}\ &\ \ \frac{2}{3}\ \\ \ \ \frac{2}{3}\ & -\frac{1}{3}\ \end{bmatrix}\begin{bmatrix}\ 1\ \\ \ 1\ \end{bmatrix} = \begin{bmatrix}\ \frac{1}{3}\ \\ \ \frac{1}{3}\ \end{bmatrix} \end{align*} $$

or Solve the linear system

$$ \begin{align*} B\mathbf{x}_B &= \mathbf{x} \\ \\ \begin{bmatrix}\ 1\ &\ 2\ \\ \ 2\ &\ 1\ \end{bmatrix}\mathbf{x}_B &= \begin{bmatrix}\ 1\ \\ \ 1\ \end{bmatrix} \\ \\ \begin{bmatrix}\ \ 1\ &\ \ 2 & | &\ 1\ \\ \ \ 2\ &\ \ 1\ & | & \ 1\ \end{bmatrix}\begin{array}{r} \\ R_2-2R_1 \end{array} &\rightarrow \begin{bmatrix}\ \ 1\ &\ \ 2 & | &\ \ 1\ \\ \ 0\ & -3\ & | & -1\ \end{bmatrix}\begin{array}{r} \\ R_2/(-3) \end{array} \rightarrow \begin{bmatrix}\ \ 1\ &\ \ 2 & | &\ \ 1\ \\ \ 0\ &\ \ 1\ & | &\ \ \frac{1}{3}\ \end{bmatrix}\begin{array}{r} R_1-2R_2 \\ \\ \end{array} \\ \\ \rightarrow \begin{bmatrix}\ 1\ &\ 0 & | &\ \frac{1}{3}\ \\ 0\ &\ 1\ & | &\ \frac{1}{3}\ \end{bmatrix} \end{align*} $$

Now if we merely compute the square root of the sum of the squares of these $B$-coordinates,

$$ \sqrt{\left(\frac{1}{3}\right)^2 + \left(\frac{1}{3}\right)^2} = \sqrt{\frac{1}{9} + \frac{1}{9}} = \sqrt{\frac{2}{9}} = \frac{\sqrt{2}}{3} \neq \sqrt{5} $$

Something is wrong here!

The familiar formula for length of a vector did not work with the vector in $B$-coordinates. What about dot product? The vector $\mathbf{x}$ is the same vector and its geometrical properties should stay the same. In standard coordinates

$$ \mathbf{x}\cdot\mathbf{x} = 1^2 + 2^2 = 5 $$

However in $B$-coordinates

$$ \left[\mathbf{x}_B\right]_1^2 + \left[\mathbf{x}_B\right]_2^2 = \frac{1}{9} + \frac{1}{9} = \frac{2}{9} \neq 5 $$

$\mathbf{x}\cdot\mathbf{x} \neq \left[\mathbf{x}_B\right]_1^2 + \left[\mathbf{x}_B\right]_2^2$ is $B$-coordinates!

Now let us choose an orthonormal basis

Example 2

Consider basis $U = \left\{\, \begin{bmatrix}\ \frac{1}{\sqrt{5}}\ \\ \ \frac{2}{\sqrt{5}}\ \end{bmatrix},\ \begin{bmatrix}\ \ \frac{2}{\sqrt{5}}\ \\ -\frac{1}{\sqrt{5}}\ \end{bmatrix} \,\right\}$, and vector $\mathbf{x} = \begin{bmatrix}\ 1\ \\ \ 1\ \end{bmatrix}$.

$$ \begin{align*} U &= \begin{bmatrix}\ \ \frac{1}{\sqrt{5}}\ &\ \ \frac{2}{\sqrt{5}}\ \\ \ \ \frac{2}{\sqrt{5}}\ & -\frac{1}{\sqrt{5}}\ \end{bmatrix} \\ \\ U^{-1} &= \frac{1}{-\frac{1}{5}-\frac{4}{5}} \begin{bmatrix} -\frac{1}{\sqrt{5}}\ & -\frac{2}{\sqrt{5}}\ \\ -\frac{2}{\sqrt{5}}\ &\ \frac{1}{\sqrt{5}}\ \end{bmatrix} = - \begin{bmatrix} -\frac{1}{\sqrt{5}}\ & -\frac{2}{\sqrt{5}}\ \\ -\frac{2}{\sqrt{5}}\ &\ \frac{1}{\sqrt{5}}\ \end{bmatrix} \\ \\ \mathbf{x}_U &= U^{-1}\mathbf{x} = \begin{bmatrix}\ \ \frac{1}{\sqrt{5}}\ &\ \ \frac{2}{\sqrt{5}}\ \\ \ \ \frac{2}{\sqrt{5}}\ & -\frac{1}{\sqrt{5}}\ \end{bmatrix}\begin{bmatrix}\ 1\ \\ \ 1\ \end{bmatrix} = \begin{bmatrix}\ \frac{3}{\sqrt{5}}\ \\ \ \frac{1}{\sqrt{5}}\ \end{bmatrix} \\ \\ \left[x_U\right]_1^2 + \left[x_U\right]_2^2 &= \left(\frac{3}{\sqrt{5}}\right)^2 + \left(\frac{1}{\sqrt{5}}\right)^2 = \frac{9}{5} + \frac{1}{5} = \frac{10}{5} = 2 = 1^2 + 1^2 = x_1^2 + x_2^2 = \mathbf{x}\cdot\mathbf{x} \end{align*} $$

In Inner Product Space $\mathbb{E}^n$, and orthonormal basis $Q = \left\{ \mathbf{q}_1, \cdots, \mathbf{q}_n \right\}$, the dot product of two vectors $\mathbf{x}_Q, \mathbf{y}_Q \in\mathbb{E}^n$ is given by

$$ \mathbf{x}_Q\cdot\mathbf{y}_Q = \sum_{k=1}^n x_ky_k $$

where $\mathbf{x}_Q = \begin{bmatrix} x_1 \\ \ddots \\ x_n \end{bmatrix}$ and $\mathbf{y}_Q = \begin{bmatrix} y_1 \\ \ddots \\ y_n \end{bmatrix}$

No matter what orthonormal basis is chosen, the dot product is the sum of the products of the corresponding coordinates in the orthonormal basis coordinate system.

Choosing an basis allows us to create a list of numbers that represents a vector in a finite dimensional inner product space. The list of numbers only helps compute magnitudes and angles when the basis is an orthogonormal basis.

The coordinates of a vector $\mathbf{x}$ in inner product space $V$ with respect an orthonormal basis $Q = \left\{ \mathbf{q}_1, \cdots, \mathbf{q}_n \right\}$ can be computed using the inner product.

For any vector $\mathbf{x}$ in an inner product space, the coefficient for basis vector $\mathbf{q}_k$ is given by

$$ [x_Q]_k = \langle \mathbf{x}, \mathbf{q}_k \rangle $$

---

5.3b.2 Invariance of Inner Product

Likewise for any inner product space with inner product $\langle\cdot,\cdot\rangle\,:\,V\times V\longrightarrow\mathbb{R}$ defined on vector space $V$, these values are invariant

• Inner Product
• Norm
• Angle

Example 3

Consider the vector space $P_3$ of polynomials of degree less than $3$ endowed with the inner product defined as follows:

$$ \langle p, q \rangle := p(-1)q(-1) + p(0)q(0) + p(1)q(1) $$

This is another sum of products-ish formula!

What is the inner product, norms, and angle between the polynomials $p(x) = x^2 + 2x - 3$ and $q(x) = 6x + 4$?

1. $$ \langle p,q \rangle = p(-1)q(-1) + p(0)q(0) + p(1)q(1) = (-4)(-2) + (-3)(4) + (0)(10) = -4 $$

2. $$ \begin{align*} \left\|p\right\| &= \left\langle p,p\right\rangle^{1/2} = \left( p(-1)^2 + p(0)^2 + p(1)^2 \right)^{1/2} \\ &= \left( (-4)^2 + (-3)^2 + 0^2 \right)^{1/2} = \sqrt{25} = 5 \\ \\ \left\|q\right\| &= \left(\langle q,q \right\rangle^{1/2} = \left( q(-1)^2 + q(0)^2 + q(1)^2 \right)^{1/2} \\ &= \left( (-2)^2 + 4^2 + 10^2 \right)^{1/2} = \sqrt{120} = 2\sqrt{30} \end{align*} $$

3. $$ \begin{align*} \cos(\alpha) &= \frac{-4}{(5)(2\sqrt{30})} \approx -0.0730 \\ \\ \alpha &\approx 94^{\circ} \end{align*} $$

Example 4

Consider the vector space of continuous functions $C[\pi,\pi]$ and the subspace spanned by

$$ Q = \left\{ \frac{1}{\sqrt{2}}, \cos(x), \sin(x), \cos(2x), \sin(2x) \right\} $$

and endowed with inner product for every $f,g\in C[-\pi,\pi]$

$$ W = \langle f, g \rangle = \frac{1}{\pi}\int_{-\pi}^{\pi} f(x)g(x)\,dx $$

Basis $Q$ is an orthonormal basis for $W$. Furthermore, for any vector (function) $f\in W$,

$$ \left\|f\right\|^2 = \langle f,f \rangle = \frac{1}{\pi}\int_{-\pi}^{\pi} f^2(x)\,dx $$

So,

$$ \begin{align*} \left\|\cos(x)\right\|^2 &= \frac{1}{\pi}\int_{-\pi}^{\pi} \cos^2(x)\,dx = \frac{1}{2\pi}\int_{-\pi}^{\pi}\left( 1 + \cos(2x) \right)\,dx \\ \\ &= \frac{1}{2\pi}\left[ x + \frac{1}{2}\sin(2x) \right]_{-\pi}^{\pi} = \frac{1}{2\pi} \left[ \pi + 0 - (-\pi + 0) \right] = 1 \\ \\ \left\| \cos(x) \right\| &= 1 \\ \\ \left\| \cos^2(x) \right\|^2 &= \frac{1}{\pi}\int_{-\pi}^{\pi} \left( \cos^2(x) \right)^2\,dx = \frac{1}{\pi}\int_{-\pi}^{\pi} \left(\frac{1 + \cos(2x)}{2} \right)^2\,dx \\ \\ &= \frac{1}{4\pi} \int_{-\pi}^{\pi} \left( 1 + 2\cos(2x) + \cos^2(2x) \right)\,dx \\ \\ &= \frac{1}{4\pi} \int_{-\pi}^{\pi} \left( 1 + 2\cos(2x) + \frac{1 + \cos(4x)}{2} \right)\,dx \\ \\ &= \frac{1}{4\pi} \int_{-\pi}^{\pi} \left( \frac{3}{2} + 2\cos(2x) + \frac{1}{2}\cos(4x) \right)\,dx \\ \\ &= \frac{1}{4\pi} \left[ \frac{3x}{2} + \sin(2x) + \frac{1}{8}\sin(4x) \right]_{-\pi}^{\pi} \\ \\ &= \frac{1}{4\pi} \left[ \frac{3\pi}{2} + 0 + 0 - \left( -\frac{3\pi}{2} + 0 + 0 \right) \right] = \frac{3}{4} \\ \\ \left| \cos^2(x) \right| &= \frac{\sqrt{3}}{2} \\ \\ \cos^2(x) &= \frac{1}{2} + \frac{1}{2}\cos(2x) = \frac{1}{\sqrt{2}}\frac{1}{\sqrt{2}} + 0\cos(x) + 0\sin(x) + \frac{1}{2}\cos(2x) + 0\sin(2x) \\ &= \begin{bmatrix}\ \frac{1}{\sqrt{2}}\ \\ \ 0\ \\ \ 0\ \\ \ \frac{1}{2}\ \\ \ 0\ \end{bmatrix}_Q \\ \\ \left\| \cos^2(x) \right\| &= \sqrt{ \left(\frac{1}{\sqrt{2}}\right)^2 + 0^2 + 0^2 + \left( \frac{1}{2} \right)^2 + 0^2 } = \sqrt{ \frac{1}{2} + \frac{1}{4} } = \sqrt{\frac{3}{4}} = \frac{\sqrt{3}}{2}\qquad{\color{green}\Large{\checkmark}} \end{align*} $$

---

5.3b.3 The Gram-Schmidt Orthogonalization Process

Theorem 5.3b.1

The Gram-Schmidt Orthogonalization Process

Let $\{\mathbf{x}_1, \mathbf{x}_2, \dots, \mathbf{x}_n\}$ be a basis for the inner product space $V$. Define

$$ \mathbf{q}_1 := \dfrac{\mathbf{x}_1}{\|\mathbf{x}_1\|} $$

and define $\mathbf{q}_2$, $\dots$, $\mathbf{1}_n$ recursively by

$$ \mathbf{q}_{k} = \dfrac{\mathbf{w}_{k}}{\|\mathbf{w}_{k}\|}, $$

where

$$ \begin{align*} \mathbf{p}_k &:= \langle\mathbf{x}_{k}, \mathbf{u}_1\rangle\mathbf{u}_1 + \langle\mathbf{x}_{k}, \mathbf{u}_2\rangle\mathbf{u}_2 + \dots + \langle\mathbf{x}_{k}, \mathbf{u}_{k-1}\rangle\mathbf{u}_{k-1} \\ \\ &= \text{Proj}_{\text{Span}\left\{ u_1, u_2, \dots, u_{k-1}\right\}} x_k \\ \\ \mathbf{w}_k &= \mathbf{x}_k - \mathbf{p}_k \end{align*} $$

is the projection of $\mathbf{x}_{k}$ onto $\text{Span}\{ \mathbf{q}_1, \cdots, \mathbf{q}_{k-1} \}$. Then the set

$$ \{ \mathbf{q}_1, \mathbf{q}_2, \dots, \mathbf{q}_n \} $$

is an orthonormal basis for $V$.

Don't memorize the formulas; memorize the process.

You have a random non-orthogonal basis $\left\{ \mathbf{v}_1,\ \mathbf{v}_2,\ \dots,\ \mathbf{v}_n \right\}$. To create the orthonormal basis you:

1. Subtract the projections onto all previous orthogonal vectors.
2. Divide the result by the norm to make the result a unit vector.
3. Repeat.

First vector

For your first vector, there are no previous orthogonal vectors ($\mathbf{q}$'s) so

1. No previous orthogonal vectors to project onto $\mathbf{w}_1 = \mathbf{v}_1 - \mathbf{0} = \mathbf{v}_1$
2. Divide the result by the norm $$ \mathbf{q}_1 = \frac{\mathbf{w}_1}{\left\| \mathbf{w}_1 \right\|} = \frac{\mathbf{v}_1}{\left\| \mathbf{v}_1 \right\|} $$
3. Repeat

Second vector

1. Subtract projections onto all previous (one) orthogonal vectors $$ \mathbf{w}_2 = \mathbf{v}_2 - \textrm{Proj}_{\mathbf{q}_1}\mathbf{v}_2 $$
2. Divide the result by the norm $$ \mathbf{q}_2 = \frac{\mathbf{w}_2}{\left\|\mathbf{w}_2\right\|} $$
3. Repeat

Third vector

1. Subtract projections onto all previous (two) orthogonal vectors $$ \mathbf{w}_3 = \mathbf{v}_3 - \textrm{Proj}_{\mathbf{q}_1}\mathbf{v}_3 - \textrm{Proj}_{\mathbf{q}_2}\mathbf{v}_3 $$
2. Divide the result by the norm $$ \mathbf{q}_3 = \frac{\mathbf{w}_3}{\left\|\mathbf{w}_3\right\|} $$
3. Repeat

$k^{\text{th}}$ vector

1. Subtract projections onto all previous ($k-1$) orthogonal vectors $$ \mathbf{w}_k = \mathbf{v}_k - \textrm{Proj}_{\mathbf{q}_1}\mathbf{v}_k - \dots - \textrm{Proj}_{\mathbf{q}_{k-1}}\mathbf{v}_k $$
2. Divide the result by the norm $$ \mathbf{q}_k = \frac{\mathbf{w}_k}{\left\|\mathbf{w}_k\right\|} $$
3. Repeat

$\dots$

Last Vector

1. Subtract projections onto all previous ($n-1$) orthogonal vectors $$ \mathbf{w}_n = \mathbf{v}_k - \textrm{Proj}_{\mathbf{q}_1}\mathbf{v}_k - \dots - \textrm{Proj}_{\mathbf{q}_{n-1}}\mathbf{v}_k $$
2. Divide the result by the norm $$ \mathbf{q}_n = \frac{\mathbf{w}_k}{\left\|\mathbf{w}_k\right\|} $$
3. No more vectors; done.

---

5.3b.4 Gram-Schmidt Orthogonalization

In most applications a basis is no longer good enough for efficient and accurate results. One needs an orthonormal basis. Given a set of basis vectors one would like to produce an orthonormal basis with the same span. The Gram-Schmidt Orthogonalization Process produces such an orthonormal basis of vectors.

Given any ordinary set of basis vectors for a vector space $\{\mathbf{x}_1, \mathbf{x}_2, \dots, \mathbf{x}_n\}$, The Gram-Schmidt Orthogonalization Process produces an orthonormal basis $\{\mathbf{q}_1, \mathbf{q}_2, \dots, \mathbf{q}_n\}$ with the same span

$$ \text{span}\{\mathbf{x}_1, \mathbf{x}_2, \dots, \mathbf{x}_n\} = \text{span}\{\mathbf{q}_1, \mathbf{q}_2, \dots, \mathbf{q}_n\} $$

The pattern for the process

You have a random non-orthogonal basis $\left\{ \mathbf{x}_1,\ \mathbf{x}_2,\ \dots,\ \mathbf{x}_n \right\}$. To create the orthonormal basis:

For each vector $\mathbf{x}_k$ in the basis

1. Subtract the projections onto all previous orthogonal vectors.
2. Divide the result by the norm to make the result a unit vector.
3. Repeat.

Example 5

Find an orthonormal basis for the inner product space $\mathbb{R}^3$, derived from the basis

$$ \{\mathbf{x}_1, \mathbf{x}_2, \mathbf{x}_3\} = \left\{\begin{bmatrix} 1 \\ 2 \\ 0 \end{bmatrix}, \begin{bmatrix} 1 \\ 0 \\ 1 \end{bmatrix},\begin{bmatrix} 2 \\ 0 \\ 1 \end{bmatrix}\right\} $$

First Vector

1. There are no previous orthonormal vectors so $\mathbf{w}_1 = \mathbf{x}_1 = \begin{bmatrix} 1 \\ 2 \\ 0 \end{bmatrix}$.
2. From the first basis vector $\mathbf{x}_1$ we obtain our first unit vector of our orthonormal basis by dividing $\mathbf{x}_1$ by its length to normalize the vector.

$$ \mathbf{q}_1 = \dfrac{\mathbf{w}_1}{\|\mathbf{w}_1\|} $$

In this example,

$$ \mathbf{q}_1 = \dfrac{1}{\sqrt{1^2 + 2^2 + 0^2}}\,\begin{bmatrix} 1 \\ 2 \\ 0 \end{bmatrix} = \begin{bmatrix} \frac{1}{\sqrt{5}} \\ \frac{2}{\sqrt{5}} \\ 0 \end{bmatrix} $$

3. Repeat the process for the next vector.

Second Vector

1. To obtain the second orthonormal vector we compute the projection vector $\mathbf{p}_2$ of $\mathbf{x}_2$ onto $\mathbf{q}_1$. The difference $\mathbf{w}_2 = \mathbf{x}_2-\mathbf{p}_2$ is the part of vector $\mathbf{x}_2$ that is orthogonal to $\mathbf{q}_1$.

$$ \mathbf{w}_2 = \mathbf{x}_2 - \mathbf{p}_2 = \mathbf{x}_2 - \text{Proj}_{\mathbf{q}_1}\mathbf{x}_2 = \mathbf{x}_2 - \dfrac{\langle\mathbf{x}_2, \mathbf{q}_1\rangle}{\langle\mathbf{q}_1, \mathbf{q}_1\rangle}\mathbf{q}_1 $$

$$ \mathbf{w}_2 = \mathbf{x}_2 - \mathbf{p}_2 = \begin{bmatrix} 1 \\ 0 \\ 1 \end{bmatrix} - \dfrac{\frac{1}{\sqrt{5}}(1) + \frac{2}{\sqrt{5}}(0) + 0(1)}{1}\,\begin{bmatrix} \frac{1}{\sqrt{5}} \\ \frac{2}{\sqrt{5}} \\ 0 \end{bmatrix} = \begin{bmatrix} 1 \\ 0 \\ 1 \end{bmatrix} - \begin{bmatrix} \frac{1}{5} \\ \frac{2}{5} \\ 0 \end{bmatrix} = \begin{bmatrix} \frac{4}{5} \\ -\frac{2}{5} \\ 1 \end{bmatrix} $$

2. The resulting vector that is orthogonal to our first orthonormal vector, however, it is not necessarily a unit vector. We need to normalize $\mathbf{x}_2 - \mathbf{p}_2$ to obtain $\mathbf{u}_2$.

$$ \mathbf{q}_2 = \dfrac{\mathbf{w}_2}{\|\mathbf{w}_2\|} = \dfrac{1}{\sqrt{\frac{16}{25} + \frac{4}{25} + 1}}\begin{bmatrix} \frac{4}{5} \\ -\frac{2}{5} \\ 1 \end{bmatrix} = \dfrac{1}{\sqrt{\frac{45}{25}}}\begin{bmatrix} \frac{4}{5} \\ -\frac{2}{5} \\ 1 \end{bmatrix} = \dfrac{5}{3\sqrt{5}}\begin{bmatrix} \frac{4}{5} \\ -\frac{2}{5} \\ 1 \end{bmatrix} = \begin{bmatrix} \frac{4}{3\sqrt{5}} \\ -\frac{2}{3\sqrt{5}} \\ \frac{5}{3\sqrt{5}} \end{bmatrix} $$

3. Repeat the process for the next vector.

Last Vector

1. Subtract projections

$$ \begin{align*} \mathbf{w}_3 = \mathbf{x}_3 - \mathbf{p}_3 &= \mathbf{x}_3 - \text{Proj}_{\mathbf{q}_1}\mathbf{x}_3 - \text{Proj}_{\mathbf{q}_2}\mathbf{x}_3 \\ \\ &= \begin{bmatrix} 2 \\ 0 \\ 1 \end{bmatrix} - \dfrac{\frac{1}{\sqrt{5}}(2) + \frac{2}{\sqrt{5}}(0) + 0(1)}{1}\begin{bmatrix} \frac{1}{\sqrt{5}} \\ \frac{2}{\sqrt{5}} \\ 0 \end{bmatrix} - \dfrac{\frac{4}{3\sqrt{5}}(2) -\frac{2}{3\sqrt{5}}(0) + \frac{5}{3\sqrt{5}}(1) }{1}\begin{bmatrix} \frac{4}{3\sqrt{5}} \\ -\frac{2}{3\sqrt{5}} \\ \frac{5}{3\sqrt{5}} \end{bmatrix} \\ \\ &= \begin{bmatrix} 2 \\ 0 \\ 1 \end{bmatrix} - \dfrac{2}{\sqrt{5}}\begin{bmatrix} \frac{1}{\sqrt{5}} \\ \frac{2}{\sqrt{5}} \\ 0 \end{bmatrix} - \dfrac{13}{3\sqrt{5}}\begin{bmatrix} \frac{4}{3\sqrt{5}} \\ -\frac{2}{3\sqrt{5}} \\ \frac{5}{3\sqrt{5}} \end{bmatrix} \\ \\ &= \begin{bmatrix} 2 \\ 0 \\ 1 \end{bmatrix} - \begin{bmatrix} \frac{2}{5} \\ \frac{4}{5} \\ 0 \end{bmatrix} - \begin{bmatrix} \frac{52}{45} \\ -\frac{26}{45} \\ \frac{65}{45} \end{bmatrix} = \begin{bmatrix} 2 \\ 0 \\ 1 \end{bmatrix} - \begin{bmatrix} \frac{70}{45} \\ \frac{10}{45} \\ \frac{65}{45} \end{bmatrix} = \begin{bmatrix} \frac{20}{45} \\ -\frac{10}{45} \\ -\frac{20}{45} \end{bmatrix} = \begin{bmatrix} \frac{4}{9} \\ -\frac{2}{9} \\ -\frac{4}{9} \end{bmatrix} \\ \end{align*} $$

2. Normalize

$$ \mathbf{q}_3 = \dfrac{\mathbf{w}_3}{\|\mathbf{w}_3\|} = \dfrac{1}{\sqrt{\frac{16}{81} + \frac{4}{81} + \frac{16}{81}}}\begin{bmatrix} \frac{4}{9} \\ -\frac{2}{9} \\ -\frac{4}{9} \end{bmatrix} = \dfrac{3}{2}\begin{bmatrix} \frac{4}{9} \\ -\frac{2}{9} \\ -\frac{4}{9} \end{bmatrix} = \begin{bmatrix} \frac{2}{3} \\ -\frac{1}{3} \\ -\frac{2}{3} \end{bmatrix} $$

3. There are no more vectors; done.

This results in a new orthonormal basis basis

$$ \{\mathbf{q}_1, \mathbf{q}_2, \mathbf{q}_3\} = \left\{ \begin{bmatrix} \frac{1}{\sqrt{5}} \\ \frac{2}{\sqrt{5}} \\ 0 \end{bmatrix}, \begin{bmatrix} \frac{4}{3\sqrt{5}} \\ -\frac{2}{3\sqrt{5}} \\ \frac{5}{3\sqrt{5}} \end{bmatrix}, \begin{bmatrix} \frac{2}{3} \\ -\frac{1}{3} \\ -\frac{2}{3} \end{bmatrix}\right\} $$

---

5.3b.5 QR Factorization

There are some other methods for computing QR. One is in the textbook; however, its wastes time and requires repeating $\dfrac{n(n-1)}{2}$ computations. The majority of the other methods either

• don't work all of the time
• are for computers and not people because they are to computationally intensive

As in the Gram-Schmidt Orthogonalization Process

Don't memorize the formula; memorize the process.

You have a random $m\times n$ matrix $A$ with $n$ linearly independent columns $A = \begin{bmatrix} \mathbf{a}_1,\ \mathbf{a}_2,\ \dots,\ \mathbf{a}_n \end{bmatrix}$. To create the QR factorization you:

1. Subtract the projections onto all previous orthogonal vectors.
2. Divide the result by the norm to make the result a unit vector.
3. Solve for original vector to get the row for matrix $R$ of coefficients
4. Repeat.

When you look at the formulas below remember that you will be filling in all of the formulas with numbers as you proceed.

First vector

For your first vector, there are no previous orthogonal vectors ($\mathbf{q}$'s) so

1. No previous orthogonal vectors to project onto $\mathbf{w}_1 = \mathbf{a}_1 - \mathbf{0} = \mathbf{a}_1$
2. Divide the result by the norm $$ \mathbf{q}_1 = \frac{\mathbf{w}_1}{\left\| \mathbf{w}_1 \right\|} = \frac{\mathbf{a}_1}{\left\| \mathbf{a}_1 \right\|} \qquad\longleftarrow \left\|\mathbf{a}_1\right\| = \left\| \mathbf{w}_1\right\| \text{ is the first diagonal element of }R $$
3. Solve for $\mathbf{a}_1$ to get the first row for matrix $R$ of coefficients $$ \begin{align*} \mathbf{a}_1 &= \left\| \mathbf{a}_1 \right\|\mathbf{q}_1 = \left\| \mathbf{a}_1 \right\|\mathbf{q}_1 + 0\mathbf{q}_2 + \cdots + 0\mathbf{q}_n \\ \\ \text{first column of }R,\ \mathbf{r}_1 &= \begin{bmatrix}\ \left\| \mathbf{a}_1 \right\|\ \\ \ 0\ \\ \ \ddots\ \\ \ 0\ \end{bmatrix} \end{align*} $$
4. Repeat

Second vector

1. Subtract projections onto all previous (one) orthogonal vectors $$ \mathbf{w}_2 = \mathbf{a}_2 - \textrm{Proj}_{\mathbf{q}_1}\mathbf{v}_2 $$
2. Divide the result by the norm $$ \mathbf{q}_2 = \frac{\mathbf{w}_2}{\left\|\mathbf{w}_2\right\|} \qquad\longleftarrow \left\|\mathbf{w}_2\right\| \text{ is the second diagonal element of }R $$
3. Solve for $\mathbf{a}_2$ to get the second row for matrix $R$ of coefficients $$ \begin{align*} \mathbf{w}_2 &= \left\| \mathbf{w}_2 \right\|\mathbf{q}_2 \\ \\ \mathbf{a}_2 - \textrm{Proj}_{\mathbf{q}_1}\mathbf{a}_2 &= \left\| \mathbf{w}_2 \right\|\mathbf{q}_2 \\ \\ \mathbf{a}_2 &= \textrm{Proj}_{\mathbf{q}_1}\mathbf{a}_2 + \left\| \mathbf{w}_2 \right\|\mathbf{q}_2 \\ \\ &\text{You should already know the formula for the projection} \\ \\ &= \frac{\langle \mathbf{a}_2, \mathbf{q}_1 \rangle}{\langle \mathbf{q}_1,\mathbf{q}_1 \rangle}\mathbf{q}_1 + \left\| \mathbf{w}_2 \right\|\mathbf{q}_2 + \cdots + 0\mathbf{q}_n \\ \\ &= \langle \mathbf{a}_2, \mathbf{q}_1 \rangle\mathbf{q}_1 + \left\| \mathbf{w}_2 \right\|\mathbf{q}_2 + \cdots + 0\mathbf{q}_n \\ \\ \langle \mathbf{q}_1,\mathbf{q}_1 \rangle &= 1 \text{ since } \mathbf{q}_1 \text{ is a unit vector} \\ \\ \text{second column of }R,\ \mathbf{r}^2 &= \begin{bmatrix}\ \langle \mathbf{a}_2, \mathbf{q}_1 \rangle\ \\ \ \left\| \mathbf{w}_2 \right\|\ \\ \ 0\ \\ \ \ddots\ \\ \ 0\ \end{bmatrix} \end{align*} $$
4. Repeat

Third vector

1. Subtract projections onto all previous (two) orthogonal vectors $$ \mathbf{w}_3 = \mathbf{a}_3 - \textrm{Proj}_{\mathbf{q}_1}\mathbf{a}_3 - \textrm{Proj}_{\mathbf{q}_2}\mathbf{a}_3 $$
2. Divide the result by the norm $$ \mathbf{q}_3 = \frac{\mathbf{w}_3}{\left\|\mathbf{w}_3\right\|}\qquad\longleftarrow \left\|\mathbf{w}_3\right\| \text{ is the third diagonal element of }R $$
3. Solve for $\mathbf{v}_3$ to get the third row for matrix $R$ of coefficients $$ \begin{align*} \mathbf{w}_3 &= \left\| \mathbf{w}_3 \right\|\mathbf{q}_3 \\ \\ \mathbf{a}_3 - \textrm{Proj}_{\mathbf{q}_1}\mathbf{a}_3 - \textrm{Proj}_{\mathbf{q}_2}\mathbf{a}_3 &= \left\| \mathbf{w}_3 \right\|\mathbf{q}_3 \\ \\ \mathbf{a}_3 &= \textrm{Proj}_{\mathbf{q}_1}\mathbf{a}_3 + \textrm{Proj}_{\mathbf{q}_2}\mathbf{a}_3 + \left\| \mathbf{w}_3 \right\|\mathbf{q}_3 \\ \\ &\text{You should already know the formula for the projection} \\ \\ &= \frac{\langle \mathbf{a}_3, \mathbf{q}_1 \rangle}{\langle \mathbf{q}_1,\mathbf{q}_1 \rangle}\mathbf{q}_1 + \frac{\langle \mathbf{a}_3, \mathbf{q}_2 \rangle}{\langle \mathbf{q}_2,\mathbf{q}_2 \rangle}\mathbf{q}_2 + \left\| \mathbf{w}_3 \right\|\mathbf{q}_3 + \cdots + 0\mathbf{q}_n \\ \\ &= \langle \mathbf{a}_3, \mathbf{q}_1 \rangle\mathbf{q}_1 + \langle \mathbf{a}_3, \mathbf{q}_2 \rangle\mathbf{q}_2 + \left\| \mathbf{w}_3 \right\|\mathbf{q}_3 + \cdots + 0\mathbf{q}_n \\ \\ \langle \mathbf{q}_1,\mathbf{q}_1 \rangle = \langle \mathbf{q}_2,\mathbf{q}_2 \rangle &= 1 \text{ since } \mathbf{q}_1 \text{ and } \mathbf{q}_2 \text{ are unit vectors} \\ \\ \text{third column of }R,\ \mathbf{r}_3 &= \begin{bmatrix}\ \langle \mathbf{a}_3, \mathbf{q}_1 \rangle\ \\ \ \langle \mathbf{a}_3, \mathbf{q}_2 \rangle\ \\ \ \left\| \mathbf{w}_3 \right\|\ \\ \ 0\ \\ \ \ddots\ \\ \ 0\ \end{bmatrix} \end{align*} $$
4. Repeat

Do you see the pattern?

$k^{\text{th}}$ vector

1. Subtract projections onto all previous ($k-1$) orthogonal vectors $$ \mathbf{w}_k = \mathbf{a}_k - \textrm{Proj}_{\mathbf{q}_1}\mathbf{a}_k - \dots - \textrm{Proj}_{\mathbf{q}_{k-1}}\mathbf{a}_k $$
2. Divide the result by the norm $$ \mathbf{q}_k = \frac{\mathbf{w}_k}{\left\|\mathbf{w}_k\right\|}\qquad\longleftarrow\left\|\mathbf{w}\right\| \text{ is the }k^{\text{th}}\text{ diagonal element of }R $$
3. Solve for $\mathbf{v}_k$ to get the $k^{\text{th}}$ row for matrix $R$ of coefficients $$ \begin{align*} \mathbf{w}_k &= \left\| \mathbf{w}_k \right\|\mathbf{q}_k \\ \\ \mathbf{a}_k - \textrm{Proj}_{\mathbf{q}_1}\mathbf{a}_k - \dots - \textrm{Proj}_{\mathbf{q}_{k-1}}\mathbf{a}_k &= \left\| \mathbf{w}_k \right\|\mathbf{q}_k \\ \\ \mathbf{a}_k &= \textrm{Proj}_{\mathbf{q}_1}\mathbf{a}_k + \dots + \textrm{Proj}_{\mathbf{q}_{k-1}}\mathbf{a}_k + \left\| \mathbf{w}_k \right\|\mathbf{q}_k \\ \\ &\text{You should already know the formula for the projection} \\ \\ &= \frac{\langle \mathbf{a}_k, \mathbf{q}_1 \rangle}{\langle \mathbf{q}_1,\mathbf{q}_1 \rangle}\mathbf{q}_1 + \cdots + \frac{\langle \mathbf{a}_k, \mathbf{q}_{k-1} \rangle}{\langle \mathbf{q}_{k-1},\mathbf{q}_{k-1} \rangle}\mathbf{q}_k + \left\| \mathbf{w}_k \right\|\mathbf{q}_k + \cdots + 0\mathbf{q}_n \\ \\ &= \langle \mathbf{a}_k, \mathbf{q}_1 \rangle\mathbf{q}_1 + \cdots + \langle \mathbf{a}_k, \mathbf{q}_{k-1} \rangle\mathbf{q}_k + \left\| \mathbf{w}_k \right\|\mathbf{q}_k + \cdots + 0\mathbf{q}_n \\ \\ \langle \mathbf{q}_j,\mathbf{q}_j \rangle &= 1 \text{ since all of the } \mathbf{q}_j \text{'s are unit vectors} \\ \\ k^{\text{th}}\text{ column of }R, \mathbf{r}^k &= \begin{bmatrix}\ \langle \mathbf{a}_k, \mathbf{q}_1 \rangle\ \\ \ddots\ \\ \ \langle \mathbf{a}_k, \mathbf{q}_{k-1} \rangle\ \\ \ \left\| \mathbf{w}_k \right\|\ \\ \ 0\ \\ \ \ddots\ \\ \ 0\ \end{bmatrix} \end{align*} $$
4. Repeat

$\ddots$

Last vector

1. Subtract projections onto all previous ($n-1$) orthogonal vectors $$ \mathbf{w}_n = \mathbf{a}_n - \textrm{Proj}_{\mathbf{q}_1}\mathbf{a}_n - \dots - \textrm{Proj}_{\mathbf{q}_{n-1}}\mathbf{a}_n $$
2. Divide the result by the norm $$ \mathbf{q}_n = \frac{\mathbf{w}_n}{\left\|\mathbf{w}_n\right\|} \qquad\longleftarrow\left\|\mathbf{w}_n\right\|\text{ is the last diagonal element of }R $$
3. Solve for $\mathbf{a}_n$ to get the $n^{\text{th}}$ (last) row for matrix $R$ of coefficients $$ \begin{align*} \mathbf{w}_n &= \left\| \mathbf{w}_n \right\|\mathbf{q}_n \\ \\ \mathbf{a}_n - \textrm{Proj}_{\mathbf{q}_1}\mathbf{a}_n - \dots - \textrm{Proj}_{\mathbf{q}_{n-1}}\mathbf{a}_n &= \left\| \mathbf{w}_n \right\|\mathbf{q}_n \\ \\ \mathbf{a}_n &= \textrm{Proj}_{\mathbf{q}_1}\mathbf{a}_n + \dots + \textrm{Proj}_{\mathbf{q}_{n-1}}\mathbf{a}_n + \left\| \mathbf{w}_n \right\|\mathbf{q}_n \\ \\ &\text{You should already know the formula for the projection} \\ \\ &= \frac{\langle \mathbf{a}_n, \mathbf{q}_1 \rangle}{\langle \mathbf{q}_1,\mathbf{q}_1 \rangle}\mathbf{q}_1 + \cdots + \frac{\langle \mathbf{a}_n, \mathbf{q}_{n-1} \rangle}{\langle \mathbf{q}_{n-1},\mathbf{q}_{n-1} \rangle}\mathbf{q}_n + \left\| \mathbf{w}_n \right\|\mathbf{q}_n \\ \\ &= \langle \mathbf{a}_n, \mathbf{q}_1 \rangle\mathbf{q}_1 + \cdots + \langle \mathbf{a}_n, \mathbf{q}_{n-1} \rangle\mathbf{q}_n + \left\| \mathbf{w}_n \right\|\mathbf{q}_n \\ \\ \langle \mathbf{q}_j,\mathbf{q}_j \rangle &= 1 \text{ since all of the } \mathbf{q}_j \text{'s are unit vectors} \\ \\ \text{last column of }R, \mathbf{r}_n &= \begin{bmatrix}\ \langle \mathbf{a}_n, \mathbf{q}_1 \rangle\ \\ \ \ddots\ \\ \ \langle \mathbf{a}_n, \mathbf{q}_{n-1} \rangle\ \\ \ \left\| \mathbf{w}_n \right\|\ \end{bmatrix} \end{align*} $$
4. No more columns of matrix $A$ - done

$$ \begin{align*} Q &= \begin{bmatrix} \mathbf{q}_1 & \mathbf{q}_2 & \cdots & \mathbf{q}_n \end{bmatrix} \\ \\ R &= \begin{bmatrix} \mathbf{r}_1 & \mathbf{r}_2 & \cdots & \mathbf{r}_n \end{bmatrix} \\ \\ &= \begin{bmatrix}\ \left\| \mathbf{a}_1 \right\|\ &\ \langle \mathbf{a}_2, \mathbf{q}_1 \rangle\ &\ \cdots\ &\ \langle \mathbf{a}_k, \mathbf{q}_1 \rangle\ &\ \cdots\ &\ \langle \mathbf{a}_{n-1}, \mathbf{q}_1 \rangle\ &\ \langle \mathbf{a}_n, \mathbf{q}_1 \rangle\ \\ \ 0\ & \left\| \mathbf{w}_2 \right\|\ &\ \cdots\ &\ \langle \mathbf{a}_k, \mathbf{q}_2 \rangle\ &\ \cdots\ &\ \langle \mathbf{a}_{n-1}, \mathbf{q}_2 \rangle\ &\ \langle \mathbf{a}_n, \mathbf{q}_2 \rangle\ \\ \ \ddots\ &\ \ddots\ &\ \ddots\ &\ \ddots\ &\ \ddots\ &\ \ddots\ &\ \ddots\ \\ \ 0\ &\ \ddots\ &\ 0\ &\ \left\| \mathbf{w}_k \right\|\ &\ \cdots\ &\ \langle \mathbf{a}_{n-1}, \mathbf{q}_k \rangle\ &\ \langle \mathbf{a}_n, \mathbf{q}_k \rangle\ \\ \ \ddots\ &\ \ddots\ &\ \ddots\ &\ \ddots\ &\ \ddots\ &\ \ddots\ &\ \ddots\ \\ \ 0\ &\ \ddots\ &\ \ddots\ &\ 0\ &\ \ddots\ &\ \left\|\mathbf{w}_{n-1}\right\|\ &\ \langle \mathbf{a}_n, \mathbf{q}_{n-1} \rangle \\ \ 0\ &\ \cdots\ &\ \cdots\ &\ 0\ &\ \cdots\ &\ 0\ &\ \left\| \mathbf{w}_n \right\|\ \end{bmatrix} \end{align*} $$

Notice

• $Q$ is an $m\times n$ matrix; the same dimensions as $A$
• The $\mathbf{q}_j$'s are the basis vectors from Gram-Schmidt
• $R$ is a square $n\times n$ matrix = number of columns of $A$.
• $R$ is upper triangular
• The diagonal elements of $R$ are the norms of the $\mathbf{w}_k$'s
• The upper triangle of $R$ are the components of each column of matrix $A$ with the $\mathbf{q}_j$'s that have been computed so far.
• You don't see denominators in the components because they are all $1$'s

In the textbook method, the denominators are not $1$'s, so it looks a lot more complicated. On page 259 of the textbook in Theorem 5.12, you compute the denominators for projection onto the previous $\mathbf{w}$'s, and then compute them again to normalize each $\mathbf{w}$ to get your orthogonal vector $\mathbf{q}$. This is the \dfrac{n(n-1)}{2} repeated computations.

---

5.3b.6 QR Factorization

In many STEM applications, the mathematical model of the constraints of a linear system does not change. Instead the current state of the linear system, or inputs change, and the model is used to extract important information from the inputs. Solving the linear system is performed repeatedly using new input date each iteration. Simplifying the process by factoring the matrices involved speeds up the process. The values of the factorization can also shed light on the mathematical model.

The pattern for the process

You have a random non-orthogonal matrix $A = \left\{ \mathbf{a}_1,\ \mathbf{a}_2,\ \dots,\ \mathbf{a}_n \right\}$. To create the QR factorization:

For each vector $\mathbf{a}_k$ in the basis

1. Subtract the projections onto all previous orthogonal vectors.
2. Divide the result by the norm to make the result a unit vector to get a column $\mathbf{q}$ of matrix $Q$.
3. Solve the result for $\mathbf{a}_k$ to obtain a column of matrix $R$.
4. Repeat.

Example 6

Consider the matrix

$$ A = \begin{bmatrix}\ 1\ &\ \ 0\ & -3\ \\ \ 1\ & -4\ &\ \ 5\ \\ \ 1\ &\ \ 0\ &\ \ 2\ \\ \ 1\ & -4\ &\ \ 0\ \end{bmatrix} $$

Verify for yourself that the columns of matrix $A$ are linearly independent. Thus the rank$(A) = 3$ and the columns of $A$ form a basis for a three dimensional subspace of $\mathbb{R}^4$. We can use the Gram-Schmidt orthogonalization process to obtain an orthonormal basis for the column space of matrix $A$,

$$ C(A) = \text{span}\{\mathbf{a}_1,\ \mathbf{a}_2,\ \mathbf{a}_3\} = \text{span}\{\mathbf{q}_1,\ \mathbf{q}_2,\ \mathbf{q}_3\} $$

The orthonormal column vectors $\{\mathbf{q}_1, \mathbf{q}_2, \mathbf{q}_3\}$ form a new matrix

$$ Q = \begin{bmatrix} \mathbf{q}_1 & \mathbf{q}_2 & \mathbf{q}_3 \end{bmatrix} $$

First Vector

1. There are no previous orthogonal vectors.

$$ \mathbf{w}_1 = \mathbf{a}_1 = \begin{bmatrix}\ 1\ \\ \ 1\ \\ \ 1\ \\ \ 1\ \end{bmatrix} $$

2. Normalize $\mathbf{w}_1$ $$ \mathbf{q}_1 = \dfrac{\mathbf{w}_1}{\|\mathbf{w}_1\|} = \dfrac{\mathbf{a}_1}{\|\mathbf{a}_1\|} $$

Performing the computations yields

$$ \begin{align*} r_{11} &= \|\mathbf{a}_1\| = \sqrt{1^2 + 1^2 + 1^2 + 1^2} = \sqrt{4} = 2 \\ \\ \mathbf{q}_1 &= \frac{1}{r_{11}}\mathbf{a}_1 = \frac{1}{2}\begin{bmatrix}\ 1\ \\ \ 1\ \\ \ 1\ \\ \ 1\ \end{bmatrix} = \begin{bmatrix} 1/2 \\ 1/2 \\ 1/2 \\ 1/2 \end{bmatrix} \end{align*} $$

3. Solve the equation for $\mathbf{w}_1$ for $\mathbf{a}_2$ to get a linear combination.

$$ \mathbf{a}_1 = \|\mathbf{a}_1\|\mathbf{q}_1 = r_{11}\mathbf{q}_1 + 0\mathbf{q}_2 + 0\mathbf{q}_3 = \begin{bmatrix} \mathbf{q}_1 & \mathbf{q}_2 & \mathbf{q}_3 \end{bmatrix}\begin{bmatrix} r_{11} \\ 0 \\ 0 \end{bmatrix} $$

Performing the computations one obtains

$$ \begin{align*} \mathbf{a}_1 &= r_{11}\mathbf{q}_1 = \| \mathbf{a}_1 \|\mathbf{q}_1 = 2\begin{bmatrix} \frac{1}{2} \\ \frac{1}{2} \\ \frac{1}{2} \\ \frac{1}{2} \end{bmatrix} + 0\mathbf{q}_2 + 0\mathbf{q}_2 \\ \\ &= 2\mathbf{q}_1 + 0\mathbf{q}_2 + 1\mathbf{q}_2 = \begin{bmatrix} \mathbf{q}_1 & \mathbf{q}_2 & \mathbf{q}_3 \end{bmatrix} \begin{bmatrix} 2 \\ 0 \\ 0 \end{bmatrix} = Q\mathbf{r}_1 \end{align*} $$

Second Column of Matrix $A$

1. Subtract projections of $\mathbf{a}_2$ onto all previous (one) orthogonal vectors.

$$ \mathbf{w}_2 = \mathbf{a}_2 - \textrm{Proj}_{\mathbf{q}_1}\mathbf{a}_2 = \mathbf{a}_2 - \dfrac{\langle\mathbf{a}_2, \mathbf{q}_1\rangle}{\langle\mathbf{q}_1, \mathbf{q}_1\rangle}\mathbf{q}_1 = \mathbf{a}_2 - \langle\mathbf{a}_2, \mathbf{q}_1\rangle\mathbf{q}_1 $$

Performing the computations

$$ \begin{align*} \langle\mathbf{a}_2, \mathbf{q}_1\rangle &= 0\left(\frac{1}{2}\right)- 4\left(\frac{1}{2}\right) + 0\left(\frac{1}{2}\right)- 4\left(\frac{1}{2}\right) = -4 \\ \\ \mathbf{w}_2 &= \begin{bmatrix}\ \ 0\ \\ -4\ \\ \ \ 0\ \\ -4\ \end{bmatrix} - (-4)\begin{bmatrix} 1/2 \\ 1/2 \\ 1/2 \\ 1/2 \end{bmatrix} = \begin{bmatrix}\ \ 0\ \\ -4\ \\ \ \ 0\ \\ -4\ \end{bmatrix} + \begin{bmatrix}\ 2\ \\ \ 2\ \\ \ 2\ \\ \ 2\ \end{bmatrix} = \begin{bmatrix}\ \ 2\ \\ -2\ \\ \ \ 2\ \\ -2\ \end{bmatrix} \end{align*} $$

3. Normalize the result

$$ \mathbf{q}_2 = \frac{\mathbf{w}_2}{\| \mathbf{w}_2 \|} = \frac{\mathbf{w}_2}{r_{22}} $$

The computations give us

$$ \begin{align*} r_{22} &= \| \mathbf{w}_2 \| = \sqrt{ 2^2 + (-2)^2 + 2^2 + (-2)^2 } = \sqrt{16} = 4 \\ \\ \mathbf{q}_2 &= \dfrac{1}{4}\begin{bmatrix}\ \ 2\ \\ -2\ \\ \ \ 2\ \\ -2\ \end{bmatrix} = \begin{bmatrix}\ \ \frac{1}{2}\ \\ -\frac{1}{2}\ \\ \ \ \frac{1}{2}\ \\ -\frac{1}{2}\ \end{bmatrix} \end{align*} $$

3. Solve the last equation for $\mathbf{a}_2$

$$ \begin{align*} r_{22}\mathbf{q}_2 &= \mathbf{w}_2 = \mathbf{a}_2 - \langle \mathbf{a}_2,\mathbf{q}_1 \rangle\mathbf{q}_1 \\ \\ \mathbf{a}_2 &= \langle \mathbf{a}_2,\mathbf{q}_1 \rangle\mathbf{q}_1 + r_{22}\mathbf{q}_2 = \langle \mathbf{a}_2,\mathbf{q}_1 \rangle\mathbf{q}_1 + r_{22}\mathbf{q}_2 + 0\mathbf{q}_3 \\ \\ &= \begin{bmatrix}\ \mathbf{q}_1\ &\ \mathbf{q}_2\ &\ \mathbf{q}_3\ \end{bmatrix}\begin{bmatrix} r_{12} \\ r_{22} \\ 0 \end{bmatrix} = Q\mathbf{r}_2 \end{align*} $$

Computationally

$$ \begin{align*} \|\mathbf{w}_2\|\mathbf{q}_2 &= \mathbf{a}_2 - \langle \mathbf{a}_2,\mathbf{q}_1 \rangle\mathbf{q}_1 \\ \\ 4\mathbf{q}_2 &= \mathbf{a}_2 - (-4)\mathbf{q}_1 \\ \\ \mathbf{a}_2 &= -4\mathbf{q}_1 + 4\mathbf{q}_2 = -4\mathbf{q}_1 + 4\mathbf{q}_2 + 0\mathbf{q}_3 \\ \\ &= \begin{bmatrix}\ \mathbf{q}_1\ &\ \mathbf{q}_2\ &\ \mathbf{q}_3\ \end{bmatrix} \begin{bmatrix} -4\ \\ \ \ 4\ \\ \ \ 0\ \end{bmatrix} = Q\mathbf{r}_2 \end{align*} $$

Last column of matrix $A$

1. Subtract projections onto all previous (two) orthogonnal vectors

$$ \begin{align*} \mathbf{w}_3 &= \mathbf{a}_3 - \textrm{Proj}_{\mathbf{q}_1}\mathbf{a}_3 - \textrm{Proj}_{\mathbf{q}_2}\mathbf{a}_3 \\ \\ &= \mathbf{a}_3 - \frac{\langle \mathbf{a}_3,\mathbf{q}_1 \rangle}{\langle \mathbf{q}_1,\mathbf{q}_1 \rangle}\mathbf{q}_1 - \frac{\langle \mathbf{a}_3,\mathbf{q}_2 \rangle}{\langle \mathbf{q}_2,\mathbf{q}_2 \rangle}\mathbf{q}_2 \\ \\ &= \mathbf{a}_3 - \langle \mathbf{a}_3,\mathbf{q}_1 \rangle\mathbf{q}_1 - \langle \mathbf{a}_3,\mathbf{q}_2 \rangle\mathbf{q}_2 \end{align*} $$

Performing the computation yields

$$ \begin{align*} \langle \mathbf{a}_3,\mathbf{q}_1 \rangle &= -3\left(\frac{1}{2}\right) + 5\left(\frac{1}{2}\right) + 2\left(\frac{1}{2}\right) + 0\left(\frac{1}{2}\right) = 2 \\ \\ \langle \mathbf{a}_3,\mathbf{q}_2 \rangle &= -3\left(\frac{1}{2}\right) + 5\left(-\frac{1}{2}\right) + 2\left(\frac{1}{2}\right) + 0\left(-\frac{1}{2}\right) = -3 \\ \\ \mathbf{w}_3 &= \begin{bmatrix} -3\ \\ \ \ 5\ \\ \ \ 2\ \\ \ \ 0\ \end{bmatrix} - 2\begin{bmatrix} \frac{1}{2} \\ \frac{1}{2} \\ \frac{1}{2} \\ \frac{1}{2} \end{bmatrix} - (-3)\begin{bmatrix}\ \ \frac{1}{2}\ \\ -\frac{1}{2}\ \\ \ \ \frac{1}{2}\ \\ -\frac{1}{2}\ \end{bmatrix} \\ \\ &= \begin{bmatrix} -3\ \\ \ \ 5\ \\ \ \ 2\ \\ \ \ 0\ \end{bmatrix} - \begin{bmatrix}\ 1\ \\ \ 1\ \\ \ 1\ \\ \ 1\ \end{bmatrix} + \begin{bmatrix}\ \ \frac{3}{2}\ \\ -\frac{3}{2}\ \\ \ \ \frac{3}{2}\ \\ -\frac{3}{2}\ \end{bmatrix} = \begin{bmatrix} -\frac{5}{2} \\ \ \ \frac{5}{2} \\ \ \ \frac{5}{2} \\ -\frac{5}{2} \end{bmatrix} \end{align*} $$

2. Normalize $\mathbf{w}_3$.

$$ \begin{align*} r_{33} &= \| \mathbf{w}_3 \| \\ \\ \mathbf{q}_3 &= \dfrac{\mathbf{w}_3}{\|\mathbf{w}_3\|} \end{align*} $$

Our computations give us

$$ \begin{align*} r_{33} &= \sqrt{\left(-\frac{5}{2}\right)^2 + \left(\frac{5}{2}\right)^2 + \left(\frac{5}{2}\right)^2 + \left(-\frac{5}{2}\right)^2} = 5 \\ \\ \mathbf{q}_3 &= \dfrac{1}{5}\begin{bmatrix} -\frac{5}{2} \\ \ \ \frac{5}{2} \\ \ \ \frac{5}{2} \\ -\frac{5}{2} \end{bmatrix} = \begin{bmatrix} -\frac{1}{2} \\ \ \ \frac{1}{2} \\ \ \ \frac{1}{2} \\ -\frac{1}{2} \end{bmatrix} \end{align*} $$

3. Solve for $\mathbf{a}_3$

$$ \begin{align*} r_{33}\mathbf{q}_3 &= \mathbf{w}_3 = \mathbf{a}_3 - \langle \mathbf{a}_3,\mathbf{q}_1 \rangle\mathbf{q}_1 - \langle \mathbf{a}_3,\mathbf{q}_2 \rangle\mathbf{q}_2 \\ \\ \mathbf{a}_3 &= \langle \mathbf{a}_3,\mathbf{q}_1 \rangle\mathbf{q}_1 + \langle \mathbf{a}_3,\mathbf{q}_2 \rangle\mathbf{q}_2 + r_{33}\mathbf{q}_3 \\ \\ \mathbf{a}_3 &= \begin{bmatrix} \mathbf{q}_1 & \mathbf{q}_2 & \mathbf{q}_3 \end{bmatrix} \begin{bmatrix}\ \langle \mathbf{a}_3,\mathbf{q}_1 \rangle\ \\ \ \langle \mathbf{a}_3,\mathbf{q}_2 \rangle\ \\ \langle \mathbf{w}_3,\mathbf{w}_3 \rangle \end{bmatrix} = Q\mathbf{r}_3 \end{align*} $$

Performing the computations

$$ \begin{align*} \mathbf{a}_3 &= 2\mathbf{q}_1 - 3\mathbf{q}_2 + 5\mathbf{q}_3 = \begin{bmatrix} \mathbf{q}_1 & \mathbf{q}_2 & \mathbf{q}_3 \end{bmatrix}\begin{bmatrix}\ \ 2\ \\ -3\ \\ \ \ 5\ \end{bmatrix} \end{align*} $$

Putting these three matrix-vector computations together

$$ \begin{align*} Q &= \begin{bmatrix} \mathbf{q}_1 & \mathbf{q}_2 & \mathbf{q}_3 \end{bmatrix} = \begin{bmatrix}\ \ \frac{1}{2}\ &\ \ \frac{1}{2}\ & -\frac{1}{2}\ \\ \ \ \frac{1}{2}\ & -\frac{1}{2}\ &\ \ \frac{1}{2}\ \\ \ \ \frac{1}{2}\ &\ \ \frac{1}{2}\ &\ \ \frac{1}{2}\ \\ \ \ \frac{1}{2}\ & -\frac{1}{2}\ & -\frac{1}{2}\ \end{bmatrix} \\ \\ R &= \begin{bmatrix} \mathbf{r}_1 & \mathbf{r}_2 & \mathbf{r}_3 \end{bmatrix} = \begin{bmatrix}\ \ 2\ & -4\ &\ \ 2\ \\ \ \ 0\ &\ \ 4\ & -3\ \\ \ \ 0\ &\ \ 0\ &\ \ 5\ \end{bmatrix} \\ \\ A &= QR \end{align*} $$

This is the $QR$ factorization of matrix $A$, the decomposition of a matrix into a product of a matrix $Q$ with orthonormal column vectors and a square matrix $R$ of coefficients of the columns of matrix $Q$.

---

5.3b.5 Gram-Schmidt or QR Factorization

Recall from Theorem 5.3.5 that if $Q$ is an orthogonal matrix, then $Q$ is square, and $\textrm{det}(Q)=\pm 1$.

Theorem 5.3b.2

The Gram-Schmidt or QR Factorization Theorem

If $A\in\mathbb{R}^{m\times n}$ is an $m\times n$ matrix, then $A$ can be factored into a product $QR$, where $Q$ is an $m\times n$ matrix with orthonormal column vectors, and $R$ is an upper triangular $n\times n$ matrix.

Notice that if $A$ is a square $n\times n$ matrix, then $Q$ is also square, and

$$ |A| = |QR| = |Q||R| = \pm\,|R| = \pm\displaystyle\prod_{k=1}^n r_{kk} $$

Corollary 5.3b.3

If $A\in\mathbb{R}^{m\times n}$ is an $m\times n$ matrix with rank $n$, then $A$ can be factored into a product $QR$, where $Q$ is an $m\times n$ matrix with orthonormal column vectors, and $R$ is an upper triangular $n\times n$ matrix with positive entries along the diagonal.

Notice that the theorem requires,

1. Matrix $A$ must be full rank, that is all of the columns of matrix $A$ must be linearly independent.

2. The coefficient matrix $R$ will always be nonsingular since the determinant of $R$ is a product of nonzero numbers.

This gives us another way to spot a matrix with a free column. If one of the diagonal elements of matrix $R$ is zero, then the corresponding column of matrix $A$ is a free column. Any linear system $A\mathbf{x}=\mathbf{b}$ will have infinitely many solutions, or no solutions.

---

5.3b.7 Exercises

Exercise 1

Consider the following matrix $$ A = \begin{bmatrix}\ 1\ &\ \ 0\ & -3\ \\ \ 1\ & -4\ &\ \ 5\ \\ \ 1\ &\ \ 0\ &\ \ 2\ \\ \ 1\ & -4\ &\ \ 0\ \end{bmatrix} $$

Compute the $QR$ factorization of matrix $A$.

Check Your Work

$$ \begin{align*} Q &= \begin{bmatrix}\ \ \frac{1}{2}\ &\ \ \frac{1}{2}\ & -\frac{1}{2}\ \\ \ \ \frac{1}{2}\ & -\frac{1}{2}\ &\ \ \frac{1}{2}\ \\ \ \ \frac{1}{2}\ &\ \ \frac{1}{2}\ &\ \ \frac{1}{2}\ \\ \ \ \frac{1}{2}\ & -\frac{1}{2}\ & -\frac{1}{2}\ \end{bmatrix} \\ \\ R &= \begin{bmatrix}\ \ 2\ & -4\ &\ \ 2\ \\ \ \ 0\ &\ \ 4\ & -3\ \\ \ \ 0\ &\ \ 0\ &\ \ 5\ \end{bmatrix} \end{align*} $$

Follow Along

$$ \begin{align*} r_{11} &= \sqrt{ 1^2 + 1^2 + 1^2 + 1^2 } = 2 \\ \\ \mathbf{q}_1 &= \frac{\mathbf{a}_1}{\left\|\mathbf{a}_1\right\|} = \frac{\mathbf{a}_1}{r_{11}} = \frac{1}{2}\begin{bmatrix}\ 1\ \\ \ 1\ \\ \ 1\ \\ \ 1\ \end{bmatrix} \\ \mathbf{r}_1 &= \begin{bmatrix}\ 2\ \\ \ 0\ \\ \ 0\ \end{bmatrix} \\ \\ \mathbf{p}_2 &= \text{Proj}_{\mathbf{q}_1}\mathbf{a}_2 = \frac{\langle\mathbf{q}_1,\mathbf{a}_2\rangle}{\langle\mathbf{q_1},\mathbf{q}_1\rangle}\mathbf{q}_1 = \frac{r_{12}}{1}\mathbf{q}_1 = \left(0\cdot\frac{1}{2} - 4\cdot\frac{1}{2} + 0\cdot\frac{1}{2} - 4\cdot\frac{1}{2} \right)\begin{bmatrix}\ \frac{1}{2}\ \\ \ \frac{1}{2}\ \\ \ \frac{1}{2}\ \\ \ \frac{1}{2}\ \end{bmatrix} = \begin{bmatrix} -2\ \\ -2\ \\ -2\ \\ -2\ \end{bmatrix} \\ \\ \mathbf{a}_2 - \mathbf{p}_2 &= \begin{bmatrix}\ \ 0\ \\ -4\ \\ \ \ 0\ \\ -4\ \end{bmatrix} - \begin{bmatrix} -2\ \\ -2\ \\ -2\ \\ -2\ \end{bmatrix} = \begin{bmatrix}\ \ 2\ \\ -2\ \\ \ \ 2\ \\ -2\ \end{bmatrix} \\ \\ \mathbf{q}_2 &= \frac{\mathbf{a}_2 - \mathbf{p}_2}{\left\|\mathbf{a}_2 - \mathbf{p}_2\right\|} = \frac{\mathbf{a}_2 - \mathbf{p}_2}{r_{22}} = \frac{1}{4}\begin{bmatrix}\ \ 2\ \\ -2\ \\ \ \ 2\ \\ -2\ \end{bmatrix} = \begin{bmatrix}\ \ \frac{1}{2}\ \\ -\frac{1}{2}\ \\ \ \ \frac{1}{2}\ \\ -\frac{1}{2}\ \end{bmatrix} \\ \mathbf{r}_2 &= \begin{bmatrix} -4\ \\ \ \ 4\ \\ \ \ 0\ \end{bmatrix} \\ \\ \mathbf{p}_3 &= \text{Proj}_{\mathbf{q}_1}\mathbf{a}_3 + \text{Proj}_{\mathbf{q}_2}\mathbf{a}_3 = \frac{\langle\mathbf{q}_1,\mathbf{a}_3\rangle}{\langle\mathbf{q_1},\mathbf{q}_1\rangle}\mathbf{q}_1 + \frac{\langle\mathbf{q}_2,\mathbf{a}_3\rangle}{\langle\mathbf{q_2},\mathbf{q}_2\rangle}\mathbf{q}_2 = r_{13}\mathbf{q_1} + r_{23}\mathbf{q}_2 \\ &= \left( -3\cdot\frac{1}{2} + 5\cdot\frac{1}{2} + 2\cdot\frac{1}{2} + 0\cdot\frac{1}{2} \right)\begin{bmatrix}\ \frac{1}{2}\ \\ \ \frac{1}{2}\ \\ \ \frac{1}{2}\ \\ \ \frac{1}{2}\ \end{bmatrix} + \left( -3\cdot\frac{1}{2} - 5\cdot\frac{1}{2} + 2\cdot\frac{1}{2} - 0\cdot\frac{1}{2} \right)\begin{bmatrix}\ \ \frac{1}{2}\ \\ -\frac{1}{2}\ \\ \ \ \frac{1}{2}\ \\ -\frac{1}{2}\ \end{bmatrix} \\ &= \begin{bmatrix}\ 1\ \\ \ 1\ \\ \ 1\ \\ \ 1\ \end{bmatrix} + \begin{bmatrix} -\frac{3}{2}\ \\ \ \ \frac{3}{2}\ \\ -\frac{3}{2}\ \\ \ \ \frac{3}{2}\ \end{bmatrix} = \begin{bmatrix} -\frac{1}{2}\ \\ \ \ \frac{5}{2}\ \\ -\frac{1}{2}\ \\ \ \ \frac{5}{2}\ \end{bmatrix} \\ \\ \mathbf{a}_3-\mathbf{p}_3 &= \begin{bmatrix} -3\ \\ \ \ 5\ \\ \ \ 2\ \\ \ \ 0\ \end{bmatrix} - \begin{bmatrix} -\frac{1}{2}\ \\ \ \ \frac{5}{2}\ \\ -\frac{1}{2}\ \\ \ \ \frac{5}{2}\ \end{bmatrix} = \begin{bmatrix} -\frac{5}{2}\ \\ \ \ \frac{5}{2}\ \\ \ \ \frac{5}{2}\ \\ -\frac{5}{2}\ \end{bmatrix} \\ \\ \mathbf{q}_3 &= \frac{\mathbf{a}_3-\mathbf{p}_3}{\left\|\mathbf{a}_3-\mathbf{p}_3\right\|} = \frac{\mathbf{a}_3-\mathbf{p}_3}{r_{33}} = \frac{1}{5}\begin{bmatrix} -\frac{5}{2}\ \\ \ \ \frac{5}{2}\ \\ \ \ \frac{5}{2}\ \\ -\frac{5}{2}\ \end{bmatrix} = \begin{bmatrix} -\frac{1}{2}\ \\ \ \ \frac{1}{2}\ \\ \ \ \frac{1}{2}\ \\ -\frac{1}{2}\ \end{bmatrix} \\ \mathbf{r}_3 &= \begin{bmatrix}\ \ 2\ \\ -3\ \\ \ \ 5\ \end{bmatrix} \\ \\ Q &= \begin{bmatrix}\ \ \frac{1}{2}\ &\ \ \frac{1}{2}\ & -\frac{1}{2}\ \\ \ \ \frac{1}{2}\ & -\frac{1}{2}\ &\ \ \frac{1}{2}\ \\ \ \ \frac{1}{2}\ &\ \ \frac{1}{2}\ &\ \ \frac{1}{2}\ \\ \ \ \frac{1}{2}\ & -\frac{1}{2}\ & -\frac{1}{2}\ \end{bmatrix} \\ \\ R &= \begin{bmatrix}\ \ 2\ & -4\ &\ \ 2\ \\ \ \ 0\ &\ \ 4\ & -3\ \\ \ \ 0\ &\ \ 0\ &\ \ 5\ \end{bmatrix} \\ \end{align*} $$

Exercise 2

Consider the following matrix $$ A = \begin{bmatrix}\ \ 3\ &\ \ 0\ & -2\ \\ \ \ 3\ & \ \ 8\ &\ \ 5\ \\ \ \ 3\ &\ \ 4\ &\ \ 0\ \\ \ \ 0\ & -4\ & -2\ \end{bmatrix} $$

Determine the $QR$ factorization of matrix $A$.

Check Your Work

$$ \begin{align*} Q &= \begin{bmatrix}\ \ \frac{1}{\sqrt{3}}\ & -\frac{1}{\sqrt{3}}\ &\ \ 0\ \\ \ \ \frac{1}{\sqrt{3}}\ &\ \ \frac{1}{\sqrt{3}}\ &\ \ \frac{1}{\sqrt{3}}\ \\ \ \ \frac{1}{\sqrt{3}}\ &\ \ 0\ & -\frac{1}{\sqrt{3}}\ \\ \ \ 0\ & -\frac{1}{\sqrt{3}}\ &\ \ \frac{1}{\sqrt{3}}\ \end{bmatrix} \\ \\ R &= \begin{bmatrix}\ \ 3\sqrt{3}\ &\ \ 4\sqrt{3}\ &\ \ \sqrt{3}\ \\ \ \ 0\ &\ \ 4\sqrt{3}\ & 3\sqrt{3}\ \\ \ \ 0\ &\ \ 0\ &\ \ \sqrt{3}\ \end{bmatrix} \end{align*} $$

Follow Along

$$ \begin{align*} r_{11} &= \sqrt{ (-3)^2 + (-3)^2 + (-3)^2 + 0^2 } = 3\sqrt{3} \\ \\ \mathbf{q}_1 &= \frac{\mathbf{a}_1}{\left\|\mathbf{a}_1\right\|} = \frac{\mathbf{a}_1}{r_{11}} = \frac{1}{\sqrt{3}}\begin{bmatrix}\ 1\ \\ \ 1\ \\ \ 1\ \\ \ 0\ \end{bmatrix} \\ \mathbf{r}_1 &= \begin{bmatrix}\ 3\sqrt{3}\ \\ \ 0\ \\ \ 0\ \end{bmatrix} \\ \\ \mathbf{p}_2 &= \text{Proj}_{\mathbf{q}_1}\mathbf{a}_2 = \frac{\langle\mathbf{q}_1,\mathbf{a}_2\rangle}{\langle\mathbf{q_1},\mathbf{q}_1\rangle}\mathbf{q}_1 = \frac{r_{12}}{1}\mathbf{q}_1 = \left(0\cdot\frac{1}{\sqrt{3}} + 8\cdot\frac{1}{\sqrt{3}} + 4\cdot\frac{1}{\sqrt{3}} - 4\cdot 0 \right)\begin{bmatrix}\ \frac{1}{\sqrt{3}}\ \\ \ \frac{1}{\sqrt{3}}\ \\ \ \frac{1}{\sqrt{3}}\ \\ \ 0\ \end{bmatrix} = \begin{bmatrix}\ 4\ \\ \ 4\ \\ \ 4\ \\ \ 0\ \end{bmatrix} \\ \\ \mathbf{a}_2 - \mathbf{p}_2 &= \begin{bmatrix}\ \ 0\ \\ \ \ 8\ \\ \ \ 4\ \\ -4\ \end{bmatrix} - \begin{bmatrix}\ 4\ \\ \ 4\ \\ \ 4\ \\ \ 0\ \end{bmatrix} = \begin{bmatrix} -4\ \\ \ \ 4\ \\ \ \ 0\ \\ -4\ \end{bmatrix} \\ \\ \mathbf{q}_2 &= \frac{\mathbf{a}_2 - \mathbf{p}_2}{\left\|\mathbf{a}_2 - \mathbf{p}_2\right\|} = \frac{\mathbf{a}_2 - \mathbf{p}_2}{r_{22}} = \frac{1}{4\sqrt{3}}\begin{bmatrix} -4\ \\ \ \ 4\ \\ \ \ 0\ \\ -4\ \end{bmatrix} = \begin{bmatrix} -\frac{1}{\sqrt{3}}\ \\ \ \ \frac{1}{\sqrt{3}}\ \\ \ \ 0\ \\ -\frac{1}{\sqrt{3}}\ \end{bmatrix} \\ \mathbf{r}_2 &= \begin{bmatrix}\ \ 4\sqrt{3}\ \\ \ \ 4\sqrt{3}\ \\ \ \ 0\ \end{bmatrix} \\ \\ \mathbf{p}_3 &= \text{Proj}_{\mathbf{q}_1}\mathbf{a}_3 + \text{Proj}_{\mathbf{q}_2}\mathbf{a}_3 = \frac{\langle\mathbf{q}_1,\mathbf{a}_3\rangle}{\langle\mathbf{q_1},\mathbf{q}_1\rangle}\mathbf{q}_1 + \frac{\langle\mathbf{q}_2,\mathbf{a}_3\rangle}{\langle\mathbf{q_2},\mathbf{q}_2\rangle}\mathbf{q}_2 = r_{13}\mathbf{q_1} + r_{23}\mathbf{q}_2 \\ &= \left( -2\cdot\frac{1}{\sqrt{3}} + 5\cdot\frac{1}{\sqrt{3}} + 0\cdot\frac{1}{\sqrt{3}} - 2\cdot 0 \right)\begin{bmatrix}\ \frac{1}{\sqrt{3}}\ \\ \ \frac{1}{\sqrt{3}}\ \\ \ \frac{1}{\sqrt{3}}\ \\ \ 0 \end{bmatrix} + \left( -2\left(-\frac{1}{\sqrt{3}}\right) + 5\cdot\frac{1}{\sqrt{3}} + 0\cdot\frac{1}{\sqrt{3}} - 2\left(-\frac{1}{\sqrt{3}}\right) \right)\begin{bmatrix} -\frac{1}{\sqrt{3}}\ \\ \ \ \frac{1}{\sqrt{3}}\ \\ \ \ 0\ \\ -\frac{1}{\sqrt{3}}\ \end{bmatrix} \\ &= \begin{bmatrix}\ 1\ \\ \ 1\ \\ \ 1\ \\ \ 0\ \end{bmatrix} + \begin{bmatrix} -3\ \\ \ \ 3\ \\ \ \ 0\ \\ -3\ \end{bmatrix} = \begin{bmatrix} -2\ \\ \ \ 4\ \\ \ \ 1\ \\ -3\ \end{bmatrix} \\ \\ \mathbf{a}_3-\mathbf{p}_3 &= \begin{bmatrix} -2\ \\ \ \ 5\ \\ \ \ 0\ \\ -2\ \end{bmatrix} - \begin{bmatrix} -2 \\ \ \ 4\ \\ \ \ 1\ \\ -3\ \end{bmatrix} = \begin{bmatrix}\ \ 0\ \\ \ \ 1\ \\ -1\ \\ \ \ 1\ \end{bmatrix} \\ \\ \mathbf{q}_3 &= \frac{\mathbf{a}_3-\mathbf{p}_3}{\left\|\mathbf{a}_3-\mathbf{p}_3\right\|} = \frac{\mathbf{a}_3-\mathbf{p}_3}{r_{33}} = \frac{1}{\sqrt{3}}\begin{bmatrix}\ \ 0\ \\ \ \ 1\ \\ -1\ \\ \ \ 1\ \end{bmatrix} = \begin{bmatrix}\ \ 0\ \\ \ \ \frac{1}{\sqrt{3}}\ \\ -\frac{1}{\sqrt{3}}\ \\ \ \ \frac{1}{\sqrt{3}}\ \end{bmatrix} \\ \mathbf{r}_3 &= \begin{bmatrix}\ \ \sqrt{3}\ \\ \ \ 3\sqrt{3}\ \\ \ \ \sqrt{3}\ \end{bmatrix} \\ \\ Q &= \begin{bmatrix}\ \ \frac{1}{\sqrt{3}}\ & -\frac{1}{\sqrt{3}}\ &\ \ 0\ \\ \ \ \frac{1}{\sqrt{3}}\ &\ \ \frac{1}{\sqrt{3}}\ &\ \ \frac{1}{\sqrt{3}}\ \\ \ \ \frac{1}{\sqrt{3}}\ &\ \ 0\ & -\frac{1}{\sqrt{3}}\ \\ \ \ 0\ & -\frac{1}{\sqrt{3}}\ &\ \ \frac{1}{\sqrt{3}}\ \end{bmatrix} \\ \\ R &= \begin{bmatrix}\ \ 3\sqrt{3}\ &\ \ 4\sqrt{3}\ &\ \ \sqrt{3}\ \\ \ \ 0\ &\ \ 4\sqrt{3}\ & 3\sqrt{3}\ \\ \ \ 0\ &\ \ 0\ &\ \ \sqrt{3}\ \end{bmatrix} \\ \\ QR &= \begin{bmatrix}\ \ \frac{1}{\sqrt{3}}\ & -\frac{1}{\sqrt{3}}\ &\ \ 0\ \\ \ \ \frac{1}{\sqrt{3}}\ &\ \ \frac{1}{\sqrt{3}}\ &\ \ \frac{1}{\sqrt{3}}\ \\ \ \ \frac{1}{\sqrt{3}}\ &\ \ 0\ & -\frac{1}{\sqrt{3}}\ \\ \ \ 0\ & -\frac{1}{\sqrt{3}}\ &\ \ \frac{1}{\sqrt{3}}\ \end{bmatrix}\begin{bmatrix}\ \ 3\sqrt{3}\ &\ \ 4\sqrt{3}\ &\ \ \sqrt{3}\ \\ \ \ 0\ &\ \ 4\sqrt{3}\ & 3\sqrt{3}\ \\ \ \ 0\ &\ \ 0\ &\ \ \sqrt{3}\ \end{bmatrix} \\ \\ &= \frac{1}{\sqrt{3}}\begin{bmatrix}\ \ 1\ & -1\ &\ \ 0\ \\ \ \ 1\ &\ \ 1\ &\ \ 1\ \\ \ \ 1\ &\ \ 0\ & -1\ \\ \ \ 0\ & -1\ &\ \ 1\ \end{bmatrix}\begin{bmatrix}\ \ 3\ &\ \ 4\ &\ \ 1\ \\ \ \ 0\ &\ \ 4\ &\ \ 3\ \\ \ \ 0\ &\ \ 0\ &\ \ 1\ \end{bmatrix}\sqrt{3} = \begin{bmatrix}\ \ 3\ &\ \ 0\ & -2\ \\ \ \ 3\ &\ \ 8\ &\ \ 5\ \\ \ \ 3\ &\ \ 4\ &\ \ 0\ \\ \ \ 0\ & -4\ & -2\ \end{bmatrix}\qquad{\color{green}\Large{\checkmark}} \end{align*} $$

---

5.3b.7 More Exercises

Exercise 3

Use the inner product defined in Example 1 for $P_3$. Find an orthonormal basis for $P_3$ with this inner product derived from the standard basis

$$ \left\{ 1, x, x^2 \right\} $$

Check Your Work

$$ \left\{\, \frac{1}{\sqrt{3}},\ \frac{x}{\sqrt{2}},\ \frac{3x^2-2}{\sqrt{6}} \,\right\} $$

Follow Along

$$ \begin{align*} \|1\| &= \langle 1, 1 \rangle^{1/2} = \sqrt{ 1^2 + 1^2 + 1^2 }^{1/2} = \sqrt{3} \\ \\ \mathbf{q}_1 &= \frac{1}{\sqrt{3}}\qquad\longleftarrow\text{a constant function} \\ \\ x - \textrm{Proj}_{\frac{1}{\sqrt{3}}} x &= x - \frac{\left( \frac{1}{\sqrt{3}}(-1) + \frac{1}{\sqrt{3}}(0) + \frac{1}{\sqrt{3}}(1) \right)}{1}\,\frac{1}{\sqrt{3}} = x - 0 = x \\ \\ \|x\| &= \sqrt{ (-1)^2 + 0^2 + 1^2 } = \sqrt{2} \\ \\ \mathbf{q}_2 &= \frac{x}{\sqrt{2}} \\ \\ x^2 - \textrm{Proj}_{\frac{1}{\sqrt{3}}} x^2 - \textrm{Proj}_{\frac{x}{\sqrt{2}}} x^2 &= x^2 - \left( (-1)^2\frac{1}{\sqrt{3}} + 0^2\frac{1}{\sqrt{3}} + 1^2\frac{1}{\sqrt{3}} \right)\frac{1}{\sqrt{3}} - \left( (-1)^2\frac{-1}{\sqrt{2}} + 0^2\frac{0}{\sqrt{2}} + 1^2\frac{1}{\sqrt{2}} \right)\frac{x}{\sqrt{2}} \\ \\ &= x^2 - \left( \frac{2}{\sqrt{3}} \right)\frac{1}{\sqrt{3}} - (0)\frac{x}{\sqrt{3}} = x^2 - \frac{2}{3} \\ \\ \left\| x^2 - \frac{2}{3} \right\| &= \left( \left( (-1)^2-\frac{2}{3} \right)^2 + \left( 0^2-\frac{2}{3} \right)^2 + \left( 1^2-\frac{2}{3} \right)^2 \right)^{1/2} \\ \\ &= \left( \left( \frac{1}{3} \right)^2 + \left( -\frac{2}{3} \right)^2 + \left( \frac{1}{3} \right)^2 \right)^{1/2} = \left( \frac{6}{9} \right)^{1/2} = \sqrt{\frac{2}{3}} \\ \\ \mathbf{q}_3 &= \sqrt{\frac{3}{2}}\left( x^2 - \frac{2}{3} \right) = \frac{\sqrt{6}}{2}\left( x^2 - \frac{2}{3} \right) = \sqrt{6}\,\frac{3x^2-2}{6} \\ \\ \end{align*} $$
An orthonormal basis for $P_3$ using this inner product is the set of polynomials
$$ \left\{\, \frac{1}{\sqrt{3}},\ \frac{x}{\sqrt{2}},\ \frac{3x^2-2}{\sqrt{6}} \,\right\} $$

Exercise 4

Determine the coordinates of $p(x) = x^2 + 2x - 3$ and $q(x) = 6x + 4$ using the orthonormal basis computed in Exercise 3.

Check Your Work

$$ \begin{align*} p &\cong \left\langle -\frac{7}{\sqrt{3}},\ 2\sqrt{2},\ \frac{2}{\sqrt{6}} \right\rangle \\ \\ q &\cong \left\langle 4\sqrt{3},\ 6\sqrt{2},\ 0 \right\rangle \end{align*} $$

Follow Along

$$ \begin{align*} \left\langle p, \frac{1}{\sqrt{3}} \right\rangle &= (-4)\frac{1}{\sqrt{3}} + (-3)\frac{1}{\sqrt{3}} + 0\frac{1}{\sqrt{3}} = -\frac{7}{\sqrt{3}} \\ \\ \left\langle p, \frac{x}{\sqrt{2}} \right\rangle &= (-4)\frac{-1}{\sqrt{2}} + (-3)\frac{0}{\sqrt{2}} + 0\frac{1}{\sqrt{2}} = \frac{4}{\sqrt{2}} = 2\sqrt{2} \\ \\ \left\langle p, \frac{3x^2-2}{\sqrt{6}} \right\rangle &= (-4)\frac{1}{\sqrt{6}} + (-3)\frac{-2}{\sqrt{6}} + 0\frac{1}{\sqrt{6}} = \frac{2}{\sqrt{6}} \\ \\ -\frac{7}{\sqrt{3}}\mathbf{q}_1 + 2\sqrt{2}\,\mathbf{q}_2 + \frac{2}{\sqrt{6}}\mathbf{q}_3 &= -\frac{7}{\sqrt{3}}\frac{1}{\sqrt{3}} + 2\sqrt{2}\frac{x}{\sqrt{2}} + \frac{2}{\sqrt{6}}\frac{3x^2-2}{\sqrt{6}} \\ \\ &= -\frac{7}{3} + 2x + \frac{3x^2-2}{3} = -\frac{7}{3} + 2x + \frac{3x^2}{3} - \frac{2}{3} \\ \\ &= x^2 + 2x - \frac{9}{3} = x^2 + 2x - 3 = p\qquad{\color{green}\Large{\checkmark}} \\ \\ p(x) &\cong \left\langle -\frac{7}{\sqrt{3}},\ 2\sqrt{2},\ \frac{2}{\sqrt{6}} \right\rangle \\ \\ \left\langle q, \frac{1}{\sqrt{3}} \right\rangle &= (-2)\frac{1}{\sqrt{3}} + 4\frac{1}{\sqrt{3}} + 10\frac{1}{\sqrt{3}} = \frac{12}{\sqrt{3}} = 4\sqrt{3} \\ \\ \left\langle p, \frac{x}{\sqrt{2}} \right\rangle &= (-2)\frac{-1}{\sqrt{2}} + 4\frac{0}{\sqrt{2}} + 10\frac{1}{\sqrt{2}} = \frac{12}{\sqrt{2}} = 6\sqrt{2} \\ \\ \left\langle p, \frac{3x^2-2}{\sqrt{6}} \right\rangle &= (-2)\frac{1}{\sqrt{6}} + (4)\frac{-2}{\sqrt{6}} + 10\frac{1}{\sqrt{6}} = \frac{0}{\sqrt{6}} = 0 \\ \\ 4\sqrt{3}\,\mathbf{q}_1 + 6\sqrt{2}\,\mathbf{q}_2 + 0\mathbf{q}_3 &= 4\sqrt{3}\frac{1}{\sqrt{3}} + 6\sqrt{2}\frac{x}{\sqrt{2}} = 4 + 6x = q\qquad{\color{green}\Large{\checkmark}} \\ \\ q(x) \cong \left\langle 4\sqrt{3},\ 6\sqrt{2},\ 0 \right\rangle \end{align*} $$

Exercise 5

Determine the inner product of $p(x) = x^2 + 2x - 3$ and $q(x) = 6x + 4$ using the orthonormal basis computed in Exercise 3.

Check Your Work

$$ \begin{align*} \langle p, q \rangle = -4 \end{align*} $$

Follow Along

$$ \begin{align*} \left\langle p, q \right\rangle &= \left\langle \begin{bmatrix} -\frac{7}{\sqrt{3}}\ \\ \ \ 2\sqrt{2}\ \\ \ \frac{2}{\sqrt{6}}\ \end{bmatrix},\ \begin{bmatrix}\ \ 4\sqrt{3}\ \\ \ \ 6\sqrt{2}\ \\ \ \ 0\ \end{bmatrix} \right\rangle = -\frac{7}{\sqrt{3}}\left(4\sqrt{3}\right) + 2\sqrt{2}\left(6\sqrt{2}\right) + \frac{2}{\sqrt{6}}\left(0\right) = -28 + 24 + 0 = -4\qquad{\color{green}\Large{\checkmark}} \\ \\ \end{align*} $$

Exercise 6

Determine the norms of $p(x) = x^2 + 2x - 3$ and $q(x) = 6x + 4$ using the orthonormal basis computed in Exercise 3.

Check Your Work

$$ \begin{align*} \|p\| &= 5 \\ \|q\| &= 2\sqrt{30} \end{align*} $$

Follow Along

$$ \begin{align*} \|p\| &= \langle p,p \rangle^{1/2} = \left( \left(-\frac{7}{\sqrt{3}}\right)^2 + \left( 2\sqrt{2} \right)^2 + \left( \frac{2}{\sqrt{6}} \right)^2 \right)^{1/2} = \left( \frac{49}{3} + 8 + \frac{4}{6} \right)^{1/2} = \sqrt{25} = 5\qquad{\Large{\checkmark}} \\ \\ \|q\| &= \langle q,q \rangle^{1/2} = \left( \left( 4\sqrt{3} \right)^2 + \left( 6\sqrt{2} \right)^2 + \left( 0 \right)^2 \right)^{1/2} = \left( 48 + 72 \right)^{1/2} = \sqrt{120} = 2\sqrt{30}\qquad{\color{green}\Large{\checkmark}} \end{align*} $$

---

Mathematics, Statistics & Physics

Your use of this self-initiated mediated course material is subject to our

Creative Commons License 4.0

---