Definition¶
The Gram-Schmidt Orthogonalization Process
Let $\{\mathbf{x}_1, \mathbf{x}_2, \dots, \mathbf{x}_n\}$ be a basis for the inner product space $V$. Define
$$ \mathbf{q}_1 := \dfrac{\mathbf{x}_1}{\|\mathbf{x}_1\|} $$
and define $\mathbf{q}_2$, $\dots$, $\mathbf{q}_n$ recursively by
$$ \mathbf{q}_{k} = \dfrac{\mathbf{w}_{k}}{\|\mathbf{w}_{k}\|}, $$
where
$$ \begin{align*} \mathbf{p}_k &:= \langle\mathbf{x}_{k}, \mathbf{q}_1\rangle\mathbf{q}_1 + \langle\mathbf{x}_{k}, \mathbf{q}_2\rangle\mathbf{q}_2 + \dots + \langle\mathbf{x}_{k}, \mathbf{q}_{k-1}\rangle\mathbf{q}_{k-1} \\ \\ &= \text{Proj}_{\text{Span}\left\{ q_1, q_2, \dots, u_{k-1}\right\}} x_k \\ \\ \mathbf{w}_k &= \mathbf{x}_k - \mathbf{p}_k \end{align*} $$
is the projection of $\mathbf{x}_{k}$ onto $\text{Span}\{ \mathbf{q}_1, \cdots, \mathbf{q}_{k-1} \}$. Then the set
$$ \{ \mathbf{q}_1, \mathbf{q}_2, \dots, \mathbf{q}_n \} $$
is an orthonormal basis for $V$.
Don't memorize the formulas; memorize the process.¶
You have a random non-orthogonal basis $\left\{ \mathbf{v}_1,\ \mathbf{v}_2,\ \dots,\ \mathbf{v}_n \right\}$. To create the orthonormal basis you:
- Subtract the projections onto all previous orthogonal vectors.
- Divide the result by the norm to make the result a unit vector.
- Repeat.
First vector¶
For your first vector, there are no previous orthogonal vectors ($\mathbf{q}$'s) so
- No previous orthogonal vectors to project onto $\mathbf{w}_1 = \mathbf{v}_1 - \mathbf{0} = \mathbf{v}_1$
- Divide the result by the norm $$ \mathbf{q}_1 = \frac{\mathbf{w}_1}{\lVert \mathbf{w}_1 \rVert} = \frac{\mathbf{v}_1}{\lVert \mathbf{v}_1 \rVert} $$
- Repeat
Second vector¶
- Subtract projections onto all previous (one) orthogonal vectors $$ \mathbf{w}_2 = \mathbf{v}_2 - \textrm{Proj}_{\mathbf{q}_1}\mathbf{v}_2 $$
- Divide the result by the norm $$ \mathbf{q}_2 = \frac{\mathbf{w}_2}{\lVert\mathbf{w}_2\rVert} $$
- Repeat
Third vector¶
- Subtract projections onto all previous (two) orthogonal vectors $$ \mathbf{w}_3 = \mathbf{v}_3 - \textrm{Proj}_{\mathbf{q}_1}\mathbf{v}_3 - \textrm{Proj}_{\mathbf{q}_2}\mathbf{v}_3 $$
- Divide the result by the norm $$ \mathbf{q}_3 = \frac{\mathbf{w}_3}{\lVert\mathbf{w}_3\rVert} $$
- Repeat
$k^{\text{th}}$ vector¶
- Subtract projections onto all previous ($k-1$) orthogonal vectors $$ \mathbf{w}_k = \mathbf{v}_k - \textrm{Proj}_{\mathbf{q}_1}\mathbf{v}_k - \dots - \textrm{Proj}_{\mathbf{q}_{k-1}}\mathbf{v}_k $$
- Divide the result by the norm $$ \mathbf{q}_k = \frac{\mathbf{w}_k}{\lVert\mathbf{w}_k\rVert} $$
- Repeat
$\dots$¶
Last Vector¶
Subtract projections onto all previous ($n-1$) orthogonal vectors $$ \mathbf{w}_n = \mathbf{v}_n - \textrm{Proj}_{\mathbf{q}_1}\mathbf{v}_n - \dots - \textrm{Proj}_{\mathbf{q}_{n-1}}\mathbf{v}_n $$
Divide the result by the norm $$ \mathbf{q}_n = \frac{\mathbf{w}_n}{\lVert\mathbf{w}_n\rVert} $$
No more vectors; done.
In most applications a basis is no longer good enough for efficient and accurate results. One needs an orthonormal basis. Given a set of basis vectors one would like to produce an orthonormal basis with the same span. The Gram-Schmidt Orthogonalization Process produces such an orthonormal basis.
Given any ordinary set of basis vectors for a vector space $\{\mathbf{x}_1, \mathbf{x}_2, \dots, \mathbf{x}_n\}$, The Gram-Schmidt Orthogonalization Process produces an orthonormal basis $\{\mathbf{q}_1, \mathbf{q}_2, \dots, \mathbf{q}_n\}$ with the same span
$$ \text{span}\{\mathbf{x}_1, \mathbf{x}_2, \dots, \mathbf{x}_n\} = \text{span}\{\mathbf{q}_1, \mathbf{q}_2, \dots, \mathbf{q}_n\} $$
The pattern for the process¶
You have a random non-orthogonal basis $\left\{ \mathbf{x}_1,\ \mathbf{x}_2,\ \dots,\ \mathbf{x}_n \right\}$. To create the orthonormal basis:
For each vector $\mathbf{x}_k$ in the basis
- Subtract the projections onto all previous orthogonal vectors.
- Divide the result by the norm to make the result a unit vector.
- Repeat.
Example 14 - Gram-Schmidt¶
Find an orthonormal basis for the inner product space $\mathbb{R}^3$, derived from the basis
$$ \{\mathbf{x}_1, \mathbf{x}_2, \mathbf{x}_3\} = \left\{\begin{bmatrix}\ \ 1\ \\ \ \ 2\ \\ \ \ 0\ \end{bmatrix}, \begin{bmatrix}\ \ 1\ \\ \ \ 0\ \\ \ \ 1\ \end{bmatrix},\begin{bmatrix}\ \ 2\ \\ \ \ 0\ \\ \ \ 1\ \end{bmatrix}\right\} $$
First Vector¶
- There are no previous orthonormal vectors so $\mathbf{w}_1 = \mathbf{x}_1 = \begin{bmatrix}\ \ 1\ \\ \ \ 2\ \\ \ \ 0\ \end{bmatrix}$.
- From the first basis vector $\mathbf{x}_1$ we obtain our first unit vector of our orthonormal basis by dividing $\mathbf{x}_1$ by its length to normalize the vector.
$$ \mathbf{q}_1 = \dfrac{\mathbf{w}_1}{\|\mathbf{w}_1\|} $$
In this example,
$$ \mathbf{q}_1 = \dfrac{1}{\sqrt{1^2 + 2^2 + 0^2}}\,\begin{bmatrix}\ \ 1\ \\ \ \ 2\ \\ \ \ 0\ \end{bmatrix} = \begin{bmatrix}\ \ \frac{1}{\sqrt{5}}\ \\ \ \ \frac{2}{\sqrt{5}}\ \\ \ \ 0\ \end{bmatrix} $$
- Repeat the process for the next vector.
Second Vector¶
- To obtain the second orthonormal vector we compute the projection vector $\mathbf{p}_2$ of $\mathbf{x}_2$ onto $\mathbf{q}_1$. The difference $\mathbf{w}_2 = \mathbf{x}_2-\mathbf{p}_2$ is the part of vector $\mathbf{x}_2$ that is orthogonal to $\mathbf{q}_1$.
$$ \mathbf{w}_2 = \mathbf{x}_2 - \mathbf{p}_2 = \mathbf{x}_2 - \text{Proj}_{\mathbf{q}_1}\mathbf{x}_2 = \mathbf{x}_2 - \dfrac{\langle\mathbf{x}_2, \mathbf{q}_1\rangle}{\langle\mathbf{q}_1, \mathbf{q}_1\rangle}\mathbf{q}_1 $$
$$ \mathbf{w}_2 = \mathbf{x}_2 - \mathbf{p}_2 = \begin{bmatrix}\ \ 1\ \\ \ \ 0\ \\ \ \ 1\ \end{bmatrix} - \dfrac{ \frac{1}{\sqrt{5}}(1) + \frac{2}{\sqrt{5}}(0) + 0(1)}{1}\,\begin{bmatrix}\ \ \frac{1}{\sqrt{5}}\ \\ \ \ \frac{2}{\sqrt{5}}\ \\ \ \ 0\ \end{bmatrix} = \begin{bmatrix}\ \ 1\ \\ \ \ 0\ \\ \ \ 1\ \end{bmatrix} - \begin{bmatrix}\ \ \frac{1}{5}\ \\ \ \ \frac{2}{5}\ \\ \ \ 0\ \end{bmatrix} = \begin{bmatrix}\ \ \frac{4}{5}\ \\ -\frac{2}{5} \\ \ \ 1\ \end{bmatrix} $$
- The resulting vector that is orthogonal to our first orthonormal vector, however, it is not necessarily a unit vector. We need to normalize $\mathbf{x}_2 - \mathbf{p}_2$ to obtain $\mathbf{q}_2$.
$$ \mathbf{q}_2 = \dfrac{\mathbf{w}_2}{\|\mathbf{w}_2\|} = \dfrac{1}{\sqrt{\frac{16}{25} + \frac{4}{25} + 1}}\begin{bmatrix}\ \ \frac{4}{5}\ \\ -\frac{2}{5}\ \\ \ \ 1\ \end{bmatrix} = \dfrac{1}{\sqrt{\frac{45}{25}}}\begin{bmatrix}\ \ \frac{4}{5}\ \\ -\frac{2}{5}\ \\ \ \ 1\ \end{bmatrix} = \dfrac{5}{3\sqrt{5}}\begin{bmatrix}\ \ \frac{4}{5}\ \\ -\frac{2}{5}\ \\ \ \ 1\ \end{bmatrix} = \begin{bmatrix}\ \ \frac{4}{3\sqrt{5}}\ \\ -\frac{2}{3\sqrt{5}}\ \\ \ \ \frac{5}{3\sqrt{5}}\ \end{bmatrix} $$
- Repeat the process for the next vector.
Last Vector¶
- Subtract projections
$$ \begin{align*} \mathbf{w}_3 = \mathbf{x}_3 - \mathbf{p}_3 &= \mathbf{x}_3 - \text{Proj}_{\mathbf{q}_1}\mathbf{x}_3 - \text{Proj}_{\mathbf{q}_2}\mathbf{x}_3 \\ \\ &= \begin{bmatrix}\ \ 2\ \\ \ \ 0\ \\ \ \ 1\ \end{bmatrix} - \dfrac{\frac{1}{\sqrt{5}}(2) + \frac{2}{\sqrt{5}}(0) + 0(1)}{1}\begin{bmatrix}\ \ \frac{1}{\sqrt{5}}\ \\ \ \ \frac{2}{\sqrt{5}}\ \\ \ \ 0\ \end{bmatrix} - \dfrac{\frac{4}{3\sqrt{5}}(2) -\frac{2}{3\sqrt{5}}(0) + \frac{5}{3\sqrt{5}}(1) }{1}\begin{bmatrix}\ \ \frac{4}{3\sqrt{5}}\ \\ -\frac{2}{3\sqrt{5}}\ \\ \ \ \frac{5}{3\sqrt{5}}\ \end{bmatrix} \\ \\ &= \begin{bmatrix}\ \ 2\ \\ \ \ 0\ \\ \ \ 1\ \end{bmatrix} - \dfrac{2}{\sqrt{5}}\begin{bmatrix}\ \ \frac{1}{\sqrt{5}}\ \\ \ \ \frac{2}{\sqrt{5}}\ \\ \ \ 0\ \end{bmatrix} - \dfrac{13}{3\sqrt{5}}\begin{bmatrix}\ \ \frac{4}{3\sqrt{5}}\ \\ -\frac{2}{3\sqrt{5}}\ \\ \ \ \frac{5}{3\sqrt{5}}\ \end{bmatrix} \\ \\ &= \begin{bmatrix}\ \ 2\ \\ \ \ 0\ \\ \ \ 1\ \end{bmatrix} - \begin{bmatrix}\ \ \frac{2}{5}\ \\ \ \ \frac{4}{5}\ \\ \ \ 0\ \end{bmatrix} - \begin{bmatrix}\ \ \frac{52}{45}\ \\ -\frac{26}{45}\ \\ \ \ \frac{65}{45}\ \end{bmatrix} = \begin{bmatrix}\ \ 2\ \\ \ \ 0\ \\ \ \ 1\ \end{bmatrix} - \begin{bmatrix}\ \ \frac{70}{45}\ \\ \ \ \frac{10}{45}\ \\ \ \ \frac{65}{45}\ \end{bmatrix} = \begin{bmatrix}\ \ \frac{20}{45}\ \\ -\frac{10}{45}\ \\ -\frac{20}{45}\ \end{bmatrix} = \begin{bmatrix}\ \ \frac{4}{9}\ \\ -\frac{2}{9}\ \\ -\frac{4}{9}\ \end{bmatrix} \end{align*} $$
- Normalize
$$ \mathbf{q}_3 = \dfrac{\mathbf{w}_3}{\|\mathbf{w}_3\|} = \dfrac{1}{\sqrt{\frac{16}{81} + \frac{4}{81} + \frac{16}{81}}}\begin{bmatrix}\ \ \frac{4}{9}\ \\ -\frac{2}{9}\ \\ -\frac{4}{9}\ \end{bmatrix} = \dfrac{3}{2}\begin{bmatrix}\ \ \frac{4}{9}\ \\ -\frac{2}{9}\ \\ -\frac{4}{9}\ \end{bmatrix} = \begin{bmatrix}\ \ \frac{2}{3}\ \\ -\frac{1}{3}\ \\ -\frac{2}{3}\ \end{bmatrix} $$
- There are no more vectors; done.
This results in a new orthonormal basis
$$ \{\mathbf{q}_1, \mathbf{q}_2, \mathbf{q}_3\} = \left\{ \begin{bmatrix}\ \ \frac{1}{\sqrt{5}}\ \\ \ \ \frac{2}{\sqrt{5}}\ \\ \ \ 0\ \end{bmatrix}, \begin{bmatrix}\ \ \frac{4}{3\sqrt{5}}\ \\ -\frac{2}{3\sqrt{5}}\ \\ \ \ \frac{5}{3\sqrt{5}}\ \end{bmatrix}, \begin{bmatrix}\ \ \frac{2}{3}\ \\ -\frac{1}{3} \\ -\frac{2}{3}\ \end{bmatrix}\right\} $$
There are several methods for computing the QR-Factorization. The one in the textbook wastes time and requires repeating $\dfrac{n(n-1)}{2}$ computations. The majority of the other methods either
don't work all of the time, or
are for computers and not people because they are too computationally intensive
The following is the most efficient method for a person to compute the QR-Factorization with pen and paper. As in the Gram-Schmidt Orthogonalization,
Don't memorize formulas; memorize the process
Definition¶
QR Factorization
You have a random $m\times n$ matrix $A$ with $n$ linearly independent columns $A = \begin{bmatrix}\ \mathbf{a}_1,\ \mathbf{a}_2,\ \dots,\ \mathbf{a}_n\ \end{bmatrix}$. To create the QR factorization you:
- Subtract the projections onto all previous orthogonal vectors.
- Divide the result by the norm to make the result a unit vector.
- Solve for original vector to get the row for matrix $R$ of coefficients
- Repeat.
When you look at the formulas below remember that you will be filling in all of the formulas with numbers as you proceed.
First vector¶
For your first vector, there are no previous orthogonal vectors ($\mathbf{q}$'s) so
No previous orthogonal vectors to project onto $\mathbf{w}_1 = \mathbf{a}_1 - \mathbf{0} = \mathbf{a}_1$
Divide the result by the norm $$ \mathbf{q}_1 = \frac{\mathbf{w}_1}{\lVert \mathbf{w}_1 \rVert} = \frac{\mathbf{a}_1}{\lVert \mathbf{a}_1 \rVert} \qquad\longleftarrow \lVert\mathbf{a}_1\rVert = \lVert \mathbf{w}_1\rVert \text{ is the first diagonal element of }R $$
Solve for $\mathbf{a}_1$ to get the first row for matrix $R$ of coefficients $$ \begin{align*} \mathbf{a}_1 &= \lVert \mathbf{a}_1 \rVert\mathbf{q}_1 = \lVert \mathbf{a}_1 \rVert\mathbf{q}_1 + 0\mathbf{q}_2 + \cdots + 0\mathbf{q}_n \\ \\ \text{first column of }R,\ \mathbf{r}_1 &= \begin{bmatrix}\ \lVert \mathbf{a}_1 \rVert\ \\ \ \ 0\ \\ \ \vdots\ \\ \ \ 0\ \end{bmatrix} \end{align*} $$
Repeat
Second vector¶
Subtract projections onto all previous (one) orthogonal vectors $$ \mathbf{w}_2 = \mathbf{a}_2 - \textrm{Proj}_{\mathbf{q}_1}\mathbf{a}_2 $$
Divide the result by the norm $$ \mathbf{q}_2 = \frac{\mathbf{w}_2}{\lVert\mathbf{w}_2\rVert} \qquad\longleftarrow \lVert\mathbf{w}_2\rVert \text{ is the second diagonal element of }R $$
Solve for $\mathbf{a}_2$ to get the second row for matrix $R$ of coefficients $$ \begin{align*} \mathbf{w}_2 &= \lVert \mathbf{w}_2 \rVert\mathbf{q}_2 \\ \\ \mathbf{a}_2 - \textrm{Proj}_{\mathbf{q}_1}\mathbf{a}_2 &= \lVert \mathbf{w}_2 \rVert\mathbf{q}_2 \\ \\ \mathbf{a}_2 &= \textrm{Proj}_{\mathbf{q}_1}\mathbf{a}_2 + \lVert \mathbf{w}_2 \rVert\mathbf{q}_2 \\ \\ &\text{You should already know the formula for the projection} \\ \\ &= \frac{\langle \mathbf{a}_2, \mathbf{q}_1 \rangle}{\langle \mathbf{q}_1,\mathbf{q}_1 \rangle}\mathbf{q}_1 + \lVert \mathbf{w}_2 \rVert\mathbf{q}_2 + \cdots + 0\mathbf{q}_n \\ \\ &= \langle \mathbf{a}_2, \mathbf{q}_1 \rangle\mathbf{q}_1 + \lVert \mathbf{w}_2 \rVert\mathbf{q}_2 + \cdots + 0\mathbf{q}_n \\ \\ \langle \mathbf{q}_1,\mathbf{q}_1 \rangle &= 1 \text{ since } \mathbf{q}_1 \text{ is a unit vector} \\ \\ \text{second column of }R,\ \mathbf{r}_2 &= \begin{bmatrix}\ \langle \mathbf{a}_2, \mathbf{q}_1 \rangle\ \\ \ \lVert \mathbf{w}_2 \rVert\ \\ \ 0\ \\ \ \vdots\ \\ \ 0\ \end{bmatrix} \end{align*} $$
Repeat
Third vector¶
Subtract projections onto all previous (two) orthogonal vectors $$ \mathbf{w}_3 = \mathbf{a}_3 - \textrm{Proj}_{\mathbf{q}_1}\mathbf{a}_3 - \textrm{Proj}_{\mathbf{q}_2}\mathbf{a}_3 $$
Divide the result by the norm $$ \mathbf{q}_3 = \frac{\mathbf{w}_3}{\lVert\mathbf{w}_3\rVert}\qquad\longleftarrow \lVert\mathbf{w}_3\rVert \text{ is the third diagonal element of }R $$
Solve for $\mathbf{a}_3$ to get the third row for matrix $R$ of coefficients $$ \begin{align*} \mathbf{w}_3 &= \lVert \mathbf{w}_3 \rVert\mathbf{q}_3 \\ \\ \mathbf{a}_3 - \textrm{Proj}_{\mathbf{q}_1}\mathbf{a}_3 - \textrm{Proj}_{\mathbf{q}_2}\mathbf{a}_3 &= \lVert \mathbf{w}_3 \rVert\mathbf{q}_3 \\ \\ \mathbf{a}_3 &= \textrm{Proj}_{\mathbf{q}_1}\mathbf{a}_3 + \textrm{Proj}_{\mathbf{q}_2}\mathbf{a}_3 + \lVert \mathbf{w}_3 \rVert\mathbf{q}_3 \\ \\ &\text{You should already know the formula for the projection} \\ \\ &= \frac{\langle \mathbf{a}_3, \mathbf{q}_1 \rangle}{\langle \mathbf{q}_1,\mathbf{q}_1 \rangle}\mathbf{q}_1 + \frac{\langle \mathbf{a}_3, \mathbf{q}_2 \rangle}{\langle \mathbf{q}_2,\mathbf{q}_2 \rangle}\mathbf{q}_2 + \lVert \mathbf{w}_3 \rVert\mathbf{q}_3 + \cdots + 0\mathbf{q}_n \\ \\ &= \langle \mathbf{a}_3, \mathbf{q}_1 \rangle\mathbf{q}_1 + \langle \mathbf{a}_3, \mathbf{q}_2 \rangle\mathbf{q}_2 + \lVert \mathbf{w}_3 \rVert\mathbf{q}_3 + \cdots + 0\mathbf{q}_n \\ \\ \langle \mathbf{q}_1,\mathbf{q}_1 \rangle = \langle \mathbf{q}_2,\mathbf{q}_2 \rangle &= 1 \text{ since } \mathbf{q}_1 \text{ and } \mathbf{q}_2 \text{ are unit vectors} \\ \\ \text{third column of }R,\ \mathbf{r}_3 &= \begin{bmatrix}\ \langle \mathbf{a}_3, \mathbf{q}_1 \rangle\ \\ \ \langle \mathbf{a}_3, \mathbf{q}_2 \rangle\ \\ \ \lVert \mathbf{w}_3 \rVert\ \\ \ \ 0\ \\ \ \vdots\ \\ \ \ 0\ \end{bmatrix} \end{align*} $$
Repeat
Do you see the pattern?¶
$k^{\text{th}}$ vector¶
Subtract projections onto all previous ($k-1$) orthogonal vectors $$ \mathbf{w}_k = \mathbf{a}_k - \textrm{Proj}_{\mathbf{q}_1}\mathbf{a}_k - \dots - \textrm{Proj}_{\mathbf{q}_{k-1}}\mathbf{a}_k $$
Divide the result by the norm $$ \mathbf{q}_k = \frac{\mathbf{w}_k}{\lVert\mathbf{w}_k\rVert}\qquad\longleftarrow\lVert\mathbf{w}_k\rVert \text{ is the }k^{\text{th}}\text{ diagonal element of }R $$
Solve for $\mathbf{a}_k$ to get the $k^{\text{th}}$ row for matrix $R$ of coefficients $$ \begin{align*} \mathbf{w}_k &= \lVert \mathbf{w}_k \rVert\mathbf{q}_k \\ \\ \mathbf{a}_k - \textrm{Proj}_{\mathbf{q}_1}\mathbf{a}_k - \dots - \textrm{Proj}_{\mathbf{q}_{k-1}}\mathbf{a}_k &= \lVert \mathbf{w}_k \rVert\mathbf{q}_k \\ \\ \mathbf{a}_k &= \textrm{Proj}_{\mathbf{q}_1}\mathbf{a}_k + \dots + \textrm{Proj}_{\mathbf{q}_{k-1}}\mathbf{a}_k + \lVert \mathbf{w}_k \rVert\mathbf{q}_k \\ \\ &\text{You should already know the formula for the projection} \\ \\ &= \frac{\langle \mathbf{a}_k, \mathbf{q}_1 \rangle}{\langle \mathbf{q}_1,\mathbf{q}_1 \rangle}\mathbf{q}_1 + \cdots + \frac{\langle \mathbf{a}_k, \mathbf{q}_{k-1} \rangle}{\langle \mathbf{q}_{k-1},\mathbf{q}_{k-1} \rangle}\mathbf{q}_{k-1} + \lVert \mathbf{w}_k \rVert\mathbf{q}_k + \cdots + 0\mathbf{q}_n \\ \\ &= \langle \mathbf{a}_k, \mathbf{q}_1 \rangle\mathbf{q}_1 + \cdots + \langle \mathbf{a}_k, \mathbf{q}_{k-1} \rangle\mathbf{q}_{k-1} + \lVert \mathbf{w}_k \rVert\mathbf{q}_k + \cdots + 0\mathbf{q}_n \\ \\ \langle \mathbf{q}_j,\mathbf{q}_j \rangle &= 1 \text{ since all of the } \mathbf{q}_j \text{'s are unit vectors} \\ \\ k^{\text{th}}\text{ column of }R, \mathbf{r}_k &= \begin{bmatrix}\ \langle \mathbf{a}_k, \mathbf{q}_1 \rangle\ \\ \vdots\ \\ \ \langle \mathbf{a}_k, \mathbf{q}_{k-1} \rangle\ \\ \ \lVert \mathbf{w}_k \rVert\ \\ \ \ 0\ \\ \ \vdots\ \\ \ \ 0\ \end{bmatrix} \end{align*} $$
Place the inner products and norms in matrix $R$ as you compute them!Repeat
$\ddots$¶
Last vector¶
Subtract projections onto all previous ($n-1$) orthogonal vectors $$ \mathbf{w}_n = \mathbf{a}_n - \textrm{Proj}_{\mathbf{q}_1}\mathbf{a}_n - \dots - \textrm{Proj}_{\mathbf{q}_{n-1}}\mathbf{a}_n $$
Divide the result by the norm $$ \mathbf{q}_n = \frac{\mathbf{w}_n}{\lVert\mathbf{w}_n\rVert} \qquad\longleftarrow\lVert\mathbf{w}_n\rVert\text{ is the last diagonal element of }R $$
Solve for $\mathbf{a}_n$ to get the $n^{\text{th}}$ (last) row for matrix $R$ of coefficients $$ \begin{align*} \mathbf{w}_n &= \lVert \mathbf{w}_n \rVert\mathbf{q}_n \\ \\ \mathbf{a}_n - \textrm{Proj}_{\mathbf{q}_1}\mathbf{a}_n - \dots - \textrm{Proj}_{\mathbf{q}_{n-1}}\mathbf{a}_n &= \lVert \mathbf{w}_n \rVert\mathbf{q}_n \\ \\ \mathbf{a}_n &= \textrm{Proj}_{\mathbf{q}_1}\mathbf{a}_n + \dots + \textrm{Proj}_{\mathbf{q}_{n-1}}\mathbf{a}_n + \lVert \mathbf{w}_n \rVert\mathbf{q}_n \\ \\ &\text{You should already know the formula for the projection} \\ \\ &= \frac{\langle \mathbf{a}_n, \mathbf{q}_1 \rangle}{\langle \mathbf{q}_1,\mathbf{q}_1 \rangle}\mathbf{q}_1 + \cdots + \frac{\langle \mathbf{a}_n, \mathbf{q}_{n-1} \rangle}{\langle \mathbf{q}_{n-1},\mathbf{q}_{n-1} \rangle}\mathbf{q}_{n-1} + \lVert \mathbf{w}_n \rVert\mathbf{q}_n \\ \\ &= \langle \mathbf{a}_n, \mathbf{q}_1 \rangle\mathbf{q}_1 + \cdots + \langle \mathbf{a}_n, \mathbf{q}_{n-1} \rangle\mathbf{q}_{n-1} + \lVert \mathbf{w}_n \rVert\mathbf{q}_n \\ \\ \langle \mathbf{q}_j,\mathbf{q}_j \rangle &= 1 \text{ since all of the } \mathbf{q}_j \text{'s are unit vectors} \\ \\ \text{last column of }R, \mathbf{r}_n &= \begin{bmatrix}\ \langle \mathbf{a}_n, \mathbf{q}_1 \rangle\ \\ \ \vdots\ \\ \ \langle \mathbf{a}_n, \mathbf{q}_{n-1} \rangle\ \\ \ \lVert \mathbf{w}_n \rVert\ \end{bmatrix} \end{align*} $$
No more columns of matrix $A$ - done
$$ \begin{align*}
Q &= \begin{bmatrix}\ \mathbf{q}_1\ &\ \mathbf{q}_2\ &\ \cdots\ &\ \mathbf{q}_n\ \end{bmatrix} \\ \\
R &= \begin{bmatrix}\ \mathbf{r}_1\ &\ \mathbf{r}_2\ &\ \cdots\ &\ \mathbf{r}_n\ \end{bmatrix} \\ \\
&= \begin{bmatrix}\ \lVert \mathbf{a}_1 \rVert\ &\ \langle \mathbf{a}_2, \mathbf{q}_1 \rangle\ &\ \cdots\ &\ \langle \mathbf{a}_k, \mathbf{q}_1 \rangle\ &\ \cdots\ &\ \langle \mathbf{a}_{n-1}, \mathbf{q}_1 \rangle\ &\ \langle \mathbf{a}_n, \mathbf{q}_1 \rangle\ \\
\ \ 0\ & \lVert \mathbf{w}_2 \rVert\ &\ \cdots\ &\ \langle \mathbf{a}_k, \mathbf{q}_2 \rangle\ &\ \cdots\ &\ \langle \mathbf{a}_{n-1}, \mathbf{q}_2 \rangle\ &\ \langle \mathbf{a}_n, \mathbf{q}_2 \rangle\ \\
\ \vdots\ &\ \ddots\ &\ \ddots\ &\ \vdots\ &\ \ddots\ &\ \vdots\ &\ \vdots\ \\
\ \ 0\ &\ \ddots\ &\ \ 0\ &\ \lVert \mathbf{w}_k \rVert\ &\ \cdots\ &\ \langle \mathbf{a}_{n-1}, \mathbf{q}_k \rangle\ &\ \langle \mathbf{a}_n, \mathbf{q}_k \rangle\ \\
\ \vdots\ &\ \ddots\ &\ \ddots\ &\ \vdots\ &\ \ddots\ &\ \vdots\ &\ \vdots\ \\
\ \ 0\ &\ \ddots\ &\ \ddots\ &\ \ 0\ &\ \ddots\ &\ \lVert\mathbf{w}_{n-1}\rVert\ &\ \langle \mathbf{a}_n, \mathbf{q}_{n-1} \rangle \\
\ \ 0\ &\ \cdots\ &\ \cdots\ &\ \ 0\ &\ \vdots\ &\ \ 0\ &\ \lVert \mathbf{w}_n \rVert\
\end{bmatrix} \end{align*} $$
Notice¶
- $Q$ is an $m\times n$ matrix; the same dimensions as $A$
- The $\mathbf{q}_j$'s are the basis vectors from Gram-Schmidt
- $R$ is a square $n\times n$ matrix = number of columns of $A$.
- $R$ is upper triangular
- The diagonal elements of $R$ are the norms of the $\mathbf{w}_k$'s
- The upper triangle of $R$ are the components of each column of matrix $A$ with the $\mathbf{q}_j$'s that have been computed so far.
- You don't see denominators in the components because they are all $1$'s
In the textbook method, the denominators are not $1$'s, so it looks a lot more complicated. On page 259 of the textbook in Theorem 5.12, you compute the denominators for projection onto the previous $\mathbf{w}$'s, and then compute them again to normalize each $\mathbf{w}$ to get your orthogonal vector $\mathbf{q}$. This is the $\dfrac{n(n-1)}{2}$ repeated computations.
In many STEM applications, the mathematical model of the constraints of a linear system do not change. Instead the current state of the linear system, or inputs change, and the model is used to extract important information from the inputs. Solving the linear system is performed repeatedly using new input data each iteration. Simplifying the process by factoring the matrices speeds up the process. The values of the factorization can also provide important insight into the mathematical model.
The pattern for the process¶
You have a random non-orthogonal matrix $A = \left[\, \mathbf{a}_1\ \dots\ \mathbf{a}_n \,\right]$. To create the QR factorization:
For each vector $\mathbf{a}_k$ in the basis
- Subtract the projections onto all previous orthogonal vectors.
- Divide the result by the norm to make the result a unit vector to get a column $\mathbf{q}$ of matrix $Q$.
- Solve the result for $\mathbf{a}_k$ to obtain a column of matrix $R$.
- Repeat.
Example 15 - QR Factorization¶
Consider the matrix
$$ A = \begin{bmatrix}\ \ 1\ &\ \ 0\ & -3\ \\ \ \ 1\ & -4\ &\ \ 5\ \\ \ \ 1\ &\ \ 0\ &\ \ 2\ \\ \ \ 1\ & -4\ &\ \ 0\ \end{bmatrix} $$
Verify for yourself that the columns of matrix $A$ are linearly independent. Thus the rank$(A) = 3$ and the columns of $A$ form a basis for a three dimensional subspace of $\mathbb{R}^4$. We can use the Gram-Schmidt orthogonalization process to obtain an orthonormal basis for the column space of matrix $A$,
$$ C(A) = \text{span}\{\, \mathbf{a}_1,\ \mathbf{a}_2,\ \mathbf{a}_3 \,\} = \text{span}\{\, \mathbf{q}_1,\ \mathbf{q}_2,\ \mathbf{q}_3 \,\} $$
The orthonormal column vectors $\{\mathbf{q}_1, \mathbf{q}_2, \mathbf{q}_3\}$ form a new matrix
$$ Q = \begin{bmatrix}\ \mathbf{q}_1\ &\ \mathbf{q}_2\ &\ \mathbf{q}_3\ \end{bmatrix} $$
First Vector¶
- There are no previous orthogonal vectors.
$$ \mathbf{w}_1 = \mathbf{a}_1 = \begin{bmatrix}\ \ 1\ \\ \ \ 1\ \\ \ \ 1\ \\ \ \ 1\ \end{bmatrix} $$
- Normalize $\mathbf{w}_1$
$$ \mathbf{q}_1 = \dfrac{\mathbf{w}_1}{\lVert\mathbf{w}_1\rVert} = \dfrac{\mathbf{a}_1}{\lVert\mathbf{a}_1\rVert} $$
Performing the computations yields
$$ \begin{align*} r_{11} &= \|\mathbf{a}_1\| = \sqrt{1^2 + 1^2 + 1^2 + 1^2} = \sqrt{4} = 2 \\ \\ \mathbf{q}_1 &= \frac{1}{r_{11}}\mathbf{a}_1 = \frac{1}{2}\begin{bmatrix}\ \ 1\ \\ \ \ 1\ \\ \ \ 1\ \\ \ \ 1\ \end{bmatrix} = \begin{bmatrix}\ \ 1/2\ \\ \ \ 1/2\ \\ \ \ 1/2\ \\ \ \ 1/2\ \end{bmatrix} \end{align*} $$
- Solve the equation for $\mathbf{w}_1$ for $\mathbf{a}_1$ to get a linear combination.
$$ \mathbf{a}_1 = \|\mathbf{a}_1\|\mathbf{q}_1 = r_{11}\mathbf{q}_1 + 0\mathbf{q}_2 + 0\mathbf{q}_3 = \begin{bmatrix}\ \mathbf{q}_1\ &\ \mathbf{q}_2\ &\ \mathbf{q}_3\ \end{bmatrix}\begin{bmatrix}\ r_{11}\ \\ \ \ 0\ \\ \ \ 0\ \end{bmatrix} $$
Performing the computations one obtains
$$ \begin{align*} \mathbf{a}_1 &= r_{11}\mathbf{q}_1 = \lVert \mathbf{a}_1 \rVert\mathbf{q}_1 = 2\begin{bmatrix}\ \frac{1}{2}\ \\ \ \frac{1}{2}\ \\ \ \frac{1}{2}\ \\ \ \frac{1}{2}\ \end{bmatrix} + 0\mathbf{q}_2 + 0\mathbf{q}_3 \\ \\ &= 2\mathbf{q}_1 + 0\mathbf{q}_2 + 0\mathbf{q}_3 = \begin{bmatrix}\ \mathbf{q}_1\ &\ \mathbf{q}_2\ &\ \mathbf{q}_3\ \end{bmatrix} \begin{bmatrix}\ \ 2\ \\ \ \ 0\ \\ \ \ 0\ \end{bmatrix} = Q\mathbf{r}_1 \end{align*} $$
Second Column of Matrix $A$¶
- Subtract projections of $\mathbf{a}_2$ onto all previous (one) orthogonal vectors.
$$ \mathbf{w}_2 = \mathbf{a}_2 - \textrm{Proj}_{\mathbf{q}_1}\mathbf{a}_2 = \mathbf{a}_2 - \dfrac{\langle\mathbf{a}_2, \mathbf{q}_1\rangle}{\langle\mathbf{q}_1, \mathbf{q}_1\rangle}\mathbf{q}_1 = \mathbf{a}_2 - \langle\mathbf{a}_2, \mathbf{q}_1\rangle\mathbf{q}_1 $$
Performing the computations
$$ \begin{align*} \langle\mathbf{a}_2, \mathbf{q}_1\rangle &= 0\left(\frac{1}{2}\right)- 4\left(\frac{1}{2}\right) + 0\left(\frac{1}{2}\right)- 4\left(\frac{1}{2}\right) = -4 \\ \\ \mathbf{w}_2 &= \begin{bmatrix}\ \ 0\ \\ -4\ \\ \ \ 0\ \\ -4\ \end{bmatrix} - (-4)\begin{bmatrix}\ \ 1/2\ \\ \ \ 1/2\ \\ \ \ 1/2\ \\ \ \ 1/2\ \end{bmatrix} = \begin{bmatrix}\ \ 0\ \\ -4\ \\ \ \ 0\ \\ -4\ \end{bmatrix} + \begin{bmatrix}\ \ 2\ \\ \ \ 2\ \\ \ \ 2\ \\ \ \ 2\ \end{bmatrix} = \begin{bmatrix}\ \ 2\ \\ -2\ \\ \ \ 2\ \\ -2\ \end{bmatrix} \end{align*} $$
- Normalize the result
$$ \mathbf{q}_2 = \frac{\mathbf{w}_2}{\lVert \mathbf{w}_2 \rVert} = \frac{\mathbf{w}_2}{r_{22}} $$
The computations give us
$$ \begin{align*} r_{22} &= \lVert \mathbf{w}_2 \rVert = \sqrt{ 2^2 + (-2)^2 + 2^2 + (-2)^2 } = \sqrt{16} = 4 \\ \\ \mathbf{q}_2 &= \dfrac{1}{4}\begin{bmatrix}\ \ 2\ \\ -2\ \\ \ \ 2\ \\ -2\ \end{bmatrix} = \begin{bmatrix}\ \ \frac{1}{2}\ \\ -\frac{1}{2}\ \\ \ \ \frac{1}{2}\ \\ -\frac{1}{2}\ \end{bmatrix} \end{align*} $$
- Solve the last equation for $\mathbf{a}_2$
$$ \begin{align*} r_{22}\mathbf{q}_2 &= \mathbf{w}_2 = \mathbf{a}_2 - \langle \mathbf{a}_2,\mathbf{q}_1 \rangle\mathbf{q}_1 \\ \\ \mathbf{a}_2 &= \langle \mathbf{a}_2,\mathbf{q}_1 \rangle\mathbf{q}_1 + r_{22}\mathbf{q}_2 = \langle \mathbf{a}_2,\mathbf{q}_1 \rangle\mathbf{q}_1 + r_{22}\mathbf{q}_2 + 0\mathbf{q}_3 \\ \\ &= \begin{bmatrix}\ \mathbf{q}_1\ &\ \mathbf{q}_2\ &\ \mathbf{q}_3\ \end{bmatrix}\begin{bmatrix}\ r_{12}\ \\ \ r_{22}\ \\ \ \ 0\ \end{bmatrix} = Q\mathbf{r}_2 \end{align*} $$
Computationally
$$ \begin{align*} \|\mathbf{w}_2\|\mathbf{q}_2 &= \mathbf{a}_2 - \langle \mathbf{a}_2,\mathbf{q}_1 \rangle\mathbf{q}_1 \\ \\ 4\mathbf{q}_2 &= \mathbf{a}_2 - (-4)\mathbf{q}_1 \\ \\ \mathbf{a}_2 &= -4\mathbf{q}_1 + 4\mathbf{q}_2 = -4\mathbf{q}_1 + 4\mathbf{q}_2 + 0\mathbf{q}_3 \\ \\ &= \begin{bmatrix}\ \mathbf{q}_1\ &\ \mathbf{q}_2\ &\ \mathbf{q}_3\ \end{bmatrix} \begin{bmatrix} -4\ \\ \ \ 4\ \\ \ \ 0\ \end{bmatrix} = Q\mathbf{r}_2 \end{align*} $$
Last column of matrix $A$¶
- Subtract projections onto all previous (two) orthogonal vectors
$$ \begin{align*} \mathbf{w}_3 &= \mathbf{a}_3 - \textrm{Proj}_{\mathbf{q}_1}\mathbf{a}_3 - \textrm{Proj}_{\mathbf{q}_2}\mathbf{a}_3 \\ \\ &= \mathbf{a}_3 - \frac{\langle \mathbf{a}_3,\mathbf{q}_1 \rangle}{\langle \mathbf{q}_1,\mathbf{q}_1 \rangle}\mathbf{q}_1 - \frac{\langle \mathbf{a}_3,\mathbf{q}_2 \rangle}{\langle \mathbf{q}_2,\mathbf{q}_2 \rangle}\mathbf{q}_2 \\ \\ &= \mathbf{a}_3 - \langle \mathbf{a}_3,\mathbf{q}_1 \rangle\mathbf{q}_1 - \langle \mathbf{a}_3,\mathbf{q}_2 \rangle\mathbf{q}_2 \end{align*} $$
Performing the computation yields
$$ \begin{align*} \langle \mathbf{a}_3,\mathbf{q}_1 \rangle &= -3\left(\frac{1}{2}\right) + 5\left(\frac{1}{2}\right) + 2\left(\frac{1}{2}\right) + 0\left(\frac{1}{2}\right) = 2 \\ \\ \langle \mathbf{a}_3,\mathbf{q}_2 \rangle &= -3\left(\frac{1}{2}\right) + 5\left(-\frac{1}{2}\right) + 2\left(\frac{1}{2}\right) + 0\left(-\frac{1}{2}\right) = -3 \\ \\ \mathbf{w}_3 &= \begin{bmatrix} -3\ \\ \ \ 5\ \\ \ \ 2\ \\ \ \ 0\ \end{bmatrix} - 2\begin{bmatrix}\ \frac{1}{2}\ \\ \ \frac{1}{2}\ \\ \ \frac{1}{2}\ \\ \ \frac{1}{2}\ \end{bmatrix} - (-3)\begin{bmatrix}\ \ \frac{1}{2}\ \\ -\frac{1}{2}\ \\ \ \ \frac{1}{2}\ \\ -\frac{1}{2}\ \end{bmatrix} \\ \\ &= \begin{bmatrix} -3\ \\ \ \ 5\ \\ \ \ 2\ \\ \ \ 0\ \end{bmatrix} - \begin{bmatrix}\ \ 1\ \\ \ \ 1\ \\ \ \ 1\ \\ \ \ 1\ \end{bmatrix} + \begin{bmatrix}\ \ \frac{3}{2}\ \\ -\frac{3}{2}\ \\ \ \ \frac{3}{2}\ \\ -\frac{3}{2}\ \end{bmatrix} = \begin{bmatrix} -\frac{5}{2} \\ \ \ \frac{5}{2} \\ \ \ \frac{5}{2} \\ -\frac{5}{2} \end{bmatrix} \end{align*} $$
- Normalize $\mathbf{w}_3$.
$$ \begin{align*} r_{33} &= \lVert \mathbf{w}_3 \rVert \\ \\ \mathbf{q}_3 &= \dfrac{\mathbf{w}_3}{\lVert \mathbf{w}_3 \rVert} \end{align*} $$
Our computations give us
$$ \begin{align*} r_{33} &= \sqrt{\left(-\frac{5}{2}\right)^2 + \left(\frac{5}{2}\right)^2 + \left(\frac{5}{2}\right)^2 + \left(-\frac{5}{2}\right)^2} = 5 \\ \\ \mathbf{q}_3 &= \dfrac{1}{5}\begin{bmatrix} -\frac{5}{2}\ \\ \ \ \frac{5}{2}\ \\ \ \ \frac{5}{2}\ \\ -\frac{5}{2}\ \end{bmatrix} = \begin{bmatrix} -\frac{1}{2}\ \\ \ \ \frac{1}{2}\ \\ \ \ \frac{1}{2}\ \\ -\frac{1}{2}\ \end{bmatrix} \end{align*} $$
- Solve for $\mathbf{a}_3$
$$ \begin{align*} r_{33}\mathbf{q}_3 &= \mathbf{w}_3 = \mathbf{a}_3 - \langle \mathbf{a}_3,\mathbf{q}_1 \rangle\mathbf{q}_1 - \langle \mathbf{a}_3,\mathbf{q}_2 \rangle\mathbf{q}_2 \\ \\ \mathbf{a}_3 &= \langle \mathbf{a}_3,\mathbf{q}_1 \rangle\mathbf{q}_1 + \langle \mathbf{a}_3,\mathbf{q}_2 \rangle\mathbf{q}_2 + r_{33}\mathbf{q}_3 \\ \\ \mathbf{a}_3 &= \begin{bmatrix}\ \mathbf{q}_1\ &\ \mathbf{q}_2 &\ \mathbf{q}_3\ \end{bmatrix} \begin{bmatrix}\ \langle \mathbf{a}_3,\mathbf{q}_1 \rangle\ \\ \ \langle \mathbf{a}_3,\mathbf{q}_2 \rangle\ \\ \ \lVert \mathbf{w}_3 \rVert\ \end{bmatrix} = Q\mathbf{r}_3 \end{align*} $$
Performing the computations
$$ \begin{align*} \mathbf{a}_3 &= 2\mathbf{q}_1 - 3\mathbf{q}_2 + 5\mathbf{q}_3 = \begin{bmatrix}\ \mathbf{q}_1\ &\ \mathbf{q}_2\ &\ \mathbf{q}_3\ \end{bmatrix}\begin{bmatrix}\ \ 2\ \\ -3\ \\ \ \ 5\ \end{bmatrix} \end{align*} $$
Putting these three matrix-vector computations together
$$ \begin{align*} Q &= \begin{bmatrix}\ \mathbf{q}_1\ &\ \mathbf{q}_2\ &\ \mathbf{q}_3\ \end{bmatrix} = \begin{bmatrix}\ \ \frac{1}{2}\ &\ \ \frac{1}{2}\ & -\frac{1}{2}\ \\ \ \ \frac{1}{2}\ & -\frac{1}{2}\ &\ \ \frac{1}{2}\ \\ \ \ \frac{1}{2}\ &\ \ \frac{1}{2}\ &\ \ \frac{1}{2}\ \\ \ \ \frac{1}{2}\ & -\frac{1}{2}\ & -\frac{1}{2}\ \end{bmatrix} \\ \\ R &= \begin{bmatrix}\ \mathbf{r}_1\ &\ \mathbf{r}_2\ &\ \mathbf{r}_3\ \end{bmatrix} = \begin{bmatrix}\ \ 2\ & -4\ &\ \ 2\ \\ \ \ 0\ &\ \ 4\ & -3\ \\ \ \ 0\ &\ \ 0\ &\ \ 5\ \end{bmatrix} \\ \\ A &= QR \end{align*} $$
This is the $QR$ factorization of matrix $A$, the decomposition of a matrix into a product of a matrix $Q$ with orthonormal column vectors and a square matrix $R$ of coefficients for the columns of matrix $Q$.
Recall from Theorem 5.3.5 that if $Q$ is an orthogonal matrix, then $Q$ is square, and $\textrm{det}(Q)=\pm 1$.
Theorem 7¶
The Gram-Schmidt or QR Factorization Theorem
If $A\in\mathbb{R}^{m\times n}$ is an $m\times n$ matrix with full rank, then $A$ can be factored into a product $QR$, where $Q$ is an $m\times n$ matrix with orthonormal column vectors, and $R$ is an upper triangular $n\times n$ matrix.
Notice that if $A$ is a square $n\times n$ matrix, then $Q$ is also square, and
$$ |A| = |QR| = |Q||R| = \pm\,|R| = \pm\displaystyle\prod_{k=1}^n r_{kk} $$
Theorem 8¶
Full Rank QR Factorization
If $A\in\mathbb{R}^{m\times n}$ has full rank, then $A$ can be factored into a product $QR$, where $Q$ is an $m\times n$ matrix with orthonormal column vectors, and $R$ is an upper triangular $n\times n$ matrix with positive entries along the diagonal.
Notice that Theorem 8 implies that the coefficient matrix $R$ will always be nonsingular since the determinant of $R$ is a product of nonzero numbers.
This gives us another way to spot a matrix with a free column. If one of the diagonal elements of matrix $R$ is zero, then the corresponding column of matrix $A$ is a free column.
Exercise 15¶
Consider the following matrix $$ A = \begin{bmatrix}\ \ 1\ &\ \ 0\ & -3\ \\ \ \ 1\ & -4\ &\ \ 5\ \\ \ \ 1\ &\ \ 0\ &\ \ 2\ \\ \ \ 1\ & -4\ &\ \ 0\ \end{bmatrix} $$
Compute the $QR$ factorization of matrix $A$.
Check Your Work
$$ \begin{align*} Q &= \begin{bmatrix}\ \ \frac{1}{2}\ &\ \ \frac{1}{2}\ & -\frac{1}{2}\ \\ \ \ \frac{1}{2}\ & -\frac{1}{2}\ &\ \ \frac{1}{2}\ \\ \ \ \frac{1}{2}\ &\ \ \frac{1}{2}\ &\ \ \frac{1}{2}\ \\ \ \ \frac{1}{2}\ & -\frac{1}{2}\ & -\frac{1}{2}\ \end{bmatrix} \\ \\ R &= \begin{bmatrix}\ \ 2\ & -4\ &\ \ 2\ \\ \ \ 0\ &\ \ 4\ & -3\ \\ \ \ 0\ &\ \ 0\ &\ \ 5\ \end{bmatrix} \end{align*} $$
Follow Along
$$ \begin{align*} r_{11} &= \sqrt{ 1^2 + 1^2 + 1^2 + 1^2 } = 2 \\ \\ \mathbf{q}_1 &= \frac{\mathbf{a}_1}{\lVert\mathbf{a}_1\rVert} = \frac{\mathbf{a}_1}{r_{11}} = \frac{1}{2}\begin{bmatrix}\ \ 1\ \\ \ \ 1\ \\ \ \ 1\ \\ \ \ 1\ \end{bmatrix} \\ \mathbf{r}_1 &= \begin{bmatrix}\ \ 2\ \\ \ \ 0\ \\ \ \ 0\ \end{bmatrix} \\ \\ \mathbf{p}_2 &= \text{Proj}_{\mathbf{q}_1}\mathbf{a}_2 = \frac{\langle\mathbf{q}_1,\mathbf{a}_2\rangle}{\langle\mathbf{q_1},\mathbf{q}_1\rangle}\mathbf{q}_1 = \frac{r_{12}}{1}\mathbf{q}_1 = \left(0\cdot\frac{1}{2} - 4\cdot\frac{1}{2} + 0\cdot\frac{1}{2} - 4\cdot\frac{1}{2} \right)\begin{bmatrix}\ \frac{1}{2}\ \\ \ \frac{1}{2}\ \\ \ \frac{1}{2}\ \\ \ \frac{1}{2}\ \end{bmatrix} = \begin{bmatrix} -2\ \\ -2\ \\ -2\ \\ -2\ \end{bmatrix} \\ \\ \mathbf{a}_2 - \mathbf{p}_2 &= \begin{bmatrix}\ \ 0\ \\ -4\ \\ \ \ 0\ \\ -4\ \end{bmatrix} - \begin{bmatrix} -2\ \\ -2\ \\ -2\ \\ -2\ \end{bmatrix} = \begin{bmatrix}\ \ 2\ \\ -2\ \\ \ \ 2\ \\ -2\ \end{bmatrix} \\ \\ \mathbf{q}_2 &= \frac{\mathbf{a}_2 - \mathbf{p}_2}{\lVert\mathbf{a}_2 - \mathbf{p}_2\rVert} = \frac{\mathbf{a}_2 - \mathbf{p}_2}{r_{22}} = \frac{1}{4}\begin{bmatrix}\ \ 2\ \\ -2\ \\ \ \ 2\ \\ -2\ \end{bmatrix} = \begin{bmatrix}\ \ \frac{1}{2}\ \\ -\frac{1}{2}\ \\ \ \ \frac{1}{2}\ \\ -\frac{1}{2}\ \end{bmatrix} \\ \mathbf{r}_2 &= \begin{bmatrix} -4\ \\ \ \ 4\ \\ \ \ 0\ \end{bmatrix} \\ \\ \mathbf{p}_3 &= \text{Proj}_{\mathbf{q}_1}\mathbf{a}_3 + \text{Proj}_{\mathbf{q}_2}\mathbf{a}_3 = \frac{\langle\mathbf{q}_1,\mathbf{a}_3\rangle}{\langle\mathbf{q_1},\mathbf{q}_1\rangle}\mathbf{q}_1 + \frac{\langle\mathbf{q}_2,\mathbf{a}_3\rangle}{\langle\mathbf{q_2},\mathbf{q}_2\rangle}\mathbf{q}_2 = r_{13}\mathbf{q_1} + r_{23}\mathbf{q}_2 \\ &= \left( -3\cdot\frac{1}{2} + 5\cdot\frac{1}{2} + 2\cdot\frac{1}{2} + 0\cdot\frac{1}{2} \right)\begin{bmatrix}\ \frac{1}{2}\ \\ \ \frac{1}{2}\ \\ \ \frac{1}{2}\ \\ \ \frac{1}{2}\ \end{bmatrix} + \left( -3\cdot\frac{1}{2} - 5\cdot\frac{1}{2} + 2\cdot\frac{1}{2} - 0\cdot\frac{1}{2} \right)\begin{bmatrix}\ \ \frac{1}{2}\ \\ -\frac{1}{2}\ \\ \ \ \frac{1}{2}\ \\ -\frac{1}{2}\ \end{bmatrix} \\ &= \begin{bmatrix}\ \ 1\ \\ \ \ 1\ \\ \ \ 1\ \\ \ \ 1\ \end{bmatrix} + \begin{bmatrix} -\frac{3}{2}\ \\ \ \ \frac{3}{2}\ \\ -\frac{3}{2}\ \\ \ \ \frac{3}{2}\ \end{bmatrix} = \begin{bmatrix} -\frac{1}{2}\ \\ \ \ \frac{5}{2}\ \\ -\frac{1}{2}\ \\ \ \ \frac{5}{2}\ \end{bmatrix} \\ \\ \mathbf{a}_3-\mathbf{p}_3 &= \begin{bmatrix} -3\ \\ \ \ 5\ \\ \ \ 2\ \\ \ \ 0\ \end{bmatrix} - \begin{bmatrix} -\frac{1}{2}\ \\ \ \ \frac{5}{2}\ \\ -\frac{1}{2}\ \\ \ \ \frac{5}{2}\ \end{bmatrix} = \begin{bmatrix} -\frac{5}{2}\ \\ \ \ \frac{5}{2}\ \\ \ \ \frac{5}{2}\ \\ -\frac{5}{2}\ \end{bmatrix} \\ \\ \mathbf{q}_3 &= \frac{\mathbf{a}_3-\mathbf{p}_3}{\lVert\mathbf{a}_3-\mathbf{p}_3\rVert} = \frac{\mathbf{a}_3-\mathbf{p}_3}{r_{33}} = \frac{1}{5}\begin{bmatrix} -\frac{5}{2}\ \\ \ \ \frac{5}{2}\ \\ \ \ \frac{5}{2}\ \\ -\frac{5}{2}\ \end{bmatrix} = \begin{bmatrix} -\frac{1}{2}\ \\ \ \ \frac{1}{2}\ \\ \ \ \frac{1}{2}\ \\ -\frac{1}{2}\ \end{bmatrix} \\ \mathbf{r}_3 &= \begin{bmatrix}\ \ 2\ \\ -3\ \\ \ \ 5\ \end{bmatrix} \\ \\ Q &= \begin{bmatrix}\ \ \frac{1}{2}\ &\ \ \frac{1}{2}\ & -\frac{1}{2}\ \\ \ \ \frac{1}{2}\ & -\frac{1}{2}\ &\ \ \frac{1}{2}\ \\ \ \ \frac{1}{2}\ &\ \ \frac{1}{2}\ &\ \ \frac{1}{2}\ \\ \ \ \frac{1}{2}\ & -\frac{1}{2}\ & -\frac{1}{2}\ \end{bmatrix} \\ \\ R &= \begin{bmatrix}\ \ 2\ & -4\ &\ \ 2\ \\ \ \ 0\ &\ \ 4\ & -3\ \\ \ \ 0\ &\ \ 0\ &\ \ 5\ \end{bmatrix} \\ \end{align*} $$
Exercise 16¶
Consider the following matrix $$ A = \begin{bmatrix}\ \ 3\ &\ \ 0\ & -2\ \\ \ \ 3\ & \ \ 8\ &\ \ 5\ \\ \ \ 3\ &\ \ 4\ &\ \ 0\ \\ \ \ 0\ & -4\ & -2\ \end{bmatrix} $$
Determine the $QR$ factorization of matrix $A$.
Check Your Work
$$ \begin{align*} Q &= \begin{bmatrix}\ \ \frac{1}{\sqrt{3}}\ & -\frac{1}{\sqrt{3}}\ &\ \ 0\ \\ \ \ \frac{1}{\sqrt{3}}\ &\ \ \frac{1}{\sqrt{3}}\ &\ \ \frac{1}{\sqrt{3}}\ \\ \ \ \frac{1}{\sqrt{3}}\ &\ \ 0\ & -\frac{1}{\sqrt{3}}\ \\ \ \ 0\ & -\frac{1}{\sqrt{3}}\ &\ \ \frac{1}{\sqrt{3}}\ \end{bmatrix} \\ \\ R &= \begin{bmatrix}\ \ 3\sqrt{3}\ &\ \ 4\sqrt{3}\ &\ \ \sqrt{3}\ \\ \ \ 0\ &\ \ 4\sqrt{3}\ & 3\sqrt{3}\ \\ \ \ 0\ &\ \ 0\ &\ \ \sqrt{3}\ \end{bmatrix} \end{align*} $$
Follow Along
$$ \begin{align*} r_{11} &= \sqrt{ 3^2 + 3^2 + 3^2 + 0^2 } = 3\sqrt{3} \\ \\ \mathbf{q}_1 &= \frac{\mathbf{a}_1}{\lVert\mathbf{a}_1\rVert} = \frac{\mathbf{a}_1}{r_{11}} = \frac{1}{\sqrt{3}}\begin{bmatrix}\ \ 1\ \\ \ \ 1\ \\ \ \ 1\ \\ \ \ 0\ \end{bmatrix} \\ \mathbf{r}_1 &= \begin{bmatrix}\ 3\sqrt{3}\ \\ \ \ 0\ \\ \ \ 0\ \end{bmatrix} \\ \\ \mathbf{p}_2 &= \text{Proj}_{\mathbf{q}_1}\mathbf{a}_2 = \frac{\langle\mathbf{q}_1,\mathbf{a}_2\rangle}{\langle\mathbf{q_1},\mathbf{q}_1\rangle}\mathbf{q}_1 = \frac{r_{12}}{1}\mathbf{q}_1 = \left(0\cdot\frac{1}{\sqrt{3}} + 8\cdot\frac{1}{\sqrt{3}} + 4\cdot\frac{1}{\sqrt{3}} - 4\cdot 0 \right)\begin{bmatrix}\ \frac{1}{\sqrt{3}}\ \\ \ \frac{1}{\sqrt{3}}\ \\ \ \frac{1}{\sqrt{3}}\ \\ \ \ 0\ \end{bmatrix} = \begin{bmatrix}\ \ 4\ \\ \ \ 4\ \\ \ \ 4\ \\ \ \ 0\ \end{bmatrix} \\ \\ \mathbf{a}_2 - \mathbf{p}_2 &= \begin{bmatrix}\ \ 0\ \\ \ \ 8\ \\ \ \ 4\ \\ -4\ \end{bmatrix} - \begin{bmatrix}\ \ 4\ \\ \ \ 4\ \\ \ \ 4\ \\ \ \ 0\ \end{bmatrix} = \begin{bmatrix} -4\ \\ \ \ 4\ \\ \ \ 0\ \\ -4\ \end{bmatrix} \\ \\ \mathbf{q}_2 &= \frac{\mathbf{a}_2 - \mathbf{p}_2}{\lVert\mathbf{a}_2 - \mathbf{p}_2\rVert} = \frac{\mathbf{a}_2 - \mathbf{p}_2}{r_{22}} = \frac{1}{4\sqrt{3}}\begin{bmatrix} -4\ \\ \ \ 4\ \\ \ \ 0\ \\ -4\ \end{bmatrix} = \begin{bmatrix} -\frac{1}{\sqrt{3}}\ \\ \ \ \frac{1}{\sqrt{3}}\ \\ \ \ 0\ \\ -\frac{1}{\sqrt{3}}\ \end{bmatrix} \\ \mathbf{r}_2 &= \begin{bmatrix}\ \ 4\sqrt{3}\ \\ \ \ 4\sqrt{3}\ \\ \ \ 0\ \end{bmatrix} \\ \\ \mathbf{p}_3 &= \text{Proj}_{\mathbf{q}_1}\mathbf{a}_3 + \text{Proj}_{\mathbf{q}_2}\mathbf{a}_3 = \frac{\langle\mathbf{q}_1,\mathbf{a}_3\rangle}{\langle\mathbf{q_1},\mathbf{q}_1\rangle}\mathbf{q}_1 + \frac{\langle\mathbf{q}_2,\mathbf{a}_3\rangle}{\langle\mathbf{q_2},\mathbf{q}_2\rangle}\mathbf{q}_2 = r_{13}\mathbf{q_1} + r_{23}\mathbf{q}_2 \\ &= \left( -2\cdot\frac{1}{\sqrt{3}} + 5\cdot\frac{1}{\sqrt{3}} + 0\cdot\frac{1}{\sqrt{3}} - 2\cdot 0 \right)\begin{bmatrix}\ \frac{1}{\sqrt{3}}\ \\ \ \frac{1}{\sqrt{3}}\ \\ \ \frac{1}{\sqrt{3}}\ \\ \ 0 \end{bmatrix} + \left( -2\left(-\frac{1}{\sqrt{3}}\right) + 5\cdot\frac{1}{\sqrt{3}} + 0\cdot\frac{1}{\sqrt{3}} - 2\left(-\frac{1}{\sqrt{3}}\right) \right)\begin{bmatrix} -\frac{1}{\sqrt{3}}\ \\ \ \ \frac{1}{\sqrt{3}}\ \\ \ \ 0\ \\ -\frac{1}{\sqrt{3}}\ \end{bmatrix} \\ &= \begin{bmatrix}\ \ 1\ \\ \ \ 1\ \\ \ \ 1\ \\ \ \ 0\ \end{bmatrix} + \begin{bmatrix} -3\ \\ \ \ 3\ \\ \ \ 0\ \\ -3\ \end{bmatrix} = \begin{bmatrix} -2\ \\ \ \ 4\ \\ \ \ 1\ \\ -3\ \end{bmatrix} \\ \\ \mathbf{a}_3-\mathbf{p}_3 &= \begin{bmatrix} -2\ \\ \ \ 5\ \\ \ \ 0\ \\ -2\ \end{bmatrix} - \begin{bmatrix} -2 \\ \ \ 4\ \\ \ \ 1\ \\ -3\ \end{bmatrix} = \begin{bmatrix}\ \ 0\ \\ \ \ 1\ \\ -1\ \\ \ \ 1\ \end{bmatrix} \\ \\ \mathbf{q}_3 &= \frac{\mathbf{a}_3-\mathbf{p}_3}{\lVert\mathbf{a}_3-\mathbf{p}_3\rVert} = \frac{\mathbf{a}_3-\mathbf{p}_3}{r_{33}} = \frac{1}{\sqrt{3}}\begin{bmatrix}\ \ 0\ \\ \ \ 1\ \\ -1\ \\ \ \ 1\ \end{bmatrix} = \begin{bmatrix}\ \ 0\ \\ \ \ \frac{1}{\sqrt{3}}\ \\ -\frac{1}{\sqrt{3}}\ \\ \ \ \frac{1}{\sqrt{3}}\ \end{bmatrix} \\ \mathbf{r}_3 &= \begin{bmatrix}\ \ \sqrt{3}\ \\ \ \ 3\sqrt{3}\ \\ \ \ \sqrt{3}\ \end{bmatrix} \\ \\ Q &= \begin{bmatrix}\ \ \frac{1}{\sqrt{3}}\ & -\frac{1}{\sqrt{3}}\ &\ \ 0\ \\ \ \ \frac{1}{\sqrt{3}}\ &\ \ \frac{1}{\sqrt{3}}\ &\ \ \frac{1}{\sqrt{3}}\ \\ \ \ \frac{1}{\sqrt{3}}\ &\ \ 0\ & -\frac{1}{\sqrt{3}}\ \\ \ \ 0\ & -\frac{1}{\sqrt{3}}\ &\ \ \frac{1}{\sqrt{3}}\ \end{bmatrix} \\ \\ R &= \begin{bmatrix}\ \ 3\sqrt{3}\ &\ \ 4\sqrt{3}\ &\ \ \sqrt{3}\ \\ \ \ 0\ &\ \ 4\sqrt{3}\ & 3\sqrt{3}\ \\ \ \ 0\ &\ \ 0\ &\ \ \sqrt{3}\ \end{bmatrix} \\ \\ QR &= \begin{bmatrix}\ \ \frac{1}{\sqrt{3}}\ & -\frac{1}{\sqrt{3}}\ &\ \ 0\ \\ \ \ \frac{1}{\sqrt{3}}\ &\ \ \frac{1}{\sqrt{3}}\ &\ \ \frac{1}{\sqrt{3}}\ \\ \ \ \frac{1}{\sqrt{3}}\ &\ \ 0\ & -\frac{1}{\sqrt{3}}\ \\ \ \ 0\ & -\frac{1}{\sqrt{3}}\ &\ \ \frac{1}{\sqrt{3}}\ \end{bmatrix}\begin{bmatrix}\ \ 3\sqrt{3}\ &\ \ 4\sqrt{3}\ &\ \ \sqrt{3}\ \\ \ \ 0\ &\ \ 4\sqrt{3}\ & 3\sqrt{3}\ \\ \ \ 0\ &\ \ 0\ &\ \ \sqrt{3}\ \end{bmatrix} \\ \\ &= \frac{1}{\sqrt{3}}\begin{bmatrix}\ \ 1\ & -1\ &\ \ 0\ \\ \ \ 1\ &\ \ 1\ &\ \ 1\ \\ \ \ 1\ &\ \ 0\ & -1\ \\ \ \ 0\ & -1\ &\ \ 1\ \end{bmatrix}\begin{bmatrix}\ \ 3\ &\ \ 4\ &\ \ 1\ \\ \ \ 0\ &\ \ 4\ &\ \ 3\ \\ \ \ 0\ &\ \ 0\ &\ \ 1\ \end{bmatrix}\sqrt{3} = \begin{bmatrix}\ \ 3\ &\ \ 0\ & -2\ \\ \ \ 3\ &\ \ 8\ &\ \ 5\ \\ \ \ 3\ &\ \ 4\ &\ \ 0\ \\ \ \ 0\ & -4\ & -2\ \end{bmatrix}\qquad{\color{green}\Large{\checkmark}} \end{align*} $$
Exercise 17¶
Use the inner product defined in Example 1 for $P_3$. Find an orthonormal basis for $P_3$ with this inner product derived from the standard basis
$$ \left\{ 1, x, x^2 \right\} $$
Check Your Work
$$ \left\{\, \frac{1}{\sqrt{3}},\ \frac{x}{\sqrt{2}},\ \frac{3x^2-2}{\sqrt{6}} \,\right\} $$
Follow Along
$$ \begin{align*} \|1\| &= \langle 1, 1 \rangle^{1/2} = \sqrt{ 1^2 + 1^2 + 1^2 } = \sqrt{3} \\ \\ \mathbf{q}_1 &= \frac{1}{\sqrt{3}}\qquad\longleftarrow\text{a constant function} \\ \\ x - \textrm{Proj}_{\frac{1}{\sqrt{3}}} x &= x - \frac{\left( \frac{1}{\sqrt{3}}(-1) + \frac{1}{\sqrt{3}}(0) + \frac{1}{\sqrt{3}}(1) \right)}{1}\,\frac{1}{\sqrt{3}} = x - 0 = x \\ \\ \|x\| &= \sqrt{ (-1)^2 + 0^2 + 1^2 } = \sqrt{2} \\ \\ \mathbf{q}_2 &= \frac{x}{\sqrt{2}} \\ \\ x^2 - \textrm{Proj}_{\frac{1}{\sqrt{3}}} x^2 - \textrm{Proj}_{\frac{x}{\sqrt{2}}} x^2 &= x^2 - \left( (-1)^2\frac{1}{\sqrt{3}} + 0^2\frac{1}{\sqrt{3}} + 1^2\frac{1}{\sqrt{3}} \right)\frac{1}{\sqrt{3}} - \left( (-1)^2\frac{-1}{\sqrt{2}} + 0^2\frac{0}{\sqrt{2}} + 1^2\frac{1}{\sqrt{2}} \right)\frac{x}{\sqrt{2}} \\ \\ &= x^2 - \left( \frac{2}{\sqrt{3}} \right)\frac{1}{\sqrt{3}} - (0)\frac{x}{\sqrt{2}} = x^2 - \frac{2}{3} \\ \\ \lVert x^2 - \frac{2}{3} \rVert &= \left( \left( (-1)^2-\frac{2}{3} \right)^2 + \left( 0^2-\frac{2}{3} \right)^2 + \left( 1^2-\frac{2}{3} \right)^2 \right)^{1/2} \\ \\ &= \left( \left( \frac{1}{3} \right)^2 + \left( -\frac{2}{3} \right)^2 + \left( \frac{1}{3} \right)^2 \right)^{1/2} = \left( \frac{6}{9} \right)^{1/2} = \sqrt{\frac{2}{3}} \\ \\ \mathbf{q}_3 &= \sqrt{\frac{3}{2}}\left( x^2 - \frac{2}{3} \right) = \frac{\sqrt{6}}{2}\left( x^2 - \frac{2}{3} \right) = \sqrt{6}\,\frac{3x^2-2}{6} \\ \\ \end{align*} $$
An orthonormal basis for $P_3$ using this inner product is the set of polynomials
$$ \left\{\, \frac{1}{\sqrt{3}},\ \frac{x}{\sqrt{2}},\ \frac{3x^2-2}{\sqrt{6}} \,\right\} $$
Exercise 18¶
Determine the coordinates of $p(x) = x^2 + 2x - 3$ and $q(x) = 6x + 4$ using the orthonormal basis computed in Exercise 17.
Check Your Work
$$ \begin{align*} p &\cong \left\langle -\frac{7}{\sqrt{3}},\ 2\sqrt{2},\ \frac{2}{\sqrt{6}} \right\rangle \\ \\ q &\cong \left\langle 4\sqrt{3},\ 6\sqrt{2},\ 0 \right\rangle \end{align*} $$
Follow Along
$$ \begin{align*} \left\langle p, \frac{1}{\sqrt{3}} \right\rangle &= (-4)\frac{1}{\sqrt{3}} + (-3)\frac{1}{\sqrt{3}} + 0\frac{1}{\sqrt{3}} = -\frac{7}{\sqrt{3}} \\ \\ \left\langle p, \frac{x}{\sqrt{2}} \right\rangle &= (-4)\frac{-1}{\sqrt{2}} + (-3)\frac{0}{\sqrt{2}} + 0\frac{1}{\sqrt{2}} = \frac{4}{\sqrt{2}} = 2\sqrt{2} \\ \\ \left\langle p, \frac{3x^2-2}{\sqrt{6}} \right\rangle &= (-4)\frac{1}{\sqrt{6}} + (-3)\frac{-2}{\sqrt{6}} + 0\frac{1}{\sqrt{6}} = \frac{2}{\sqrt{6}} \\ \\ -\frac{7}{\sqrt{3}}\mathbf{q}_1 + 2\sqrt{2}\,\mathbf{q}_2 + \frac{2}{\sqrt{6}}\mathbf{q}_3 &= -\frac{7}{\sqrt{3}}\frac{1}{\sqrt{3}} + 2\sqrt{2}\frac{x}{\sqrt{2}} + \frac{2}{\sqrt{6}}\frac{3x^2-2}{\sqrt{6}} \\ \\ &= -\frac{7}{3} + 2x + \frac{3x^2-2}{3} = -\frac{7}{3} + 2x + \frac{3x^2}{3} - \frac{2}{3} \\ \\ &= x^2 + 2x - \frac{9}{3} = x^2 + 2x - 3 = p\qquad{\color{green}\Large{\checkmark}} \\ \\ p(x) &\cong \left\langle -\frac{7}{\sqrt{3}},\ 2\sqrt{2},\ \frac{2}{\sqrt{6}} \right\rangle \\ \\ \left\langle q, \frac{1}{\sqrt{3}} \right\rangle &= (-2)\frac{1}{\sqrt{3}} + 4\frac{1}{\sqrt{3}} + 10\frac{1}{\sqrt{3}} = \frac{12}{\sqrt{3}} = 4\sqrt{3} \\ \\ \left\langle q, \frac{x}{\sqrt{2}} \right\rangle &= (-2)\frac{-1}{\sqrt{2}} + 4\frac{0}{\sqrt{2}} + 10\frac{1}{\sqrt{2}} = \frac{12}{\sqrt{2}} = 6\sqrt{2} \\ \\ \left\langle q, \frac{3x^2-2}{\sqrt{6}} \right\rangle &= (-2)\frac{1}{\sqrt{6}} + (4)\frac{-2}{\sqrt{6}} + 10\frac{1}{\sqrt{6}} = \frac{0}{\sqrt{6}} = 0 \\ \\ 4\sqrt{3}\,\mathbf{q}_1 + 6\sqrt{2}\,\mathbf{q}_2 + 0\mathbf{q}_3 &= 4\sqrt{3}\frac{1}{\sqrt{3}} + 6\sqrt{2}\frac{x}{\sqrt{2}} = 4 + 6x = q\qquad{\color{green}\Large{\checkmark}} \\ \\ q(x) \cong \left\langle 4\sqrt{3},\ 6\sqrt{2},\ 0 \right\rangle \end{align*} $$
Exercise 19¶
Determine the inner product of $p(x) = x^2 + 2x - 3$ and $q(x) = 6x + 4$ using the orthonormal basis computed in Exercise 17.
Check Your Work
$$ \begin{align*} \langle p, q \rangle = -4 \end{align*} $$
Follow Along
$$ \begin{align*} \left\langle p, q \right\rangle &= \left\langle \begin{bmatrix} -\frac{7}{\sqrt{3}}\ \\ \ \ 2\sqrt{2}\ \\ \ \ \frac{2}{\sqrt{6}}\ \end{bmatrix},\ \begin{bmatrix}\ \ 4\sqrt{3}\ \\ \ \ 6\sqrt{2}\ \\ \ \ 0\ \end{bmatrix} \right\rangle = -\frac{7}{\sqrt{3}}\left(4\sqrt{3}\right) + 2\sqrt{2}\left(6\sqrt{2}\right) + \frac{2}{\sqrt{6}}\left(0\right) = -28 + 24 + 0 = -4\qquad{\color{green}\Large{\checkmark}} \\ \\ \end{align*} $$
Exercise 20¶
Determine the norms of $p(x) = x^2 + 2x - 3$ and $q(x) = 6x + 4$ using the orthonormal basis computed in Exercise 17.
Check Your Work
$$ \begin{align*} \|p\| &= 5 \\ \|q\| &= 2\sqrt{30} \end{align*} $$
Follow Along
$$ \begin{align*} \|p\| &= \langle p,p \rangle^{1/2} = \left( \left(-\frac{7}{\sqrt{3}}\right)^2 + \left( 2\sqrt{2} \right)^2 + \left( \frac{2}{\sqrt{6}} \right)^2 \right)^{1/2} = \left( \frac{49}{3} + 8 + \frac{4}{6} \right)^{1/2} = \sqrt{25} = 5\qquad{\color{green}\Large{\checkmark}} \\ \\ \|q\| &= \langle q,q \rangle^{1/2} = \left( \left( 4\sqrt{3} \right)^2 + \left( 6\sqrt{2} \right)^2 + \left( 0 \right)^2 \right)^{1/2} = \left( 48 + 72 \right)^{1/2} = \sqrt{120} = 2\sqrt{30}\qquad{\color{green}\Large{\checkmark}} \end{align*} $$