Math 511: Linear Algebra

6.4 Similar Matrices

6.4.1 Change of Basis¶

$$ \require{color} \definecolor{brightblue}{rgb}{.267, .298, .812} \definecolor{darkblue}{rgb}{0.0, 0.0, 1.0} \definecolor{palepink}{rgb}{1, .73, .8} \definecolor{softmagenta}{rgb}{.99,.34,.86} \definecolor{blueviolet}{rgb}{.537,.192,.937} \definecolor{jonquil}{rgb}{.949,.792,.098} \definecolor{shockingpink}{rgb}{1, 0, .741} \definecolor{royalblue}{rgb}{0, .341, .914} \definecolor{alien}{rgb}{.529,.914,.067} \definecolor{crimson}{rgb}{1, .094, .271} \def\ihat{\mathbf{\hat{\unicode{x0131}}}} \def\jhat{\mathbf{\hat{\unicode{x0237}}}} \def\khat{\mathrm{\hat{k}}} \def\tombstone{\unicode{x220E}} \def\contradiction{\unicode{x2A33}} $$

Similar matrices represent the same linear transformation on $\mathbb{R}^n$, however with respect to a different choice of basis for both the domain and codomain. This is an important aspect of similar matrices:

Although similar matrices are typically made up of a different array of numbers, they represent the same linear transformation.

Now might be a good time to review Change of basis

6.4.2 Similarity¶

Borrowing the example from the Grant Sanderson's video, suppose we have a linear operator on $\mathbb{R}^2$ given by

$$ L(\mathbf{x}) = \left[ \begin{array}{r} -x_2 \\ x_1 \end{array} \right] $$

The operator's effect on the standard basis in $\mathbb{R}^2$ $\left\{\mathbf{e}_1,\mathbf{e}_2\right\}$ is

$$ L(\mathbf{e}_1) = \left[ \begin{array}{r} 0 \\ 1 \end{array} \right] \qquad\qquad L(\mathbf{e}_2) = \left[ \begin{array}{r} -1 \\ 0 \end{array} \right] $$

so its matrix representation with respect to the standard basis is given by

$$ A = \left[ \begin{array}{r} 0 & -1 \\ 1 & 0 \end{array} \right] $$

The question is what happens if we want to represent $L$ with respect to a different basis, say

$$ \mathbf{u}_1 = \left[ \begin{array}{r} 2 \\ 1 \end{array} \right] \qquad\qquad \mathbf{u}_2 = \left[ \begin{array}{r} -1 \\ 1 \end{array} \right] $$

Since $A$ represents the linear operator $L$ with respect to the standard basis, we may use $A$ to compute $L(\mathbf{u}_1)$ and $L(\mathbf{u}_2)$:

$$ \begin{align*} L(\mathbf{u}_1) &= A\mathbf{u}_1 = \left[ \begin{array}{r} 0 & -1 \\ 1 & 0 \end{array} \right]\left[ \begin{array}{r} \ \ \, 2 \\ 1 \end{array} \right] = \left[ \begin{array}{r} -1 \\ 2 \end{array} \right] \\ \\ L(\mathbf{u}_2) &= A\mathbf{u}_2 = \left[ \begin{array}{r} 0 & -1 \\ 1 & 0 \end{array} \right]\left[ \begin{array}{r} -1 \\ 1 \end{array} \right] = \left[ \begin{array}{r} 1 \\ -1 \end{array} \right] \end{align*} $$

The transition matrix from $\left\{\mathbf{u}_1,\mathbf{u}_2\right\}$ to the standard basis $\left\{\mathbf{e}_1,\mathbf{e}_2\right\}$ is given by

$$ U = \left[ \begin{array}{rr} 2 & 1 \\ -1 & 1 \end{array} \right] $$

and we compute its inverse $U^{-1}$ to find the transition matrix from $\left\{\mathbf{e}_1,\mathbf{e}_2\right\}$ to $\left\{\mathbf{u}_1,\mathbf{u}_2\right\}$

$$ U^{-1} = \left[ \begin{array}{rr} \frac{1}{3} & -\frac{1}{3} \\ \frac{1}{3} & \frac{2}{3} \end{array} \right] $$

To find our operator $L$ with respect to $\left\{\mathbf{u}_1,\mathbf{u}_2\right\}$, we multiply our previous results by $U^{-1}$

$$ \begin{align*} U^{-1}L(\mathbf{u}_1) &= U^{-1}A\mathbf{u}_1 = \left[ \begin{array}{rr} \frac{1}{3} & \frac{1}{3} \\ -\frac{1}{3} & \frac{2}{3} \end{array} \right]\left[ \begin{array}{r} -1 \\ 2 \end{array}\right] = \left[ \begin{array}{r} \frac{1}{3} \\ \frac{5}{3} \end{array} \right] \\ \\ U^{-1}L(\mathbf{u}_2) &= U^{-1}A\mathbf{u}_2 = \left[ \begin{array}{rr} \frac{1}{3} & \frac{1}{3} \\ -\frac{1}{3} & \frac{2}{3} \end{array} \right]\left[ \begin{array}{r} 1 \\ -1 \end{array}\right] = \left[ \begin{array}{r} 0 \\ -1 \end{array} \right] \end{align*} $$

Hence,

$$ \begin{align*} L(\mathbf{u}_1) &= \frac{1}{3}\mathbf{u}_1 + \frac{5}{3}\mathbf{u}_2 \\ \\ L(\mathbf{u}_2) &= \ 0\ \mathbf{u}_1 - 1\ \mathbf{u}_2 \end{align*} $$

and the matrix representation of $L$ with respect to $\left\{\mathbf{u}_1,\mathbf{u}_2\right\}$ is

$$ B = \left[ \begin{array}{rr} \frac{1}{3} & 0 \\ \frac{5}{3} & -1 \end{array} \right] $$

What we want to determine now is the relationship between matrices $A$ and $B$. Since the columns of $B$ are

$$ \left[ \begin{array}{r} \frac{1}{3} \\ \frac{5}{3} \end{array} \right] = U^{-1}A\mathbf{u}_1 \qquad\qquad \left[ \begin{array}{r} 0 \\ -1 \end{array} \right] = U^{-1}A\mathbf{u}_2 $$

we have that

$$ B = \left[\ U^{-1}A\mathbf{u}_1\ U^{-1}A\mathbf{u}_2\ \right] = U^{-1}A\left[ \ \mathbf{u}_1\ \mathbf{u}_2\ \right] = U^{-1}AU $$

This relates $B$ to $A$ where

(i) $B$ is the matrix representing linear operator $L$ with respect to $\left\{\mathbf{u}_1,\mathbf{u}_2\right\}$

(ii) $A$ is the matrix representing linear operator $L$ with respect to $\left\{\mathbf{e}_1,\mathbf{e}_2\right\}$

(iii) $U$ is the transition matrix for a change of basis from $\left\{\mathbf{u}_1,\mathbf{u}_2\right\}$ to $\left\{\mathbf{e}_1,\mathbf{e}_2\right\}$

In general, we say that $A$ and $B$ are similar matrices.

Definition of Similarity¶

The matrices $A,B\in\mathbb{R}^n$ are similar if there exists a nonsingular matrix $\ U$ such that

$$ B = U^{-1}AU $$

Note that this relation works in both directions since it follows immediately that

$$ B = U^{-1}AU $$

implies

$$ A = \left(U^{-1}\right)^{-1}BU^{-1} = UBU^{-1} $$

The Main Idea¶

Choosing a different ordered basis for a vector space is choosing a different coordinate system.
Similar matrices represent the same linear transformation on a vector space, expressed with respect to different choices for coordinates.

6.4.3 Main Theorem¶

Theorem 6.5.1¶

Matrix Representations of a Linear Operator With Respect to Different Bases are Similar

Let $E = \left\{\mathbf{v}_1,\mathbf{v}_2,\ldots,\mathbf{v}_n\right\}$ and $F = \left\{\mathbf{w}_1,\mathbf{w}_2,\ldots,\mathbf{w}_n\right\}$ be ordered bases on a vector space $V$, $L$ be a linear operator on $V$, and $S$ be the transition matrix from bases $F$ to $E$. If $A$ is the matrix representing $L$ with respect to $E$ and $B$ is the matrix representing $L$ with respect to $F$, then $A$ is similar to $B$, so

$$ B = S^{-1}AS $$

Proof:¶

Let $\mathbf{x}\in\mathbb{R}^n$ be arbitrary vector in vector space $V$. Vector $\mathbf{x}$ can be represented in the standard coordinate system

$$ \mathbf{x} = x_1\mathbf{e}_1 + x_2\mathbf{e}_2 + \ldots + x_n\mathbf{e}_n $$

Vector $\mathbf{x}$ can also be represented in "Jennifer's Language", that is using the $E$ basis as

$$ \mathbf{x} = \chi_1\mathbf{v}_1 + \chi_2\mathbf{v}_2 + \ldots + \chi_n\mathbf{v}_n = \begin{bmatrix}\mathbf{x}\end{bmatrix}_E $$

Finally vector $\mathbf{x}$ can be represented using "Bob's Language", that is using the $F$ basis

$$ \mathbf{x} = \xi_1\mathbf{w}_1 + \xi_2\mathbf{w}_2 + \ldots + \xi_n\mathbf{w}_n = \begin{bmatrix}\mathbf{x}\end{bmatrix}_F $$

Let us define vector $\mathbf{y}$ so that $\mathbf{y} = L(\mathbf{x})$. In our notation we write the vector and its representation in standard coordinates the same. However, since $A$ is the representation of $L$ with respect to basis $E$ we have

$$ \begin{bmatrix}\mathbf{y}\end{bmatrix}_E = A\begin{bmatrix}\mathbf{x}\end{bmatrix}_E = \begin{bmatrix} L(\mathbf{x}) \end{bmatrix}_E $$

Likewse $B$ is the representation of $L$ with respect to $F$,

$$ \begin{bmatrix}\mathbf{y}\end{bmatrix}_F = B\begin{bmatrix}\mathbf{x}\end{bmatrix}_F = \begin{bmatrix} L(\mathbf{x}) \end{bmatrix}_F $$

Matrix $S$ is the transition matrix from $F$ to $E$, so

$$ \begin{align*} \begin{bmatrix}\mathbf{x}\end{bmatrix}_E &= S\begin{bmatrix}\mathbf{x}\end{bmatrix}_F \\ \\ A\begin{bmatrix}\mathbf{x}\end{bmatrix}_E &= A\left(S\begin{bmatrix}\mathbf{x}\end{bmatrix}_F\right) = \left(AS\right)\begin{bmatrix}\mathbf{x}\end{bmatrix}_F \\ \\ \begin{bmatrix}\mathbf{y}\end{bmatrix}_E &= \left(AS\right)\begin{bmatrix}\mathbf{x}\end{bmatrix}_F \\ \\ \begin{bmatrix}\mathbf{y}\end{bmatrix}_F &= S^{-1}\begin{bmatrix}\mathbf{y}\end{bmatrix}_E \\ \\ \begin{bmatrix}\mathbf{y}\end{bmatrix}_F &= S^{-1}\left[ \left(AS\right)\begin{bmatrix}\mathbf{x}\end{bmatrix}_F\right] = \left(S^{-1}AS\right)\begin{bmatrix}\mathbf{x}\end{bmatrix}_F \\ \\ B\begin{bmatrix}\mathbf{x}\end{bmatrix}_F &= \left(S^{-1}AS\right)\begin{bmatrix}\mathbf{x}\end{bmatrix}_F \end{align*} $$

Since vector $\mathbf{x}\in V$ was chosen arbitrarily, the last equation holds for all vectors in vector space $V$. Hence

$$ B = S^{-1}AS $$

A commutative diagram showing the relationship between a matrix A and a similar matrix B through the use of the transition matrices between bases.

$\tombstone$

Linear Transformation as an Operator¶

This theorem may also be thought of in terms of the linear operator $L$ and transition operators with respect to $E$ and $F$. If we denote the transition operator from basis $F$ to basis $E$ represented by matrix $S$ using $S_{FE}$, then

$$ L = S_{FE}^{-1}\circ L\circ S_{FE} $$

A commutative diagram showing the relationship between identity operations between bases E and F and the linear operator L.

6.4.4 Exercises¶

Exercise 1¶

Let $L$ be a linear operator on $\mathbb{R}^3$ given by

$$ L(\mathbf{x}) = \begin{bmatrix} 2 x_1 - x_2 -x_3 \\ 2 x_2 - x_1 - x_3 \\ 2x_3 - x_1 - x_2 \end{bmatrix} $$

and $A$ be the matrix representation of $L$ with respect to the standard basis. If

$$ \mathbf{u}_1 = \left[\begin{array}{r} 1 \\ 1 \\ 0 \end{array}\right]\qquad \mathbf{u}_2 = \left[\begin{array}{r} 1 \\ 0 \\ 1 \end{array}\right]\qquad \mathbf{u}_3 = \left[\begin{array}{r} 0 \\ 1 \\ 1 \end{array}\right] $$

then $\left\{\mathbf{u}_1,\mathbf{u}_2,\mathbf{u}_3\right\}$ is an ordered basis for $\mathbb{R}^3$ and $U = \left[\ \mathbf{u}_1\ \mathbf{u}_2\ \mathbf{u}_3\ \right]$ is the transition matrix from $\left\{\mathbf{u}_1,\mathbf{u}_2,\mathbf{u}_3\right\}$ to the standard basis in $\mathbb{R}^3$. Determine the matrix representation $B$ of $L$ with respect to $\left\{\mathbf{u}_1,\mathbf{u}_2,\mathbf{u}_3\right\}$ by computing $U^{-1}AU$.

Check Your Work

$$ B = \left[\begin{array}{rrr} 2 & -1 & -1\\ -1 & 2 & -1 \\ -1 & -1 & 2 \end{array} \right] $$

Follow Along

The transition matrix from $\left\{\mathbf{u}_1,\mathbf{u}_2,\mathbf{u}_3\right\}$ to the standard basis is
$$ U = \left[\begin{array}{rrr} 1 & 1 & 0 \\ 1 & 0 & 1 \\ 0 & 1 & 1 \end{array}\right] $$
We need to find the inverse of this matrix, so we construct the augmented matrix $\left[\ U\ |\ I\ \right]$ and perform row operations until we have $\left[\ I\ |\ U^{-1}\ \right]$
$$ \left[\ \begin{array}{rrr|rrr} 1 & 1 & 0 & 1 & 0 & 0 \\ 1 & 0 & 1 & 0 & 1 & 0 \\ 0 & 1 & 1 & 0 & 0 & 1\end{array}\ \right] $$
and place it into reduced row echelon form.

$$ \begin{align*} \left[\ \begin{array}{rrr|rrr} 1 & 1 & 0 & \ \ \,1 & 0 & 0 \\ 1 & 0 & 1 & 0 & 1 & 0 \\ 0 & 1 & 1 & 0 & 0 & 1\end{array}\ \right]& \begin{array}{l} \ \\ -R_1 + R_2 \\ \ \end{array} \\ \\ \left[\ \begin{array}{rrr|rrr} 1 & 1 & 0 & 1 & 0 & 0 \\ 0 & -1 & 1 & -1 & 1 & 0 \\ 0 & 1 & 1 & 0 & 0 & 1\end{array}\ \right]& \begin{array}{l} \ \\ \ \ \,R_2 + R_3 \\ \ \end{array} \\ \\ \left[\ \begin{array}{rrr|rrr} 1 & 1 & 0 & 1 & 0 & 0 \\ 0 & -1 & 1 & -1 & 1 & 0 \\ 0 & 0 & 2 & -1 & 1 & 1\end{array}\ \right]& \begin{array}{l} \ \\ \ \\ \frac{1}{2}R_3 \end{array} \\ \\ \left[\ \begin{array}{rrr|rrr} 1 & 1 & 0 & 1 & 0 & 0 \\ 0 & -1 & 1 & -1 & 1 & 0 \\ 0 & 0 & 1 & -\frac{1}{2} & \frac{1}{2} & \frac{1}{2}\end{array}\ \right]& \begin{array}{l} \ \\ -R_3 + R_2 \\ \ \end{array} \\ \\ \left[\ \begin{array}{rrr|rrr} 1 & 1 & 0 & 1 & 0 & 0 \\ 0 & -1 & 0 & -\frac{1}{2} & \frac{1}{2} & -\frac{1}{2} \\ 0 & 0 & 1 & -\frac{1}{2} & \frac{1}{2} & \frac{1}{2}\end{array}\ \right]& \begin{array}{l} \ \\ -R_2 \\ \ \end{array} \\ \\ \left[\ \begin{array}{rrr|rrr} 1 & 1 & 0 & 1 & 0 & 0 \\ 0 & 1 & 0 & \frac{1}{2} & -\frac{1}{2} & \frac{1}{2} \\ 0 & 0 & 1 & -\frac{1}{2} & \frac{1}{2} & \frac{1}{2}\end{array}\ \right]& \begin{array}{l} -R_2 \\ \ \\ \ \end{array} \\ \\ \left[\ \begin{array}{rrr|rrr} 1 & 0 & 0 & \frac{1}{2} & \frac{1}{2} & -\frac{1}{2} \\ 0 & 1 & 0 & \frac{1}{2} & -\frac{1}{2} & \frac{1}{2} \\ 0 & 0 & 1 & -\frac{1}{2} & \frac{1}{2} & \frac{1}{2}\end{array}\ \right]& \end{align*} $$
Hence, the matrix $U^{-1}$ is given by
$$ U^{-1} = \left[\begin{array}{rrr} \frac{1}{2} & \frac{1}{2} & -\frac{1}{2} \\ \frac{1}{2} & -\frac{1}{2} & \frac{1}{2} \\ -\frac{1}{2} & \frac{1}{2} & \frac{1}{2} \\ \end{array} \right] $$
and $B$ may be found by performing the matrix multiplication $B = U^{-1}AU$ where $A$ is found by inspection of $L$ to be

$$ A = \left[\begin{array}{rrr} 2 & -1 & -1\\ -1 & 2 & -1 \\ -1 & -1 & 2 \end{array} \right] $$

$$ \begin{align*} B &= U^{-1}AU \\ \\ &= \left[\begin{array}{rrr} \frac{1}{2} & \frac{1}{2} & -\frac{1}{2} \\ \frac{1}{2} & -\frac{1}{2} & \frac{1}{2} \\ -\frac{1}{2} & \frac{1}{2} & \frac{1}{2} \\ \end{array} \right] \left[\begin{array}{rrr} 2 & -1 & -1\\ -1 & 2 & -1 \\ -1 & -1 & 2 \end{array} \right] \left[\begin{array}{rrr} 1 & 1 & 0 \\ 1 & 0 & 1 \\ 0 & 1 & 1 \end{array}\right]\\ \\ &= \left[\begin{array}{rrr} 2 & -1 & -1\\ -1 & 2 & -1 \\ -1 & -1 & 2 \end{array} \right] \end{align*} $$

It is pure coincidence that $A = B$.

Exercise 2¶

Let matrices $A,B\in\mathbb{R}^n$ be similar. Show that

(a) $A^T$ and $B^T$ are similar.

(b) $A^n$ and $B^n$ are similar for every positive integer $n$.

Check Your Work

(a) $A^T$ and $B^T$ are similar.

We completed previous exercises where we showed that if $S$ is a nonsingular matrix, then $S^T$ is also nonsingular, and $\left(S^T\right)^{-1} = \left(S^{-1}\right)^T$. $$ \begin{align*} B &= S^{-1}AS \\ \\ B^T &= \left(S^{-1}AS\right)^T \\ \\ &= S^TA^T\left(S^{-1}\right)^T \\ \\ &= S^TA^T\left(S^T\right)^{-1} \end{align*} $$
So $A^T$ is similar to $B^T$ via transition matrix $S^T$.

(b) $A^n$ and $B^n$ are similar for every positive integer $n$.

$$ \begin{align*} B^n &= \left(S^{-1}AS\right)^n \\ \\ &= \left(S^{-1}AS\right)\left(S^{-1}AS\right)\left(S^{-1}AS\right) \cdots \left(S^{-1}AS\right) \\ \\ &= S^{-1}A\left(SS^{-1}\right)A\left(SS^{-1}\right)A\left(S \cdots S^{-1}\right)AS \\ \\ &= S^{-1}A^nS \end{align*} $$
Hence $A^n$ is similar to $B^n$.

Your use of this self-initiated mediated course material is subject to our $An international nonprofit organization that empowers people to grow and sustain the thriving commons of shared knowledge and culture.$ Creative Commons License 4.0