Each Section and Topic title are links. At the end of each section, there is a link back to this table.
- 2.2.1: Matrix Algebra
- 2.2.2: Matrix Multiplication is Associative
- 2.2.3: Left Distribution Over Addition
- 2.2.4: Powers of Square Matrices
- 2.2.5: The Identity Matrix
- 2.2.6: Matrix Transpose
- 2.2.7: Algebra of Matrix Transpose
- 2.2.8: Symmetric and Antisymmetric Matrices
- 2.2.9: Exercises
- copyleft
The rules for the algebra of matrices is similar, but not the same as the rules for the algebra of real numbers. For example, if $a$ and $b$ are real numbers, then
$$a + b = b + a\qquad\text{and}\qquad ab = ba $$
We say that the operations of addition and multiplication of real numbers are commutative. We have already seen that multiplication of matrices is not commutative. If
$$ A = \begin{bmatrix} 1 & 2 & 3 \\ 4 & 5 & 6 \end{bmatrix}\qquad\text{and}\qquad B = \begin{bmatrix} 1 & 2 & 3 \\ 4 & 5 & 6 \\ 7 & 8 & 9 \end{bmatrix}, $$
then we can compute the matrix product $AB$, however the product $BA$ does not exist.
Addition of matrices is commutative because for two $m\times n$ matrices, $A$, $B\ \in\mathbb{R}^{m\times n}$,
$$ A + B = B + A. $$
How can we show that this is a true statement about addition of two $m\times n$ matrices?
We can write out the matrices using patterns and dots,
$$ A = \begin{bmatrix} \ a_{11} & \ a_{12} & \ \cdots & \ a_{1n} \\ \ a_{21} & \ a_{22} & \ \cdots &\ a_{2n} \\ \ \ddots & \ \ddots & \ \cdots & \ \ddots \\ \ a_{m1} & \ a_{m2} & \ \cdots & \ a_{mn} \end{bmatrix}\qquad\text{and}\qquad B = \begin{bmatrix} \ b_{11} & \ b_{12} & \ \cdots & \ b_{1n} \\ \ b_{21} & \ b_{22} & \ \cdots &\ b_{2n} \\ \ \ddots & \ \ddots & \ \cdots & \ \ddots \\ \ b_{m1} & \ b_{m2} & \ \cdots & \ b_{mn} \end{bmatrix} $$
so
$$ \begin{align*} A + B &= \begin{bmatrix} \ a_{11} & \ a_{12} & \ \cdots & \ a_{1n} \\ \ a_{21} & \ a_{22} & \ \cdots &\ a_{2n} \\ \ \ddots & \ \ddots & \ \cdots & \ \ddots \\ \ a_{m1} & \ a_{m2} & \ \cdots & \ a_{mn} \end{bmatrix} + \begin{bmatrix} \ b_{11} & \ b_{12} & \ \cdots & \ b_{1n} \\ \ b_{21} & \ b_{22} & \ \cdots &\ b_{2n} \\ \ \ddots & \ \ddots & \ \cdots & \ \ddots \\ \ b_{m1} & \ b_{m2} & \ \cdots & \ b_{mn} \end{bmatrix} \\ \\ &= \begin{bmatrix} \ a_{11} + b_{11} & \ a_{12} + b_{12} & \ \cdots & \ a_{1n} + b_{1n} \\ \ a_{21} + b_{21} & \ a_{22} + b_{22} & \ \cdots &\ a_{2n} + b_{2n} \\ \ \ddots & \ \ddots & \ \cdots & \ \ddots \\ \ a_{m1} + b_{m1} & \ a_{m2} + b_{m2} & \ \cdots & \ a_{mn} + b_{mn} \end{bmatrix} \\ \\ &= \begin{bmatrix} \ b_{11} + a_{11} & \ b_{12} + a_{12} & \ \cdots & \ b_{1n} + a_{1n} \\ \ b_{21} + a_{21} & \ b_{22} + a_{22} & \ \cdots &\ b_{2n} + a_{2n} \\ \ \ddots & \ \ddots & \ \cdots & \ \ddots \\ \ b_{m1} + a_{m1} & \ b_{m2} + a_{m2} & \ \cdots & \ b_{mn} + a_{mn} \end{bmatrix} \\ \\ &= \begin{bmatrix} \ b_{11} & \ b_{12} & \ \cdots & \ b_{1n} \\ \ b_{21} & \ b_{22} & \ \cdots &\ b_{2n} \\ \ \ddots & \ \ddots & \ \cdots & \ \ddots \\ \ b_{m1} & \ b_{m2} & \ \cdots & \ b_{mn} \end{bmatrix} + \begin{bmatrix} \ a_{11} & \ a_{12} & \ \cdots & \ a_{1n} \\ \ a_{21} & \ a_{22} & \ \cdots &\ a_{2n} \\ \ \ddots & \ \ddots & \ \cdots & \ \ddots \\ \ a_{m1} & \ a_{m2} & \ \cdots & \ a_{mn} \end{bmatrix} \\ \\ &= B + A \end{align*} $$
The problem is that this is just property 1 of Theorem 1 and there are 8 more that need to be proved!
Theorem 1¶
Algebra of Matrices
Each of the following statements is valid for any scalars $\alpha$ and $\beta$ and for any matrices $A$, $B$, and $C$ for which the indicated operations are defined.
- $A+B = B+A$
- $(A+B) + C = A + (B+C)$
- $(AB)C = A(BC)$
- $A(B+C) = AB + AC$
- $(A+B)C = AC + BC$
- $(\alpha\beta)A = \alpha(\beta A)$
- $\alpha (AB) = (\alpha A)B = A (\alpha B)$
- $(\alpha + \beta)A = \alpha A + \beta A$
- $\alpha (A+B) = \alpha A + \alpha B$
Proof - Property 1 of Theorem 1¶
Commutative Property of Matrix Addition
We talked about a more compact notation for denoting the $m\,n$ elements of an $m\times n$ matrix,
$$ A = [a_{ij}]\qquad\text{and}\qquad B = [b_{ij}],\qquad\text{for $1\le i\le m$ and $1\le j\le n$.} $$
Now we can prove the commutative property as follows:
$$ A + B = [a_{ij}] + [b_{ij}] = [a_{ij} + b_{ij}] = [b_{ij} + a_{ij}] = [b_{ij}] + [a_{ij}] = B + A, $$
where $1\le i\le m$ and $1\le j\le n$. $\blacksquare$
Both methods of proof highlight the sum of two matrices as just the sum of $m\cdot n$ real numbers, and each of the sums of real numbers commutes.
We can prove property 2 of Theorem 1 is true as well.
Proof - Property 2 of Theorem 1¶
Associative Property of Matrix Addition
If $A$, $B$ and $C$ are $m\times n$ matrices, then
$$\begin{align*} (A + B) + C &= \left([a_{ij}] + [b_{ij}]\right) + [c_{ij}] \\ \\ &= \left([a_{ij} + b_{ij}]\right) + [c_{ij}] \\ \\ &= [a_{ij} + b_{ij}] + [c_{ij}] \\ \\ &= \left[(a_{ij} + b_{ij}) + c_{ij}\right], \\ \end{align*}$$
where $1\le i\le m$ and $1\le j\le n$.
So the sum of three $m\times n$ matrices is just $m\cdot n$ sums of three real numbers. Since addition of real numbers is associative we have that
$$\begin{align*} \left[(a_{ij} + b_{ij}) + c_{ij}\right] &= \left[a_{ij} + (b_{ij} + c_{ij})\right] \\ \\ &= [a_{ij}] + [b_{ij} + c_{ij}] \\ \\ &= [a_{ij}] + \left([b_{ij} + c_{ij}]\right) \\ \\ &= [a_{ij}] + \left([b_{ij}] + [c_{ij}]\right) \\ \\ &= A + (B + C), \\ \end{align*}$$
where $1\le i\le m$ and $1\le j\le n$. Therefore,
$$(A+B)+C = A+(B+C)$$
and addition of $m\times n$ matrices is associative. $\blacksquare$
This demonstrates the utility of representing a matrix as an array of numbers $A = [a_{ij}]$.
Skip to the end to avoid this very tedious proof for a much easier one!
Proving property 3 of Theorem 1 is more tedious. We resort to the ugly version of the definition of matrix multiplication. If $A\in\mathbb{R}^{m\times n}$, $B\in\mathbb{R}^{n\times r}$ and $C\in\mathbb{R}^{r\times s}$, then
$$ A = [a_{ik}],\ B = [b_{kl}]\ \text{and}\ D = AB = [d_{il}], $$
where $1\le i\le m$, $1\le k\le n$ and $1\le l\le r$. Using the definition of matrix multiplication we have that
$$ d_{il} = \displaystyle\sum_{k=1}^n a_{ik}b_{kl}, $$
where $1\le i\le m$ and $1\le l\le r$, and $D$ is a $m\times r$ matrix. So
$$ D = [d_{il}],\ C = [c_{lj}]\ \text{and}\ (AB)C = DC = F = [f_{ij}], $$
where $1\le i\le m$, $1\le l\le r$ and $1\le j\le s$. The definition of matrix multiplication gives us that
$$ f_{ij} = \displaystyle\sum_{l=1}^r d_{il}c_{lj}, $$
On the other hand we can define our multiplication using
$$ B = [b_{kl}],\ C = [c_{lj}]\ \text{and}\ BC = E = [e_{kj}], $$
where $1\le k\le n$, $1\le l\le r$ and $1\le j\le s$. The definition of matrix multiplication gives us that
$$ e_{kj} = \displaystyle\sum_{l=1}^r b_{kl}c_{lj}, $$
where $1\le k\le n$ and $1\le j\le s$. Therefore
$$ A(BC) = AE = G = [g_{ij}], $$
where $1\le i\le m$ and $1\le j\le s$. Using the definition of matrix multiplication again
$$ g_{ij} = \displaystyle\sum_{k=1}^n a_{ik}e_{kj}, $$
where $1\le i\le m$ and $1\le j\le s$. Now we are ready!
$$ \begin{align*} (AB)C &= DC = F = [f_{ij} ] = \left[ \displaystyle\sum_{l=1}^r d_{il}c_{lj} \right] \\ \\ &= \left[ \displaystyle\sum_{l=1}^r \left(\displaystyle\sum_{k=1}^n a_{ik}b_{kl}\right)c_{lj} \right] \\ \\ &= \left[ \displaystyle\sum_{l=1}^r \left(\displaystyle\sum_{k=1}^n a_{ik}b_{kl}c_{lj}\right) \right] \\ \\ &= \displaystyle\sum_{l=1}^r \displaystyle\sum_{k=1}^n a_{ik}b_{kl}c_{lj} \\ \\ &= \displaystyle\sum_{k=1}^n \displaystyle\sum_{l=1}^r a_{ik}b_{kl}c_{lj} \\ \\ &= \left[ \displaystyle\sum_{k=1}^n \left(\displaystyle\sum_{l=1}^r a_{ik}b_{kl}c_{lj}\right) \right] \\ \\ &= \left[ \displaystyle\sum_{k=1}^n a_{ik}\left(\displaystyle\sum_{l=1}^r b_{kl}c_{lj}\right) \right] \\ \\ &= \left[ \displaystyle\sum_{k=1}^n a_{ik}\left(e_{kj}\right) \right] \\ \\ &= \left[ \displaystyle\sum_{k=1}^n a_{ik}e_{kj} \right] = [ g_{ij} ] = G = AE = A(BC). \\ \end{align*} $$
This is very hard to read and very hard to follow.
Let's try again. This time let's remember that in section 2.1.4, we learned that matrix multiplication corresponds to function composition of the linear transformations represented by the matrices.
Proof - Property 3 of Theorem 1¶
Associative Property of Matrix Multiplication
If $A\in\mathbb{R}^{m\times n}$, $B\in\mathbb{R}^{n\times p}$ and $C\in\mathbb{R}^{p\times r}$, then let us define functions
$$ \begin{array}{lcl} \text{Function} & \text{by} & \text{Definition} \\ \hline f:\mathbb{R}^n\rightarrow\mathbb{R}^m & & \mathbf{w} = f(\mathbf{z}) = A\mathbf{z} \\ g:\mathbb{R}^p\rightarrow\mathbb{R}^n & & \mathbf{z} = g(\mathbf{y}) = B\mathbf{y} \\ h:\mathbb{R}^r\rightarrow\mathbb{R}^p & & \mathbf{y} = h(\mathbf{x}) = C\mathbf{x} \end{array} $$
Then for all $x\in\mathbb{R}^r$ we have
$$ \begin{align*} A(BC)\mathbf{x} &= f\left( (BC)\mathbf{x} \right) \\ \\ &= f\left( (g\circ h)(\mathbf{x}) \right) \\ \\ &= \left( f\circ \left( g\circ h\right) \right)(\mathbf{x}) \\ \\ &= \left( f\circ g\circ h \right)(\mathbf{x}) \\ \\ &= \left(\left(f\circ g\right)\circ h\right)(\mathbf{x}) \\ \\ &= \left(f\circ g\right)\left(h(\mathbf{x})\right) \\ \\ &= \left( f\circ g \right)\left( C\mathbf{x} \right) \\ \\ &= (AB)C\mathbf{x} \end{align*} $$
Since $A(BC)\mathbf{x} = (AB)C\mathbf{x}$ for all $\mathbf{x}\in\mathbb{R}^r$, they represent the same linear transformation and must be the same matrices. $\blacksquare$
This reveals the utility of thinking of matrix multiplication as function composition. We will see uses for all four of our new ways of visualizing matrix multiplication in this chapter.
We will prove property 4 of Theorem 1 and then the remaining 5 properties are an exercise for you to prove.
Proof - Property 4 of Theorem 1¶
Again, let us prove property 4 using the definition of function composition. If $A\in\mathbb{R}^{m\times n}$ and $B,C\in\mathbb{R}^{n\times p}$, then each matrix represents a linear transformation
$$ \begin{array}{lcl} \text{Function} & \text{by} & \text{Definition} \\ \hline f:\mathbb{R}^n\rightarrow\mathbb{R}^m & & \mathbf{z} = f(\mathbf{y}) = A\mathbf{y} \\ g:\mathbb{R}^p\rightarrow\mathbb{R}^n & & \mathbf{u} = g(\mathbf{x}) = B\mathbf{x} \\ h:\mathbb{R}^p\rightarrow\mathbb{R}^n & & \mathbf{v} = h(\mathbf{x}) = C\mathbf{x} \end{array} $$
For all $x\in\mathbb{R}^p$ we have
$$ \begin{align*} A(B + C)\mathbf{x} &= \left(f\circ(g+h)\right)(\mathbf{x}) \\ &= f\left((g+h)(\mathbf{x})\right) \\ &= f\left(g(\mathbf{x}) + h(\mathbf{x})\right) \\ &= f\left(g(\mathbf{x})\right) + f\left(h(\mathbf{x})\right) \\ &= (f\circ g)\left(\mathbf{x}\right) + (f\circ h)\left(\mathbf{x}\right) \\ &= (AB)\mathbf{x} + (AC)\mathbf{x} \\ &= (AB + AC)\mathbf{x} \end{align*} $$
Since $A(B+C)\mathbf{x} = (AB+AC)\mathbf{x}$ for all $\mathbf{x}\in\mathbb{R}^p$, they represent the same linear transformation and must be the same matrices. We have $A(B+C)=AB+AC$. $\blacksquare$
Exercise 1 - Right Distributive Property¶
Prove property 5 of Theorem 1 using function composition. For any $m\times n$ matrices $A$ and $B$, and any $n\times p$ matrix $C$,
$$(A + B)C = AC + BC$$
View Solution
If $A,B\in\mathbb{R}^{m\times n}$ and $C\in\mathbb{R}^{n\times p}$, then each matrix represents a linear transformation$$ \begin{array}{lcl} \text{Function} & \text{by} & \text{Definition} \\ \hline f:\mathbb{R}^n\rightarrow\mathbb{R}^m & & \mathbf{u} = f(\mathbf{x}) = A\mathbf{x} \\ g:\mathbb{R}^n\rightarrow\mathbb{R}^m & & \mathbf{v} = g(\mathbf{x}) = B\mathbf{x} \\ h:\mathbb{R}^p\rightarrow\mathbb{R}^n & & \mathbf{x} = h(\mathbf{z}) = C\mathbf{z} \end{array} $$
For all $z\in\mathbb{R}^p$ we have
$$ \begin{align*} (A + B)C\mathbf{z} &= \left( (f+g)\circ h \right)(\mathbf{z}) \\ &= (f+g)\left( h(\mathbf{z}) \right) \\ &= f\left( h(\mathbf{z}) \right) + g\left( h(\mathbf{z}) \right) \\ &= (f\circ h)(\mathbf{z}) + (g\circ h)(\mathbf{z}) \\ &= AC\mathbf{z} + BC\mathbf{z} \\ &= (AC + BC)\mathbf{z} \end{align*} $$
Since $(A+B)C\mathbf{z} = (AC + BC)\mathbf{z}$ for all $\mathbf{z}\in\mathbb{R}^p$, they represent the same linear transformation and must be the same matrices. We have $(A+B)C = AC+BC$. $\blacksquare$
Exercise 2 - Scalar Distribution¶
Prove that scalar multiplication distributes of matrix addition (Property 9). If $A$ and $B$ are $m\times n$ matrices and $t$ is a scalar, then
$$t(A+B) = tA + tB$$
View Solution
If $A$ and $B$ are $m\times n$ matrices and $t$ is a scalar, then $$ \begin{align*} t(A+B) &= t\left( \left[a_{ij}\right] + \left[b_{ij}\right] \right) \\ &= t\left[ a_{ij} + b_{ij} \right] \\ &= \left[ t\left( a_{ij} + b_{ij} \right) \right] \\ &= \left[ ta_{ij} + tb_{ij} \right] \\\ &= \left[ ta_{ij} \right] + \left[ tb_{ij} \right] \\ &= t\left[ a_{ij} \right] + t\left[ b_{ij} \right] \\ &= tA + tB \end{align*} $$ Therefore $t(A+B) = tA + tB$. $\blacksquare$If $A\in\mathbb{R}^{n\times n}$, then we say that $A$ is a square matrix; it has the same number of rows and columns. We can multiply a square matrix by itself and use the familiar superscript notation for the number of times that a square matrix is multiplied by itself.
$$ A^2 = AA,\ A^k = AA\cdots A\ (k\text{ times}) $$
If
$$ A = \begin{bmatrix} 1 & 1 \\ 1 & 1 \end{bmatrix} $$
then
$$ \begin{align*} A^2 &= \begin{bmatrix} 1 & 1 \\ 1 & 1 \end{bmatrix}\begin{bmatrix} 1 & 1 \\ 1 & 1 \end{bmatrix} = \begin{bmatrix} 2 & 2 \\ 2 & 2 \end{bmatrix}, \\ \\ A^3 &= AAA = (AA)A = A^2A = \begin{bmatrix} 2 & 2 \\ 2 & 2 \end{bmatrix}\begin{bmatrix} 1 & 1 \\ 1 & 1 \end{bmatrix} = \begin{bmatrix} 4 & 4 \\ 4 & 4 \end{bmatrix} \\ \\ A^4 &= AAAA = (AAA)A = A^3A = \begin{bmatrix} 4 & 4 \\ 4 & 4 \end{bmatrix}\begin{bmatrix} 1 & 1 \\ 1 & 1 \end{bmatrix} = \begin{bmatrix} 8 & 8 \\ 8 & 8 \end{bmatrix} \\ \\ A^k &= \begin{bmatrix} 2^{k-1} & 2^{k-1} \\ 2^{k-1} & 2^{k-1} \end{bmatrix} = 2^{k-1}\begin{bmatrix} 1 & 1 \\ 1 & 1 \end{bmatrix}\\ \\ A^{k+1} &= A^kA = \left(2^{k-1}\begin{bmatrix} 1 & 1 \\ 1 & 1 \end{bmatrix}\right)\begin{bmatrix} 1 & 1 \\ 1 & 1 \end{bmatrix} \\ \\ &= 2^{k-1}\left(\begin{bmatrix} 1 & 1 \\ 1 & 1 \end{bmatrix}\begin{bmatrix} 1 & 1 \\ 1 & 1 \end{bmatrix}\right) \\ \\ &= 2^{k-1}\begin{bmatrix} 2 & 2 \\ 2 & 2 \end{bmatrix} = 2^k\begin{bmatrix} 1 & 1 \\ 1 & 1 \end{bmatrix} = 2^kA.\\ \end{align*} $$
Exercise 3 - Matrix Powers¶
Using matrix $A = \begin{bmatrix}\ \ 1\ &\ \ 0\ &\ \ 0\ &\ \ 0\ \\ \ \ 0\ & -3\ &\ \ 0\ &\ \ 0\ \\ \ \ 0\ &\ \ 0\ & -2\ &\ \ 0\ \\ \ \ 0\ &\ \ 0\ &\ \ 0\ &\ \ 0\ \end{bmatrix}$, compute $A^9$, $A^0$, and $\dfrac{A^5}{5!}$.
View Solution
Since matrix $A$ is a diagonal matrix,$$ \begin{align*} A^9 &= \begin{bmatrix}\ \ 1^9\ &\ \ 0\ &\ \ 0\ &\ \ 0\ \\ \ \ 0\ & (-3)^9\ &\ \ 0\ &\ \ 0\ \\ \ \ 0\ &\ \ 0\ & (-2)^9\ &\ \ 0\ \\ \ \ 0\ &\ \ 0\ &\ \ 0\ &\ \ 0^9\ \end{bmatrix} \\ &= \begin{bmatrix}\ \ 1\ &\ \ 0\ &\ \ 0\ &\ \ 0\ \\ \ \ 0\ & -19683\ &\ \ 0\ &\ \ 0\ \\ \ \ 0\ &\ \ 0\ & -512\ &\ \ 0\ \\ \ \ 0\ &\ \ 0\ &\ \ 0\ &\ \ 0\ \end{bmatrix} \\ \\ A^0 &= I_4 \\ \\ \frac{A^5}{5!} &= \frac{1}{5!}\begin{bmatrix}\ \ 1^5\ &\ \ 0\ &\ \ 0\ &\ \ 0\ \\ \ \ 0\ & (-3)^5\ &\ \ 0\ &\ \ 0\ \\ \ \ 0\ &\ \ 0\ & (-2)^5\ &\ \ 0\ \\ \ \ 0\ &\ \ 0\ &\ \ 0\ &\ \ 0^5\ \end{bmatrix} \\ &= \frac{1}{120}\begin{bmatrix}\ \ 1\ &\ \ 0\ &\ \ 0\ &\ \ 0\ \\ \ \ 0\ & -243\ &\ \ 0\ &\ \ 0\ \\ \ \ 0\ &\ \ 0\ & -32\ &\ \ 0\ \\ \ \ 0\ &\ \ 0\ &\ \ 0\ &\ \ 0\ \end{bmatrix} \\ &= \begin{bmatrix}\ \ \frac{1}{120}\ &\ \ 0\ &\ \ 0\ &\ \ 0\ \\ \ \ 0\ & -\frac{81}{40}\ &\ \ 0\ &\ \ 0\ \\ \ \ 0\ &\ \ 0\ & -\frac{4}{15}\ &\ \ 0\ \\ \ \ 0\ &\ \ 0\ &\ \ 0\ &\ \ 0\ \end{bmatrix} \end{align*} $$
In scalar arithmetic, the multiplicative identity element $1$ has the property,
$$ a\cdot1 = 1\cdot a = a $$
for every real number $a\in\mathbb{R}$. Likewise, the identity function maps every element of its domain to itself.
$$ \mathfrak{I}(x) = x $$
for every $x\in $ the domain of $\mathfrak{I}$. If one composes the identity function $\mathfrak{I}$ with another function, the composition has the property,
$$ \left(\mathfrak{I}\circ f\right)(x) = \left(f\circ\mathfrak{I}\right)(x) = f(x), $$
for every $x$ in the domain of function $f$. Similarly, there is a square matrix $I\in\mathbb{R}^{n\times n}$, called the identity matrix, with the property that for every vector $\mathbf{x}\in\mathbb{R}^n$,
$$ I\mathbf{x} = \mathbf{x}. $$
If one multiplies the identity matrix times any other matrix $A\in\mathbb{R}^{n\times n}$, one obtains
$$ AI = IA = A. $$
Recall that when we multiply $AI$ we may also view this multiplication as
$$ AI = A[\mathbf{i}_1\ \mathbf{i}_2\ \dots\ \mathbf{i}_n] = [A\mathbf{i}_1\ A\mathbf{i}_2\ \dots\ A\mathbf{i}_n] = A = [\mathbf{a}_1\ \mathbf{a}_2\ \dots\ \mathbf{a}_n] $$
So for $1\le j\le n$ we have
$$ \begin{align*} A\mathbf{i}_j &= \mathbf{a}_j \\ \\ [\mathbf{a}_1\ \mathbf{a}_2\ \dots\ \mathbf{a}_n]\begin{bmatrix} i_{1j} \\ i_{2j} \\ \vdots \\ i_{nj} \end{bmatrix} &= \mathbf{a}_j \\ \\ i_{1j}\mathbf{a}_1 + i_{2j}\mathbf{a}_2 + \dots + i_{jj}\mathbf{a}_j + \dots + i_{nj}\mathbf{a}_n &= \mathbf{a}_j. \\ \\ \end{align*} $$
So for column $j$ in identity matrix $I$, $i_{kj} = 0$ for $1\le k\le n$ except for $k=j$, $i_{jj}=1$. We can write the $j^{\text{th}}$ column of $I$,
$$ \mathbf{i}_j = \begin{bmatrix} 0 \\ 0 \\ \vdots \\ 0 \\ 1 \\ 0 \\ \vdots \\ 0 \end{bmatrix},\qquad i_{kj} = \left\{\begin{array}{cc} 0,& k\neq j \\ 1,& k=j \end{array}\right. $$
Definition¶
Identity matrix
$$ I = [\mathbf{i}_1\ \mathbf{i}_2\ \dots \mathbf{i}_n ] = \begin{bmatrix} 1 & 0 & 0 & \dots & 0 \\ 0 & 1 & 0 & \dots & 0 \\ 0 & 0 & 1 & \dots & 0 \\ \ddots & \ddots & \ddots & \ddots & \ddots \\ 0 & 0 & 0 & \dots & 1 \end{bmatrix} $$
We define the symbol $\delta_{ij}$ (the Kronecker delta)
$$ \delta_{ij} = \left\{\begin{array}{cc} 1, & i=j \\ 0, & i\neq j \end{array}\right. $$
In this way we can write the identity matrix as $I = [\delta_{ij}]$.
In calculus, physics and engineering, we often denote the three basis vectors of our 3 dimensional space $\mathbb{R}^3$
$$ \ihat = \begin{bmatrix} 1 \\ 0 \\ 0 \end{bmatrix},\quad \jhat = \begin{bmatrix} 0 \\ 1 \\ 0 \end{bmatrix},\quad\khat = \begin{bmatrix} 0 \\ 0 \\ 1 \end{bmatrix}. $$
Our three dimensional space $\mathbb{R}^3$ is also called Euclidean $3$-space, and the identity matrix in $\mathbb{R}^3$ is
$$ I_3 = \begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{bmatrix} $$
For a positive integer $n$ and Euclidean $n$-space we may need more than 3 elementary basis vectors so we denote these vectors
$$ \mathbf{e}_1,\ \mathbf{e}_2,\ \mathbf{e}_3,\ \dots\ ,\mathbf{e}_n. $$
Here
$$ \mathbf{e}_j = \begin{bmatrix} 0 \\ 0 \\ \vdots \\ 0 \\ 1 \\ 0 \\ \vdots \\ 0 \end{bmatrix}\qquad\text{and}\qquad I = [\mathbf{e}_1\ \mathbf{e}_2\ \mathbf{e}_3\ \dots\ \mathbf{e}_n] = [\delta_{ij}] $$
Example 1 - Zero Power¶
If matrix $A$ is $n\times n$ (square), then what should we use for $A^0$?
$$ A = A^1 = A^{1+0} = A^1\left(A^0\right) = A\cdot\left(A^0\right) $$
Unlike nonzero scalars, not all square matrices have multiplicative inverses. One cannot algebraically divide or multiply both sides of this equation by a matrix that cancels matrix $A$. Also,
$$ A = A^1 = A^{0+1} = \left(A^0\right)A^1 = \left(A^0\right)A $$
We are in the same predicament that we have with scalars. If $x$ is nonzero, then $x^0=1$. As we will see in the remainder of Chapter 2, some square matrices do not have multiplication inverses. The only scalar that has no multiplicative inverse is $0$. For a variety of reasons mathematicians define $0^0=1$.
Likewise, even if matrix $A$ has no multiplicative inverse, we will define $A^0=I$.
The transpose of an $m\times n$ matrix A is an $n\times m$ matrix whose columns are the rows of $A$ and whose rows are the columns of $A$. for example if
$$ A = \begin{bmatrix} -2 &\ \ 6 &\ \ 0 \\ \ 6 & -7 & -1 \end{bmatrix}, $$
then
$$ A^T = \begin{bmatrix} -2 &\ \ 6 \\ \ \ 6 & -7 \\ \ \ 0 & -1 \end{bmatrix}. $$
Definition¶
Transpose
For any $m\times n$ matrix $A$, the transpose of matrix $A$ is an $n\times m$ matrix $A^T$ with
$$ A^T = \begin{bmatrix} a_{ij} \end{bmatrix}^T = \begin{bmatrix} a_{ji} \end{bmatrix}. $$
Notice that in the last matrix, the indices are reversed. This indicates that the element in position $(1,2)$ is now in position $(2,1)$, and the element in position $(i,j)$ is now in position $(j,i)$. This relationship defines the operation of Transpose much better than talking about exchanging rows or columns. Clearly the transpose of a scalar is just the scalar in position $(1,1)$ only.
Definition¶
Symmetric Matrices
If $A^T = A$, then we say that matrix $A$ is symmetric.
If matrix $A$ is symmetric, then $A$ must have the same number of rows and columns. The reflections of entries across the diagonal of a symmetric matrix are the same.
The following matrices are symmetric.
$$ \begin{bmatrix} 3 & \ \ 4 & \ \ 5 \\ 4 & \ \ 4 & -3 \\ 5 & -3 & \ \ 0 \end{bmatrix},\qquad\begin{bmatrix} 10 & 6 \\ 6 & 2 \end{bmatrix},\qquad\begin{bmatrix} 8 & 2 & 6 \\ 2 & 9 & 0 \\ 6 & 0 & 1 \end{bmatrix}. $$
Exercise 4 - Identify Symmetric Matrices¶
Which of the following matrices are symmetric?
$$ \begin{alignat}{2} A &= \begin{bmatrix}\ \ 3\ & -6\ & -8\ \\ \ \ 5\ &\ \ 4\ &\ \ 6\ \\ \ \ 5\ & -9\ &\ \ 4\ \\ -2\ & -4\ & -3\ \\ \ \ 3\ & -9\ &\ \ 9\ \end{bmatrix}, & B &= \begin{bmatrix}\ \ 0\ &\ \ 2\ & -5\ \\ \ \ 9\ & -5\ &\ \ 0\ \\ -3\ &\ \ 5\ &\ \ 4\ \end{bmatrix}, \\ \\ C &= \begin{bmatrix}\ \ 7\ &\ \ 1\ &\ \ 4\ \\ \ \ 1\ & -7\ & -5\ \\ \ \ 4\ & -5\ &\ \ 6\ \end{bmatrix}, & D &= \begin{bmatrix} -9\ &\ \ 6\ &\ \ 3\ &\ \ 3\ \\ -1\ & -6\ &\ \ 4\ &\ \ 3\ \\ -2\ & \ \ 0\ &\ \ 5\ & -6\ \\ \ \ 5\ & -1\ & -4\ & -7\ \end{bmatrix}, \\ \\ E &= \begin{bmatrix}\ \ 5\ &\ \ 2\ &\ \ 2\ & -5\ \\ \ \ 2\ &\ \ 2\ &\ \ 1\ &\ \ 1 \ \\ \ \ 2\ &\ \ 1\ &\ \ 1\ &\ \ 7\ \\ -5\ &\ \ 1\ &\ \ 7\ & -2\ \end{bmatrix} \end{alignat} $$
View Solution
C,EThe transpose of an $m\times n$ matrix $A$ is an $n\times m$ matrix whose rows are the columns of $A$ and whose columns are the rows of $A$. We can express this algebraically.
The $j^{\text{th}}$ column of matrix $A$ is the $j^{\text{th}}$ row of matrix $A^T$ is written:
$$ \begin{align*} \mathbf{a}_j &= \begin{bmatrix} a_{1j} \\ \vdots \\ a_{mj} \end{bmatrix} \\ \\ (\mathbf{a}^T)^j &= \left[\, a_{1j}\ \dots\ a_{mj} \,\right] = \left[\, (a^T)_{j1}\ \dots\ (a^T)_{jm} \,\right] \\ \\ (\mathbf{a}_j)^T &= (\mathbf{a}^T)^j \end{align*} $$
The $i^{\text{th}}$ row of matrix $A$ is the $i^{\text{th}}$ column of matrix $A^T$ is written:
$$ (\mathbf{a}^i)^T = (\mathbf{a}^T)_i $$
The transpose of $m\times n$ matrix $A$ with columns $\mathbf{a}_j$ is $n\times m$ matrix $A^T$ with rows $\left(\mathbf{a}_j\right)^T$. $A^T$ is a matrix made up of dual vectors. The transpose of a matrix is also the matrix dual of the matrix.
We need to understand how transpose interacts with our other matrix algebra operations.
Theorem 2¶
Algebraic Properties of Matrix Transpose
For any $m\times n$ matrices $A$ and $B$ and scalar $t$:
- $\left(A^T\right)^T = \left([a_{ij}]^T\right)^T = \left([a_{ji}]\right)^T = [a_{ji}]^T = [a_{ij}] = A$
- $\left(t A\right)^T = t A^T$
- $\left(A + B\right)^T = \left([a_{ij}] + [b_{ij}] \right)^T = [a_{ij} + b_{ij}]^T = [a_{ji} + b_{ji}] = [a_{ji}] + [b_{ji}] = A^T + B^T$
For any $m\times n$ matrix $A$ and $n\times p$ matrix $B$:
- $\left(AB\right)^T = B^TA^T$
Proof (1-3):¶
If $A$ and $B$ are $m\times n$ matrices and $t$ is a scalar, then:
$\left(A^T\right)^T = A$ $$ \begin{align*} \left(A^T\right)^T &= \left(\begin{bmatrix} a_{ij} \end{bmatrix}^T\right)^T = \left(\begin{bmatrix} a_{ji} \end{bmatrix}\right)^T = \begin{bmatrix} a_{ij} \end{bmatrix} = A \\ \end{align*} $$
$(tA)^T = tA^T$ $$ (tA)^T = \left[ t\mathbf{a}_j \right]^T = \left[ ( t\mathbf{a}_j )^T \right] = \left[ (t\mathbf{a}^T)^j \right] = \left[ t(\mathbf{a}^T)^j \right] = t \left[ (\mathbf{a}^T)^j \right] = tA^T $$
$(A + B)^T = A^T + B^T$ $$ \begin{align*} A^T + B^T &= \begin{bmatrix} a_{ij} \end{bmatrix}^T + \begin{bmatrix} b_{ij} \end{bmatrix}^T = \begin{bmatrix} a_{ji} \end{bmatrix} + \begin{bmatrix} b_{ji} \end{bmatrix} \\ &= \begin{bmatrix} a_{ji} + b_{ji} \end{bmatrix} = \begin{bmatrix} a_{ij} + b_{ij} \end{bmatrix}^T \\ &= \left( A + B \right)^T \\ \end{align*} $$
This proves identities 1, 2, and 3 of Theorem 2. $\blacksquare$
Proving identity 4 of Theorem 2 requires use of our understanding of matrix multiplication. Consider first the matrix product $\mathbf{x}^T\mathbf{y}$ of two $n\times 1$ vectors $x$ and $y$.
$$ \begin{align*} \mathbf{x}^T\mathbf{y} &= \begin{bmatrix} x_1 & x_2 & \cdots & x_n \end{bmatrix}\begin{bmatrix} y_1 \\ y_2 \\ \vdots \\ y_n \end{bmatrix} \\ \\ &= x_1y_1 + x_2y_2 + \cdots + x_ny_n \\ \\ &= y_1x_1 + y_2x_2 + \cdots + y_nx_n \\ \\ &= \begin{bmatrix} y_1 & y_2 & \cdots & y_n \end{bmatrix}\begin{bmatrix} x_1 \\ x_2 \\ \vdots \\ x_n \end{bmatrix} = \mathbf{y}^T\mathbf{x}. \end{align*} $$
Notice that when rewriting the transpose of a product as the product of transposes, the factors had to switch places. This is necessary because we need the inner dimensions to match for matrix product.
Exercise 5 - Transpose of a Product¶
Use the inner product view of matrix multiplication to prove identity 4 of Theorem 2. If $A$ is an $m\times n$ matrix and $B$ is an $n\times p$ matrix, then:
$$(AB)^T = B^TA^T.$$
View Solution
The product of matrix $m\times n$ matrix $A$ and $n\times p$ matrix $B$ is $m\times p$ matrix $C$ so that the elements of $C$ are given by the $m\times p$ products$$ \begin{bmatrix} c_{ij} \end{bmatrix} = \mathbf{a}^i\mathbf{b}_j. $$
We showed above that
$$ \left( \mathbf{a}^i\mathbf{b}_j \right)^T = (\mathbf{b}_j)^T(\mathbf{a}^i)^T $$
So the transpose of $C$ is
$$ C^T = \left[ c_{ij} \right]^T = \left[ c_{ji} \right] = \left[ \mathbf{a}^j\mathbf{b}_i \right] = \left[ \left( \mathbf{b}_i \right)^T\,\left( \mathbf{a}^j \right)^T \right] = \left[ \left( \mathbf{b}^T \right)^i\,\left( \mathbf{a}^T \right)_j \right] = B^TA^T. $$
This completes the proof. $\blacksquare$
Definition¶
Antisymmetric Matrices
If a square $n\times n$ matrix has the property that $A^T = -A$, then we call matrix $A$ antisymmetric.
Antisymmetric matrices are also referred to as skew-symmetric.
We can show that a few useful matrices are symmetric or antisymmetric.
For instance, the matrix $B = A^T A$ is symmetric since
$$ B^T = \left(A^T A\right)^T = A^T\left(A^T\right)^T = A^T A = B. $$
Likewise, $C = AA^T$ is symmetric because
$$ C^T = \left(AA^T\right)^T = \left(A^T\right)^T A^T = AA^T = C. $$
Every square $n\times n$ matrix may be written as the sum of a symmetric matrix and an antisymmetric matrix
Theorem 3¶
Decomposition into symmetric and skew-symmetric
Any square $n\times n$ matrix $A$, can be decomposed to symmetric and skew-symmetric terms.
$$ \begin{align*} A_+ &= \dfrac{A+A^T}{2} \text{ is a symmetric matrix, and} \\ A_- &= \dfrac{A-A^T}{2} \text{ is a skew-symmetric matrix} \end{align*} $$
We may use these definitions to write $A$ and $A^T$ as
$$ \begin{align*} A &= A_+ + A_- \\ \\ A^T &= A_+ - A_- \end{align*} $$
Exercise 6 - Prove Theorem 3¶
Prove Theorem 3.
View Solution
First we prove that $A_+$ is a symmetric matrix. $$ \begin{align*} (A_+)^T &= \left(\frac{A+A^T}{2}\right)^T = \dfrac{A^T+\left(A^T\right)^T}{2} = \dfrac{A^T+A}{2} = \dfrac{A+A^T}{2} = A_+ \end{align*} $$ Next we show that $A_-$ is skew-symmetric (antisymmetric). $$ \begin{align*} (A_-)^T &= \left(\frac{A-A^T}{2}\right)^T = \dfrac{A^T-\left(A^T\right)^T}{2} = \dfrac{A^T-A}{2} = -\dfrac{A-A^T}{2} = -A_- \\ \end{align*} $$ Then we show that the sum of $A_+$ and $A_-$ is $A$. $$ \begin{align*} A_+ + A_- &= \frac{A+A^T}{2} + \frac{A-A^T}{2} = \frac{A}{2} + \frac{A^T}{2} + \frac{A}{2} - \frac{A^T}{2} = A \end{align*} $$ Finally we show that the difference of $A_+$ and $A_-$ is $A^T$. $$ \begin{align*} A_+ - A_- &= \frac{A+A^T}{2} - \frac{A-A^T}{2} = \frac{A}{2} + \frac{A^T}{2} - \frac{A}{2} + \frac{A^T}{2} = A^T \end{align*} $$Matrix Algebra Exercises¶
Exercise 7 - Compute the Following¶
Suppose $A = \begin{bmatrix}\ \ 1\ & -2\ \\ -3\ &\ \ 0\ \end{bmatrix}$, $B=\begin{bmatrix}\ \ 1\ &\ \ 2\ \\ \ \ 2\ & -1\ \end{bmatrix}$, and $C = \begin{bmatrix}\ \ 1\ &\ \ 1\ \\ -2\ & -3\ \end{bmatrix}$. Compute the following:
- $A(B + C)$
- $(A + B)C$
- $2A^T + B^T$
- $\left(A^TB^T\right)C$
View Solution
$$ \begin{align*} A(B + C) &= \begin{bmatrix}\ \ 1\ & -2\ \\ -3\ &\ \ 0\ \end{bmatrix}\left( \begin{bmatrix}\ \ 1\ &\ \ 2\ \\ \ \ 2\ & -1\ \end{bmatrix} + \begin{bmatrix}\ \ 1\ &\ \ 1\ \\ -2\ & -3\ \end{bmatrix} \right) \\ \\ &= \begin{bmatrix}\ \ 1\ & -2\ \\ -3\ &\ \ 0\ \end{bmatrix}\begin{bmatrix}\ \ 2\ &\ \ 3\ \\ \ \ 0\ & -4\ \end{bmatrix} = \begin{bmatrix}\ \ 2\ &\ \ 11\ \\ -6\ & -9\ \end{bmatrix} \\ \\ (A + B)C &= \left( \begin{bmatrix}\ \ 1\ & -2\ \\ -3\ &\ \ 0\ \end{bmatrix} + \begin{bmatrix}\ \ 1\ &\ \ 2\ \\ \ \ 2\ & -1\ \end{bmatrix} \right) \begin{bmatrix}\ \ 1\ &\ \ 1\ \\ -2\ & -3\ \end{bmatrix} \\ \\ &= \begin{bmatrix}\ \ 2\ &\ \ 0\ \\ -1\ & -1\ \end{bmatrix}\begin{bmatrix}\ \ 1\ &\ \ 1\ \\ -2\ & -3\ \end{bmatrix} = \begin{bmatrix}\ \ 2\ &\ \ 2\ \\ \ \ 1\ &\ \ 2\ \end{bmatrix} \\ \\ 2A^T + B^T &= 2\begin{bmatrix}\ \ 1\ & -2\ \\ -3\ &\ \ 0\ \end{bmatrix}^T + \begin{bmatrix}\ \ 1\ &\ \ 2\ \\ \ \ 2\ & -1\ \end{bmatrix}^T \\ \\ &= \begin{bmatrix}\ \ 2\ & -6\ \\ -4\ &\ \ 0\ \end{bmatrix} + \begin{bmatrix}\ \ 1\ &\ \ 2\ \\ \ \ 2\ & -1\ \end{bmatrix} = \begin{bmatrix}\ \ 3\ & -4\ \\ -2\ & -1\ \end{bmatrix} \\ \\ \left(A^TB^T\right)C &= \left(BA\right)^TC = \left( \begin{bmatrix}\ \ 1\ &\ \ 2\ \\ \ \ 2\ & -1\ \end{bmatrix}\begin{bmatrix}\ \ 1\ & -2\ \\ -3\ &\ \ 0\ \end{bmatrix} \right)^T\begin{bmatrix}\ \ 1\ &\ \ 1\ \\ -2\ & -3\ \end{bmatrix} \\ \\ &= \begin{bmatrix} -5\ & -2\ \\ \ \ 5\ & -4\ \end{bmatrix}^T\begin{bmatrix}\ \ 1\ &\ \ 1\ \\ -2\ & -3\ \end{bmatrix} \\ \\ &= \begin{bmatrix} -5\ &\ \ 5\ \\ -2\ &\ -4\ \end{bmatrix}\begin{bmatrix}\ \ 1\ &\ \ 1\ \\ -2\ & -3\ \end{bmatrix} = \begin{bmatrix} -15\ & -20\ \\ \ \ 6\ &\ \ 10\ \end{bmatrix} \end{align*} $$
Exercise 8 - Find All¶
Find all $2\times 2$ symmetric matrices $A$ so that $A^TA = O$.
View Solution
If $2\times 2$ matrix $A$ is symmetric, then $A^T = A$, or $\begin{bmatrix}\ a\ &\ b\ \\ \ c\ &\ d\ \end{bmatrix} = \begin{bmatrix}\ a\ &\ b\ \\ \ c\ &\ d\ \end{bmatrix}$. This equation implies that $b=c$, so we can represent $2\times 2$ symmetric matrices using only three unknowns, $A = \begin{bmatrix}\ a\ &\ b\ \\ \ b\ &\ d\ \end{bmatrix}$. Hence
$$ A^TA = AA = A^2 = \begin{bmatrix}\ a\ &\ b\ \\ \ b\ &\ d\ \end{bmatrix}\begin{bmatrix}\ a\ &\ b\ \\ \ b\ &\ d\ \end{bmatrix} = \begin{bmatrix}\ a^2+b^2\ &\ ab+bd\ \\ \ ab+bd\ &\ b^2+d^2\ \end{bmatrix} = \begin{bmatrix}\ 0\ &\ 0\ \\ \ 0\ &\ 0\ \end{bmatrix} $$
Now $a^2 + b^2 = 0$ is the circle of radius zero centered at the origin of the $ab$-plane, so $a=0$ and $b=0$. Hence $b^2+d^2=0$ gives that $d=0$. Finally if $a=b=d=0$, then $ab+bd=0$. The only $2\times 2$ matrix whose product $A^TA=O$ is the zero matrix $A=O$.
Exercise 9 - Idempotent¶
An $n\times n$ matrix $A$ is called idempotent if and only if $A^2 = AA = A$. Suppose that $A$ and $B$ are both idempotent $n\times n$ matrices. Show that if $AB = BA$, then $AB$ is idempotent as well.
View Solution
If $A$ and $B$ are both $n\times n$ idempotent matrices, then $A^2=A$ and $B^2=B$. If we also know that $AB=BA$, then
$$ (AB)^2 = (AB)(AB) = A(BA)B = A(AB)B = (AA)(BB) = (A)(B) = AB $$
Hence, $AB$ is an $n\times n$ idempotent matrix.
Exercise 10 - Trace¶
The trace of an $n\times n$ matrix is the sum of the diagonal elements of the matrix, denoted $\textrm{tr}(A)$.
$$ \textrm{tr}(A) = \textrm{tr}\left( [ a_{ij} ] \right) = \sum_{j=1}^n a_{jj} $$
Show that
(a) If $A$ and $B$ are $n\times n$ matrices, then $\textrm{tr}(A+B) = \textrm{tr}(A) + \textrm{tr}(B)$.
(b) If $A$ and $B$ are $n\times n$ matrices and $c\in\mathbb{R}$ is a scalar, then $\textrm{tr}(cA) = c\,\textrm{tr}(A)$.
View Solution
(a) If $A$ and $B$ are $n\times n$ matrices, then$$ \textrm{tr}(A+B) = \textrm{tr}\left( [ a_{ij}+b_{ij} ] \right) = \sum_{j=1}^n \left( a_{jj}+b_{jj} \right) = \sum_{j=1}^n a_{jj} + \sum_{j=1}^n b_{jj} = \textrm{tr}(A) + \textrm{tr}(B) $$
(b) If $A$ is and $n\times n$ matrix and $c\in\mathbb{R}$ is a scalar, then
$$ \textrm{tr}(cA) = \textrm{tr}\left( [ ca_{ij} ] \right) = \sum_{j=1}^n ca_{jj} = c\sum_{j=1}^n a_{jj} = c\,\textrm{tr}(A) $$
