Inverse matrices¶

Let $f$ be a linear function that has an inverse function $f^{-1}$ (not all functions have an inverse function). We now have $\overrightarrow{v}=f(f^{-1}(\overrightarrow{v}))$ for any vector $\overrightarrow{v}$. Using this, we can show that the inverse function $f^{-1}$ is also linear:

• Because $f$ is linear, we get $$\begin{array}{rl}\overrightarrow{v}+\overrightarrow{w}& =f(f^{-1}(\overrightarrow{v}))+f(f^{-1}(\overrightarrow{w}))\\ & =f(f^{-1}(\overrightarrow{v})+f^{-1}(\overrightarrow{w})),\end{array}$$ and applying $f^{-1}$ on both sides gives $$f^{-1}(\overrightarrow{v}+\overrightarrow{w})=f^{-1}(\overrightarrow{v})+f^{-1}(\overrightarrow{w}).$$
• Because $f$ is linear, we can also move any number $a$ inside it, so $$a\overrightarrow{v}=af(f^{-1}(\overrightarrow{v}))=f(a{f}^{-1}(\overrightarrow{v})),$$ and applying $f^{-1}$ on both sides gives $f^{-1}(a\overrightarrow{v})=a{f}^{-1}(\overrightarrow{v})$.

Recall that any matrix $A$ corresponds with a linear function. If that function has an inverse function, we say that the matrix $A$ is invertible, and its inverse matrix $A^{-1}$ is the matrix of the inverse function of $A$'s linear function.

Examples:

• Let $$R(\theta )=\left[\begin{array}{cc}\cos (\theta )& -\sin (\theta )\\ \sin (\theta )& \cos (\theta )\end{array}\right]$$ be the matrix that rotates by an angle $\theta$. Its inverse matrix rotates by the same angle, but in the opposite direction, so $$R(\theta {)}^{-1}=R(-\theta ).$$
• The inverse function of "do nothing" is "do nothing", so the inverse of the identity matrix is the identity matrix, $I^{-1}=I$.
• The inverse of the zero matrix $O$ is undefined, because the linear function corresponding to the zero matrix always outputs the zero vector, regardless of what the input was.
• The matrix $5I$ multiplies a vector by 5, because $5I\overrightarrow{v}=5\overrightarrow{v}$ for any vector $\overrightarrow{v}$ (see here for more details). The inverse matrix should divide the vector by 5, so it is $\frac{1}{5}I$.

When does an inverse matrix exist?¶

As explained here, the inverse function of $f$ exists if and only if both of these conditions are met:

1. The function $f$ gives different outputs for different inputs.
2. The function $f$ produces each output with some input.

Let $f$ be the linear function corresponding to a matrix $A$. Let's look at what it would mean to get the same output for two different inputs, $A\overrightarrow{v}=A\overrightarrow{w}$ with $\overrightarrow{v}\ne \overrightarrow{w}$. As shown here, $A\overrightarrow{v}$ is a linear combination of $A$'s columns, with numbers from $\overrightarrow{v}$ as coefficients. So for example, if $\overrightarrow{v}=(6,0)$, $\overrightarrow{w}=(3,1)$ and $$A=\left[\begin{array}{cc}1& 3\\ 2& 6\end{array}\right],$$ we would have $$6\left[\begin{array}{c}1\\ 2\end{array}\right]+0\left[\begin{array}{c}3\\ 6\end{array}\right]=3\left[\begin{array}{c}1\\ 2\end{array}\right]+1\left[\begin{array}{c}3\\ 6\end{array}\right].$$ Because $\overrightarrow{v}\ne \overrightarrow{w}$, the linear combinations don't have the same coefficients. This is not possible with linearly independent vectors, so if the columns are linearly independent, it is not possible to have $A\overrightarrow{v}=A\overrightarrow{w}$ with $\overrightarrow{v}\ne \overrightarrow{w}$.

If the columns of a matrix are linearly dependent, it means that one of them is a linear combination of others, which then means that you can multiply the matrix with two different vectors and get the same resulting vector. For example, if a matrix $A$ has 3 columns $\overrightarrow{c_1},\overrightarrow{c_2},\overrightarrow{c_3}$ and $\overrightarrow{c_1}=4\overrightarrow{c_2}-3\overrightarrow{c_3}$, we get $$A\left[\begin{array}{c}1\\ 0\\ 0\end{array}\right]=A\left[\begin{array}{c}0\\ 4\\ -3\end{array}\right].$$ So if the columns are linearly independent, there are no two different vectors that produce the same vector when multiplied with $A$, and if the columns are linearly dependent, there are such vectors.

In other words, linear independence is needed for a matrix to be invertible, and it is enough to take care of condition 1.

Let's look at the second condition. Consider the set of all output vectors of the matrix; that is, the set of all vectors $A\overrightarrow{v}$, where $\overrightarrow{v}$ is any vector. Because $A\overrightarrow{v}$ can be any linear combination of $A$'s columns, this is in fact the span of $A$'s columns.

From here, we split this into 3 cases. We can assume that the columns are linearly independent, because we will need that anyway for the first condition.

• If the matrix is an $n\times n$ matrix (same width and height, i.e. a square matrix), then independentness alone is enough to guarantee that the matrix is invertible: there are $n$ columns and each column is an $n$-dimensional vector, and the span of $n$ linearly independent $n$-dimensional vectors is the set of all $n$-dimensional vectors.
• If $\text{width}>\text{height}$, the columns are $n$-dimensional vectors and there are more than $n$ columns, where $n$ is the height of the matrix. This means that they can't be linearly independent, so matrices like this are not invertible.
• If $A$ is invertible, then the inverse $A^{-1}$ is also invertible, with the original matrix $A$ as its inverse. Consider a matrix that has width 3 and height 4. It takes in 3D vectors and outputs 4D vectors, so its inverse $A^{-1}$, if it exists, takes in 4D vectors and outputs 3D vectors, and would have $A$ as its inverse matrix. So $A^{-1}$ would be an invertible matrix with width 4 and height 3. As shown above, this isn't possible, and therefore matrices with $\text{width}<\text{height}$ can't be invertible.

We can now summarize invertibility into a very simple condition: