Inverse matrices¶

Let $\blue f$ be a linear function that has an inverse function $\red{f^{-1}}$ (not all functions have an inverse function). We now have $\vec v = \blue{f(}\red{f^{-1}(}\vec v\red)\blue)$ for any vector $\vec v$. Using this, we can show that the inverse function $\red{f^{-1}}$ is also linear:

• Because $\blue f$ is linear, we get $$ \begin{align} \vec v+\vec w &= \blue{f(}\red{f^{-1}(}\vec v\red)\blue)+\blue{f(}\red{f^{-1}(}\vec w\red)\blue) \\ &= \blue{f(}\red{f^{-1}(}\vec v\red)+\red{f^{-1}(}\vec w\red)\blue), \end{align} $$ and applying $\red{f^{-1}}$ on both sides gives $$ \red{f^{-1}(}\vec v+\vec w\red) = \red{f^{-1}(}\vec v\red) + \red{f^{-1}(}\vec w\red). $$
• Because $\blue f$ is linear, we can also move any number $a$ inside it, so $$ a\vec v = a\blue{f(}\red{f^{-1}(}\vec v\red)\blue)=\blue{f(}a\red{f^{-1}(}\vec v\red)\blue), $$ and applying $\red{f^{-1}}$ on both sides gives $\red{f^{-1}(}a\vec v\red) = a\red{f^{-1}(}\vec v\red)$.

Recall that any matrix $A$ corresponds with a linear function. If that function has an inverse function, we say that the matrix $A$ is invertible, and its inverse matrix $A^{-1}$ is the matrix of the inverse function of $A$'s linear function.

Examples:

• Let $$ R(\theta) = \begin{bmatrix} \cos(\theta) & -\sin(\theta) \\ \sin(\theta) & \cos(\theta) \end{bmatrix} $$ be the matrix that rotates by an angle $\theta$. Its inverse matrix rotates by the same angle, but in the opposite direction, so $$ R(\theta)^{-1} = R(-\theta). $$
• The inverse function of "do nothing" is "do nothing", so the inverse of the identity matrix is the identity matrix, $I^{-1} = I$.
• The inverse of the zero matrix $O$ is undefined, because the linear function corresponding to the zero matrix always outputs the zero vector, regardless of what the input was.
• The matrix $5I$ multiplies a vector by 5, because $5I\vec v = 5\vec v$ for any vector $\vec v$ (see here for more details). The inverse matrix should divide the vector by 5, so it is $\frac{1}{5}I$.

When does an inverse matrix exist?¶

As explained here, the inverse function of $f$ exists if and only if both of these conditions are met:

1. The function $f$ gives different outputs for different inputs.
2. The function $f$ produces each output with some input.

Let $f$ be the linear function corresponding to a matrix $A$. Let's look at what it would mean to get the same output for two different inputs, $A\red{\vec v} = A\blue{\vec w}$ with $\red{\vec v} \ne \blue{\vec w}$. As shown here, $A\red{\vec v}$ is a linear combination of $A$'s columns, with numbers from $\red{\vec v}$ as coefficients. So for example, if $\red{\vec v = (6,0)}$, $\blue{\vec w = (3,1)}$ and $$ A = \begin{bmatrix} \magenta1 & \green3 \\ \magenta2 & \green6 \end{bmatrix}, $$ we would have $$ \red6\begin{bmatrix} \magenta1 \\ \magenta2 \end{bmatrix} +\red0\begin{bmatrix} \green3 \\ \green6 \end{bmatrix} =\blue3\begin{bmatrix} \magenta1 \\ \magenta2 \end{bmatrix} +\blue1\begin{bmatrix} \green3 \\ \green6 \end{bmatrix}. $$ Because $\red{\vec v} \ne \blue{\vec w}$, the linear combinations don't have the same coefficients. This is not possible with linearly independent vectors, so if the columns are linearly independent, it is not possible to have $A\red{\vec v} = A\blue{\vec w}$ with $\red{\vec v} \ne \blue{\vec w}$.

If the columns of a matrix are linearly dependent, it means that one of them is a linear combination of others, which then means that you can multiply the matrix with two different vectors and get the same resulting vector. For example, if a matrix $A$ has 3 columns $\vec{c_1},\vec{c_2},\vec{c_3}$ and $\vec{c_1} = 4\vec{c_2}-3\vec{c_3}$, we get $$ A\begin{bmatrix}1\\0\\0\end{bmatrix} = A\begin{bmatrix}0\\4\\-3\end{bmatrix}. $$ So if the columns are linearly independent, there are no two different vectors that produce the same vector when multiplied with $A$, and if the columns are linearly dependent, there are such vectors.

In other words, linear independence is needed for a matrix to be invertible, and it is enough to take care of condition 1.

Let's look at the second condition. Consider the set of all output vectors of the matrix; that is, the set of all vectors $A\vec v$, where $\vec v$ is any vector. Because $A\vec v$ can be any linear combination of $A$'s columns, this is in fact the span of $A$'s columns.

From here, we split this into 3 cases. We can assume that the columns are linearly independent, because we will need that anyway for the first condition.

• If the matrix is an $n \times n$ matrix (same width and height, i.e. a square matrix), then independentness alone is enough to guarantee that the matrix is invertible: there are $n$ columns and each column is an $n$-dimensional vector, and the span of $n$ linearly independent $n$-dimensional vectors is the set of all $n$-dimensional vectors.
• If $\text{width} > \text{height}$, the columns are $n$-dimensional vectors and there are more than $n$ columns, where $n$ is the height of the matrix. This means that they can't be linearly independent, so matrices like this are not invertible.
• If $A$ is invertible, then the inverse $A^{-1}$ is also invertible, with the original matrix $A$ as its inverse. Consider a matrix that has width 3 and height 4. It takes in 3D vectors and outputs 4D vectors, so its inverse $A^{-1}$, if it exists, takes in 4D vectors and outputs 3D vectors, and would have $A$ as its inverse matrix. So $A^{-1}$ would be an invertible matrix with width 4 and height 3. As shown above, this isn't possible, and therefore matrices with $\text{width} < \text{height}$ can't be invertible.

We can now summarize invertibility into a very simple condition: