More Derivatives

This chapter contains derivative stuff that most people probably find boring, but you might like some of it. It assumes that you have read the first derivative chapter.

Derivative Rules

Libraries like sympy are sure handy for finding derivatives, but we can also find the derivative of pretty much anything by hand. In the first derivative chapter I showed you a bunch of derivative rules and I told you we'd prove them later. Let's do it.

The proofs are ordered so that they don't use anything that has not been proved before them. In these proofs, $c$ can be any constant, $f$ and $g$ are functions and $f'(x)=\frac{d}{dx}f(x)$ and $g'(x)=\frac{d}{dx}g(x)$ are their derivatives.

$\frac{d}{dx} x = 1$

The graph $y=x$ is a straight line with slope 1.

We can also plug in $f(x)=x$ to the definition of derivative:

$$\begin{align}\frac{d}{dx} x &= \lim_{h\to0} \frac{f(x+h)-f(x)}{h} \\ &= \lim_{h\to0} \frac{(\rcancel x+h)\rcancel{-x}}{h} \\ &= \lim_{h\to0} \frac h h \\ &= 1\end{align}$$

We can also use the $\lim_{\Delta x \to 0} \frac{\Delta y}{\Delta x}$ thing with $y=x$:

$$\begin{align}\frac{d}{dx} x = \lim_{\Delta x \to 0} \frac{\Delta x}{\Delta x} = 1\end{align}$$

$\frac{d}{dx} x = c$

The graph $y=c$ is a horizontal line at height $c$, so its slope is $0$.

We can also plug in $f(x)=c$ to the definition of derivative:

$$\begin{align}\frac{d}{dx} x &= \lim_{h\to0} \frac{f(x+h)-f(x)}{h} \\ &= \lim_{h\to0} \frac{c-c}{h} \\ &= \lim_{h\to0} \frac 0 h \\ &= 0\end{align}$$

We can use the $\displaystyle\lim_{\Delta x \to 0} \tfrac{\Delta y}{\Delta x}$ thing by setting $y=c$. Then $\Delta y = 0$ because $y$ is a constant; its value doesn't change.

$$\begin{align}\lim_{\Delta x \to 0} \frac{\Delta y}{\Delta x} = \lim_{\Delta x \to 0} \frac{0}{\Delta x} = 0\end{align}$$

$\frac{d}{dx}(c \cdot f(x)) = c \cdot \frac{d}{dx}(f(x))$

Let's plug in $c \cdot f(x)$ to the definition of derivative.

$$\begin{align}\frac{d}{dx}(c \cdot f(x)) &= \lim_{h\to0} \frac{\green c \cdot f(x+h) - \green c \cdot f(x)}{h} \\ &= \lim_{h\to0} \frac{\green c \cdot\bigl(f(x+h)-f(x)\bigr)}{h} \\ &= \lim_{h\to0} \left(c \cdot\frac{f(x+h)-f(x)}{h}\right) \\ &= c \cdot\lim_{h\to0} \frac{f(x+h)-f(x)}{h} \\ &= c \cdot f'(x)\end{align}$$

Here's a nice consequence of this rule.

$$\begin{align}\frac{d}{dx}(-f(x)) &= \frac{d}{dx}((-1)f(x)) \\ &= (-1)f'(x) \\ &= -f'(x)\end{align}$$

$\text{A bunch of stuff added and substracted}$

We did this in the previous chapter:

$$\begin{align}\frac{d}{dx} (6x^2+7x-123) = \frac{d}{dx}(6x^2) + \frac{d}{dx}(7x) - \frac{d}{dx}(123)\end{align}$$

If these things work, then the stuff above is correct:

This is because adding and substracting a big pile of stuff can be always turned into simpler adds and substractions...

$$\begin{align}a+b-c-d+e = (((a+b)-c)-d)+e\end{align}$$

...and then we could apply the above rules to one thing at a time, starting with $a+b$.

Let's start by proving the $+$ thing with the definition of derivative. It's kind of messy, but it works. I'll color $f$'s with blue and $g$'s with green so the mess should be a bit less unreadable.

$$\begin{align}\frac{d}{dx}(\blue{f(x)}+\green{g(x)}) &= \lim_{h\to0} \frac{(\blue{f(x+h)}+\green{g(x+h)})-(\blue{f(x)}+\green{g(x)})}{h} \\ &= \lim_{h\to0} \frac{\blue{f(x+h)}+\green{g(x+h)}-\blue{f(x)}-\green{g(x)}}{h} \\ &= \lim_{h\to0} \frac{\blue{f(x+h)}-\blue{f(x)}+\green{g(x+h)}-\green{g(x)}}{h} \\ &= \lim_{h\to0} \frac{\bigl(\blue{f(x+h)}-\blue{f(x)}\bigr)+\bigl(\green{g(x+h)}-\green{g(x)}\bigr)}{h} \\ &= \lim_{h\to0} \left(\frac{\blue{f(x+h)}-\blue{f(x)}}{h}+\frac{\green{g(x+h)}-\green{g(x)}}{h}\right) \\ &= \left(\lim_{h\to0}\frac{\blue{f(x+h)}-\blue{f(x)}}{h}\right)+\left(\lim_{h\to0}\frac{\green{g(x+h)}-\green{g(x)}}{h}\right) \\ &= \blue{f'(x)} + \green{g'(x)}\end{align}$$

We can prove the minus thing using stuff that we have proved earlier:

$$\begin{align}\frac{d}{dx}(f(x)-g(x)) &= \frac{d}{dx}\bigl(f(x)+(-g(x))\bigr) \\ &= \frac{d}{dx}f(x)+\frac{d}{dx}(-g(x)) \\ &= f'(x)+(-g'(x)) \\ &= f'(x)-g'(x)\end{align}$$

$\frac{d}{dx} f(g(x)) = f'(g(x))g'(x)$

Let's first assume that $g(x+h) \ne g(x)$ with a small $h$; that is, $g(x+h)-g(x) \ne 0$ in a $h \to 0$ limit. This means that we can divide stuff by $\green{g(x+h)-g(x)}$. We'll do the $g(x+h)=g(x)$ case separately.

Multiplying top and bottom by the green stuff is a nice and clever way to get started with this:

$$\begin{align}\frac{d}{dx} f(g(x)) &= \lim_{h\to0} \frac{f(g(x+h))-f(g(x))}{h} \\ &= \lim_{h\to0} \frac{\bigl(f(g(x+h))-f(g(x))\bigr)\green{\bigl(g(x+h)-g(x)\bigr)}} {h\cdot\green{\bigl(g(x+h)-g(x)\bigr)}} \\ &= \lim_{h\to0} \left(\frac{f(g(x+h))-f(g(x))}{\green{g(x+h)-g(x)}} \cdot \frac{\green{g(x+h)-g(x)}}{h} \right) \\ &= \left(\lim_{h\to0} \frac{f(g(x+h))-f(g(x))}{\green{g(x+h)-g(x)}} \right) \left(\lim_{h\to0} \frac{\green{g(x+h)-g(x)}}{h} \right) \\ &= \left(\lim_{h\to0} \frac{f(g(x+h))-f(g(x))}{\green{g(x+h)-g(x)}} \right) g'(x)\end{align}$$

The stuff with green on the bottom looks kind of like something that could be turned into the definition of derivative, but the defintion has just $h$ on the bottom instead of the green mess. Let's define another variable $\Delta g = \green{g(x+h)-g(x)}$. We assumed that the derivative of $g$ exists, so $g$ must be continuous. So if $h \to 0$, then $g(x+h) \to g(x)$ and $\bigl(\green{g(x+h)-g(x)}\bigr) \to 0$.

$$\begin{align}\Delta g &= \green{g(x+h)-g(x)} \\ g(x) + \Delta g &= \blue{g(x+h)}\end{align}$$$$\begin{align}\frac{d}{dx} f(g(x)) &= \left(\lim_{h\to0} \frac{f(\blue{g(x+h)})-f(g(x))}{\green{g(x+h)-g(x)}} \right) g'(x) \\ &= \left(\lim_{\Delta g \to 0} \frac{f(\blue{g(x) + \Delta g})-f(g(x))}{\green{\Delta g}}\right)g'(x) \\ &= f'(g(x))g'(x)\end{align}$$

If $g(x+h)=g(x)$ when $h \to 0$, then we can just replace $g(x+h)$ with $g(x)$ everywhere:

$$\begin{align}\frac{d}{dx} f(g(x)) &= \lim_{h\to0} \frac{f(g(x+h))-f(g(x))}{h} \\ &= \lim_{h\to0} \frac{\rcancel{f(g(x))}-\rcancel{f(g(x))}}{h} \\ &= \lim_{h\to0} \frac 0 h \\ &= 0\end{align}$$$$\begin{align}f(g(x))g'(x) &= f(g(x)) \left( \lim_{h \to 0} \frac{g(x+h)-g(x)}{h} \right) \\ &= f(g(x)) \biggl( \lim_{h \to 0} \frac{\rcancel{g(x)}-\rcancel{g(x)}}{h} \biggr) \\ &= f(g(x)) \left( \lim_{h \to 0} \frac{0}{h} \right) \\ &= f(g(x)) \cdot 0 \\ &= 0\end{align}$$

We are now done with the proof, but I also want to show you what this rule looks like with the $\displaystyle\lim_{\Delta x \to 0}$ thing. For convenience, we'll abbreviate $f(g(x))$ as $f$ and $g(x)$ as $g$. Now $f'(x) = \displaystyle\lim_{\Delta x\to0}\frac{\Delta f}{\Delta x}$, but note that $f'(\maroon{g(x)}) = \displaystyle\lim_{\Delta g \to 0}\frac{\Delta f}{\Delta\maroon g}$ because we're putting $g(x)$ into $f'(\quad)$.

$$\begin{align}\lim_{\Delta x\to0}\frac{\Delta f}{\Delta x} &= \biggl(\lim_{\Delta g\to0}\frac{\Delta f}{\Delta g}\biggr) \biggl(\lim_{\Delta x\to0}\frac{\Delta g}{\Delta x}\biggr) \\ &= \lim_{\Delta x\to0}\biggl(\frac{\Delta f}{\green{\Delta g}} \cdot \frac{\green{\Delta g}}{\Delta x}\biggr)\end{align}$$

It looks like the $\Delta g$'s just cancel out, and in fact, the green stuff we added in the beginning of the proof was $\Delta g$. This is how I knew that adding it would do something useful.

Also note that this notation doesn't handle the corner case with $g$ being a constant because then $\green{\Delta g} = 0$ and $\frac{\Delta f}{\Delta g}$ is dividing by zero.

$\frac{d}{dx} (f(x)g(x)) = f'(x)g(x) + f(x)g'(x)$

Plugging into the definition of derivative gives this:

$$\begin{align}\frac{d}{dx}(f(x)g(x)) = \lim_{h\to0} \frac{f(x+h)g(x+h)-f(x)g(x)}{h}\end{align}$$

If the top was slightly different, we could just group it nicely.

$$\begin{align}f(x+h)\green{g(x+h)}-f(x)\green{g(x+h)} = \bigl(f(x+h)-f(x)\bigr)\green{g(x+h)}\end{align}$$

We have $-f(x)g(x)$ instead of $-f(x)g(x+h)$, but we can just put a $-f(x)g(x+h)$ there if we also add $+f(x)g(x+h)$ to cancel it out. The blue parts add up to $0$:

$$\begin{align}\frac{d}{dx} (f(x)g(x)) &= \lim_{h\to0} \frac{\green{f(x+h)g(x+h)-f(x)g(x)}}{h} \\ &= \lim_{h\to0} \frac{ \\ \overbrace{\green{f(x+h)g(x+h)}\blue{-f(x)g(x+h)}}^{\text{we can group }g(x+h)} \blue{+}\overbrace{\blue{f(x)g(x+h)}\green{-f(x)g(x)}}^{\text{we can group }f(x)}}{h} \\ &= \lim_{h\to0} \frac{\bigl(\green{f(x+h)}\blue{-f(x)}\bigr)g(x+h) + f(x)\bigl(\blue{g(x+h)}\green{-g(x)}\bigr)}{h} \\ &= \lim_{h\to0} \left(\frac{\bigl(\green{f(x+h)}\blue{-f(x)}\bigr)g(x+h)}{h} + \frac{f(x)\bigl(\blue{g(x+h)}\green{-g(x)}\bigr)}{h}\right) \\ &= \left(\lim_{h\to0} \frac{\green{f(x+h)}\blue{-f(x)}}{h}\right) \left(\lim_{h\to0}g(x+h)\right) + f(x)\left(\lim_{h\to0}\frac{\blue{g(x+h)}\green{-g(x)}}{h}\right) \\ &= f'(x)g(x) + f(x)g'(x)\end{align}$$

In the previous chapter we checked the $x^2$ derivative with a fun square area thing, and we can do the same thing here if we assume that $f(x)$ and $g(x)$ are positive. The area of an $f(x)$ by $g(x)$ rectangle is $f(x)g(x)$.

$$\begin{align}\Delta(f(x)g(x)) &= \blue{\text{blue}} + \green{\text{green}} + \red{\text{red}} \\ &= \blue{\Delta f \cdot g(x)} + \green{f(x) \cdot \Delta g} + \red{\Delta f \cdot \Delta g} \\ \frac{\Delta(f(x)g(x))}{\Delta x} &= \frac{\blue{\Delta f \cdot g(x)}}{\Delta x} + \frac{\green{f(x) \cdot \Delta g}}{\Delta x} + \frac{\red{\Delta f \cdot \Delta g}}{\Delta x} \\ &= \frac{\blue{\Delta f}}{\Delta x}\blue{g(x)} + \green{f(x)}\frac{\green{\Delta g}}{\Delta x} + \frac{\red{\Delta f}}{\Delta x}\red{\Delta g}\end{align}$$

Now taking $\Delta x \to 0$ limits gives $\dfrac{d(f(x)g(x))}{dx} = f'(x)g(x) + f(x)g'(x) + f'(x)\cdot\red0$ where $\red0=\displaystyle\lim_{\Delta x \to 0}\red{\Delta g}$.

Exercise

The same rule with three functions looks like this:

$$\begin{align}\frac{d}{dx}(f(x)g(x)h(x)) = f'(x)g(x)h(x) + f(x)g'(x)h(x) + f(x)g(x)h'(x)\end{align}$$

Check this rule geometrically.

$\frac{d}{dx} x^c = c\ x^{c-1}$

It's easy to prove that this works for an individual $c$ value. For example, our very first derivative example shows that $\frac{d}{dx} x^2 = 2x$, and plugging in $c=2$ gives $2x^{2-1}=2x^1=2x$. Let's prove that this works with all positive integers instead of just 2 using a powerful technique known as induction.

Let's start by showing that this works with $c=1$:

$$\begin{align}\frac{d}{dx} x^1 = \frac{d}{dx} x = 1 = 1x^0 = 1x^{1-1}\end{align}$$

Next we'll prove that if the rule works at $c=k$ then it also works at $c=k+1$ where $k$ is a positive integer. Let's write things down just to be clear:

We assume: $\frac{d}{dx} x^k = k x^{k-1}$

We'll prove: $\frac{d}{dx} x^{k+1} = (k+1)x^{(k+1)-1}$

Let's use the $\frac{d}{dx}(f(x)g(x))$ and $\frac{d}{dx} x$ rules we proved above and the assumption.

$$\begin{align}\frac{d}{dx} x^{k+1} &= \frac{d}{dx} (x^k x^1) \\ &= \frac{d}{dx} (\green{x} \cdot \blue{x^k}) \\ &= \left(\frac{d}{dx} \green{x}\right) \cdot \blue{x^k} + \green{x} \cdot \left(\frac{d}{dx} \blue{x^k}\right) \\ &= 1x^k + x \cdot k \maroon{x^{k-1}} \\ &= 1x^k + \rcancel x \cdot k\maroon{\frac{x^k}{\rcancel{\maroon x}}} \\ &= 1\green{x^k} + k\green{x^k} \\ &= (1+k)\green{x^k} \\ &= (k+1)x^{(k+1)-1}\end{align}$$

We proved that if $\frac{d}{dx} x^k = k x^{k-1}$ then $\frac{d}{dx} x^{k+1} = (k+1)x^{(k+1)-1}$. Now we know that the rule works when $c=1$, and then if we plug in $k=1$ we know it works when $c=2$, and so on.

Note that we only proved that the rule works when $c$ is a positive integer, but it also works when $c$ is e.g. $\frac{1}{2}$. This tutorial also contains another proof that doesn't have this restriction.

Handy thing: $a^2-b^2=(a-b)(a+b)$

Proof using $(a+b)c=ac+bc$ and $(a-b)c=ac-bc$:

$$\begin{align}& \ (a-b)(a+b) \\ =&\ a(a+b)-b(a+b) \\ =&\ (aa+ab)-(ba+bb) \\ =&\ aa\rcancel{+ab}\rcancel{-ab}-bb \\ =&\ a^2 - b^2\end{align}$$

In this case we need to plug in $a=\sqrt{x+h}$ and $b=\sqrt x$.

$\frac{d}{dx} \sqrt x = \displaystyle \frac{1}{2\ \sqrt x}$

We could prove this with the $\frac{d}{dx} x^c$ rule because $\sqrt x = x^{1/2}$, but we proved the $x^c$ rule only for positive integers. Let's survive without it.

$$\begin{align}\frac{d}{dx} \sqrt x &= \lim_{h\to0} \frac{\sqrt{x+h}-\sqrt x}{h} \\ &= \lim_{h\to0} \frac{ \bigl(\sqrt{x+h}-\sqrt x\ \bigr)\green{\bigl(\sqrt{x+h}+\sqrt x\ \bigr)}}{ h \cdot \green{\bigl(\sqrt{x+h} + \sqrt x\ \bigr)}} \\ &= \lim_{h\to0} \frac{\bigl(\sqrt{x+h}\ \bigr)^2-\bigl(\sqrt x\ \bigr)^2}{ h \cdot \bigl(\sqrt{x+h} + \sqrt x\ \bigr)} \\ &= \lim_{h\to0} \frac{(\rcancel x+h)\rcancel{-x}}{h\cdot\bigl(\sqrt{x+h} + \sqrt x\ \bigr)} \\ &= \lim_{h\to0} \frac{\rcancel h}{\rcancel h\cdot\bigl(\sqrt{x+h} + \sqrt x\ \bigr)} \\ &= \lim_{h\to0} \frac{1}{\sqrt{x+h} + \sqrt x} \\ &= \frac{1}{\sqrt x + \sqrt x} \\ &= \frac{1}{2\ \sqrt x}\end{align}$$

You're probably wondering how the heck I knew that multiplying top and bottom by the green stuff would result in something useful, but this is actually a well-known trick with square roots and limits.

$\frac{d}{dx}(1/x) = \displaystyle \frac{-1}{x^2}$

We could use the $\frac{d}{dx} x^c$ rule again because $1/x$ is actually $x^{-1}$, but as before, we haven't proved that it works at $c=-1$ so we'll do this without it. Adding the green and blue stuff means that the bottoms are the same and we can combine stuff together.

$$\begin{align}\frac{d}{dx} (1/x) &= \lim_{h\to0} \frac{\frac{1}{x+h} - \frac 1 x}{h} \\ &= \lim_{h\to0} \frac{\frac{\green x}{(x+h)\green x} - \frac{\blue{x+h}}{\blue{(x+h)}x}}{h} \\ &= \lim_{h\to0} \frac{\left(\frac{x-(x+h)}{(x+h)x}\right)}{h} \\ &= \lim_{h\to0} \frac{x-(x+h)}{(x+h)xh} \\ &= \lim_{h\to0} \frac{\rcancel{x-x}-h}{(x+h)xh} \\ &= \lim_{h\to0} \frac{-\rcancel h}{(x+h)x\rcancel h} \\ &= \lim_{h\to0} \frac{-1}{(x+h)x} \\ &= \frac{-1}{x^2}\end{align}$$

The $-1$ is kind of surprising, but as you can see, it comes from the $-\frac 1 x$ on the top. $x^2$ is never negative so $\frac{1}{x^2}$ is also never negative, and $\frac{-1}{x^2}$ is always negative (it can't be $0$ because $x$ is finite). So it seems like $\frac 1 x$ is decreasing; that is, its values always get smaller as $x$ gets bigger.

>>> 1/2
0.5
>>> 1/3
0.3333333333333333
>>> 1/4
0.25
>>> 1/5
0.2

Which functions have derivatives?

This stuff is boring and it's here mostly to avoid mathematicians complaining about not having it here.

In the previous chapter we calculated some derivatives, but we just assumed that it's possible and that was it. Pretty much all functions in this tutorial have derivatives, but the derivative does not exist in these cases:

Let's see what the definition of derivative gives for $\frac{d}{dx}|x|$ at $x=0$:

$$\begin{align}\lim_{h\to0} \frac{|0+h| - |0|}{h} &= \lim_{h\to0} \frac{|h|}{h} \\ &= \lim_{h\to0} \begin{cases} \frac h h = 1 & \text{ if }h > 0 \\ \frac{-h}{h} = -1 & \text{ if }h < 0 \\ \end{cases}\end{align}$$

But $h\to0$ means that we must get the same answer from both cases. The limit does not exist.

$$\begin{align}f(x) = \begin{cases} 1 & \text{ if } x > 0 \\ 0 & \text{ if } x = 0 \\ -1 & \text{ if } x < 0 \\ \end{cases}\end{align}$$

Let's check with the definition of derivative.

$$\begin{align}f'(0) &= \lim_{h\to0} \frac{f(0+h)-f(0)}{h} \\ &= \lim_{h\to0} \frac{f(h)-0}{h} \\ &= \lim_{h\to0} \begin{cases} \frac{1}{h} & \text{ if } h > 0 \\ \frac{-1}{h}=-\frac 1 h & \text{ if } h < 0 \\ \end{cases}\end{align}$$

If $h\to0^+$, $1/h$ is dividing by something very small and positive, so it goes to $\infty$. Similarly, in the second case $h$ is negative and $1/h$ goes to $-\infty$... but $-1/h$ goes to $\infty$:

$$\begin{align}f'(0) &\mathop{=}^{\red ?} \begin{cases} \infty & \text{ if } h > 0 \\ \infty & \text{ if } h < 0 \\ \end{cases} \\ f'(0) &\mathop{=}^{\red ?} \infty\end{align}$$

Most mathematicians say that the derivative cannot be $\infty$ or $-\infty$ because many ways to use derivatives only work with finite values. In fact, limits at infinity are a kind of corner case anyway, and some people mean "the limit exists and is finite" when they say "the limit exists".

Let's update our definition of derivative:

$$\begin{align}f'(x) = \lim_{h\to0} \frac{f(x+h)-f(x)}{h} \quad \text{if the limit exists and is finite}\end{align}$$

For people who are interested in it, here's the definition of continuity:

Saying that a function $f$ is continuous at $x$ means $\displaystyle\lim_{h\to0} f(x+h) = f(x)$. Saying that $f$ is continuous means that it's continuous at every $x$ where $\displaystyle\lim_{h\to0} f(x+h)$ and $f(x)$ are defined.

The "where blablabla are defined" thing seems confusing. Let's look at an example:

Let's plug in $f(x)=\sqrt x$ and $x=0$. We can't do $\displaystyle\lim_{h\to0} \sqrt{0+h}$ because that would mean doing $\displaystyle\lim_{h\to0^-}\sqrt h$, but we can't take square roots of negative numbers. So, it doesn't really make sense to think about the continuity of $\sqrt x$ at $x=0$ because it isn't defined for $x < 0$, but $\sqrt\quad$ is nice and continuous elsewhere, so we say that $\sqrt\quad$ is continuous.