How to Take the Derivative of the Softmax Function

This blog post is inspired by MLDawn’s video on the same topic.

Suppose we have a vector: $[z_1, z_2, z_3]$ . Softmax function is in the form of:

$$S(z_1) = \frac{e^{z_1}}{e^{z_1} + e^{z_2} + e^{z_3}}$$

$$S(z_2) = \frac{e^{z_2}}{e^{z_1} + e^{z_2} + e^{z_3}}$$

$$S(z_3) = \frac{e^{z_3}}{e^{z_1} + e^{z_2} + e^{z_3}}$$

Now, we want to take the derivative of $S(z_1)$ . …