Motivationn & Definition
In this post, we explain the concept of Log-Sum-Exp .
Suppose we have three extremely small numbers: $a_1 = e^{-200}, a_2 = e^{-201}, a_3 = e^{-202}$. We are interested in the log of their sum.
There are two issues to solve here:
- The sum will still be very small. To make the result more readible, we want to take the logarithm of it.
- Dealing with extremely small numbers may lead to <a …