Graph of the Softplus Function $y = \ln(1 + e^x)$

$y = \ln(1 + e^x)$ is called the softplus function. Since $e^x > 0$ gives $1 + e^x > 1$, the logarithm is always positive, so the domain is all real numbers and the range is $y > 0$. The inside $1 + e^x$ increases monotonically and $\ln$ is increasing, so the whole function increases monotonically.

Two asymptotes

The behavior on the left and on the right could hardly be more different.

Flat toward $0$ on the left and tending to the line of slope $1$ on the right, it is a smoothly bent shape.

The derivative is the sigmoid

The derivative is exactly the logistic, or sigmoid, function.

Its value lies between $0$ and $1$, tending to $0$ as $x \to -\infty$ and to $1$ as $x \to +\infty$. The second derivative is positive, so the graph is convex throughout.

Values

For large $x$ the output is almost equal to $x$. The gap is exactly $\ln(1 + e^{-x})$, and the values appear in swapped order down the two columns because $y - x$ equals the value at $-x$.

Relation to ReLU

The ReLU $\max(0, x)$, common in machine learning, has a corner at the origin where it is not differentiable; softplus is a smooth approximation that rounds that corner off, staying rounded near the origin while nearly coinciding with ReLU far away. It is used as an activation function in neural networks, and to parametrize quantities that must stay positive, such as a variance or a population count. That its derivative is the sigmoid, equivalently that softplus is the antiderivative of the sigmoid, also simplifies gradient computations.

A trick for numerical work

Computing $\ln(1 + e^x)$ directly overflows when $x$ is large, because $e^x$ does. Implementations commonly rewrite it in the following form for numerical stability.

The exponent is never positive, so nothing overflows.