y = \dfrac{1}{1+e^{-x}}

The Logistic (Sigmoid) Function $y = \dfrac{1}{1+e^{-x}}$

The logistic function $y = \dfrac{1}{1+e^{-x}}$ , also called the standard sigmoid, is the archetypal S-shaped curve¹. It confines the output between $0$ and $1$ whatever the real input, which makes it well suited to representing a probability or a degree of being on or off.

Domain and range

The domain is all real numbers
The range is the open interval $(0, 1)$
It increases monotonically
The lines $y = 0$ and $y = 1$ are horizontal asymptotes

The denominator $1 + e^{-x}$ is always positive, and the values at the two ends are never reached.

Symmetry

Since $f(-x) = 1 - f(x)$ , the graph has point symmetry about $\left( 0, \dfrac{1}{2} \right)$ .

$x$	$f(x)$	$f(-x)$	Sum
$1$	$\approx 0.7311$	$\approx 0.2689$	$1$
$2$	$\approx 0.8808$	$\approx 0.1192$	$1$
$4$	$\approx 0.9820$	$\approx 0.0180$	$1$

Monotonicity

The function has the striking property that its derivative can be written in terms of itself.

f'(x) = f(x)\bigl(1 - f(x)\bigr) = \frac{e^{-x}}{(1+e^{-x})^2}

It is always positive, so the function increases over the whole line. The slope is greatest at the center $x = 0$ , where it equals $\dfrac{1}{4}$ . Where the value is near $0$ or $1$ the slope is almost $0$ , and that is what leads to the vanishing gradient in deep networks.

Inflection point

The second derivative is $f'' = f(1-f)(1-2f)$ , whose sign changes at $f = \dfrac{1}{2}$ , that is at $x = 0$ . The point $\left( 0, \dfrac{1}{2} \right)$ is the only inflection point, and the curve turns there from concave up to concave down.

Relation to the hyperbolic tangent

f(x) = \frac{1}{2}\left(1 + \tanh\frac{x}{2}\right)

It coincides with $\tanh$ widened by a factor of $2$ horizontally, halved vertically and lifted by $\dfrac{1}{2}$ .

Item	$\dfrac{1}{1+e^{-x}}$	$\tanh x$
Range	$(0, 1)$	$(-1, 1)$
Center of symmetry	$\left( 0, \dfrac{1}{2} \right)$	$(0, 0)$
Slope at the origin	$\dfrac{1}{4}$	$1$

They are the same shape, differing only in range and scale.

Applications and history

In the nineteenth century the Belgian mathematician Verhulst introduced it as the solution of the logistic equation, which models the growth of a population in an environment of limited resources.

The model of a probability in logistic regression
An activation function in neural networks
Population growth and the progress of a chemical reaction

Logistic function, Wikipedia

The Logistic (Sigmoid) Function y=11+e−xy = \dfrac{1}{1+e^{-x}}y=1+e−x1​