Tutorial on the logistic function

The Logistic Function


In [1]:
%load_ext version_information
%version_information theano, numpy


Out[1]:
SoftwareVersion
Python2.7.6 64bit [GCC 4.8.2]
IPython4.0.1
OSLinux 4.1.13 boot2docker x86_64 with Ubuntu 14.04 trusty
theano0.7.0.dev-30cc6380863b08a3a90ecbe083ddfb629a56161d
numpy1.8.2
Tue Dec 08 03:33:04 2015 UTC

We want to compute the logistic function:

$$s(x) = \frac{1}{1 + e^{-x}}$$

We apply the function element-wise on the tensor like so.


In [2]:
import theano
import theano.tensor as T
import numpy as np

# Same recipe as last tutorial: 
x = T.dmatrix("x")                  # Define variables
s = 1 / (1 + T.exp(-x))             # Build symbolic expression
logistic = theano.function([x], s)  # Compile function

my_matrix = np.array([[0, 1], [-1, -2]])
logistic(my_matrix)


Out[2]:
array([[ 0.5       ,  0.73105858],
       [ 0.26894142,  0.11920292]])

There's another definition of the logistic function, using the hyperbolic tangent.

$$s(x) = \frac{1}{1 + e^{-x}} = \frac{1 + \tanh(x / 2)}{2}$$

Which we can use to get the same result.


In [3]:
s2 =(1 + T.tanh(x / 2)) / 2
logistic2 = theano.function([x], s2)
logistic2(my_matrix)


Out[3]:
array([[ 0.5       ,  0.73105858],
       [ 0.26894142,  0.11920292]])

Using Shared Variables