notebook.community
Edit and run
Neural Networks
$$h_{w,b}(x) = f\left(\sum_{i=1}^{3} w_i x_i + b\right)$$
where $f$ is called the
activation function
.
Common choices for activation function
1. Linear
2. Binary threshold
3. Rectified Linear Unit (ReLU)
4. Sigmoid activation function
$$f(z) = \frac{1}{1 + \exp(-z)}$$
output range:
$0$ to $1$
5. Hyperbolic tangent (tanh)
$$f(z) = \tanh(z) = \frac{e^z - e^{-z}}{e^z + e^{-z}}$$
output range:
$-1$ to $1$