Neural networks are a form of machine learning inspired by the biological function of the human brain. By giving the program known inputs and outputs of a function, the computer learns the pattern to an uknown function. Once the program has "learned", it is tested against testing inputs in where it predicts the solution. If the user known the solution to the testing inputs, the accuracy of the program can be computed.
In the base code of this program, the computer is given a set of 1797 hand written digits from zero to nine. Each 64 pixel digit is given as a 2D array of values correspoding to the pixel shade; similar images will have similar arrays. Each of thse 1797 images also has a "solution" set of the correspond correct answers. After a training period is completed using a fraction of the 1797 image-solution pairs, the computer will be tested on the numerical value of the remaining 797 images. With careful construction of the neural network, the computer's accuracy on the testing set will be well above that of random guessing.
The first additional question I am exploring was exploring how one would program multiple hidden layers, and if accuracy increases with multiple hidden layers. It will also be interesting to compare the number of neurons in a single hidden layer versus the number of neurons in two hidden layers to achieve maximum accuracy.
The second additional question I am addressing is how many iterations through the training set will it take for the neural net to memorize a "perfect" data set. In this case, I'm defining a "perfect" data set to be one in which there is a unique input for every output. This then means that the neural net will be tested on the exact material it was trained on.
In [1]:
import NNpix as npx
from IPython.display import HTML
from IPython.display import display
The value that a neuron receives is the product of the incoming signal and the weights. The neuron will fire if its "threshold" value is met.
In [2]:
npx.neuron_signal
Out[2]:
In [3]:
npx.neuron_no_signal
Out[3]:
The "threshold" value can be thought of as a step function. We refer to this as the activation function. If the values mutiply to a value greater than 0, the neuron fires. Otherwise it does not. Because we want a smooth function, we can use the sigmoid function as the activate function.
In [4]:
npx.activation
Out[4]:
In [5]:
npx.derivative1
Out[5]:
In [6]:
f = open("HTMLTable.html")
In [7]:
display(HTML(f.read()))
In [8]:
f.close()
The main difference is arrays are used and the input for each nueron is the sum of all the previous neurons multiplied by their respective weights. If working with arrays, once all the arrays cast together as expected, the neural net may only require minor to no debugging.
In [9]:
npx.cneuron1
Out[9]:
In [ ]: