Load MNIST Data


In [5]:
import mnist_loader

In [6]:
training_data, validation_data, test_data = mnist_loader.load_data_wrapper()

Set up Network


In [9]:
import network

In [10]:
# 784 (28 x 28 pixel images) input neurons; 30 hidden neurons; 10 output neurons
net = network.Network([784, 30, 10])

Train Network


In [12]:
# Use stochastic gradient descent over 30 epochs, with mini-batch size of 10, learning rate of 3.0
net.SGD(training_data, 30, 10, 3.0, test_data=test_data)


Epoch 0: 9041 / 10000
Epoch 1: 9220 / 10000
Epoch 2: 9290 / 10000
Epoch 3: 9366 / 10000
Epoch 4: 9365 / 10000
Epoch 5: 9386 / 10000
Epoch 6: 9367 / 10000
Epoch 7: 9443 / 10000
Epoch 8: 9415 / 10000
Epoch 9: 9454 / 10000
Epoch 10: 9426 / 10000
Epoch 11: 9426 / 10000
Epoch 12: 9461 / 10000
Epoch 13: 9475 / 10000
Epoch 14: 9472 / 10000
Epoch 15: 9474 / 10000
Epoch 16: 9453 / 10000
Epoch 17: 9469 / 10000
Epoch 18: 9485 / 10000
Epoch 19: 9465 / 10000
Epoch 20: 9458 / 10000
Epoch 21: 9465 / 10000
Epoch 22: 9455 / 10000
Epoch 23: 9490 / 10000
Epoch 24: 9464 / 10000
Epoch 25: 9481 / 10000
Epoch 26: 9488 / 10000
Epoch 27: 9481 / 10000
Epoch 28: 9500 / 10000
Epoch 29: 9510 / 10000

Exercise: Create network with just two layers


In [13]:
two_layer_net = network.Network([784, 10])

In [14]:
two_layer_net.SGD(training_data, 10, 10, 1.0, test_data=test_data)


Epoch 0: 3284 / 10000
Epoch 1: 3776 / 10000
Epoch 2: 4564 / 10000
Epoch 3: 4613 / 10000
Epoch 4: 4637 / 10000
Epoch 5: 4668 / 10000
Epoch 6: 5743 / 10000
Epoch 7: 5764 / 10000
Epoch 8: 6508 / 10000
Epoch 9: 6632 / 10000

In [15]:
two_layer_net.SGD(training_data, 10, 10, 2.0, test_data=test_data)


Epoch 0: 6602 / 10000
Epoch 1: 6623 / 10000
Epoch 2: 6647 / 10000
Epoch 3: 6681 / 10000
Epoch 4: 7274 / 10000
Epoch 5: 7380 / 10000
Epoch 6: 7422 / 10000
Epoch 7: 7440 / 10000
Epoch 8: 7420 / 10000
Epoch 9: 7441 / 10000

In [16]:
two_layer_net.SGD(training_data, 10, 10, 3.0, test_data=test_data)


Epoch 0: 7408 / 10000
Epoch 1: 7400 / 10000
Epoch 2: 7422 / 10000
Epoch 3: 7416 / 10000
Epoch 4: 7459 / 10000
Epoch 5: 8294 / 10000
Epoch 6: 8305 / 10000
Epoch 7: 8264 / 10000
Epoch 8: 8284 / 10000
Epoch 9: 8314 / 10000

In [17]:
two_layer_net.SGD(training_data, 10, 10, 4.0, test_data=test_data)


Epoch 0: 8255 / 10000
Epoch 1: 8278 / 10000
Epoch 2: 8255 / 10000
Epoch 3: 8246 / 10000
Epoch 4: 8260 / 10000
Epoch 5: 8272 / 10000
Epoch 6: 8289 / 10000
Epoch 7: 8309 / 10000
Epoch 8: 8284 / 10000
Epoch 9: 8287 / 10000

In [18]:
two_layer_net.SGD(training_data, 20, 10, 3.0, test_data=test_data)


Epoch 0: 8311 / 10000
Epoch 1: 8277 / 10000
Epoch 2: 8303 / 10000
Epoch 3: 8300 / 10000
Epoch 4: 8293 / 10000
Epoch 5: 8301 / 10000
Epoch 6: 8295 / 10000
Epoch 7: 8292 / 10000
Epoch 8: 8305 / 10000
Epoch 9: 8289 / 10000
Epoch 10: 8307 / 10000
Epoch 11: 8304 / 10000
Epoch 12: 8294 / 10000
Epoch 13: 8291 / 10000
Epoch 14: 8299 / 10000
Epoch 15: 8309 / 10000
Epoch 16: 8314 / 10000
Epoch 17: 8323 / 10000
Epoch 18: 8301 / 10000
Epoch 19: 8311 / 10000

In [ ]: