In [1]:
%matplotlib inline
Starting with a brief benchmark below, which was found in the forums we can use this to solve the digit recognizer problem with 0.96 accuracy score in less than a minute of learning. However, a neural net would have better accuracy.
In [2]:
from sklearn.ensemble import RandomForestClassifier
import numpy as np
import pandas as pd
# create the training & test sets, skipping the header row with [1:]
dataset = pd.read_csv("../data/train.csv")
target = dataset[[0]].values.ravel()
train = dataset.iloc[:,1:].values
test = pd.read_csv("../data/test.csv").values
# create and train the random forest
# multi-core CPUs can use: rf = RandomForestClassifier(n_estimators=100, n_jobs=2)
#rf = RandomForestClassifier(n_estimators=100)
#rf.fit(train, target)
#pred = rf.predict(test)
#np.savetxt('submission_rand_forest.csv', np.c_[range(1,len(test)+1),pred], delimiter=',', header = 'ImageId,Label', comments = '', fmt='%d')
In [3]:
dataset.head()
Out[3]:
Convert each data variable from a python list into a numpy array. For the target array I specify the integer data type. The train set has 42k rows and 784 columns, so its shape is (42k,784). Each row is a 28 by 28 pixel picture. I will reshape the train set to have (42k,1) shape, i.e. each row will contain a 28 by 28 matrix of pixel color values. Same for the test set.
In [8]:
# convert to array, specify data type, and reshape
target = target.astype(np.uint8)
train = np.array(train).reshape(-1,1, 28, 28) # -1 means the value is inferred from the original array 42k in this case
test = np.array(test).reshape((-1, 1, 28, 28)).astype(np.uint8)
Now, we can actually plot those pixel color values and see what a sample picture of a digit looks like. Below is the picture of a digit in the 1700th row:
In [9]:
import matplotlib.pyplot as plt
import matplotlib.cm as cm
figs, axes = plt.subplots(4, 4, figsize=(6, 6))
for i in range(4):
for j in range(4):
axes[i, j].imshow(-train[i + 4 * j].reshape(28, 28), cmap='gray', interpolation='none')
axes[i, j].set_xticks([])
axes[i, j].set_yticks([])
axes[i, j].set_title("Label: {}".format(target[i + 4 * j]))
axes[i, j].axis('off')
# plt.imshow(train[1700][0], cmap=cm.binary) # draw the picture
Below is a simple NN set up. Supposedly, it should give >0.9 accuracy score. I had trouble with figuring out the training part in that the accuracy I was getting would not change during the training process. All I had to do was to decrease the learning rate from 0.01 to 0.0001, nota bene.
In [10]:
import lasagne
from lasagne import layers
from lasagne.updates import nesterov_momentum
from nolearn.lasagne import NeuralNet
from nolearn.lasagne import visualize
net1 = NeuralNet(
layers = [
# layer dealing with the input data
(InputLayer, {'shape': (None, X.shape[1], X.shape[2], X.shape[3])}),
# first stage of our convolutional layers
(Conv2DLayer, {'num_filters': 96, 'filter_size': 5}),
(Conv2DLayer, {'num_filters': 96, 'filter_size': 3}),
(Conv2DLayer, {'num_filters': 96, 'filter_size': 3}),
(Conv2DLayer, {'num_filters': 96, 'filter_size': 3}),
(Conv2DLayer, {'num_filters': 96, 'filter_size': 3}),
(MaxPool2DLayer, {'pool_size': 2}),
# second stage of our convolutional layers
(Conv2DLayer, {'num_filters': 128, 'filter_size': 3}),
(Conv2DLayer, {'num_filters': 128, 'filter_size': 3}),
(Conv2DLayer, {'num_filters': 128, 'filter_size': 3}),
(MaxPool2DLayer, {'pool_size': 2}),
# two dense layers with dropout
(DenseLayer, {'num_units': 64}),
(DropoutLayer, {}),
(DenseLayer, {'num_units': 64}),
# the output layer
(DenseLayer, {'num_units': 10, 'nonlinearity': softmax}),
],
# layer parameters:
input_shape=(None,1,28,28),
hidden_num_units=1000, # number of units in 'hidden' layer
output_nonlinearity=lasagne.nonlinearities.softmax,
output_num_units=10, # 10 target values for the digits 0, 1, 2, ..., 9
# optimization method:
update=nesterov_momentum,
update_learning_rate=0.0001,
update_momentum=0.9,
max_epochs=15,
verbose=1,
)
In [11]:
# Train the network
net1.fit(train, target)
In [ ]: