Get the Class names

This file contains a subset of the quick draw classes. I choose around 100 classes from the dataset.


In [1]:
!wget 'https://raw.githubusercontent.com/zaidalyafeai/zaidalyafeai.github.io/master/sketcher/mini_classes.txt'


--2018-07-03 18:55:10--  https://raw.githubusercontent.com/zaidalyafeai/zaidalyafeai.github.io/master/sketcher/mini_classes.txt
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.0.133, 151.101.64.133, 151.101.128.133, ...
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.0.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 760 [text/plain]
Saving to: ‘mini_classes.txt.3’

mini_classes.txt.3  100%[===================>]     760  --.-KB/s    in 0s      

2018-07-03 18:55:10 (74.2 MB/s) - ‘mini_classes.txt.3’ saved [760/760]

Read the classes names


In [0]:
f = open("mini_classes.txt","r")
# And for reading use
classes = f.readlines()
f.close()

In [0]:
classes = [c.replace('\n','').replace(' ','_') for c in classes]

Download the Dataset

Loop over the classes and download the currospondent data


In [0]:
!mkdir data

In [0]:
import urllib.request
def download():
  
  base = 'https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/'
  for c in classes:
    cls_url = c.replace('_', '%20')
    path = base+cls_url+'.npy'
    print(path)
    urllib.request.urlretrieve(path, 'data/'+c+'.npy')

In [6]:
download()


https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/drums.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/sun.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/laptop.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/anvil.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/baseball%20bat.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/ladder.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/eyeglasses.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/grapes.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/book.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/dumbbell.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/traffic%20light.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/wristwatch.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/wheel.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/shovel.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/bread.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/table.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/tennis%20racquet.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/cloud.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/chair.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/headphones.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/face.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/eye.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/airplane.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/snake.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/lollipop.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/power%20outlet.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/pants.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/mushroom.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/star.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/sword.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/clock.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/hot%20dog.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/syringe.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/stop%20sign.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/mountain.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/smiley%20face.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/apple.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/bed.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/shorts.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/broom.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/diving%20board.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/flower.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/spider.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/cell%20phone.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/car.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/camera.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/tree.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/square.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/moon.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/radio.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/hat.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/pizza.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/axe.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/door.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/tent.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/umbrella.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/line.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/cup.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/fan.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/triangle.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/basketball.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/pillow.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/scissors.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/t-shirt.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/tooth.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/alarm%20clock.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/paper%20clip.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/spoon.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/microphone.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/candle.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/pencil.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/envelope.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/saw.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/frying%20pan.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/screwdriver.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/helmet.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/bridge.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/light%20bulb.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/ceiling%20fan.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/key.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/donut.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/bird.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/circle.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/beard.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/coffee%20cup.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/butterfly.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/bench.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/rifle.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/cat.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/sock.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/ice%20cream.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/moustache.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/suitcase.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/hammer.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/rainbow.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/knife.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/cookie.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/baseball.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/lightning.npy
https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/bicycle.npy

Imports


In [0]:
import os
import glob
import numpy as np
from tensorflow.keras import layers
from tensorflow import keras 
import tensorflow as tf

Load the Data

Each class contains different number samples of arrays stored as .npy format. Since we have some memory limitations we only load 5000 images per class.


In [0]:
def load_data(root, vfold_ratio=0.2, max_items_per_class= 4000 ):
    all_files = glob.glob(os.path.join(root, '*.npy'))

    #initialize variables 
    x = np.empty([0, 784])
    y = np.empty([0])
    class_names = []

    #load each data file 
    for idx, file in enumerate(all_files):
        data = np.load(file)
        data = data[0: max_items_per_class, :]
        labels = np.full(data.shape[0], idx)

        x = np.concatenate((x, data), axis=0)
        y = np.append(y, labels)

        class_name, ext = os.path.splitext(os.path.basename(file))
        class_names.append(class_name)

    data = None
    labels = None
    
    #randomize the dataset 
    permutation = np.random.permutation(y.shape[0])
    x = x[permutation, :]
    y = y[permutation]

    #separate into training and testing 
    vfold_size = int(x.shape[0]/100*(vfold_ratio*100))

    x_test = x[0:vfold_size, :]
    y_test = y[0:vfold_size]

    x_train = x[vfold_size:x.shape[0], :]
    y_train = y[vfold_size:y.shape[0]]
    return x_train, y_train, x_test, y_test, class_names

In [0]:
x_train, y_train, x_test, y_test, class_names = load_data('data')
num_classes = len(class_names)
image_size = 28

In [7]:
print(len(x_train))


400000

Show some random data


In [8]:
import matplotlib.pyplot as plt
from random import randint
%matplotlib inline  
idx = randint(0, len(x_train))
plt.imshow(x_train[idx].reshape(28,28)) 
print(class_names[int(y_train[idx].item())])


shovel

Preprocess the Data


In [0]:
# Reshape and normalize
x_train = x_train.reshape(x_train.shape[0], image_size, image_size, 1).astype('float32')
x_test = x_test.reshape(x_test.shape[0], image_size, image_size, 1).astype('float32')

x_train /= 255.0
x_test /= 255.0

# Convert class vectors to class matrices
y_train = keras.utils.to_categorical(y_train, num_classes)
y_test = keras.utils.to_categorical(y_test, num_classes)

The Model


In [16]:
# Define model
model = keras.Sequential()
model.add(layers.Convolution2D(16, (3, 3),
                        padding='same',
                        input_shape=x_train.shape[1:], activation='relu'))
model.add(layers.MaxPooling2D(pool_size=(2, 2)))
model.add(layers.Convolution2D(32, (3, 3), padding='same', activation= 'relu'))
model.add(layers.MaxPooling2D(pool_size=(2, 2)))
model.add(layers.Convolution2D(64, (3, 3), padding='same', activation= 'relu'))
model.add(layers.MaxPooling2D(pool_size =(2,2)))
model.add(layers.Flatten())
model.add(layers.Dense(128, activation='relu'))
model.add(layers.Dense(100, activation='softmax')) 
# Train model
adam = tf.train.AdamOptimizer()
model.compile(loss='categorical_crossentropy',
              optimizer=adam,
              metrics=['top_k_categorical_accuracy'])
print(model.summary())


_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
conv2d_9 (Conv2D)            (None, 28, 28, 16)        160       
_________________________________________________________________
max_pooling2d_9 (MaxPooling2 (None, 14, 14, 16)        0         
_________________________________________________________________
conv2d_10 (Conv2D)           (None, 14, 14, 32)        4640      
_________________________________________________________________
max_pooling2d_10 (MaxPooling (None, 7, 7, 32)          0         
_________________________________________________________________
conv2d_11 (Conv2D)           (None, 7, 7, 64)          18496     
_________________________________________________________________
max_pooling2d_11 (MaxPooling (None, 3, 3, 64)          0         
_________________________________________________________________
flatten_3 (Flatten)          (None, 576)               0         
_________________________________________________________________
dense_6 (Dense)              (None, 128)               73856     
_________________________________________________________________
dense_7 (Dense)              (None, 100)               12900     
=================================================================
Total params: 110,052
Trainable params: 110,052
Non-trainable params: 0
_________________________________________________________________
None

Training


In [17]:
model.fit(x = x_train, y = y_train, validation_split=0.1, batch_size = 256, verbose=2, epochs=5)


Train on 288000 samples, validate on 32000 samples
Epoch 1/5
 - 14s - loss: 1.8775 - top_k_categorical_accuracy: 0.7930 - val_loss: 1.3543 - val_top_k_categorical_accuracy: 0.8811
Epoch 2/5
 - 13s - loss: 1.2097 - top_k_categorical_accuracy: 0.8960 - val_loss: 1.1262 - val_top_k_categorical_accuracy: 0.9063
Epoch 3/5
 - 13s - loss: 1.0523 - top_k_categorical_accuracy: 0.9132 - val_loss: 1.0283 - val_top_k_categorical_accuracy: 0.9166
Epoch 4/5
 - 13s - loss: 0.9646 - top_k_categorical_accuracy: 0.9220 - val_loss: 0.9908 - val_top_k_categorical_accuracy: 0.9186
Epoch 5/5
 - 13s - loss: 0.9038 - top_k_categorical_accuracy: 0.9282 - val_loss: 0.9460 - val_top_k_categorical_accuracy: 0.9235
Out[17]:
<tensorflow.python.keras.callbacks.History at 0x7f1a2f22be10>

Testing


In [18]:
score = model.evaluate(x_test, y_test, verbose=0)
print('Test accuarcy: {:0.2f}%'.format(score[1] * 100))


Test accuarcy: 92.20%

Inference


In [8]:
import matplotlib.pyplot as plt
from random import randint
%matplotlib inline  
idx = randint(0, len(x_test))
img = x_test[idx]
plt.imshow(img.squeeze()) 
pred = model.predict(np.expand_dims(img, axis=0))[0]
ind = (-pred).argsort()[:5]
latex = [class_names[x] for x in ind]
print(latex)


['alarm_clock', 'bread', 't-shirt', 'butterfly', 'bird']

Store the classes


In [0]:
with open('class_names.txt', 'w') as file_handler:
    for item in class_names:
        file_handler.write("{}\n".format(item))

Install TensorFlowJS


In [10]:
!pip install tensorflowjs


Collecting tensorflowjs
  Downloading https://files.pythonhosted.org/packages/b2/fd/39f5e1709a543cdce74f2ff6423d70800dbb785494ff66765464feeb67a5/tensorflowjs-0.4.1-py3-none-any.whl
Requirement already satisfied: tensorflow==1.8.0 in /usr/local/lib/python3.6/dist-packages (from tensorflowjs) (1.8.0)
Requirement already satisfied: six==1.11.0 in /usr/local/lib/python3.6/dist-packages (from tensorflowjs) (1.11.0)
Collecting keras==2.1.4 (from tensorflowjs)
  Downloading https://files.pythonhosted.org/packages/86/45/a273fe3f8fe931a11da34fba1cb74013cfc70dcf93e5d8d329c951dc44c5/Keras-2.1.4-py2.py3-none-any.whl (322kB)
    100% |████████████████████████████████| 327kB 6.4MB/s 
Requirement already satisfied: h5py==2.7.1 in /usr/local/lib/python3.6/dist-packages (from tensorflowjs) (2.7.1)
Collecting numpy==1.14.1 (from tensorflowjs)
  Downloading https://files.pythonhosted.org/packages/de/7d/348c5d8d44443656e76285aa97b828b6dbd9c10e5b9c0f7f98eff0ff70e4/numpy-1.14.1-cp36-cp36m-manylinux1_x86_64.whl (12.2MB)
    100% |████████████████████████████████| 12.2MB 3.5MB/s 
Requirement already satisfied: tensorflow-hub==0.1.0 in /usr/local/lib/python3.6/dist-packages (from tensorflowjs) (0.1.0)
Requirement already satisfied: astor>=0.6.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.8.0->tensorflowjs) (0.6.2)
Requirement already satisfied: termcolor>=1.1.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.8.0->tensorflowjs) (1.1.0)
Requirement already satisfied: protobuf>=3.4.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.8.0->tensorflowjs) (3.5.2.post1)
Requirement already satisfied: absl-py>=0.1.6 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.8.0->tensorflowjs) (0.2.2)
Requirement already satisfied: grpcio>=1.8.6 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.8.0->tensorflowjs) (1.12.1)
Requirement already satisfied: wheel>=0.26 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.8.0->tensorflowjs) (0.31.1)
Requirement already satisfied: gast>=0.2.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.8.0->tensorflowjs) (0.2.0)
Requirement already satisfied: tensorboard<1.9.0,>=1.8.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.8.0->tensorflowjs) (1.8.0)
Requirement already satisfied: scipy>=0.14 in /usr/local/lib/python3.6/dist-packages (from keras==2.1.4->tensorflowjs) (0.19.1)
Requirement already satisfied: pyyaml in /usr/local/lib/python3.6/dist-packages (from keras==2.1.4->tensorflowjs) (3.12)
Requirement already satisfied: setuptools in /usr/local/lib/python3.6/dist-packages (from protobuf>=3.4.0->tensorflow==1.8.0->tensorflowjs) (39.2.0)
Requirement already satisfied: bleach==1.5.0 in /usr/local/lib/python3.6/dist-packages (from tensorboard<1.9.0,>=1.8.0->tensorflow==1.8.0->tensorflowjs) (1.5.0)
Requirement already satisfied: markdown>=2.6.8 in /usr/local/lib/python3.6/dist-packages (from tensorboard<1.9.0,>=1.8.0->tensorflow==1.8.0->tensorflowjs) (2.6.11)
Requirement already satisfied: werkzeug>=0.11.10 in /usr/local/lib/python3.6/dist-packages (from tensorboard<1.9.0,>=1.8.0->tensorflow==1.8.0->tensorflowjs) (0.14.1)
Requirement already satisfied: html5lib==0.9999999 in /usr/local/lib/python3.6/dist-packages (from tensorboard<1.9.0,>=1.8.0->tensorflow==1.8.0->tensorflowjs) (0.9999999)
Installing collected packages: numpy, keras, tensorflowjs
  Found existing installation: numpy 1.14.3
    Uninstalling numpy-1.14.3:
      Successfully uninstalled numpy-1.14.3
  Found existing installation: Keras 2.1.6
    Uninstalling Keras-2.1.6:
      Successfully uninstalled Keras-2.1.6
Successfully installed keras-2.1.4 numpy-1.14.1 tensorflowjs-0.4.1

Save and Convert


In [0]:
model.save('keras.h5')

In [13]:
!mkdir model
!tensorflowjs_converter --input_format keras keras.h5 model/


mkdir: cannot create directory ‘model’: File exists
/usr/local/lib/python3.6/dist-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
  from ._conv import register_converters as _register_converters
Using TensorFlow backend.

Zip and Download


In [0]:
!cp class_names.txt model/class_names.txt

In [15]:
!zip -r model.zip model


  adding: model/ (stored 0%)
  adding: model/group5-shard1of1 (deflated 7%)
  adding: model/model.json (deflated 82%)
  adding: model/group2-shard1of1 (deflated 7%)
  adding: model/group3-shard1of1 (deflated 7%)
  adding: model/class_names.txt (deflated 41%)
  adding: model/group1-shard1of1 (stored 0%)
  adding: model/group4-shard1of1 (deflated 7%)

In [0]:
from google.colab import files
files.download('model.zip')