Retrain a CNN, part training from scratch

https://blog.keras.io/building-powerful-image-classification-models-using-very-little-data.html



In [1]:

    
import warnings
warnings.filterwarnings('ignore')



In [2]:

    
%matplotlib inline
%pylab inline









    



Populating the interactive namespace from numpy and matplotlib



In [3]:

    
import matplotlib.pylab as plt
import numpy as np



In [4]:

    
from distutils.version import StrictVersion



In [5]:

    
import sklearn
print(sklearn.__version__)

assert StrictVersion(sklearn.__version__ ) >= StrictVersion('0.18.1')



In [6]:

    
import tensorflow as tf
tf.logging.set_verbosity(tf.logging.ERROR)
print(tf.__version__)

assert StrictVersion(tf.__version__) >= StrictVersion('1.1.0')



In [7]:

    
import keras
print(keras.__version__)

assert StrictVersion(keras.__version__) >= StrictVersion('2.0.0')









    



Using TensorFlow backend.






    



2.0.6

This script goes along the blog post "Building powerful image classification models using very little data" from blog.keras.io. It uses data that can be downloaded at: https://www.kaggle.com/c/dogs-vs-cats/data In our setup, we:

created a data/ folder
created train/ and validation/ subfolders inside data/
created cats/ and dogs/ subfolders inside train/ and validation/
put the cat pictures index 0-999 in data/train/cats
put the cat pictures index 1000-1400 in data/validation/cats
put the dogs pictures index 12500-13499 in data/train/dogs

put the dog pictures index 13500-13900 in data/validation/dogs So that we have 1000 training examples for each class, and 400 validation examples for each class. In summary, this is our directory structure:

data/
  train/
      dogs/
          dog001.jpg
          dog002.jpg
          ...
      cats/
          cat001.jpg
          cat002.jpg
          ...
  validation/
      dogs/
          dog001.jpg
          dog002.jpg
          ...
      cats/
          cat001.jpg
          cat002.jpg
          ...



In [8]:

    
!ls -lh data









    



total 8.0K
drwxrwxr-x 4 ubuntu ubuntu 4.0K Aug 31 18:52 train
drwxrwxr-x 4 ubuntu ubuntu 4.0K Aug 31 18:52 validation



In [9]:

    
from keras.preprocessing.image import ImageDataGenerator
from keras.models import Sequential
from keras.layers import Conv2D, MaxPooling2D
from keras.layers import Activation, Dropout, Flatten, Dense
from keras import backend as K



In [10]:

    
# dimensions of our images.
img_width, img_height = 150, 150

train_data_dir = 'data/train'
validation_data_dir = 'data/validation'
nb_train_samples = 2000
nb_validation_samples = 800

if K.image_data_format() == 'channels_first':
    input_shape = (3, img_width, img_height)
else:
    input_shape = (img_width, img_height, 3)



In [11]:

    
model = Sequential()
model.add(Conv2D(32, (3, 3), input_shape=input_shape))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))

model.add(Conv2D(32, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))

model.add(Conv2D(64, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))

model.add(Flatten())
model.add(Dense(64))
model.add(Activation('relu'))
model.add(Dropout(0.5))
model.add(Dense(1))
model.add(Activation('sigmoid'))

model.compile(loss='binary_crossentropy',
              optimizer='rmsprop',
              metrics=['accuracy'])



In [12]:

    
model.summary()









    



_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
conv2d_1 (Conv2D)            (None, 148, 148, 32)      896       
_________________________________________________________________
activation_1 (Activation)    (None, 148, 148, 32)      0         
_________________________________________________________________
max_pooling2d_1 (MaxPooling2 (None, 74, 74, 32)        0         
_________________________________________________________________
conv2d_2 (Conv2D)            (None, 72, 72, 32)        9248      
_________________________________________________________________
activation_2 (Activation)    (None, 72, 72, 32)        0         
_________________________________________________________________
max_pooling2d_2 (MaxPooling2 (None, 36, 36, 32)        0         
_________________________________________________________________
conv2d_3 (Conv2D)            (None, 34, 34, 64)        18496     
_________________________________________________________________
activation_3 (Activation)    (None, 34, 34, 64)        0         
_________________________________________________________________
max_pooling2d_3 (MaxPooling2 (None, 17, 17, 64)        0         
_________________________________________________________________
flatten_1 (Flatten)          (None, 18496)             0         
_________________________________________________________________
dense_1 (Dense)              (None, 64)                1183808   
_________________________________________________________________
activation_4 (Activation)    (None, 64)                0         
_________________________________________________________________
dropout_1 (Dropout)          (None, 64)                0         
_________________________________________________________________
dense_2 (Dense)              (None, 1)                 65        
_________________________________________________________________
activation_5 (Activation)    (None, 1)                 0         
=================================================================
Total params: 1,212,513
Trainable params: 1,212,513
Non-trainable params: 0
_________________________________________________________________



In [12]:

    
tb_callback = keras.callbacks.TensorBoard(log_dir='./tf_log')



In [13]:

    
# this is the augmentation configuration we will use for training
train_datagen = ImageDataGenerator(
    rescale=1. / 255,
    shear_range=0.2,
    zoom_range=0.2,
    horizontal_flip=True)

# this is the augmentation configuration we will use for testing:
# only rescaling
test_datagen = ImageDataGenerator(rescale=1. / 255)

epochs = 50
batch_size = 16

train_generator = train_datagen.flow_from_directory(
    train_data_dir,
    target_size=(img_width, img_height),
    batch_size=batch_size,
    class_mode='binary')

validation_generator = test_datagen.flow_from_directory(
    validation_data_dir,
    target_size=(img_width, img_height),
    batch_size=batch_size,
    class_mode='binary')

model.fit_generator(
    train_generator,
    steps_per_epoch=nb_train_samples // batch_size,
    epochs=epochs,
    validation_data=validation_generator,
    validation_steps=nb_validation_samples // batch_size,
    callbacks=[tb_callback])









    



Found 2000 images belonging to 2 classes.
Found 802 images belonging to 2 classes.
Epoch 1/50
125/125 [==============================] - 16s - loss: 0.7377 - acc: 0.5200 - val_loss: 0.6848 - val_acc: 0.6162
Epoch 2/50
125/125 [==============================] - 14s - loss: 0.6744 - acc: 0.5740 - val_loss: 0.6554 - val_acc: 0.6590
Epoch 3/50
125/125 [==============================] - 14s - loss: 0.6402 - acc: 0.6420 - val_loss: 0.6137 - val_acc: 0.6718
Epoch 4/50
125/125 [==============================] - 14s - loss: 0.6278 - acc: 0.6705 - val_loss: 0.5937 - val_acc: 0.6654
Epoch 5/50
125/125 [==============================] - 14s - loss: 0.5873 - acc: 0.6930 - val_loss: 0.6949 - val_acc: 0.6170
Epoch 6/50
125/125 [==============================] - 14s - loss: 0.5924 - acc: 0.7050 - val_loss: 0.5733 - val_acc: 0.6679
Epoch 7/50
125/125 [==============================] - 14s - loss: 0.5667 - acc: 0.7170 - val_loss: 0.5804 - val_acc: 0.6959
Epoch 8/50
125/125 [==============================] - 15s - loss: 0.5539 - acc: 0.7265 - val_loss: 0.5534 - val_acc: 0.7239
Epoch 9/50
125/125 [==============================] - 14s - loss: 0.5475 - acc: 0.7335 - val_loss: 0.5408 - val_acc: 0.7048
Epoch 10/50
125/125 [==============================] - 14s - loss: 0.5133 - acc: 0.7450 - val_loss: 0.5671 - val_acc: 0.7328
Epoch 11/50
125/125 [==============================] - 14s - loss: 0.5348 - acc: 0.7560 - val_loss: 0.5298 - val_acc: 0.7316
Epoch 12/50
125/125 [==============================] - 14s - loss: 0.5034 - acc: 0.7600 - val_loss: 0.5041 - val_acc: 0.7545
Epoch 13/50
125/125 [==============================] - 14s - loss: 0.5000 - acc: 0.7665 - val_loss: 0.5197 - val_acc: 0.7443
Epoch 14/50
125/125 [==============================] - 14s - loss: 0.4912 - acc: 0.7680 - val_loss: 0.5788 - val_acc: 0.7061
Epoch 15/50
125/125 [==============================] - 14s - loss: 0.4826 - acc: 0.7750 - val_loss: 0.4822 - val_acc: 0.7761
Epoch 16/50
125/125 [==============================] - 14s - loss: 0.4816 - acc: 0.7750 - val_loss: 0.5343 - val_acc: 0.7379
Epoch 17/50
125/125 [==============================] - 14s - loss: 0.4701 - acc: 0.7835 - val_loss: 0.5388 - val_acc: 0.7417
Epoch 18/50
125/125 [==============================] - 14s - loss: 0.4669 - acc: 0.7810 - val_loss: 0.5625 - val_acc: 0.7379
Epoch 19/50
125/125 [==============================] - 14s - loss: 0.4747 - acc: 0.7880 - val_loss: 0.4631 - val_acc: 0.7697
Epoch 20/50
125/125 [==============================] - 14s - loss: 0.4655 - acc: 0.7925 - val_loss: 0.5017 - val_acc: 0.7481
Epoch 21/50
125/125 [==============================] - 14s - loss: 0.4630 - acc: 0.7950 - val_loss: 0.5188 - val_acc: 0.7417
Epoch 22/50
125/125 [==============================] - 15s - loss: 0.4501 - acc: 0.8015 - val_loss: 0.4845 - val_acc: 0.7646
Epoch 23/50
125/125 [==============================] - 14s - loss: 0.4595 - acc: 0.7925 - val_loss: 0.8829 - val_acc: 0.7328s
Epoch 24/50
125/125 [==============================] - 14s - loss: 0.4569 - acc: 0.7955 - val_loss: 0.5194 - val_acc: 0.7341
Epoch 25/50
125/125 [==============================] - 15s - loss: 0.4353 - acc: 0.7985 - val_loss: 0.4421 - val_acc: 0.7888
Epoch 26/50
125/125 [==============================] - 14s - loss: 0.4332 - acc: 0.8155 - val_loss: 0.5578 - val_acc: 0.7723
Epoch 27/50
125/125 [==============================] - 14s - loss: 0.4392 - acc: 0.8090 - val_loss: 0.4636 - val_acc: 0.7850
Epoch 28/50
125/125 [==============================] - 14s - loss: 0.4460 - acc: 0.8165 - val_loss: 0.4521 - val_acc: 0.8003
Epoch 29/50
125/125 [==============================] - 14s - loss: 0.4332 - acc: 0.8110 - val_loss: 0.4741 - val_acc: 0.7748
Epoch 30/50
125/125 [==============================] - 14s - loss: 0.4430 - acc: 0.8135 - val_loss: 0.5109 - val_acc: 0.7888
Epoch 31/50
125/125 [==============================] - 15s - loss: 0.4418 - acc: 0.8170 - val_loss: 0.5010 - val_acc: 0.7799
Epoch 32/50
125/125 [==============================] - 14s - loss: 0.4288 - acc: 0.8145 - val_loss: 0.4587 - val_acc: 0.7824.
Epoch 33/50
125/125 [==============================] - 15s - loss: 0.4440 - acc: 0.8110 - val_loss: 0.4930 - val_acc: 0.7863
Epoch 34/50
125/125 [==============================] - 14s - loss: 0.4286 - acc: 0.8110 - val_loss: 0.4863 - val_acc: 0.8041
Epoch 35/50
125/125 [==============================] - 14s - loss: 0.4224 - acc: 0.8180 - val_loss: 0.4894 - val_acc: 0.7824
Epoch 36/50
125/125 [==============================] - 15s - loss: 0.4235 - acc: 0.8170 - val_loss: 0.4804 - val_acc: 0.7939
Epoch 37/50
125/125 [==============================] - 15s - loss: 0.4123 - acc: 0.8145 - val_loss: 0.5654 - val_acc: 0.7277
Epoch 38/50
125/125 [==============================] - 14s - loss: 0.4292 - acc: 0.8275 - val_loss: 0.4979 - val_acc: 0.7926
Epoch 39/50
125/125 [==============================] - 14s - loss: 0.4145 - acc: 0.8250 - val_loss: 0.5422 - val_acc: 0.7837
Epoch 40/50
125/125 [==============================] - 14s - loss: 0.4278 - acc: 0.8145 - val_loss: 0.4614 - val_acc: 0.8003
Epoch 41/50
125/125 [==============================] - 14s - loss: 0.4327 - acc: 0.8200 - val_loss: 0.5279 - val_acc: 0.7850
Epoch 42/50
125/125 [==============================] - 14s - loss: 0.4167 - acc: 0.8265 - val_loss: 0.5805 - val_acc: 0.7875
Epoch 43/50
125/125 [==============================] - 14s - loss: 0.4310 - acc: 0.8170 - val_loss: 0.5371 - val_acc: 0.7926
Epoch 44/50
125/125 [==============================] - 14s - loss: 0.4062 - acc: 0.8210 - val_loss: 0.4673 - val_acc: 0.7990
Epoch 45/50
125/125 [==============================] - 14s - loss: 0.4154 - acc: 0.8220 - val_loss: 0.5180 - val_acc: 0.732838 - acc: 0.823
Epoch 46/50
125/125 [==============================] - 14s - loss: 0.4242 - acc: 0.8210 - val_loss: 0.5720 - val_acc: 0.7723
Epoch 47/50
125/125 [==============================] - 14s - loss: 0.4191 - acc: 0.8240 - val_loss: 0.5071 - val_acc: 0.7837
Epoch 48/50
125/125 [==============================] - 14s - loss: 0.3993 - acc: 0.8315 - val_loss: 0.4662 - val_acc: 0.7939
Epoch 49/50
125/125 [==============================] - 14s - loss: 0.4053 - acc: 0.8315 - val_loss: 0.5089 - val_acc: 0.7964
Epoch 50/50
125/125 [==============================] - 14s - loss: 0.4219 - acc: 0.8210 - val_loss: 0.4933 - val_acc: 0.8003






    Out[13]:





<keras.callbacks.History at 0x7fa439903940>



In [14]:

    
model.save_weights('first_try.h5')



In [ ]: