Transfer Learning in Keras

In this notebook, we'll cover how to load a pre-trained model (in this case, VGGNet19) and finetune it for a new task: detecting hot dogs.



In [1]:

    
import os
os.environ["CUDA_DEVICE_ORDER"] = "PCI_BUS_ID"
os.environ["CUDA_VISIBLE_DEVICES"] = "1"

Load dependencies



In [2]:

    
from keras.applications.vgg19 import VGG19
from keras.models import Sequential
from keras.layers import Dense, Dropout, Flatten
from keras.preprocessing.image import ImageDataGenerator
from keras.callbacks import ModelCheckpoint









    



Using TensorFlow backend.

Load the pre-trained VGG19 model



In [3]:

    
vgg19 = VGG19(include_top=False,
              weights='imagenet',
              input_shape=(224,224,3),
              pooling=None)

Freeze all the layers in the base VGGNet19 model



In [4]:

    
vgg19.layers









    Out[4]:





[<keras.engine.topology.InputLayer at 0x7ff177a59a58>,
 <keras.layers.convolutional.Conv2D at 0x7ff177a59c18>,
 <keras.layers.convolutional.Conv2D at 0x7ff177a59d68>,
 <keras.layers.pooling.MaxPooling2D at 0x7ff177a73e10>,
 <keras.layers.convolutional.Conv2D at 0x7ff1777cd358>,
 <keras.layers.convolutional.Conv2D at 0x7ff1777cdd30>,
 <keras.layers.pooling.MaxPooling2D at 0x7ff1777ef5f8>,
 <keras.layers.convolutional.Conv2D at 0x7ff177780c50>,
 <keras.layers.convolutional.Conv2D at 0x7ff1777942b0>,
 <keras.layers.convolutional.Conv2D at 0x7ff1777a4588>,
 <keras.layers.convolutional.Conv2D at 0x7ff177747780>,
 <keras.layers.pooling.MaxPooling2D at 0x7ff177a53da0>,
 <keras.layers.convolutional.Conv2D at 0x7ff1e84856d8>,
 <keras.layers.convolutional.Conv2D at 0x7ff1e84510b8>,
 <keras.layers.convolutional.Conv2D at 0x7ff1e8464eb8>,
 <keras.layers.convolutional.Conv2D at 0x7ff20115f780>,
 <keras.layers.pooling.MaxPooling2D at 0x7ff1776e2208>,
 <keras.layers.convolutional.Conv2D at 0x7ff1776890f0>,
 <keras.layers.convolutional.Conv2D at 0x7ff177689b70>,
 <keras.layers.convolutional.Conv2D at 0x7ff17769b668>,
 <keras.layers.convolutional.Conv2D at 0x7ff17763d0b8>,
 <keras.layers.pooling.MaxPooling2D at 0x7ff17765f710>]



In [5]:

    
vgg19.summary()









    



_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
input_1 (InputLayer)         (None, 224, 224, 3)       0         
_________________________________________________________________
block1_conv1 (Conv2D)        (None, 224, 224, 64)      1792      
_________________________________________________________________
block1_conv2 (Conv2D)        (None, 224, 224, 64)      36928     
_________________________________________________________________
block1_pool (MaxPooling2D)   (None, 112, 112, 64)      0         
_________________________________________________________________
block2_conv1 (Conv2D)        (None, 112, 112, 128)     73856     
_________________________________________________________________
block2_conv2 (Conv2D)        (None, 112, 112, 128)     147584    
_________________________________________________________________
block2_pool (MaxPooling2D)   (None, 56, 56, 128)       0         
_________________________________________________________________
block3_conv1 (Conv2D)        (None, 56, 56, 256)       295168    
_________________________________________________________________
block3_conv2 (Conv2D)        (None, 56, 56, 256)       590080    
_________________________________________________________________
block3_conv3 (Conv2D)        (None, 56, 56, 256)       590080    
_________________________________________________________________
block3_conv4 (Conv2D)        (None, 56, 56, 256)       590080    
_________________________________________________________________
block3_pool (MaxPooling2D)   (None, 28, 28, 256)       0         
_________________________________________________________________
block4_conv1 (Conv2D)        (None, 28, 28, 512)       1180160   
_________________________________________________________________
block4_conv2 (Conv2D)        (None, 28, 28, 512)       2359808   
_________________________________________________________________
block4_conv3 (Conv2D)        (None, 28, 28, 512)       2359808   
_________________________________________________________________
block4_conv4 (Conv2D)        (None, 28, 28, 512)       2359808   
_________________________________________________________________
block4_pool (MaxPooling2D)   (None, 14, 14, 512)       0         
_________________________________________________________________
block5_conv1 (Conv2D)        (None, 14, 14, 512)       2359808   
_________________________________________________________________
block5_conv2 (Conv2D)        (None, 14, 14, 512)       2359808   
_________________________________________________________________
block5_conv3 (Conv2D)        (None, 14, 14, 512)       2359808   
_________________________________________________________________
block5_conv4 (Conv2D)        (None, 14, 14, 512)       2359808   
_________________________________________________________________
block5_pool (MaxPooling2D)   (None, 7, 7, 512)         0         
=================================================================
Total params: 20,024,384
Trainable params: 20,024,384
Non-trainable params: 0
_________________________________________________________________



In [6]:

    
for layer in vgg19.layers:
    layer.trainable = False

Add custom classification layers



In [7]:

    
# Instantiate the sequential model and add the VGG19 model: 
model = Sequential()
model.add(vgg19)

# Add the custom layers atop the VGG19 model: 
model.add(Flatten(name='flattened'))
model.add(Dropout(0.5, name='dropout'))
model.add(Dense(2, activation='softmax', name='predictions'))



In [8]:

    
model.summary()









    



_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
vgg19 (Model)                (None, 7, 7, 512)         20024384  
_________________________________________________________________
flattened (Flatten)          (None, 25088)             0         
_________________________________________________________________
dropout (Dropout)            (None, 25088)             0         
_________________________________________________________________
predictions (Dense)          (None, 2)                 50178     
=================================================================
Total params: 20,074,562
Trainable params: 50,178
Non-trainable params: 20,024,384
_________________________________________________________________

Compile the model for training



In [9]:

    
model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])



In [10]:

    
output_dir = 'model_output/transfer_VGG'



In [11]:

    
modelcheckpoint = ModelCheckpoint(filepath=output_dir+
                                 "/weights.{epoch:02d}.hdf5")



In [12]:

    
if not os.path.exists(output_dir):
    os.makedirs(output_dir)

Prepare the data for training

The dataset is available for download here. You should download the zipfile and extract the contents into a folder called hot-dog-not-hot-dog in the notebooks directory.



In [13]:

    
# Instantiate two image generator classes:
train_datagen = ImageDataGenerator(
    rescale=1.0/255,
    data_format='channels_last',
    rotation_range=30,
    horizontal_flip=True,
    fill_mode='reflect')

valid_datagen = ImageDataGenerator(
    rescale=1.0/255,
    data_format='channels_last')



In [14]:

    
# Define the batch size:
batch_size=32



In [15]:

    
# Define the train and validation generators: 
train_generator = train_datagen.flow_from_directory(
    directory='./hot-dog-not-hot-dog/train',
    target_size=(224, 224),
    classes=['hot_dog','not_hot_dog'],
    class_mode='categorical',
    batch_size=batch_size,
    shuffle=True,
    seed=42)

valid_generator = valid_datagen.flow_from_directory(
    directory='./hot-dog-not-hot-dog/test',
    target_size=(224, 224),
    classes=['hot_dog','not_hot_dog'],
    class_mode='categorical',
    batch_size=batch_size,
    shuffle=True,
    seed=42)









    



Found 498 images belonging to 2 classes.
Found 500 images belonging to 2 classes.

Train!



In [16]:

    
model.fit_generator(train_generator, steps_per_epoch=15, 
                    epochs=16, validation_data=valid_generator, 
                    validation_steps=15, callbacks=[modelcheckpoint])









    



Epoch 1/16
15/15 [==============================] - 66s - loss: 0.9554 - acc: 0.6125 - val_loss: 0.6407 - val_acc: 0.6813
Epoch 2/16
15/15 [==============================] - 6s - loss: 0.6735 - acc: 0.6653 - val_loss: 0.4638 - val_acc: 0.7708
Epoch 3/16
15/15 [==============================] - 8s - loss: 0.4699 - acc: 0.7749 - val_loss: 0.4216 - val_acc: 0.8034
Epoch 4/16
15/15 [==============================] - 7s - loss: 0.4460 - acc: 0.7905 - val_loss: 0.4306 - val_acc: 0.8034
Epoch 5/16
15/15 [==============================] - 8s - loss: 0.3581 - acc: 0.8359 - val_loss: 0.4347 - val_acc: 0.8013
Epoch 6/16
15/15 [==============================] - 7s - loss: 0.3165 - acc: 0.8560 - val_loss: 0.4827 - val_acc: 0.7415
Epoch 7/16
15/15 [==============================] - 8s - loss: 0.3269 - acc: 0.8425 - val_loss: 0.5908 - val_acc: 0.7628
Epoch 8/16
15/15 [==============================] - 8s - loss: 0.3220 - acc: 0.8461 - val_loss: 0.5413 - val_acc: 0.7692
Epoch 9/16
15/15 [==============================] - 8s - loss: 0.2519 - acc: 0.8822 - val_loss: 0.5079 - val_acc: 0.7778
Epoch 10/16
15/15 [==============================] - 7s - loss: 0.3160 - acc: 0.8628 - val_loss: 0.4417 - val_acc: 0.7885
Epoch 11/16
15/15 [==============================] - 8s - loss: 0.3001 - acc: 0.8729 - val_loss: 0.5363 - val_acc: 0.7714
Epoch 12/16
15/15 [==============================] - 7s - loss: 0.3297 - acc: 0.8410 - val_loss: 0.5267 - val_acc: 0.7927
Epoch 13/16
15/15 [==============================] - 8s - loss: 0.2733 - acc: 0.8780 - val_loss: 0.4656 - val_acc: 0.8013
Epoch 14/16
15/15 [==============================] - 7s - loss: 0.2471 - acc: 0.8899 - val_loss: 0.6002 - val_acc: 0.7842
Epoch 15/16
15/15 [==============================] - 7s - loss: 0.2397 - acc: 0.8932 - val_loss: 0.5060 - val_acc: 0.7949
Epoch 16/16
15/15 [==============================] - 8s - loss: 0.2694 - acc: 0.8958 - val_loss: 0.9579 - val_acc: 0.7179






    Out[16]:





<keras.callbacks.History at 0x7ff1774b07f0>



In [17]:

    
model.load_weights('model_output/transfer_VGG/weights.02.hdf5')



In [ ]: