Transfer Learning

Most of the time you won't want to train a whole convolutional network yourself. Modern ConvNets training on huge datasets like ImageNet take weeks on multiple GPUs. Instead, most people use a pretrained network either as a fixed feature extractor, or as an initial network to fine tune. In this notebook, you'll be using VGGNet trained on the ImageNet dataset as a feature extractor. Below is a diagram of the VGGNet architecture.

VGGNet is great because it's simple and has great performance, coming in second in the ImageNet competition. The idea here is that we keep all the convolutional layers, but replace the final fully connected layers with our own classifier. This way we can use VGGNet as a feature extractor for our images then easily train a simple classifier on top of that. What we'll do is take the first fully connected layer with 4096 units, including thresholding with ReLUs. We can use those values as a code for each image, then build a classifier on top of those codes.

You can read more about transfer learning from the CS231n course notes.

Pretrained VGGNet

We'll be using a pretrained network from https://github.com/machrisaa/tensorflow-vgg.

This is a really nice implementation of VGGNet, quite easy to work with. The network has already been trained and the parameters are available from this link.


In [1]:
from urllib.request import urlretrieve
from os.path import isfile, isdir
from tqdm import tqdm

vgg_dir = 'tensorflow_vgg/'
# Make sure vgg exists
if not isdir(vgg_dir):
    raise Exception("VGG directory doesn't exist!")

class DLProgress(tqdm):
    last_block = 0

    def hook(self, block_num=1, block_size=1, total_size=None):
        self.total = total_size
        self.update((block_num - self.last_block) * block_size)
        self.last_block = block_num

if not isfile(vgg_dir + "vgg16.npy"):
    with DLProgress(unit='B', unit_scale=True, miniters=1, desc='VGG16 Parameters') as pbar:
        urlretrieve(
            'https://s3.amazonaws.com/content.udacity-data.com/nd101/vgg16.npy',
            vgg_dir + 'vgg16.npy',
            pbar.hook)
else:
    print("Parameter file already exists!")


VGG16 Parameters: 553MB [09:07, 1.01MB/s]                                                                              

Flower power

Here we'll be using VGGNet to classify images of flowers. To get the flower dataset, run the cell below. This dataset comes from the TensorFlow inception tutorial.


In [2]:
import tarfile

dataset_folder_path = 'flower_photos'

class DLProgress(tqdm):
    last_block = 0

    def hook(self, block_num=1, block_size=1, total_size=None):
        self.total = total_size
        self.update((block_num - self.last_block) * block_size)
        self.last_block = block_num

if not isfile('flower_photos.tar.gz'):
    with DLProgress(unit='B', unit_scale=True, miniters=1, desc='Flowers Dataset') as pbar:
        urlretrieve(
            'http://download.tensorflow.org/example_images/flower_photos.tgz',
            'flower_photos.tar.gz',
            pbar.hook)

if not isdir(dataset_folder_path):
    with tarfile.open('flower_photos.tar.gz') as tar:
        tar.extractall()
        tar.close()


Flowers Dataset: 229MB [03:25, 1.12MB/s]                                                                               

ConvNet Codes

Below, we'll run through all the images in our dataset and get codes for each of them. That is, we'll run the images through the VGGNet convolutional layers and record the values of the first fully connected layer. We can then write these to a file for later when we build our own classifier.

Here we're using the vgg16 module from tensorflow_vgg. The network takes images of size $244 \times 224 \times 3$ as input. Then it has 5 sets of convolutional layers. The network implemented here has this structure (copied from the source code:

self.conv1_1 = self.conv_layer(bgr, "conv1_1")
self.conv1_2 = self.conv_layer(self.conv1_1, "conv1_2")
self.pool1 = self.max_pool(self.conv1_2, 'pool1')

self.conv2_1 = self.conv_layer(self.pool1, "conv2_1")
self.conv2_2 = self.conv_layer(self.conv2_1, "conv2_2")
self.pool2 = self.max_pool(self.conv2_2, 'pool2')

self.conv3_1 = self.conv_layer(self.pool2, "conv3_1")
self.conv3_2 = self.conv_layer(self.conv3_1, "conv3_2")
self.conv3_3 = self.conv_layer(self.conv3_2, "conv3_3")
self.pool3 = self.max_pool(self.conv3_3, 'pool3')

self.conv4_1 = self.conv_layer(self.pool3, "conv4_1")
self.conv4_2 = self.conv_layer(self.conv4_1, "conv4_2")
self.conv4_3 = self.conv_layer(self.conv4_2, "conv4_3")
self.pool4 = self.max_pool(self.conv4_3, 'pool4')

self.conv5_1 = self.conv_layer(self.pool4, "conv5_1")
self.conv5_2 = self.conv_layer(self.conv5_1, "conv5_2")
self.conv5_3 = self.conv_layer(self.conv5_2, "conv5_3")
self.pool5 = self.max_pool(self.conv5_3, 'pool5')

self.fc6 = self.fc_layer(self.pool5, "fc6")
self.relu6 = tf.nn.relu(self.fc6)

So what we want are the values of the first fully connected layer, after being ReLUd (self.relu6). To build the network, we use

with tf.Session() as sess:
    vgg = vgg16.Vgg16()
    input_ = tf.placeholder(tf.float32, [None, 224, 224, 3])
    with tf.name_scope("content_vgg"):
        vgg.build(input_)

This creates the vgg object, then builds the graph with vgg.build(input_). Then to get the values from the layer,

feed_dict = {input_: images}
codes = sess.run(vgg.relu6, feed_dict=feed_dict)

In [2]:
import os

import numpy as np
import tensorflow as tf

from tensorflow_vgg import vgg16
from tensorflow_vgg import utils

In [3]:
data_dir = 'flower_photos/'
contents = os.listdir(data_dir)
classes = [each for each in contents if os.path.isdir(data_dir + each)]

Below I'm running images through the VGG network in batches.


In [5]:
# Set the batch size higher if you can fit in in your GPU memory
batch_size = 10
codes_list = []
labels = []
batch = []

codes = None

with tf.Session() as sess:
    vgg = vgg16.Vgg16()
    input_ = tf.placeholder(tf.float32, [None, 224, 224, 3])
    with tf.name_scope("content_vgg"):
        vgg.build(input_)

    for each in classes:
        print("Starting {} images".format(each))
        class_path = data_dir + each
        files = os.listdir(class_path)
        for ii, file in enumerate(files, 1):
            # Add images to the current batch
            # utils.load_image crops the input images for us, from the center
            img = utils.load_image(os.path.join(class_path, file))
            batch.append(img.reshape((1, 224, 224, 3)))
            labels.append(each)
            
            # Running the batch through the network to get the codes
            if ii % batch_size == 0 or ii == len(files):
                images = np.concatenate(batch)

                feed_dict = {input_: images}
                codes_batch = sess.run(vgg.relu6, feed_dict=feed_dict)
                
                # Here I'm building an array of the codes
                if codes is None:
                    codes = codes_batch
                else:
                    codes = np.concatenate((codes, codes_batch))
                
                # Reset to start building the next batch
                batch = []
                print('{} images processed'.format(ii))


F:\AllProjects\deep-learning\transfer-learning\tensorflow_vgg\vgg16.npy
npy file loaded
build model started
build model finished: 0s
Starting daisy images
10 images processed
20 images processed
30 images processed
40 images processed
50 images processed
60 images processed
70 images processed
80 images processed
90 images processed
100 images processed
110 images processed
120 images processed
130 images processed
140 images processed
150 images processed
160 images processed
170 images processed
180 images processed
190 images processed
200 images processed
210 images processed
220 images processed
230 images processed
240 images processed
250 images processed
260 images processed
270 images processed
280 images processed
290 images processed
300 images processed
310 images processed
320 images processed
330 images processed
340 images processed
350 images processed
360 images processed
370 images processed
380 images processed
390 images processed
400 images processed
410 images processed
420 images processed
430 images processed
440 images processed
450 images processed
460 images processed
470 images processed
480 images processed
490 images processed
500 images processed
510 images processed
520 images processed
530 images processed
540 images processed
550 images processed
560 images processed
570 images processed
580 images processed
590 images processed
600 images processed
610 images processed
620 images processed
630 images processed
633 images processed
Starting dandelion images
10 images processed
20 images processed
30 images processed
40 images processed
50 images processed
60 images processed
70 images processed
80 images processed
90 images processed
100 images processed
110 images processed
120 images processed
130 images processed
140 images processed
150 images processed
160 images processed
170 images processed
180 images processed
190 images processed
200 images processed
210 images processed
220 images processed
230 images processed
240 images processed
250 images processed
260 images processed
270 images processed
280 images processed
290 images processed
300 images processed
310 images processed
320 images processed
330 images processed
340 images processed
350 images processed
360 images processed
370 images processed
380 images processed
390 images processed
400 images processed
410 images processed
420 images processed
430 images processed
440 images processed
450 images processed
460 images processed
470 images processed
480 images processed
490 images processed
500 images processed
510 images processed
520 images processed
530 images processed
540 images processed
550 images processed
560 images processed
570 images processed
580 images processed
590 images processed
600 images processed
610 images processed
620 images processed
630 images processed
640 images processed
650 images processed
660 images processed
670 images processed
680 images processed
690 images processed
700 images processed
710 images processed
720 images processed
730 images processed
740 images processed
750 images processed
760 images processed
770 images processed
780 images processed
790 images processed
800 images processed
810 images processed
820 images processed
830 images processed
840 images processed
850 images processed
860 images processed
870 images processed
880 images processed
890 images processed
898 images processed
Starting roses images
10 images processed
20 images processed
30 images processed
40 images processed
50 images processed
60 images processed
70 images processed
80 images processed
90 images processed
100 images processed
110 images processed
120 images processed
130 images processed
140 images processed
150 images processed
160 images processed
170 images processed
180 images processed
190 images processed
200 images processed
210 images processed
220 images processed
230 images processed
240 images processed
250 images processed
260 images processed
270 images processed
280 images processed
290 images processed
300 images processed
310 images processed
320 images processed
330 images processed
340 images processed
350 images processed
360 images processed
370 images processed
380 images processed
390 images processed
400 images processed
410 images processed
420 images processed
430 images processed
440 images processed
450 images processed
460 images processed
470 images processed
480 images processed
490 images processed
500 images processed
510 images processed
520 images processed
530 images processed
540 images processed
550 images processed
560 images processed
570 images processed
580 images processed
590 images processed
600 images processed
610 images processed
620 images processed
630 images processed
640 images processed
641 images processed
Starting sunflowers images
10 images processed
20 images processed
30 images processed
40 images processed
50 images processed
60 images processed
70 images processed
80 images processed
90 images processed
100 images processed
110 images processed
120 images processed
130 images processed
140 images processed
150 images processed
160 images processed
170 images processed
180 images processed
190 images processed
200 images processed
210 images processed
220 images processed
230 images processed
240 images processed
250 images processed
260 images processed
270 images processed
280 images processed
290 images processed
300 images processed
310 images processed
320 images processed
330 images processed
340 images processed
350 images processed
360 images processed
370 images processed
380 images processed
390 images processed
400 images processed
410 images processed
420 images processed
430 images processed
440 images processed
450 images processed
460 images processed
470 images processed
480 images processed
490 images processed
500 images processed
510 images processed
520 images processed
530 images processed
540 images processed
550 images processed
560 images processed
570 images processed
580 images processed
590 images processed
600 images processed
610 images processed
620 images processed
630 images processed
640 images processed
650 images processed
660 images processed
670 images processed
680 images processed
690 images processed
699 images processed
Starting tulips images
10 images processed
20 images processed
30 images processed
40 images processed
50 images processed
60 images processed
70 images processed
80 images processed
90 images processed
100 images processed
110 images processed
120 images processed
130 images processed
140 images processed
150 images processed
160 images processed
170 images processed
180 images processed
190 images processed
200 images processed
210 images processed
220 images processed
230 images processed
240 images processed
250 images processed
260 images processed
270 images processed
280 images processed
290 images processed
300 images processed
310 images processed
320 images processed
330 images processed
340 images processed
350 images processed
360 images processed
370 images processed
380 images processed
390 images processed
400 images processed
410 images processed
420 images processed
430 images processed
440 images processed
450 images processed
460 images processed
470 images processed
480 images processed
490 images processed
500 images processed
510 images processed
520 images processed
530 images processed
540 images processed
550 images processed
560 images processed
570 images processed
580 images processed
590 images processed
600 images processed
610 images processed
620 images processed
630 images processed
640 images processed
650 images processed
660 images processed
670 images processed
680 images processed
690 images processed
700 images processed
710 images processed
720 images processed
730 images processed
740 images processed
750 images processed
760 images processed
770 images processed
780 images processed
790 images processed
799 images processed

In [6]:
# write codes to file
with open('codes', 'w') as f:
    codes.tofile(f)
    
# write labels to file
import csv
with open('labels', 'w') as f:
    writer = csv.writer(f, delimiter='\n')
    writer.writerow(labels)

Building the Classifier

Now that we have codes for all the images, we can build a simple classifier on top of them. The codes behave just like normal input into a simple neural network. Below I'm going to have you do most of the work.


In [7]:
# read codes and labels from file
import csv

with open('labels') as f:
    reader = csv.reader(f, delimiter='\n')
    labels = np.array([each for each in reader if len(each) > 0]).squeeze()
with open('codes') as f:
    codes = np.fromfile(f, dtype=np.float32)
    codes = codes.reshape((len(labels), -1))

Data prep

As usual, now we need to one-hot encode our labels and create validation/test sets. First up, creating our labels!

Exercise: From scikit-learn, use LabelBinarizer to create one-hot encoded vectors from the labels.


In [8]:
from sklearn.preprocessing import LabelBinarizer

lb = LabelBinarizer()
lb.fit(labels)

labels_vecs = lb.transform(labels)

Now you'll want to create your training, validation, and test sets. An important thing to note here is that our labels and data aren't randomized yet. We'll want to shuffle our data so the validation and test sets contain data from all classes. Otherwise, you could end up with testing sets that are all one class. Typically, you'll also want to make sure that each smaller set has the same the distribution of classes as it is for the whole data set. The easiest way to accomplish both these goals is to use StratifiedShuffleSplit from scikit-learn.

You can create the splitter like so:

ss = StratifiedShuffleSplit(n_splits=1, test_size=0.2)

Then split the data with

splitter = ss.split(x, y)

ss.split returns a generator of indices. You can pass the indices into the arrays to get the split sets. The fact that it's a generator means you either need to iterate over it, or use next(splitter) to get the indices. Be sure to read the documentation and the user guide.

Exercise: Use StratifiedShuffleSplit to split the codes and labels into training, validation, and test sets.


In [9]:
from sklearn.model_selection import StratifiedShuffleSplit

ss = StratifiedShuffleSplit(n_splits=1, test_size=0.2)

train_idx, val_idx = next(ss.split(codes, labels_vecs))

half_val_len = int(len(val_idx)/2)
val_idx, test_idx = val_idx[:half_val_len], val_idx[half_val_len:]

train_x, train_y = codes[train_idx], labels_vecs[train_idx]
val_x, val_y = codes[val_idx], labels_vecs[val_idx]
test_x, test_y = codes[test_idx], labels_vecs[test_idx]

In [10]:
print("Train shapes (x, y):", train_x.shape, train_y.shape)
print("Validation shapes (x, y):", val_x.shape, val_y.shape)
print("Test shapes (x, y):", test_x.shape, test_y.shape)


Train shapes (x, y): (2936, 4096) (2936, 5)
Validation shapes (x, y): (367, 4096) (367, 5)
Test shapes (x, y): (367, 4096) (367, 5)

If you did it right, you should see these sizes for the training sets:

Train shapes (x, y): (2936, 4096) (2936, 5)
Validation shapes (x, y): (367, 4096) (367, 5)
Test shapes (x, y): (367, 4096) (367, 5)

Classifier layers

Once you have the convolutional codes, you just need to build a classfier from some fully connected layers. You use the codes as the inputs and the image labels as targets. Otherwise the classifier is a typical neural network.

Exercise: With the codes and labels loaded, build the classifier. Consider the codes as your inputs, each of them are 4096D vectors. You'll want to use a hidden layer and an output layer as your classifier. Remember that the output layer needs to have one unit for each class and a softmax activation function. Use the cross entropy to calculate the cost.


In [11]:
inputs_ = tf.placeholder(tf.float32, shape=[None, codes.shape[1]])
labels_ = tf.placeholder(tf.int64, shape=[None, labels_vecs.shape[1]])

fc = tf.contrib.layers.fully_connected(inputs_, 256)
    
logits = tf.contrib.layers.fully_connected(fc, labels_vecs.shape[1], activation_fn=None)
cross_entropy = tf.nn.softmax_cross_entropy_with_logits(labels=labels_, logits=logits)
cost = tf.reduce_mean(cross_entropy)

optimizer = tf.train.AdamOptimizer().minimize(cost)

predicted = tf.nn.softmax(logits)
correct_pred = tf.equal(tf.argmax(predicted, 1), tf.argmax(labels_, 1))
accuracy = tf.reduce_mean(tf.cast(correct_pred, tf.float32))

Batches!

Here is just a simple way to do batches. I've written it so that it includes all the data. Sometimes you'll throw out some data at the end to make sure you have full batches. Here I just extend the last batch to include the remaining data.


In [12]:
def get_batches(x, y, n_batches=10):
    """ Return a generator that yields batches from arrays x and y. """
    batch_size = len(x)//n_batches
    
    for ii in range(0, n_batches*batch_size, batch_size):
        # If we're not on the last batch, grab data with size batch_size
        if ii != (n_batches-1)*batch_size:
            X, Y = x[ii: ii+batch_size], y[ii: ii+batch_size] 
        # On the last batch, grab the rest of the data
        else:
            X, Y = x[ii:], y[ii:]
        # I love generators
        yield X, Y

Training

Here, we'll train the network.

Exercise: So far we've been providing the training code for you. Here, I'm going to give you a bit more of a challenge and have you write the code to train the network. Of course, you'll be able to see my solution if you need help.


In [13]:
epochs = 10
iteration = 0
saver = tf.train.Saver()
with tf.Session() as sess:
    
    sess.run(tf.global_variables_initializer())
    for e in range(epochs):
        for x, y in get_batches(train_x, train_y):
            feed = {inputs_: x,
                    labels_: y}
            loss, _ = sess.run([cost, optimizer], feed_dict=feed)
            print("Epoch: {}/{}".format(e+1, epochs),
                  "Iteration: {}".format(iteration),
                  "Training loss: {:.5f}".format(loss))
            iteration += 1
            
            if iteration % 5 == 0:
                feed = {inputs_: val_x,
                        labels_: val_y}
                val_acc = sess.run(accuracy, feed_dict=feed)
                print("Epoch: {}/{}".format(e, epochs),
                      "Iteration: {}".format(iteration),
                      "Validation Acc: {:.4f}".format(val_acc))
    saver.save(sess, "checkpoints/flowers.ckpt")


Epoch: 1/10 Iteration: 0 Training loss: 6.83105
Epoch: 1/10 Iteration: 1 Training loss: 16.12787
Epoch: 1/10 Iteration: 2 Training loss: 18.87531
Epoch: 1/10 Iteration: 3 Training loss: 10.62253
Epoch: 1/10 Iteration: 4 Training loss: 8.27784
Epoch: 0/10 Iteration: 5 Validation Acc: 0.5450
Epoch: 1/10 Iteration: 5 Training loss: 5.30857
Epoch: 1/10 Iteration: 6 Training loss: 3.01656
Epoch: 1/10 Iteration: 7 Training loss: 1.91641
Epoch: 1/10 Iteration: 8 Training loss: 2.67971
Epoch: 1/10 Iteration: 9 Training loss: 3.31814
Epoch: 0/10 Iteration: 10 Validation Acc: 0.6376
Epoch: 2/10 Iteration: 10 Training loss: 3.01398
Epoch: 2/10 Iteration: 11 Training loss: 2.86086
Epoch: 2/10 Iteration: 12 Training loss: 1.55826
Epoch: 2/10 Iteration: 13 Training loss: 1.26445
Epoch: 2/10 Iteration: 14 Training loss: 1.22664
Epoch: 1/10 Iteration: 15 Validation Acc: 0.7984
Epoch: 2/10 Iteration: 15 Training loss: 0.69025
Epoch: 2/10 Iteration: 16 Training loss: 0.93449
Epoch: 2/10 Iteration: 17 Training loss: 1.07787
Epoch: 2/10 Iteration: 18 Training loss: 1.55226
Epoch: 2/10 Iteration: 19 Training loss: 0.94599
Epoch: 1/10 Iteration: 20 Validation Acc: 0.7548
Epoch: 3/10 Iteration: 20 Training loss: 1.19399
Epoch: 3/10 Iteration: 21 Training loss: 0.80513
Epoch: 3/10 Iteration: 22 Training loss: 0.99141
Epoch: 3/10 Iteration: 23 Training loss: 0.86765
Epoch: 3/10 Iteration: 24 Training loss: 0.54006
Epoch: 2/10 Iteration: 25 Validation Acc: 0.8392
Epoch: 3/10 Iteration: 25 Training loss: 0.56149
Epoch: 3/10 Iteration: 26 Training loss: 0.51853
Epoch: 3/10 Iteration: 27 Training loss: 0.36491
Epoch: 3/10 Iteration: 28 Training loss: 0.39325
Epoch: 3/10 Iteration: 29 Training loss: 0.41676
Epoch: 2/10 Iteration: 30 Validation Acc: 0.8392
Epoch: 4/10 Iteration: 30 Training loss: 0.46002
Epoch: 4/10 Iteration: 31 Training loss: 0.55004
Epoch: 4/10 Iteration: 32 Training loss: 0.36509
Epoch: 4/10 Iteration: 33 Training loss: 0.35646
Epoch: 4/10 Iteration: 34 Training loss: 0.51689
Epoch: 3/10 Iteration: 35 Validation Acc: 0.8256
Epoch: 4/10 Iteration: 35 Training loss: 0.30043
Epoch: 4/10 Iteration: 36 Training loss: 0.45037
Epoch: 4/10 Iteration: 37 Training loss: 0.27634
Epoch: 4/10 Iteration: 38 Training loss: 0.25237
Epoch: 4/10 Iteration: 39 Training loss: 0.16054
Epoch: 3/10 Iteration: 40 Validation Acc: 0.8610
Epoch: 5/10 Iteration: 40 Training loss: 0.21953
Epoch: 5/10 Iteration: 41 Training loss: 0.19416
Epoch: 5/10 Iteration: 42 Training loss: 0.20036
Epoch: 5/10 Iteration: 43 Training loss: 0.20402
Epoch: 5/10 Iteration: 44 Training loss: 0.16661
Epoch: 4/10 Iteration: 45 Validation Acc: 0.8747
Epoch: 5/10 Iteration: 45 Training loss: 0.14164
Epoch: 5/10 Iteration: 46 Training loss: 0.25398
Epoch: 5/10 Iteration: 47 Training loss: 0.22019
Epoch: 5/10 Iteration: 48 Training loss: 0.17126
Epoch: 5/10 Iteration: 49 Training loss: 0.13203
Epoch: 4/10 Iteration: 50 Validation Acc: 0.8883
Epoch: 6/10 Iteration: 50 Training loss: 0.20429
Epoch: 6/10 Iteration: 51 Training loss: 0.17949
Epoch: 6/10 Iteration: 52 Training loss: 0.13337
Epoch: 6/10 Iteration: 53 Training loss: 0.14060
Epoch: 6/10 Iteration: 54 Training loss: 0.13124
Epoch: 5/10 Iteration: 55 Validation Acc: 0.8856
Epoch: 6/10 Iteration: 55 Training loss: 0.10118
Epoch: 6/10 Iteration: 56 Training loss: 0.19599
Epoch: 6/10 Iteration: 57 Training loss: 0.12400
Epoch: 6/10 Iteration: 58 Training loss: 0.09383
Epoch: 6/10 Iteration: 59 Training loss: 0.06692
Epoch: 5/10 Iteration: 60 Validation Acc: 0.8719
Epoch: 7/10 Iteration: 60 Training loss: 0.12170
Epoch: 7/10 Iteration: 61 Training loss: 0.13112
Epoch: 7/10 Iteration: 62 Training loss: 0.11522
Epoch: 7/10 Iteration: 63 Training loss: 0.10736
Epoch: 7/10 Iteration: 64 Training loss: 0.09849
Epoch: 6/10 Iteration: 65 Validation Acc: 0.8774
Epoch: 7/10 Iteration: 65 Training loss: 0.08137
Epoch: 7/10 Iteration: 66 Training loss: 0.14627
Epoch: 7/10 Iteration: 67 Training loss: 0.09552
Epoch: 7/10 Iteration: 68 Training loss: 0.05922
Epoch: 7/10 Iteration: 69 Training loss: 0.05270
Epoch: 6/10 Iteration: 70 Validation Acc: 0.8992
Epoch: 8/10 Iteration: 70 Training loss: 0.09773
Epoch: 8/10 Iteration: 71 Training loss: 0.08882
Epoch: 8/10 Iteration: 72 Training loss: 0.07773
Epoch: 8/10 Iteration: 73 Training loss: 0.08307
Epoch: 8/10 Iteration: 74 Training loss: 0.06658
Epoch: 7/10 Iteration: 75 Validation Acc: 0.8937
Epoch: 8/10 Iteration: 75 Training loss: 0.06538
Epoch: 8/10 Iteration: 76 Training loss: 0.10554
Epoch: 8/10 Iteration: 77 Training loss: 0.08278
Epoch: 8/10 Iteration: 78 Training loss: 0.04470
Epoch: 8/10 Iteration: 79 Training loss: 0.04466
Epoch: 7/10 Iteration: 80 Validation Acc: 0.8937
Epoch: 9/10 Iteration: 80 Training loss: 0.08180
Epoch: 9/10 Iteration: 81 Training loss: 0.06043
Epoch: 9/10 Iteration: 82 Training loss: 0.06415
Epoch: 9/10 Iteration: 83 Training loss: 0.06631
Epoch: 9/10 Iteration: 84 Training loss: 0.05151
Epoch: 8/10 Iteration: 85 Validation Acc: 0.8965
Epoch: 9/10 Iteration: 85 Training loss: 0.05481
Epoch: 9/10 Iteration: 86 Training loss: 0.08841
Epoch: 9/10 Iteration: 87 Training loss: 0.06416
Epoch: 9/10 Iteration: 88 Training loss: 0.03447
Epoch: 9/10 Iteration: 89 Training loss: 0.03428
Epoch: 8/10 Iteration: 90 Validation Acc: 0.8937
Epoch: 10/10 Iteration: 90 Training loss: 0.05629
Epoch: 10/10 Iteration: 91 Training loss: 0.05080
Epoch: 10/10 Iteration: 92 Training loss: 0.05108
Epoch: 10/10 Iteration: 93 Training loss: 0.05086
Epoch: 10/10 Iteration: 94 Training loss: 0.04208
Epoch: 9/10 Iteration: 95 Validation Acc: 0.8910
Epoch: 10/10 Iteration: 95 Training loss: 0.04360
Epoch: 10/10 Iteration: 96 Training loss: 0.06878
Epoch: 10/10 Iteration: 97 Training loss: 0.05376
Epoch: 10/10 Iteration: 98 Training loss: 0.02781
Epoch: 10/10 Iteration: 99 Training loss: 0.02976
Epoch: 9/10 Iteration: 100 Validation Acc: 0.8965

Testing

Below you see the test accuracy. You can also see the predictions returned for images.


In [14]:
with tf.Session() as sess:
    saver.restore(sess, tf.train.latest_checkpoint('checkpoints'))
    
    feed = {inputs_: test_x,
            labels_: test_y}
    test_acc = sess.run(accuracy, feed_dict=feed)
    print("Test accuracy: {:.4f}".format(test_acc))


INFO:tensorflow:Restoring parameters from checkpoints\flowers.ckpt
Test accuracy: 0.9046

In [18]:
%matplotlib inline

import matplotlib.pyplot as plt
from scipy.ndimage import imread

Below, feel free to choose images and see how the trained classifier predicts the flowers in them.


In [19]:
test_img_path = 'flower_photos/20151210160455_XPaji.jpg'
test_img = imread(test_img_path)
plt.imshow(test_img)


Out[19]:
<matplotlib.image.AxesImage at 0x2225b590160>

In [20]:
# Run this cell if you don't have a vgg graph built
with tf.Session() as sess:
    input_ = tf.placeholder(tf.float32, [None, 224, 224, 3])
    vgg = vgg16.Vgg16()
    vgg.build(input_)


F:\AllProjects\deep-learning\transfer-learning\tensorflow_vgg\vgg16.npy
npy file loaded
build model started
build model finished: 0s

In [21]:
with tf.Session() as sess:
    img = utils.load_image(test_img_path)
    img = img.reshape((1, 224, 224, 3))

    feed_dict = {input_: img}
    code = sess.run(vgg.relu6, feed_dict=feed_dict)
        
saver = tf.train.Saver()
with tf.Session() as sess:
    saver.restore(sess, tf.train.latest_checkpoint('checkpoints'))
    
    feed = {inputs_: code}
    prediction = sess.run(predicted, feed_dict=feed).squeeze()


INFO:tensorflow:Restoring parameters from checkpoints\flowers.ckpt

In [22]:
plt.imshow(test_img)


Out[22]:
<matplotlib.image.AxesImage at 0x2225d7064e0>

In [23]:
plt.barh(np.arange(5), prediction)
_ = plt.yticks(np.arange(5), lb.classes_)



In [ ]: