Einführung in CNNs


In [1]:
import warnings
warnings.filterwarnings('ignore')

In [2]:
%matplotlib inline
%pylab inline


Populating the interactive namespace from numpy and matplotlib

In [3]:
import matplotlib.pylab as plt
import numpy as np

In [4]:
from distutils.version import StrictVersion

In [5]:
import sklearn
print(sklearn.__version__)

assert StrictVersion(sklearn.__version__ ) >= StrictVersion('0.18.1')


0.19.0

In [6]:
import tensorflow as tf
tf.logging.set_verbosity(tf.logging.ERROR)
print(tf.__version__)

assert StrictVersion(tf.__version__) >= StrictVersion('1.1.0')


1.2.1

In [7]:
# We need keras 2.0.6 or later as this is the version our GPU based system used to create some models
!pip install keras --upgrade
# after installation call Restart & Run All in Kernel menu


Requirement already up-to-date: keras in /home/nbcommon/anaconda3_410/lib/python3.5/site-packages
Requirement already up-to-date: pyyaml in /home/nbcommon/anaconda3_410/lib/python3.5/site-packages (from keras)
Requirement already up-to-date: scipy>=0.14 in /home/nbcommon/anaconda3_410/lib/python3.5/site-packages (from keras)
Requirement already up-to-date: six>=1.9.0 in /home/nbcommon/anaconda3_410/lib/python3.5/site-packages (from keras)
Requirement already up-to-date: numpy>=1.9.1 in /home/nbcommon/anaconda3_410/lib/python3.5/site-packages (from keras)

In [7]:
import keras
print(keras.__version__)

assert StrictVersion(keras.__version__) >= StrictVersion('2.0.6')


Using TensorFlow backend.
2.0.8

In [9]:
import pandas as pd
print(pd.__version__)

assert StrictVersion(pd.__version__) >= StrictVersion('0.19.0')


0.19.2

Laden und Vorbereiten der Bild-Daten


In [ ]:
!curl -O https://raw.githubusercontent.com/DJCordhose/speed-limit-signs/master/data/speed-limit-signs.zip
from zipfile import ZipFile
zip = ZipFile(r'speed-limit-signs.zip')
zip.extractall('.')


  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 1810k  100 1810k    0     0  1496k      0  0:00:01  0:00:01 --:--:-- 1498k

In [12]:
!ls -l speed-limit-signs


total 0
drwxr-xr-x 2 nbuser nbuser   0 Sep 22 14:08 0
drwxr-xr-x 2 nbuser nbuser   0 Sep 22 14:08 1
drwxr-xr-x 2 nbuser nbuser   0 Sep 22 14:08 2
drwxr-xr-x 2 nbuser nbuser   0 Sep 22 14:08 3
drwxr-xr-x 2 nbuser nbuser   0 Sep 22 14:08 4
drwxr-xr-x 2 nbuser nbuser   0 Sep 22 14:09 5
-rw-r--r-- 1 nbuser nbuser 380 Sep 22 14:09 README.md

In [13]:
!cat speed-limit-signs/README.md


Data extracted from http://benchmark.ini.rub.de/?section=gtsdb&subsection=dataset

From http://benchmark.ini.rub.de/Dataset_GTSDB/FullIJCNN2013.zip just the samples for the speed limit signs 

## Format

https://en.wikipedia.org/wiki/Netpbm_format

Can be previewed on a Mac and be processed by http://scikit-image.org/

## Labels
- 0: 30
- 1: 50
- 2: 70
- 3: 80
- 4: 100
- 5: 120

Big Kudos to Waleed Abdulla for providing the initial idea and many of the functions used to prepare and display the images: https://medium.com/@waleedka/traffic-sign-recognition-with-tensorflow-629dffc391a6#.i728o84ib


In [14]:
import os
import skimage.data
import skimage.transform
from keras.utils.np_utils import to_categorical
import numpy as np

def load_data(data_dir, type=".ppm"):
    num_categories = 6

    # Get all subdirectories of data_dir. Each represents a label.
    directories = [d for d in os.listdir(data_dir) 
                   if os.path.isdir(os.path.join(data_dir, d))]
    # Loop through the label directories and collect the data in
    # two lists, labels and images.
    labels = []
    images = []
    for d in directories:
        label_dir = os.path.join(data_dir, d)
        file_names = [os.path.join(label_dir, f) for f in os.listdir(label_dir) if f.endswith(type)]
        # For each label, load it's images and add them to the images list.
        # And add the label number (i.e. directory name) to the labels list.
        for f in file_names:
            images.append(skimage.data.imread(f))
            labels.append(int(d))
    images64 = [skimage.transform.resize(image, (64, 64)) for image in images]
    return images64, labels

In [15]:
# Load datasets.
ROOT_PATH = "./"
original_dir = os.path.join(ROOT_PATH, "speed-limit-signs")
images, labels = load_data(original_dir, type=".ppm")

In [16]:
import matplotlib
import matplotlib.pyplot as plt

def display_images_and_labels(images, labels):
    """Display the first image of each label."""
    unique_labels = set(labels)
    plt.figure(figsize=(15, 15))
    i = 1
    for label in unique_labels:
        # Pick the first image for each label.
        image = images[labels.index(label)]
        plt.subplot(8, 8, i)  # A grid of 8 rows x 8 columns
        plt.axis('off')
        plt.title("Label {0} ({1})".format(label, labels.count(label)))
        i += 1
        _ = plt.imshow(image)

In [17]:
display_images_and_labels(images, labels)


Modell-Architektur

http://cs231n.github.io/neural-networks-1/#power

Layout of a typical CNN

http://cs231n.github.io/convolutional-networks/

Classic VGG like Architecture

  • we use a VGG like architecture
  • based on https://arxiv.org/abs/1409.1556
  • basic idea: sequential, deep, small convolutional filters, use dropouts to reduce overfitting
  • 16/19 layers are typical
  • we choose less layers, because we have limited resources

Convolutional Blocks: Cascading many Convolutional Layers having down sampling in between

http://cs231n.github.io/convolutional-networks/#conv

Example of a Convolution

Original Image

Many convolutional filters applied over all channels

http://cs.stanford.edu/people/karpathy/convnetjs/demo/cifar10.html

Downlsampling Layer: Reduces data sizes and risk of overfitting

http://cs231n.github.io/convolutional-networks/#pool

Wähle aus einer von 2 Hands-Ons


Hands-On Alternative 1 (Überblick)

Mache Erfahrungen mit allen Schichten: https://transcranial.github.io/keras-js/#/mnist-cnn

  • Die Architektur für MNIST ist einfacher, aber enthält alle Arten von Schichten, die wir auch hier nutzen
  • Zeichne ein paar Zahlen und sieh dir die Zwischenergebnisse in allen Schichten an

Nebenbei: Keras.js kann deine Keras-Modelle auch im Browser ausführen



Hands-On Alternative 2 (Funktionsweise)

Probiere Filter Kernel für CNNs aus: http://setosa.io/ev/image-kernels/



In [18]:
# again a little bit of feature engeneering

y = np.array(labels)
X = np.array(images)
from keras.utils.np_utils import to_categorical

num_categories = 6

y = to_categorical(y, num_categories)

In [68]:
from keras.models import Model
from keras.layers import Dense, Dropout, Flatten, Input
from keras.layers import Convolution2D, MaxPooling2D

# input tensor for a 3-channel 64x64 image
inputs = Input(shape=(64, 64, 3))

# one block of convolutional layers
x = Convolution2D(64, 3, activation='relu', padding='same')(inputs)
x = Convolution2D(64, 3, activation='relu', padding='same')(x)
x = Convolution2D(64, 3, activation='relu', padding='same')(x)
x = MaxPooling2D(pool_size=(2, 2))(x)

# one more block
x = Convolution2D(128, 3, activation='relu', padding='same')(x)
x = Convolution2D(128, 3, activation='relu', padding='same')(x)
x = MaxPooling2D(pool_size=(2, 2))(x)

# one more block
x = Convolution2D(256, 3, activation='relu', padding='same')(x)
x = MaxPooling2D(pool_size=(2, 2))(x)

x = Flatten()(x)
x = Dense(256, activation='relu')(x)

# softmax activation, 6 categories
predictions = Dense(6, activation='softmax')(x)

In [69]:
model = Model(input=inputs, output=predictions)
model.summary()


_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
input_9 (InputLayer)         (None, 64, 64, 3)         0         
_________________________________________________________________
conv2d_19 (Conv2D)           (None, 64, 64, 64)        1792      
_________________________________________________________________
conv2d_20 (Conv2D)           (None, 64, 64, 64)        36928     
_________________________________________________________________
conv2d_21 (Conv2D)           (None, 64, 64, 64)        36928     
_________________________________________________________________
max_pooling2d_9 (MaxPooling2 (None, 32, 32, 64)        0         
_________________________________________________________________
conv2d_22 (Conv2D)           (None, 32, 32, 128)       73856     
_________________________________________________________________
conv2d_23 (Conv2D)           (None, 32, 32, 128)       147584    
_________________________________________________________________
max_pooling2d_10 (MaxPooling (None, 16, 16, 128)       0         
_________________________________________________________________
conv2d_24 (Conv2D)           (None, 16, 16, 256)       295168    
_________________________________________________________________
max_pooling2d_11 (MaxPooling (None, 8, 8, 256)         0         
_________________________________________________________________
flatten_9 (Flatten)          (None, 16384)             0         
_________________________________________________________________
dense_17 (Dense)             (None, 256)               4194560   
_________________________________________________________________
dense_18 (Dense)             (None, 6)                 1542      
=================================================================
Total params: 4,788,358
Trainable params: 4,788,358
Non-trainable params: 0
_________________________________________________________________

In [70]:
model.compile(optimizer='adam',
              loss='categorical_crossentropy',
              metrics=['accuracy'])

Optimizers: Adam and RMSprop seem nice

http://cs231n.github.io/neural-networks-3/#ada

Zuerst testen wir unser Modell, ob wir es überhaupt trainiert bekommen, indem wir auf einer kleinen Menge von Daten overfitten

http://cs231n.github.io/neural-networks-3/#sanitycheck


In [71]:
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.9, random_state=42, stratify=y)

In [72]:
X_train.shape, y_train.shape


Out[72]:
((37, 64, 64, 3), (37, 6))

In [ ]:
# %time model.fit(X_train, y_train, epochs=50, validation_split=0.3)
%time model.fit(X_train, y_train, epochs=100, validation_split=0.3)


Train on 25 samples, validate on 12 samples
Epoch 1/100
25/25 [==============================] - 1s - loss: 1.7933 - acc: 0.0400 - val_loss: 1.6831 - val_acc: 0.4167
Epoch 2/100
25/25 [==============================] - 1s - loss: 1.7293 - acc: 0.2000 - val_loss: 1.9382 - val_acc: 0.0000e+00
Epoch 3/100
25/25 [==============================] - 1s - loss: 1.7610 - acc: 0.2400 - val_loss: 1.8848 - val_acc: 0.0000e+00
Epoch 4/100
25/25 [==============================] - 1s - loss: 1.7379 - acc: 0.3200 - val_loss: 1.8775 - val_acc: 0.0000e+00
Epoch 5/100
25/25 [==============================] - 1s - loss: 1.6711 - acc: 0.3200 - val_loss: 1.9892 - val_acc: 0.4167
Epoch 6/100
25/25 [==============================] - 1s - loss: 1.6185 - acc: 0.2400 - val_loss: 2.0577 - val_acc: 0.4167
Epoch 7/100
25/25 [==============================] - 1s - loss: 1.5515 - acc: 0.3200 - val_loss: 2.3862 - val_acc: 0.1667
Epoch 8/100
25/25 [==============================] - 1s - loss: 1.5036 - acc: 0.5200 - val_loss: 2.3639 - val_acc: 0.4167
Epoch 9/100
25/25 [==============================] - 1s - loss: 1.4309 - acc: 0.3600 - val_loss: 3.8796 - val_acc: 0.0000e+00
Epoch 10/100
25/25 [==============================] - 1s - loss: 1.6168 - acc: 0.4800 - val_loss: 2.1618 - val_acc: 0.3333
Epoch 11/100
25/25 [==============================] - 1s - loss: 1.3474 - acc: 0.4800 - val_loss: 1.8899 - val_acc: 0.3333
Epoch 12/100
25/25 [==============================] - 1s - loss: 1.4186 - acc: 0.4000 - val_loss: 1.8527 - val_acc: 0.5000
Epoch 13/100
25/25 [==============================] - 1s - loss: 1.3726 - acc: 0.4000 - val_loss: 2.0050 - val_acc: 0.4167
Epoch 14/100
25/25 [==============================] - 1s - loss: 1.2986 - acc: 0.4800 - val_loss: 2.5170 - val_acc: 0.2500
Epoch 15/100
25/25 [==============================] - 1s - loss: 1.1719 - acc: 0.6800 - val_loss: 3.7760 - val_acc: 0.0833
Epoch 16/100
25/25 [==============================] - 1s - loss: 1.0438 - acc: 0.5600 - val_loss: 4.0980 - val_acc: 0.5833
Epoch 17/100
25/25 [==============================] - 1s - loss: 1.0190 - acc: 0.6000 - val_loss: 6.3938 - val_acc: 0.2500
Epoch 18/100
25/25 [==============================] - 1s - loss: 1.3366 - acc: 0.5200 - val_loss: 4.2336 - val_acc: 0.3333
Epoch 19/100
25/25 [==============================] - 1s - loss: 0.7783 - acc: 0.6800 - val_loss: 3.4941 - val_acc: 0.3333
Epoch 20/100
25/25 [==============================] - 1s - loss: 0.8403 - acc: 0.6800 - val_loss: 3.2433 - val_acc: 0.2500
Epoch 21/100
25/25 [==============================] - 1s - loss: 0.8473 - acc: 0.6800 - val_loss: 3.3329 - val_acc: 0.2500
Epoch 22/100
25/25 [==============================] - 1s - loss: 0.7572 - acc: 0.8400 - val_loss: 3.9150 - val_acc: 0.1667
Epoch 23/100
25/25 [==============================] - 1s - loss: 0.5710 - acc: 0.8000 - val_loss: 4.8760 - val_acc: 0.1667
Epoch 24/100
25/25 [==============================] - 1s - loss: 0.4464 - acc: 0.8000 - val_loss: 4.8956 - val_acc: 0.3333
Epoch 25/100
25/25 [==============================] - 1s - loss: 0.2761 - acc: 0.8400 - val_loss: 4.3892 - val_acc: 0.5000
Epoch 26/100
25/25 [==============================] - 1s - loss: 0.2302 - acc: 0.9600 - val_loss: 4.9575 - val_acc: 0.3333
Epoch 27/100
25/25 [==============================] - 1s - loss: 0.1661 - acc: 0.9600 - val_loss: 5.6919 - val_acc: 0.4167
Epoch 28/100
25/25 [==============================] - 1s - loss: 0.5877 - acc: 0.8000 - val_loss: 6.2074 - val_acc: 0.2500
Epoch 29/100
25/25 [==============================] - 1s - loss: 0.5383 - acc: 0.8000 - val_loss: 6.7891 - val_acc: 0.3333
Epoch 31/100
25/25 [==============================] - 1s - loss: 0.4905 - acc: 0.8800 - val_loss: 5.4787 - val_acc: 0.3333
Epoch 32/100
25/25 [==============================] - 1s - loss: 0.4796 - acc: 0.9200 - val_loss: 5.2441 - val_acc: 0.1667
Epoch 33/100
25/25 [==============================] - 1s - loss: 0.4107 - acc: 0.8800 - val_loss: 3.9080 - val_acc: 0.1667
Epoch 34/100
25/25 [==============================] - 1s - loss: 0.3148 - acc: 0.9200 - val_loss: 3.3647 - val_acc: 0.3333
Epoch 35/100
25/25 [==============================] - 1s - loss: 0.3200 - acc: 0.9600 - val_loss: 3.4183 - val_acc: 0.3333
Epoch 36/100
25/25 [==============================] - 1s - loss: 0.2948 - acc: 0.9200 - val_loss: 3.9785 - val_acc: 0.2500
Epoch 37/100
25/25 [==============================] - 1s - loss: 0.2421 - acc: 0.9200 - val_loss: 4.6968 - val_acc: 0.1667
Epoch 38/100
25/25 [==============================] - 1s - loss: 0.2350 - acc: 0.8400 - val_loss: 4.4759 - val_acc: 0.3333
Epoch 39/100
25/25 [==============================] - 1s - loss: 0.1561 - acc: 0.9600 - val_loss: 4.1572 - val_acc: 0.5833
Epoch 40/100
25/25 [==============================] - 1s - loss: 0.0943 - acc: 1.0000 - val_loss: 4.2319 - val_acc: 0.5833
Epoch 41/100
25/25 [==============================] - 1s - loss: 0.0627 - acc: 1.0000 - val_loss: 4.6920 - val_acc: 0.5000
Epoch 42/100
25/25 [==============================] - 1s - loss: 0.0349 - acc: 1.0000 - val_loss: 5.6762 - val_acc: 0.4167
Epoch 43/100
25/25 [==============================] - 1s - loss: 0.0143 - acc: 1.0000 - val_loss: 6.6847 - val_acc: 0.4167
Epoch 44/100
25/25 [==============================] - 1s - loss: 0.0281 - acc: 1.0000 - val_loss: 5.1815 - val_acc: 0.5000
Epoch 45/100
25/25 [==============================] - 1s - loss: 0.0030 - acc: 1.0000 - val_loss: 4.2801 - val_acc: 0.5000
Epoch 46/100
25/25 [==============================] - 1s - loss: 0.0225 - acc: 1.0000 - val_loss: 4.1837 - val_acc: 0.6667
Epoch 47/100
25/25 [==============================] - 1s - loss: 0.0137 - acc: 1.0000 - val_loss: 4.5031 - val_acc: 0.5833
Epoch 48/100
25/25 [==============================] - 1s - loss: 9.1717e-04 - acc: 1.0000 - val_loss: 5.1536 - val_acc: 0.5000
Epoch 49/100
25/25 [==============================] - 1s - loss: 4.8559e-04 - acc: 1.0000 - val_loss: 6.3512 - val_acc: 0.3333
Epoch 50/100
25/25 [==============================] - 1s - loss: 0.0011 - acc: 1.0000 - val_loss: 7.6370 - val_acc: 0.3333
Epoch 51/100
25/25 [==============================] - 1s - loss: 0.0030 - acc: 1.0000 - val_loss: 8.5163 - val_acc: 0.3333
Epoch 52/100
25/25 [==============================] - 1s - loss: 0.0025 - acc: 1.0000 - val_loss: 8.9259 - val_acc: 0.1667
Epoch 53/100
25/25 [==============================] - 1s - loss: 0.0637 - acc: 0.9600 - val_loss: 4.4127 - val_acc: 0.5833
Epoch 54/100
25/25 [==============================] - 1s - loss: 1.7844 - acc: 0.6000 - val_loss: 8.7619 - val_acc: 0.2500
Epoch 55/100
25/25 [==============================] - 1s - loss: 0.0321 - acc: 0.9600 - val_loss: 13.3513 - val_acc: 0.0000e+00
Epoch 56/100
25/25 [==============================] - 1s - loss: 2.7935 - acc: 0.7200 - val_loss: 11.5809 - val_acc: 0.1667
Epoch 57/100
25/25 [==============================] - 1s - loss: 1.9827 - acc: 0.8400 - val_loss: 9.9685 - val_acc: 0.2500
Epoch 58/100
25/25 [==============================] - 1s - loss: 1.8238 - acc: 0.6400 - val_loss: 8.9416 - val_acc: 0.3333
Epoch 59/100
25/25 [==============================] - 1s - loss: 0.0277 - acc: 1.0000 - val_loss: 8.7894 - val_acc: 0.0833
Epoch 60/100
25/25 [==============================] - 1s - loss: 0.5200 - acc: 0.8800 - val_loss: 8.4947 - val_acc: 0.0833
Epoch 61/100
25/25 [==============================] - 1s - loss: 0.5783 - acc: 0.8800 - val_loss: 7.6639 - val_acc: 0.0833
Epoch 62/100
25/25 [==============================] - 1s - loss: 0.2498 - acc: 0.8800 - val_loss: 7.1862 - val_acc: 0.2500
Epoch 63/100
25/25 [==============================] - 1s - loss: 0.0693 - acc: 1.0000 - val_loss: 7.0393 - val_acc: 0.2500
Epoch 64/100
25/25 [==============================] - 1s - loss: 0.0368 - acc: 1.0000 - val_loss: 6.9619 - val_acc: 0.2500
Epoch 65/100
25/25 [==============================] - 1s - loss: 0.0405 - acc: 1.0000 - val_loss: 6.8821 - val_acc: 0.3333
Epoch 66/100
25/25 [==============================] - 1s - loss: 0.0739 - acc: 1.0000 - val_loss: 6.8064 - val_acc: 0.3333
Epoch 67/100
25/25 [==============================] - 1s - loss: 0.1376 - acc: 0.9200 - val_loss: 6.7576 - val_acc: 0.3333
Epoch 68/100
25/25 [==============================] - 1s - loss: 0.1690 - acc: 0.9200 - val_loss: 6.7342 - val_acc: 0.3333
Epoch 69/100
25/25 [==============================] - 1s - loss: 0.1414 - acc: 0.9200 - val_loss: 6.7353 - val_acc: 0.3333
Epoch 70/100
25/25 [==============================] - 1s - loss: 0.0936 - acc: 1.0000 - val_loss: 6.7705 - val_acc: 0.3333
Epoch 71/100
25/25 [==============================] - 1s - loss: 0.0604 - acc: 1.0000 - val_loss: 6.8503 - val_acc: 0.3333
Epoch 72/100
25/25 [==============================] - 1s - loss: 0.0408 - acc: 1.0000 - val_loss: 7.0036 - val_acc: 0.3333
Epoch 73/100
25/25 [==============================] - 1s - loss: 0.0293 - acc: 1.0000 - val_loss: 7.1662 - val_acc: 0.2500
Epoch 74/100
25/25 [==============================] - 1s - loss: 0.0215 - acc: 1.0000 - val_loss: 7.2242 - val_acc: 0.2500
Epoch 75/100
25/25 [==============================] - 1s - loss: 0.0154 - acc: 1.0000 - val_loss: 7.3098 - val_acc: 0.2500
Epoch 76/100
25/25 [==============================] - 1s - loss: 0.0101 - acc: 1.0000 - val_loss: 7.4342 - val_acc: 0.3333
Epoch 77/100
25/25 [==============================] - 1s - loss: 0.0061 - acc: 1.0000 - val_loss: 7.6955 - val_acc: 0.3333
Epoch 78/100
25/25 [==============================] - 1s - loss: 0.0045 - acc: 1.0000 - val_loss: 7.7929 - val_acc: 0.4167
Epoch 79/100
25/25 [==============================] - 1s - loss: 0.0028 - acc: 1.0000 - val_loss: 7.7913 - val_acc: 0.3333
Epoch 80/100
25/25 [==============================] - 1s - loss: 0.0012 - acc: 1.0000 - val_loss: 7.8167 - val_acc: 0.3333
Epoch 81/100
25/25 [==============================] - 1s - loss: 5.5407e-04 - acc: 1.0000 - val_loss: 7.8733 - val_acc: 0.3333
Epoch 82/100
25/25 [==============================] - 1s - loss: 2.7586e-04 - acc: 1.0000 - val_loss: 7.9352 - val_acc: 0.3333
Epoch 83/100
25/25 [==============================] - 1s - loss: 1.5266e-04 - acc: 1.0000 - val_loss: 7.9501 - val_acc: 0.3333
Epoch 84/100
25/25 [==============================] - 1s - loss: 1.0352e-04 - acc: 1.0000 - val_loss: 7.9713 - val_acc: 0.3333
Epoch 85/100
25/25 [==============================] - 1s - loss: 9.1040e-05 - acc: 1.0000 - val_loss: 7.9897 - val_acc: 0.3333
Epoch 86/100
25/25 [==============================] - 1s - loss: 9.3615e-05 - acc: 1.0000 - val_loss: 8.0008 - val_acc: 0.3333
Epoch 87/100
25/25 [==============================] - 1s - loss: 8.8113e-05 - acc: 1.0000 - val_loss: 8.0053 - val_acc: 0.3333
Epoch 88/100
25/25 [==============================] - 1s - loss: 6.7538e-05 - acc: 1.0000 - val_loss: 8.0060 - val_acc: 0.3333
Epoch 89/100
25/25 [==============================] - 1s - loss: 4.5456e-05 - acc: 1.0000 - val_loss: 8.0059 - val_acc: 0.3333
Epoch 90/100
25/25 [==============================] - 1s - loss: 3.1327e-05 - acc: 1.0000 - val_loss: 8.0033 - val_acc: 0.3333
Epoch 91/100
25/25 [==============================] - 1s - loss: 2.5860e-05 - acc: 1.0000 - val_loss: 7.9970 - val_acc: 0.3333
Epoch 92/100
25/25 [==============================] - 1s - loss: 2.5214e-05 - acc: 1.0000 - val_loss: 7.9865 - val_acc: 0.3333
Epoch 93/100
25/25 [==============================] - 1s - loss: 2.3608e-05 - acc: 1.0000 - val_loss: 7.9721 - val_acc: 0.3333
Epoch 94/100
25/25 [==============================] - 1s - loss: 1.8960e-05 - acc: 1.0000 - val_loss: 7.9549 - val_acc: 0.3333
Epoch 95/100
25/25 [==============================] - 1s - loss: 1.3501e-05 - acc: 1.0000 - val_loss: 7.9368 - val_acc: 0.3333
Epoch 96/100
25/25 [==============================] - 1s - loss: 9.0486e-06 - acc: 1.0000 - val_loss: 7.9195 - val_acc: 0.3333
Epoch 97/100
25/25 [==============================] - 1s - loss: 6.1180e-06 - acc: 1.0000 - val_loss: 7.9036 - val_acc: 0.3333
Epoch 98/100
25/25 [==============================] - 1s - loss: 4.3822e-06 - acc: 1.0000 - val_loss: 7.8894 - val_acc: 0.3333
Epoch 99/100
25/25 [==============================] - 1s - loss: 3.3951e-06 - acc: 1.0000 - val_loss: 7.8767 - val_acc: 0.3333
Epoch 100/100
25/25 [==============================] - 1s - loss: 2.8515e-06 - acc: 1.0000 - val_loss: 7.8657 - val_acc: 0.3333
CPU times: user 11min 57s, sys: 38 s, total: 12min 35s
Wall time: 2min 11s
Out[ ]:
<keras.callbacks.History at 0x7fe4a022b438>

Verlauf der Metriken beim Overfitting

Accuracy

Validation Accuracy


Hands-On: Hauptübung

Vereinfache die Architektur so weit, bis sie nicht mehr overfitten kann

  • reduziere dazu entweder die Layers
  • oder die Feature Channels
  • Reduziere die Epochen auf ca. 50, damit du schnell experimentieren kannst

Ein solches Modell kann extrem einfach sein und schon dadurch Overfittung verhindern.

Wir verfolgen hier aber eine andere Philosophie: Wir machen das Modell so komplex wie es unsere Hardware zulässt und nutzen eine andere Methode, um Overfitting zu verhindern.


Overfitting vermeiden mit Dropout


In [74]:
from keras.models import Model
from keras.layers import Dense, Dropout, Activation, Flatten, Input
from keras.layers import Convolution2D, MaxPooling2D

# this is important, try and vary between .4 and .75
drop_out = 0.7

# input tensor for a 3-channel 64x64 image
inputs = Input(shape=(64, 64, 3))

# one block of convolutional layers
x = Convolution2D(64, 3, activation='relu', padding='same')(inputs)
x = Convolution2D(64, 3, activation='relu', padding='same')(x)
x = Convolution2D(64, 3, activation='relu', padding='same')(x)
x = MaxPooling2D(pool_size=(2, 2))(x)
x = Dropout(drop_out)(x)

# one more block
x = Convolution2D(128, 3, activation='relu', padding='same')(x)
x = Convolution2D(128, 3, activation='relu', padding='same')(x)
x = MaxPooling2D(pool_size=(2, 2))(x)
x = Dropout(drop_out)(x)

# one more block
x = Convolution2D(256, 3, activation='relu', padding='same')(x)
x = MaxPooling2D(pool_size=(2, 2))(x)
x = Dropout(drop_out)(x)

x = Flatten()(x)
x = Dense(256, activation='relu')(x)
x = Dropout(drop_out)(x)

# softmax activation, 6 categories
predictions = Dense(6, activation='softmax')(x)

In [75]:
model = Model(input=inputs, output=predictions)
model.summary()


_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
input_10 (InputLayer)        (None, 64, 64, 3)         0         
_________________________________________________________________
conv2d_25 (Conv2D)           (None, 64, 64, 64)        1792      
_________________________________________________________________
conv2d_26 (Conv2D)           (None, 64, 64, 64)        36928     
_________________________________________________________________
conv2d_27 (Conv2D)           (None, 64, 64, 64)        36928     
_________________________________________________________________
max_pooling2d_12 (MaxPooling (None, 32, 32, 64)        0         
_________________________________________________________________
dropout_1 (Dropout)          (None, 32, 32, 64)        0         
_________________________________________________________________
conv2d_28 (Conv2D)           (None, 32, 32, 128)       73856     
_________________________________________________________________
conv2d_29 (Conv2D)           (None, 32, 32, 128)       147584    
_________________________________________________________________
max_pooling2d_13 (MaxPooling (None, 16, 16, 128)       0         
_________________________________________________________________
dropout_2 (Dropout)          (None, 16, 16, 128)       0         
_________________________________________________________________
conv2d_30 (Conv2D)           (None, 16, 16, 256)       295168    
_________________________________________________________________
max_pooling2d_14 (MaxPooling (None, 8, 8, 256)         0         
_________________________________________________________________
dropout_3 (Dropout)          (None, 8, 8, 256)         0         
_________________________________________________________________
flatten_10 (Flatten)         (None, 16384)             0         
_________________________________________________________________
dense_19 (Dense)             (None, 256)               4194560   
_________________________________________________________________
dropout_4 (Dropout)          (None, 256)               0         
_________________________________________________________________
dense_20 (Dense)             (None, 6)                 1542      
=================================================================
Total params: 4,788,358
Trainable params: 4,788,358
Non-trainable params: 0
_________________________________________________________________

In [42]:
model.compile(optimizer='adam',
              loss='categorical_crossentropy',
              metrics=['accuracy'])

In [43]:
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42, stratify=y)

Training

  • Auf einem GPU basierten System geht das in ein paar Minuten
  • Azure Rechner sind relativ schnell, haben aber keine GPU
  • Hier dauert jede Epoche ca. 10 Sekunden, bei 500 Epochen = 5000 Sekunden = 1,5 Stunden
  • Das können wir nicht warten
  • 2 Möglichkeiten:
    1. Du trainierst mit deinem vereinfachten Modell
    2. Wir trainieren mit diesem Notebook gemeinsam ein Modell auf einer K80 GPU
      • dieses Modell wird dann geteilt und jeder macht dann die Auswertung wieder in seinem Notebook
      • Während das Modell trainiert (dauert auf K80 GPU nur ein paar Minuten) sehen wir uns das TensorBoard an und verfolgen das Training
        • Loss, Accuracy, Validation Loss, Validation Accuracy

In [44]:
# https://keras.io/callbacks/#tensorboard
tb_callback = keras.callbacks.TensorBoard(log_dir='./tf_log')
# To start tensorboard
# tensorboard --logdir=/mnt/c/Users/olive/Development/ml/tf_log
# open http://localhost:6006

In [49]:
early_stopping_callback = keras.callbacks.EarlyStopping(monitor='val_loss', patience=100, verbose=1)

In [54]:
checkpoint_callback = keras.callbacks.ModelCheckpoint('./model-checkpoints/weights.epoch-{epoch:02d}-val_loss-{val_loss:.2f}.hdf5');

In [51]:
!rm -r tf_log

In [81]:
# Depends on harware GPU architecture, set as high as possible (this works well on K80)
BATCH_SIZE = 500

In [52]:
# %time model.fit(X_train, y_train, epochs=500, batch_size=BATCH_SIZE, validation_split=0.2, callbacks=[tb_callback, early_stopping_callback])
%time model.fit(X_train, y_train, epochs=500, batch_size=BATCH_SIZE, validation_split=0.2, callbacks=[tb_callback])
# %time model.fit(X_train, y_train, epochs=500, batch_size=BATCH_SIZE, validation_split=0.2)


Train on 242 samples, validate on 61 samples
Epoch 1/500
242/242 [==============================] - 1s - loss: 1.1399 - acc: 0.5537 - val_loss: 1.8481 - val_acc: 0.3934
Epoch 2/500
242/242 [==============================] - 0s - loss: 1.1650 - acc: 0.5083 - val_loss: 1.9291 - val_acc: 0.4590
Epoch 3/500
242/242 [==============================] - 0s - loss: 1.1924 - acc: 0.5207 - val_loss: 1.8771 - val_acc: 0.4918
Epoch 4/500
242/242 [==============================] - 0s - loss: 1.1175 - acc: 0.5455 - val_loss: 1.9865 - val_acc: 0.4426
Epoch 5/500
242/242 [==============================] - 0s - loss: 1.1362 - acc: 0.4959 - val_loss: 2.1604 - val_acc: 0.4426
Epoch 6/500
242/242 [==============================] - 0s - loss: 1.0366 - acc: 0.5537 - val_loss: 2.2037 - val_acc: 0.4590
Epoch 7/500
242/242 [==============================] - 0s - loss: 1.2770 - acc: 0.5083 - val_loss: 1.7027 - val_acc: 0.4754
Epoch 8/500
242/242 [==============================] - 0s - loss: 1.1607 - acc: 0.5041 - val_loss: 1.5693 - val_acc: 0.5246
Epoch 9/500
242/242 [==============================] - 0s - loss: 1.1738 - acc: 0.5248 - val_loss: 1.6129 - val_acc: 0.5410
Epoch 10/500
242/242 [==============================] - 0s - loss: 1.1461 - acc: 0.5331 - val_loss: 1.7696 - val_acc: 0.4754
Epoch 11/500
242/242 [==============================] - 0s - loss: 1.1274 - acc: 0.4793 - val_loss: 1.8349 - val_acc: 0.4754
Epoch 12/500
242/242 [==============================] - 0s - loss: 1.0630 - acc: 0.5372 - val_loss: 2.0153 - val_acc: 0.3770
Epoch 13/500
242/242 [==============================] - 0s - loss: 1.0856 - acc: 0.5289 - val_loss: 2.0708 - val_acc: 0.3934
Epoch 14/500
242/242 [==============================] - 0s - loss: 1.0674 - acc: 0.5620 - val_loss: 2.0049 - val_acc: 0.4098
Epoch 15/500
242/242 [==============================] - 0s - loss: 1.1161 - acc: 0.5413 - val_loss: 1.9009 - val_acc: 0.4754
Epoch 16/500
242/242 [==============================] - 0s - loss: 1.0510 - acc: 0.5620 - val_loss: 1.8349 - val_acc: 0.5082
Epoch 17/500
242/242 [==============================] - 0s - loss: 1.0444 - acc: 0.5702 - val_loss: 1.7589 - val_acc: 0.5082
Epoch 18/500
242/242 [==============================] - 0s - loss: 1.0213 - acc: 0.5785 - val_loss: 1.7356 - val_acc: 0.5246
Epoch 19/500
242/242 [==============================] - 0s - loss: 0.9496 - acc: 0.6570 - val_loss: 1.8008 - val_acc: 0.4754
Epoch 20/500
242/242 [==============================] - 0s - loss: 0.9947 - acc: 0.6033 - val_loss: 1.8557 - val_acc: 0.4590
Epoch 21/500
242/242 [==============================] - 0s - loss: 0.9562 - acc: 0.6446 - val_loss: 1.9570 - val_acc: 0.4262
Epoch 22/500
242/242 [==============================] - 0s - loss: 0.9855 - acc: 0.5826 - val_loss: 2.0134 - val_acc: 0.4918
Epoch 23/500
242/242 [==============================] - 0s - loss: 0.9191 - acc: 0.5992 - val_loss: 2.0331 - val_acc: 0.4918
Epoch 24/500
242/242 [==============================] - 0s - loss: 0.9244 - acc: 0.6157 - val_loss: 2.1062 - val_acc: 0.4754
Epoch 25/500
242/242 [==============================] - 0s - loss: 0.9089 - acc: 0.5992 - val_loss: 2.1733 - val_acc: 0.4590
Epoch 26/500
242/242 [==============================] - 0s - loss: 0.9767 - acc: 0.6116 - val_loss: 2.2174 - val_acc: 0.4754
Epoch 27/500
242/242 [==============================] - 0s - loss: 0.8826 - acc: 0.6198 - val_loss: 2.1408 - val_acc: 0.5082
Epoch 28/500
242/242 [==============================] - 0s - loss: 0.8790 - acc: 0.6364 - val_loss: 2.1015 - val_acc: 0.5246
Epoch 29/500
242/242 [==============================] - 0s - loss: 0.9147 - acc: 0.6446 - val_loss: 2.0182 - val_acc: 0.5246
Epoch 30/500
242/242 [==============================] - 0s - loss: 0.8856 - acc: 0.6281 - val_loss: 2.0788 - val_acc: 0.5574
Epoch 31/500
242/242 [==============================] - 0s - loss: 0.8812 - acc: 0.6074 - val_loss: 2.1176 - val_acc: 0.5410
Epoch 32/500
242/242 [==============================] - 0s - loss: 0.9472 - acc: 0.6033 - val_loss: 2.1301 - val_acc: 0.5246
Epoch 33/500
242/242 [==============================] - 0s - loss: 0.8449 - acc: 0.6529 - val_loss: 2.3474 - val_acc: 0.4426
Epoch 34/500
242/242 [==============================] - 0s - loss: 0.8740 - acc: 0.6736 - val_loss: 2.2575 - val_acc: 0.4754
Epoch 35/500
242/242 [==============================] - 0s - loss: 0.7674 - acc: 0.6901 - val_loss: 2.0163 - val_acc: 0.5246
Epoch 36/500
242/242 [==============================] - 0s - loss: 0.8574 - acc: 0.6777 - val_loss: 2.4293 - val_acc: 0.4918
Epoch 37/500
242/242 [==============================] - 0s - loss: 1.0311 - acc: 0.6157 - val_loss: 2.1469 - val_acc: 0.4754
Epoch 38/500
242/242 [==============================] - 0s - loss: 0.9567 - acc: 0.6612 - val_loss: 2.0580 - val_acc: 0.4918
Epoch 39/500
242/242 [==============================] - 0s - loss: 1.0135 - acc: 0.5868 - val_loss: 1.9396 - val_acc: 0.5246
Epoch 40/500
242/242 [==============================] - 0s - loss: 0.8794 - acc: 0.6198 - val_loss: 2.1498 - val_acc: 0.5246
Epoch 41/500
242/242 [==============================] - 0s - loss: 0.9597 - acc: 0.6405 - val_loss: 2.2993 - val_acc: 0.5082
Epoch 42/500
242/242 [==============================] - 0s - loss: 0.8719 - acc: 0.6240 - val_loss: 2.2203 - val_acc: 0.5082
Epoch 43/500
242/242 [==============================] - 0s - loss: 0.9071 - acc: 0.6364 - val_loss: 2.0764 - val_acc: 0.5410
Epoch 44/500
242/242 [==============================] - 0s - loss: 0.9769 - acc: 0.6281 - val_loss: 2.0285 - val_acc: 0.5082
Epoch 45/500
242/242 [==============================] - 0s - loss: 0.9988 - acc: 0.6116 - val_loss: 2.0342 - val_acc: 0.5574
Epoch 46/500
242/242 [==============================] - 0s - loss: 0.8629 - acc: 0.6488 - val_loss: 2.1600 - val_acc: 0.5410
Epoch 47/500
242/242 [==============================] - 0s - loss: 0.9847 - acc: 0.6198 - val_loss: 1.9753 - val_acc: 0.4918
Epoch 48/500
242/242 [==============================] - 0s - loss: 0.8783 - acc: 0.6488 - val_loss: 2.0191 - val_acc: 0.3770
Epoch 49/500
242/242 [==============================] - 0s - loss: 0.9287 - acc: 0.6281 - val_loss: 1.8864 - val_acc: 0.4098
Epoch 50/500
242/242 [==============================] - 0s - loss: 0.8966 - acc: 0.6818 - val_loss: 1.7654 - val_acc: 0.4754
Epoch 51/500
242/242 [==============================] - 0s - loss: 0.8657 - acc: 0.6612 - val_loss: 1.8648 - val_acc: 0.4918
Epoch 52/500
242/242 [==============================] - 0s - loss: 0.8837 - acc: 0.6157 - val_loss: 1.9028 - val_acc: 0.4754
Epoch 53/500
242/242 [==============================] - 0s - loss: 0.8629 - acc: 0.6736 - val_loss: 1.8718 - val_acc: 0.4754
Epoch 54/500
242/242 [==============================] - 0s - loss: 0.7821 - acc: 0.6694 - val_loss: 1.9813 - val_acc: 0.4590
Epoch 55/500
242/242 [==============================] - 0s - loss: 0.8502 - acc: 0.6364 - val_loss: 2.0582 - val_acc: 0.4754
Epoch 56/500
242/242 [==============================] - 0s - loss: 0.8183 - acc: 0.7149 - val_loss: 2.2045 - val_acc: 0.5082
Epoch 57/500
242/242 [==============================] - 0s - loss: 0.8377 - acc: 0.6612 - val_loss: 2.3473 - val_acc: 0.5738
Epoch 58/500
242/242 [==============================] - 0s - loss: 0.8917 - acc: 0.7231 - val_loss: 2.2515 - val_acc: 0.5082
Epoch 59/500
242/242 [==============================] - 0s - loss: 0.7495 - acc: 0.7149 - val_loss: 2.1030 - val_acc: 0.4918
Epoch 60/500
242/242 [==============================] - 0s - loss: 0.8097 - acc: 0.7066 - val_loss: 2.0530 - val_acc: 0.4918
Epoch 61/500
242/242 [==============================] - 0s - loss: 0.8091 - acc: 0.6901 - val_loss: 2.0718 - val_acc: 0.4754
Epoch 62/500
242/242 [==============================] - 0s - loss: 0.7005 - acc: 0.7273 - val_loss: 2.0658 - val_acc: 0.4754
Epoch 63/500
242/242 [==============================] - 0s - loss: 0.8185 - acc: 0.6901 - val_loss: 1.9286 - val_acc: 0.4918
Epoch 64/500
242/242 [==============================] - 0s - loss: 0.7603 - acc: 0.7025 - val_loss: 1.8138 - val_acc: 0.5246
Epoch 65/500
242/242 [==============================] - 0s - loss: 0.7594 - acc: 0.7025 - val_loss: 1.7751 - val_acc: 0.5246
Epoch 66/500
242/242 [==============================] - 0s - loss: 0.6630 - acc: 0.7603 - val_loss: 1.8482 - val_acc: 0.5246
Epoch 67/500
242/242 [==============================] - 0s - loss: 0.6585 - acc: 0.7603 - val_loss: 1.9853 - val_acc: 0.5902
Epoch 68/500
242/242 [==============================] - 0s - loss: 0.6643 - acc: 0.7397 - val_loss: 1.9533 - val_acc: 0.5902
Epoch 69/500
242/242 [==============================] - 0s - loss: 0.6805 - acc: 0.7355 - val_loss: 1.8779 - val_acc: 0.5902
Epoch 70/500
242/242 [==============================] - 0s - loss: 0.6655 - acc: 0.7438 - val_loss: 1.8835 - val_acc: 0.5738
Epoch 71/500
242/242 [==============================] - 0s - loss: 0.6672 - acc: 0.7603 - val_loss: 1.9669 - val_acc: 0.5902
Epoch 72/500
242/242 [==============================] - 0s - loss: 0.6879 - acc: 0.7107 - val_loss: 2.0078 - val_acc: 0.5410
Epoch 73/500
242/242 [==============================] - 0s - loss: 0.7238 - acc: 0.7107 - val_loss: 1.7524 - val_acc: 0.5410
Epoch 74/500
242/242 [==============================] - 0s - loss: 0.6376 - acc: 0.7603 - val_loss: 1.6599 - val_acc: 0.5246
Epoch 75/500
242/242 [==============================] - 0s - loss: 0.6122 - acc: 0.7934 - val_loss: 1.7117 - val_acc: 0.5410
Epoch 76/500
242/242 [==============================] - 0s - loss: 0.6209 - acc: 0.7769 - val_loss: 1.8659 - val_acc: 0.5082
Epoch 77/500
242/242 [==============================] - 0s - loss: 0.6147 - acc: 0.7810 - val_loss: 1.9167 - val_acc: 0.5246
Epoch 78/500
242/242 [==============================] - 0s - loss: 0.6062 - acc: 0.7934 - val_loss: 1.9883 - val_acc: 0.5738
Epoch 79/500
242/242 [==============================] - 0s - loss: 0.5574 - acc: 0.7769 - val_loss: 2.0635 - val_acc: 0.5574
Epoch 80/500
242/242 [==============================] - 0s - loss: 0.6465 - acc: 0.7645 - val_loss: 2.0038 - val_acc: 0.5082
Epoch 81/500
242/242 [==============================] - 0s - loss: 0.5869 - acc: 0.7810 - val_loss: 2.0189 - val_acc: 0.4754
Epoch 82/500
242/242 [==============================] - 0s - loss: 0.5172 - acc: 0.8058 - val_loss: 2.0489 - val_acc: 0.4918
Epoch 83/500
242/242 [==============================] - 0s - loss: 0.5638 - acc: 0.7893 - val_loss: 2.1778 - val_acc: 0.5246
Epoch 84/500
242/242 [==============================] - 0s - loss: 0.5871 - acc: 0.7686 - val_loss: 2.2709 - val_acc: 0.5410
Epoch 85/500
242/242 [==============================] - 0s - loss: 0.5515 - acc: 0.7562 - val_loss: 2.1105 - val_acc: 0.5246
Epoch 86/500
242/242 [==============================] - 0s - loss: 0.4946 - acc: 0.8099 - val_loss: 2.0532 - val_acc: 0.5410
Epoch 87/500
242/242 [==============================] - 0s - loss: 0.5339 - acc: 0.8223 - val_loss: 2.1360 - val_acc: 0.4754
Epoch 88/500
242/242 [==============================] - 0s - loss: 0.5377 - acc: 0.7934 - val_loss: 2.2908 - val_acc: 0.5246
Epoch 89/500
242/242 [==============================] - 0s - loss: 0.5741 - acc: 0.7769 - val_loss: 2.2413 - val_acc: 0.5246
Epoch 90/500
242/242 [==============================] - 0s - loss: 0.5421 - acc: 0.7893 - val_loss: 2.1797 - val_acc: 0.5902
Epoch 91/500
242/242 [==============================] - 0s - loss: 0.6201 - acc: 0.7686 - val_loss: 2.0873 - val_acc: 0.5902
Epoch 92/500
242/242 [==============================] - 0s - loss: 0.4695 - acc: 0.8182 - val_loss: 1.8516 - val_acc: 0.5902
Epoch 93/500
242/242 [==============================] - 0s - loss: 0.5138 - acc: 0.8264 - val_loss: 1.8117 - val_acc: 0.6066
Epoch 94/500
242/242 [==============================] - 0s - loss: 0.5531 - acc: 0.8140 - val_loss: 1.8809 - val_acc: 0.6066
Epoch 95/500
242/242 [==============================] - 0s - loss: 0.5110 - acc: 0.8058 - val_loss: 2.0241 - val_acc: 0.5902
Epoch 96/500
242/242 [==============================] - 0s - loss: 0.5286 - acc: 0.8140 - val_loss: 2.0678 - val_acc: 0.5902
Epoch 97/500
242/242 [==============================] - 0s - loss: 0.4523 - acc: 0.8306 - val_loss: 2.1462 - val_acc: 0.5738
Epoch 98/500
242/242 [==============================] - 0s - loss: 0.5038 - acc: 0.8264 - val_loss: 2.0905 - val_acc: 0.5738
Epoch 99/500
242/242 [==============================] - 0s - loss: 0.6250 - acc: 0.8017 - val_loss: 1.7533 - val_acc: 0.5410
Epoch 100/500
242/242 [==============================] - 0s - loss: 0.4054 - acc: 0.8471 - val_loss: 1.6308 - val_acc: 0.5574
Epoch 101/500
242/242 [==============================] - 0s - loss: 0.4746 - acc: 0.8264 - val_loss: 1.6487 - val_acc: 0.6066
Epoch 102/500
242/242 [==============================] - 0s - loss: 0.5577 - acc: 0.8099 - val_loss: 1.8301 - val_acc: 0.5410
Epoch 103/500
242/242 [==============================] - 0s - loss: 0.3918 - acc: 0.8926 - val_loss: 2.0447 - val_acc: 0.5738
Epoch 104/500
242/242 [==============================] - 0s - loss: 0.5260 - acc: 0.8430 - val_loss: 2.1282 - val_acc: 0.5410
Epoch 105/500
242/242 [==============================] - 0s - loss: 0.5186 - acc: 0.8223 - val_loss: 2.2109 - val_acc: 0.5574
Epoch 106/500
242/242 [==============================] - 0s - loss: 0.4690 - acc: 0.8430 - val_loss: 2.0630 - val_acc: 0.6066
Epoch 107/500
242/242 [==============================] - 0s - loss: 0.4325 - acc: 0.8223 - val_loss: 1.9274 - val_acc: 0.6066
Epoch 108/500
242/242 [==============================] - 0s - loss: 0.3774 - acc: 0.8636 - val_loss: 1.8551 - val_acc: 0.5738
Epoch 109/500
242/242 [==============================] - 0s - loss: 0.4533 - acc: 0.8347 - val_loss: 1.9684 - val_acc: 0.5902
Epoch 00108: early stopping
CPU times: user 56.7 s, sys: 10.4 s, total: 1min 7s
Wall time: 1min 39s
Out[52]:
<keras.callbacks.History at 0x7f5367e83208>

Idealer Verlauf der Metriken beim vollen Training

100% bei Training und über 95% bei Validation sind möglich, sind bei der Datenmenge aber mit Vorsicht zu genießen

Accuracy

Validation Accuracy

Sichern des Modells (falls in diesem Notebook trainiert wurde)

  • unser Modell ist 55 MB groß, das ist ein wirklich großes Modell

In [34]:
model.save('conv-vgg.hdf5')

In [77]:
!ls -lh


total 0
-rw-r--r-- 1 nbuser nbuser   63 Aug 27 07:32 README.html
drwxr-xr-x 2 nbuser nbuser    0 Sep 22 14:08 __MACOSX
-rw-r--r-- 1 nbuser nbuser 140K Aug 30 11:44 cnn-augmentation.ipynb
-rw-r--r-- 1 nbuser nbuser 1.6M Aug 30 17:27 cnn-comparing-all-models.ipynb
-rw-r--r-- 1 nbuser nbuser 319K Sep 22 15:06 cnn-intro.ipynb
-rw-r--r-- 1 nbuser nbuser 1.3M Sep  7 13:56 cnn-prediction.ipynb
-rw-r--r-- 1 nbuser nbuser 198K Aug 30 17:19 cnn-train-augmented.ipynb
-rw-r--r-- 1 nbuser nbuser  55M Sep 22 15:09 conv-vgg.hdf5
-rw-r--r-- 1 nbuser nbuser 113K Sep 22 14:29 nn-intro.ipynb
drwxr-xr-x 2 nbuser nbuser    0 Sep 22 14:08 speed-limit-signs
-rw-r--r-- 1 nbuser nbuser 1.8M Sep 22 14:08 speed-limit-signs.zip
-rw-r--r-- 1 nbuser nbuser 5.5K Sep 22 14:28 workshop.ipynb

In [78]:
# https://transfer.sh/
# Speichert eure Daten für 14 Tage
!curl --upload-file conv-vgg.hdf5 https://transfer.sh/conv-vgg.hdf5

# Vortrainiertes Modell
# loss: 0.0310 - acc: 0.9917 - val_loss: 0.4075 - val_acc: 0.9508
# https://transfer.sh/B1W8e/conv-vgg.hdf5


https://transfer.sh/M5SOs/conv-vgg.hdf5

ODER Laden des trainierten Modells


In [59]:
!ls -lh


total 2.0M
-rw-rw-r-- 1 ubuntu ubuntu 131K Aug 29 08:29 cnn-intro.ipynb
drwxrwxr-x 3 ubuntu ubuntu 4.0K Aug 29 07:45 __MACOSX
-rw------- 1 ubuntu ubuntu 4.8K Aug 29 08:29 nohup.out
drwxrwxr-x 8 ubuntu ubuntu 4.0K Aug 29 07:45 speed-limit-signs
-rw-rw-r-- 1 ubuntu ubuntu 1.8M Aug 29 07:45 speed-limit-signs.zip
drwxr-xr-x 2 ubuntu ubuntu 4.0K Aug 29 08:17 tf_log

In [58]:
!rm conv-vgg.hdf5

In [1]:
# anpassen an aktuelles Modell
# Nachricht an Olli: Liegt auch local auf Ollis Rechner und kann zur Not von da hochgeladen werden (ai/models/conv-vgg.hdf5)
!curl -O https://transfer.sh/B1W8e/conv-vgg.hdf5


  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 54.8M  100 54.8M    0     0  4515k      0  0:00:12  0:00:12 --:--:-- 3854k

In [61]:
!ls -lh


total 57M
-rw-rw-r-- 1 ubuntu ubuntu 131K Aug 29 08:31 cnn-intro.ipynb
-rw-rw-r-- 1 ubuntu ubuntu  55M Aug 29 08:30 conv-vgg.hdf5
drwxrwxr-x 3 ubuntu ubuntu 4.0K Aug 29 07:45 __MACOSX
-rw------- 1 ubuntu ubuntu 4.9K Aug 29 08:31 nohup.out
drwxrwxr-x 8 ubuntu ubuntu 4.0K Aug 29 07:45 speed-limit-signs
-rw-rw-r-- 1 ubuntu ubuntu 1.8M Aug 29 07:45 speed-limit-signs.zip
drwxr-xr-x 2 ubuntu ubuntu 4.0K Aug 29 08:17 tf_log

In [79]:
from keras.models import load_model
model = load_model('conv-vgg.hdf5')

Bewertung


In [82]:
train_loss, train_accuracy = model.evaluate(X_train, y_train, batch_size=BATCH_SIZE)
train_loss, train_accuracy


37/37 [==============================] - 0s
Out[82]:
(0.43861934542655945, 0.97297298908233643)

In [84]:
test_loss, test_accuracy = model.evaluate(X_test, y_test, batch_size=BATCH_SIZE)
test_loss, test_accuracy


342/342 [==============================] - 7s
Out[84]:
(0.28564885258674622, 0.95029240846633911)

Ausprobieren auf ein paar Test-Daten


In [86]:
import random

# Pick 10 random images for test data set
random.seed(4) # to make this deterministic
sample_indexes = random.sample(range(len(X_test)), 10)
sample_images = [X_test[i] for i in sample_indexes]
sample_labels = [y_test[i] for i in sample_indexes]

In [88]:
ground_truth = np.argmax(sample_labels, axis=1)
ground_truth


Out[88]:
array([1, 1, 2, 0, 2, 1, 3, 3, 4, 4])

In [89]:
X_sample = np.array(sample_images)
prediction = model.predict(X_sample)
predicted_categories = np.argmax(prediction, axis=1)
predicted_categories


Out[89]:
array([0, 1, 2, 0, 2, 1, 3, 3, 4, 4])

In [90]:
# Display the predictions and the ground truth visually.
def display_prediction (images, true_labels, predicted_labels):
    fig = plt.figure(figsize=(10, 10))
    for i in range(len(true_labels)):
        truth = true_labels[i]
        prediction = predicted_labels[i]
        plt.subplot(5, 2,1+i)
        plt.axis('off')
        color='green' if truth == prediction else 'red'
        plt.text(80, 10, "Truth:        {0}\nPrediction: {1}".format(truth, prediction), 
                 fontsize=12, color=color)
        plt.imshow(images[i])

In [91]:
display_prediction(sample_images, ground_truth, predicted_categories)