Lesson 2 Assignment JNB -- Kaggle Galaxy Zoo

10 May 2017 - Wayne H Nixalo

Plan:

Build a Linear Model (no activations) from scratch
Build a 1-Layer Neural Network using linear model layers + activations
Build a finetuned DLNN atop VGG16

experiment w/ SGD vs RMSprop
experiment w/ sigmoid vs tanh vs ReLU
compare scores of ea. model
use utils.py & vgg16.py source code + Keras.io documentation for help

Note: I'm pretty sure that by "from scratch" what J. Howard means is a fresh linear model atop Vgg16.. Creating a straight Linear Model for image classification... does not sound... very sound..

Whatever, build break learn.



In [2]:

    
import keras









    



Using Theano backend.



In [ ]:

    
# Constants
Num_Classes = Num_Classes
batch_size = 4
lr = 0.01



In [ ]:

    
# Helper Functions

# get_batches(..) copied from utils.py
# gen.flow_from_directory() is an iterator that yields batches of images
# from a directory indefinitely.
from keras.preprocessing import image
def get_batches(dirname, gen=image.ImageDataGenerator(), shuffle=True, batche_size=4, class_mode='categorical',
                target_size=(224,224)):
    return gen.flow_from_directory(dirname, target_size=target_size,
            class_mode=class_mode, shuffle=shuffle, batch_size=batch_size)

# fast array saving/loading
import bcolz
def save_array(fname, arr): c=bcolz.carray(arr, rootdir=fname, mode='w'); c.flush()
def load_array(fname): return bcolz.open(fname)[:]

# One-Hot Encoding for Keras
from sklearn.preprocessing import OneHotEncoder
def onehot(x): return np.array(OneHotEncoder().fit_transform(x.reshape(-1, 1))).todense()
# should I use that or from Keras?
# def onehot(x): return keras.utils.np_utils.to_categorical(x)

# from utils.py -- retrieving data saved by bcolz
def get_data(path, target_size=(224,224)):
    batches = get_batches(path, shuffle=False, batch_size=1, class_mode=None, target_size=target_size)
    return np.concatenate([batches.next() for i in range(batches.nb_sample)])

1. Basic Linear Model



In [ ]:

    
LM = keras.model.Sequential([Dense(Num_Classes, input_shape=(784,))])

LM.compile(optimizer=SGD(lr=0.01), loss='mse')
# LM.compile(optimizer=RMSprop(lr=0.01), loss='mse')



In [ ]:

2. 1-Layer Neural Network

3. Finetuned VGG16



In [ ]:

    
import os, sys



In [ ]:

    
sys.path.insert(os.path.join(1, '../utils/'))
import Vgg16