DATA622 Final Project

Daniel Dittenhafer

This notebook proceeds with using the trained convolutional neural network to predict emotion for each of the face images in the provided test set as part of the Emotion Detection Form Facial Expressions Kaggle competition.

Import References



In [38]:

    
import csv
import random
import numpy as np
import emotion_model
import dwdii_transforms

random.seed(20275)

Definitions

The model was trained on 150x150 images, therefore we will use this size of the test data as well.



In [62]:

    
imgDimension = []
imgDimension.append(150)
imgDimension.append(150)

Load Kaggle Test Data

In this section, the test data is loaded from the previously cloned facial_expressions GitHub repo.



In [63]:

    
testDataPath = "../../facial_expressions/test"



In [64]:

    
import os
ls  = os.listdir(testDataPath)
len(ls)









    Out[64]:





263



In [65]:

    
testData, fileList = dwdii_transforms.load_test_data(testDataPath, imgResize=imgDimension, verboseFreq=20)









    



0.000000: y38.jpg
0.076046: y06.jpg
0.152091: BhanuPriya_60.jpg
0.228137: NM.HA2.96.jpg
0.304183: BomanIrani_9.jpg
0.380228: 98a.jpg
0.456274: KL.SA2.162.jpg
0.532319: KL.DI2.171.jpg
0.608365: 140a.jpg
0.684411: Dileep_60.jpg
0.760456: YM.AN2.62.jpg
0.836502: KM.SU2.15.jpg
0.912548: y28.jpg
0.988593: y09.jpg



In [66]:

    
testData.shape









    Out[66]:





(263, 150, 150)



In [67]:

    
testX = testData.reshape(testData.shape[0], 1, testData.shape[1],testData.shape[2])

Define and Load Trained Model



In [80]:

    
# Load the emotion array for our count in the model definition
emotions = dwdii_transforms.emotionNumerics()
print emotions
print len(emotions)
# Construct the model using our help function
model = emotion_model.emotion_model_jh_v5(len(emotions), verbose=True, 
                                        input_shape=(1,imgDimension[0],imgDimension[1]))









    



{'sadness': 6, 'neutral': 2, 'contempt': 7, 'disgust': 1, 'anger': 0, 'surprise': 4, 'fear': 5, 'happiness': 3}
8
____________________________________________________________________________________________________
Layer (type)                       Output Shape        Param #     Connected to                     
====================================================================================================
convolution2d_17 (Convolution2D)   (None, 32, 143, 143)2080        convolution2d_input_5[0][0]      
____________________________________________________________________________________________________
activation_25 (Activation)         (None, 32, 143, 143)0           convolution2d_17[0][0]           
____________________________________________________________________________________________________
maxpooling2d_17 (MaxPooling2D)     (None, 32, 71, 71)  0           activation_25[0][0]              
____________________________________________________________________________________________________
convolution2d_18 (Convolution2D)   (None, 32, 67, 67)  25632       maxpooling2d_17[0][0]            
____________________________________________________________________________________________________
activation_26 (Activation)         (None, 32, 67, 67)  0           convolution2d_18[0][0]           
____________________________________________________________________________________________________
maxpooling2d_18 (MaxPooling2D)     (None, 32, 33, 33)  0           activation_26[0][0]              
____________________________________________________________________________________________________
convolution2d_19 (Convolution2D)   (None, 64, 31, 31)  18496       maxpooling2d_18[0][0]            
____________________________________________________________________________________________________
activation_27 (Activation)         (None, 64, 31, 31)  0           convolution2d_19[0][0]           
____________________________________________________________________________________________________
maxpooling2d_19 (MaxPooling2D)     (None, 64, 15, 15)  0           activation_27[0][0]              
____________________________________________________________________________________________________
convolution2d_20 (Convolution2D)   (None, 64, 14, 14)  16448       maxpooling2d_19[0][0]            
____________________________________________________________________________________________________
activation_28 (Activation)         (None, 64, 14, 14)  0           convolution2d_20[0][0]           
____________________________________________________________________________________________________
maxpooling2d_20 (MaxPooling2D)     (None, 64, 7, 7)    0           activation_28[0][0]              
____________________________________________________________________________________________________
flatten_5 (Flatten)                (None, 3136)        0           maxpooling2d_20[0][0]            
____________________________________________________________________________________________________
dense_9 (Dense)                    (None, 64)          200768      flatten_5[0][0]                  
____________________________________________________________________________________________________
activation_29 (Activation)         (None, 64)          0           dense_9[0][0]                    
____________________________________________________________________________________________________
dense_10 (Dense)                   (None, 8)           520         activation_29[0][0]              
____________________________________________________________________________________________________
activation_30 (Activation)         (None, 8)           0           dense_10[0][0]                   
====================================================================================================
Total params: 263944
____________________________________________________________________________________________________
None



In [81]:

    
loadWeights = True
if loadWeights:
    #model.load_weights("dwdii-emo-150-jhv5a-Cloud.hdf5")
    #model.load_weights("dwdii-emo-150-jhv5-9tf-Cloud.hdf5")
    model.load_weights("dwdii-emo-150-jhv5-19tf-25e-Cloud.hdf5")

Produce Predictions



In [82]:

    
predictOutput = model.predict(testX, batch_size=32, verbose=1)









    



263/263 [==============================] - 7s



In [83]:

    
# How many predictions?
len(predictOutput)









    Out[83]:





263



In [84]:

    
# Show an example of the prediction output
print predictOutput[0]









    



[  7.42519587e-18   1.14439990e-25   9.99990165e-01   9.70902147e-06
   7.69798614e-10   2.67830899e-22   1.11349941e-07   7.61658962e-22]



In [85]:

    
np.array(predictOutput[0]).argmax()









    Out[85]:





2



In [86]:

    
numEmo = dwdii_transforms.numericEmotions()
numEmo[2]









    Out[86]:





'neutral'



In [87]:

    
fileList[0]









    Out[87]:





'y38.jpg'

Persist Predictions

In this section we save the prediction results to a file in the prescribed Kaggle competition format.



In [88]:

    
# For each prediction output, build the row for the CSV and add to our list for saving.
outputData = []
outputData.append(["Image", "Emotion"])
for i in range(len(predictOutput)):
    arPred = np.array(predictOutput[i])
    predictionProb = arPred.max()
    predictionNdx = arPred.argmax()
    predictedEmo = numEmo[predictionNdx]
    fileName = fileList[i]
    outputData.append([fileName,predictedEmo])



In [89]:

    
print outputData[0:10]









    



[['Image', 'Emotion'], ['y38.jpg', 'neutral'], ['KA.AN2.40.jpg', 'neutral'], ['KajalAgarwal_22.jpg', 'sadness'], ['FaridaJalal_149.jpg', 'neutral'], ['NA.HA2.203.jpg', 'neutral'], ['KM.FE2.24.jpg', 'happiness'], ['JayaBhaduri_42.jpg', 'neutral'], ['FaridaJalal_315.jpg', 'happiness'], ['186a.jpg', 'sadness']]



In [90]:

    
with open('../output/dwdii_predictions-19tf-25e.csv', 'w') as mycsvfile:
    dw = csv.writer(mycsvfile)
    for row in outputData:
        dw.writerow(row)

Appendix: Sanity Check Code

I wanted to make sure the resizing didn't cause significant distortion for images which werent' square already. The following code helped with this sanity check.



In [ ]:

    
from scipy import misc
path = testDataPath + "/JayaBhaduri_42.jpg"
img = misc.imread(path)
img = misc.imresize(img, (150, 150))

misc.imsave("test1000.jpg", img)



In [ ]: