Notes



In [1]:

    
%pylab inline









    



Populating the interactive namespace from numpy and matplotlib



In [2]:

    
from ggplot import *
import pandas as pd



In [11]:

    
data = pd.DataFrame(
    dict(epoch=range(1,11)+range(1,11)+range(1,11)+range(1,8)+range(1,11)+range(1,11),
         model=hstack([repeat("char-3-grow", 10), 
                       repeat("char-1", 10),
                       repeat("char-3", 10),
                       repeat("visual", 7), 
                       repeat("multitask",10), 
                       repeat("sum", 10)]),
         recall=[#char-3-grow lw0222.uvt.nl:/home/gchrupala/reimaginet/run-110-phon
                 0.097281087565,
                 0.140863654538,
                 0.161015593762,
                 0.173410635746,
                 0.176969212315,
                 0.175529788085,
                 0.175089964014,
                 0.174010395842,
                 0.173370651739,
                 0.173050779688,
                 # char-1 yellow.uvt.nl:/home/gchrupala/repos/reimagine/run-200-phon
                 0.100919632147,
                 0.127588964414,
                 0.140583766493,
                 0.148300679728,
                 0.150739704118,
                 0.153338664534,
                 0.156657337065,
                 0.159016393443,
                 0.159056377449,
                 0.160655737705,
                 # char-3 yellow.uvt.nl:/home/gchrupala/repos/reimagine/run-201-phon
                 0.078368652539,
                 0.125789684126,
                 0.148140743703,
                 0.158216713315,
                 0.163694522191,
                 0.168612554978,
                 0.172570971611,
                 0.17181127549,
                 0.171531387445,
                 0.170611755298,

                 # visual 
                 0.160015993603,
                 0.184406237505,
                 0.193202718912,
                 0.19956017593,
                 0.201079568173,
                 0.201719312275,
                 0.19944022391,
                 # multitask
                 0.16093562575, 
                 0.185525789684,
                 0.194482207117,
                 0.202758896441,
                 0.203558576569,
                 0.20243902439,
                 0.199240303878,
                 0.195361855258,
                 0.193242702919,
                 0.189924030388,
                 # sum
                 0.137984806078,
                 0.145581767293,
                 0.149340263894,
                 0.151819272291,
                 0.152898840464,
                 0.154218312675,
                 0.155257896841,
                 0.155697720912,
                 0.15637744902,
                 0.156657337065
            ]))



In [6]:

    
def standardize(x):
    return (x-numpy.mean(x))/numpy.std(x)

Image retrieval evaluation

Models:

Sum - additively composed word embeddings (1024 dimensions)
Visual - Imaginet with disabled textual pathway (1024 embeddings + 1 x 1024 hidden
Multitask - Full Imaginet model (1024 embeddings + 1 x 1024 hidden)
Char-1 - Model similar to imaginet, but trained on character-level. Captions are lowecases, with spaces removed. The model has 256 character embeddings + 3 layers of 1024 recurrent hidden layers.
Char-3 - Like above, but 3 GRU layers
Char-3-grow. Like above, but layers >1 initialized to pre-trained approximate identity

Remarks:

Models NOT trained on extra train data (restval)



In [21]:

    
ggplot(data.loc[data['model'].isin(['sum','char-1','char-3','char-3-grow','multitask'])], 
       aes(x='epoch', y='recall', color='model')) + geom_line(size=3)  + theme()









    












    Out[21]:





<ggplot: (8759760442429)>



In [19]:

    
ggplot(data.loc[data['model'].isin(['visual','multitask','sum'])], 
       aes(x='epoch', y='recall', color='model')) + geom_line(size=3)  + theme()









    












    Out[19]:





<ggplot: (8759814358949)>



In [8]:

    
data_grow = pd.DataFrame(dict(epoch=range(1,11)+range(1,11),
                         model=hstack([repeat("gru-2-grow", 10),repeat("gru-1", 10)]),
                         recall=[#gru-1
                                 0.170971611355,
                                 0.192163134746,
                                 0.206797281088,
                                 0.211355457817,
                                 0.21331467413,
                                 0.218992403039,
                                 0.214674130348,
                                 0.214634146341,
                                 0.214434226309,
                                 0.212115153938,
                                 # gru-2-grow
                                 0.173730507797,
                                 0.198320671731,
                                 0.206117552979,
                                 0.211715313874,
                                 0.212914834066,
                                 0.211915233906,
                                 0.209956017593,
                                 0.210795681727,
                                 0.209076369452,
                                 0.208996401439
                                 ]))

Models:

GRU-1 - Imaginet (1024 emb + 1 x 1024 hidden)
GRU-2 grow - Imaginet (1024 emb + 2 x 1024 hidden)

Remarks:

Models trained on extra train data (restval)
Layers >1 initialized to pre-trained approximate identity



In [10]:

    
ggplot(data_grow, aes(x='epoch', y='recall', color='model')) + geom_line(size=3)  + theme()









    












    Out[10]:





<ggplot: (8759760487093)>



In [ ]:

Notes

Development and evaluation of imaginet and related models.

Image retrieval evaluation