notebook.community

Edit and run

Net surgery to produce a Graph model for regressing poselets





In [17]:



    


%matplotlib inline

from keras.optimizers import SGD

import numpy as np

import models
from vggnet.upgrade_weights import upgrade_weights
from vggnet.vgg16_keras import VGG_16



    











In [12]:



    


models = reload(models)
solver = SGD()
rgb_shape = (6, 224, 224)
flow_shape = (2, 224, 224)
regressor_outputs = 6
init = 'glorot_normal'



    











In [3]:



    


huge = models.vggnet16_poselet_class_flow_norm_bn({
    'images': (6, 224, 224),
    'flow': (2, 224, 224),
    'poselet': (901,)
}, solver, init)



    











In [4]:



    


flow_seq = huge.nodes['flow_conv']
rgb_seq = huge.nodes['rgb_conv']
ilsvrc_weights_path = './vggnet/vgg16_weights.h5'
ilsvrc_model = VGG_16(ilsvrc_weights_path)



    











In [5]:



    


ol1 = upgrade_weights(flow_seq.layers, ilsvrc_model.layers)
ol2 = upgrade_weights(rgb_seq.layers, ilsvrc_model.layers)
assert ol1 == ol2
old_left = ol1 # Number of layers remaining in the old model



    


















    






Concatenating old L1 dimensions to change shape, (64, 3, 3, 3)->(64, 1, 3, 3)
First layer done
WARNING: layer count differs (24 new v. 37 old)
Skipping BatchNormalization layer in new model
Changing ZeroPadding2D
Changing Convolution2D
Skipping BatchNormalization layer in new model
Changing MaxPooling2D
Changing ZeroPadding2D
Changing Convolution2D
Skipping BatchNormalization layer in new model
Changing ZeroPadding2D
Changing Convolution2D
Skipping BatchNormalization layer in new model
Changing MaxPooling2D
Changing ZeroPadding2D
Changing Convolution2D
Skipping BatchNormalization layer in new model
Changing ZeroPadding2D
Changing Convolution2D
Skipping BatchNormalization layer in new model
Changing ZeroPadding2D
Changing Convolution2D
Skipping BatchNormalization layer in new model
Changing MaxPooling2D
0 new layers ignored and 20 old layers ignored
Concatenating old L1 dimensions to change shape, (64, 3, 3, 3)->(64, 6, 3, 3)
First layer done
WARNING: layer count differs (24 new v. 37 old)
Skipping BatchNormalization layer in new model
Changing ZeroPadding2D
Changing Convolution2D
Skipping BatchNormalization layer in new model
Changing MaxPooling2D
Changing ZeroPadding2D
Changing Convolution2D
Skipping BatchNormalization layer in new model
Changing ZeroPadding2D
Changing Convolution2D
Skipping BatchNormalization layer in new model
Changing MaxPooling2D
Changing ZeroPadding2D
Changing Convolution2D
Skipping BatchNormalization layer in new model
Changing ZeroPadding2D
Changing Convolution2D
Skipping BatchNormalization layer in new model
Changing ZeroPadding2D
Changing Convolution2D
Skipping BatchNormalization layer in new model
Changing MaxPooling2D
0 new layers ignored and 20 old layers ignored

















In [30]:



    


front_layers = len(flow_seq.layers)
assert front_layers == len(rgb_seq.layers), "Flow and RGB pipelines should be same length"
back_ilsvrc_layers = ilsvrc_model.layers[-old_left:]
back_seq = huge.nodes['shared_layers']
upgrade_weights(back_seq.layers, back_ilsvrc_layers)



    


















    






Concatenating old L1 dimensions to change shape, (512, 256, 3, 3)->(512, 512, 3, 3)
First layer done
WARNING: layer count differs (25 new v. 20 old)
Skipping BatchNormalization layer in new model
Changing ZeroPadding2D
Changing Convolution2D
Skipping BatchNormalization layer in new model
Changing ZeroPadding2D
Changing Convolution2D
Skipping BatchNormalization layer in new model
Changing MaxPooling2D
Changing ZeroPadding2D
Changing Convolution2D
Skipping BatchNormalization layer in new model
Changing ZeroPadding2D
Changing Convolution2D
Skipping BatchNormalization layer in new model
Changing ZeroPadding2D
Changing Convolution2D
Skipping BatchNormalization layer in new model
Changing MaxPooling2D
Changing Flatten
Changing Dense
Changing Dropout
Changing Dense
Changing Dropout
0 new layers ignored and 1 old layers ignored

















In [31]:



    


huge.save_weights('vggnet/vgg16-2stream-pslt-clas-norm-bn.h5')



    











In [19]:



    


from keras.utils.visualize_util import to_graph

from IPython.display import SVG



    











In [32]:



    


SVG(to_graph(huge, show_shape=True).create_svg())



    


















    
Out[32]:








G


layer0

poselet (keras.layers.core.Dense)

input:

output:

(None, 4096)

(None, 901)


layer1

(keras.layers.core.Dropout)

input:

output:

(None, 4096)

(None, 4096)


layer1->layer0




layer2

(keras.layers.core.Dense)

input:

output:

(None, 4096)

(None, 4096)


layer2->layer1




layer3

(keras.layers.core.Dropout)

input:

output:

(None, 4096)

(None, 4096)


layer3->layer2




layer4

(keras.layers.core.Dense)

input:

output:

(None, 25088)

(None, 4096)


layer4->layer3




layer5

(keras.layers.core.Flatten)

input:

output:

(None, 512, 7, 7)

(None, 25088)


layer5->layer4




layer6

(keras.layers.convolutional.MaxPooling2D)

input:

output:

(None, 512, 14, 14)

(None, 512, 7, 7)


layer6->layer5




layer7

(keras.layers.normalization.BatchNormalization)

input:

output:

(None, 512, 14, 14)

(None, 512, 14, 14)


layer7->layer6




layer8

(keras.layers.convolutional.Convolution2D)

input:

output:

(None, 512, 16, 16)

(None, 512, 14, 14)


layer8->layer7




layer9

(keras.layers.convolutional.ZeroPadding2D)

input:

output:

(None, 512, 14, 14)

(None, 512, 16, 16)


layer9->layer8




layer10

(keras.layers.normalization.BatchNormalization)

input:

output:

(None, 512, 14, 14)

(None, 512, 14, 14)


layer10->layer9




layer11

(keras.layers.convolutional.Convolution2D)

input:

output:

(None, 512, 16, 16)

(None, 512, 14, 14)


layer11->layer10




layer12

(keras.layers.convolutional.ZeroPadding2D)

input:

output:

(None, 512, 14, 14)

(None, 512, 16, 16)


layer12->layer11




layer13

(keras.layers.normalization.BatchNormalization)

input:

output:

(None, 512, 14, 14)

(None, 512, 14, 14)


layer13->layer12




layer14

(keras.layers.convolutional.Convolution2D)

input:

output:

(None, 512, 16, 16)

(None, 512, 14, 14)


layer14->layer13




layer15

(keras.layers.convolutional.ZeroPadding2D)

input:

output:

(None, 512, 14, 14)

(None, 512, 16, 16)


layer15->layer14




layer16

(keras.layers.convolutional.MaxPooling2D)

input:

output:

(None, 512, 28, 28)

(None, 512, 14, 14)


layer16->layer15




layer17

(keras.layers.normalization.BatchNormalization)

input:

output:

(None, 512, 28, 28)

(None, 512, 28, 28)


layer17->layer16




layer18

(keras.layers.convolutional.Convolution2D)

input:

output:

(None, 512, 30, 30)

(None, 512, 28, 28)


layer18->layer17




layer19

(keras.layers.convolutional.ZeroPadding2D)

input:

output:

(None, 512, 28, 28)

(None, 512, 30, 30)


layer19->layer18




layer20

(keras.layers.normalization.BatchNormalization)

input:

output:

(None, 512, 28, 28)

(None, 512, 28, 28)


layer20->layer19




layer21

(keras.layers.convolutional.Convolution2D)

input:

output:

(None, 512, 30, 30)

(None, 512, 28, 28)


layer21->layer20




layer22

(keras.layers.convolutional.ZeroPadding2D)

input:

output:

(None, 512, 28, 28)

(None, 512, 30, 30)


layer22->layer21




layer23

(keras.layers.normalization.BatchNormalization)

input:

output:

(None, 512, 28, 28)

(None, 512, 28, 28)


layer23->layer22




layer24

(keras.layers.convolutional.Convolution2D)

input:

output:

(None, 512, 30, 30)

(None, 512, 28, 28)


layer24->layer23




layer25

(keras.layers.convolutional.ZeroPadding2D)

input:

output:

(None, 512, 28, 28)

(None, 512, 30, 30)


layer25->layer24




layer26

(keras.layers.core.Merge)

input:

output:

 

(None, 512, 28, 28)


layer26->layer25




layer27

(keras.layers.convolutional.MaxPooling2D)

input:

output:

(None, 256, 56, 56)

(None, 256, 28, 28)


layer27->layer26




layer28

(keras.layers.normalization.BatchNormalization)

input:

output:

(None, 256, 56, 56)

(None, 256, 56, 56)


layer28->layer27




layer29

(keras.layers.convolutional.Convolution2D)

input:

output:

(None, 256, 58, 58)

(None, 256, 56, 56)


layer29->layer28




layer30

(keras.layers.convolutional.ZeroPadding2D)

input:

output:

(None, 256, 56, 56)

(None, 256, 58, 58)


layer30->layer29




layer31

(keras.layers.normalization.BatchNormalization)

input:

output:

(None, 256, 56, 56)

(None, 256, 56, 56)


layer31->layer30




layer32

(keras.layers.convolutional.Convolution2D)

input:

output:

(None, 256, 58, 58)

(None, 256, 56, 56)


layer32->layer31




layer33

(keras.layers.convolutional.ZeroPadding2D)

input:

output:

(None, 256, 56, 56)

(None, 256, 58, 58)


layer33->layer32




layer34

(keras.layers.normalization.BatchNormalization)

input:

output:

(None, 256, 56, 56)

(None, 256, 56, 56)


layer34->layer33




layer35

(keras.layers.convolutional.Convolution2D)

input:

output:

(None, 128, 58, 58)

(None, 256, 56, 56)


layer35->layer34




layer36

(keras.layers.convolutional.ZeroPadding2D)

input:

output:

(None, 128, 56, 56)

(None, 128, 58, 58)


layer36->layer35




layer37

(keras.layers.convolutional.MaxPooling2D)

input:

output:

(None, 128, 112, 112)

(None, 128, 56, 56)


layer37->layer36




layer38

(keras.layers.normalization.BatchNormalization)

input:

output:

(None, 128, 112, 112)

(None, 128, 112, 112)


layer38->layer37




layer39

(keras.layers.convolutional.Convolution2D)

input:

output:

(None, 128, 114, 114)

(None, 128, 112, 112)


layer39->layer38




layer40

(keras.layers.convolutional.ZeroPadding2D)

input:

output:

(None, 128, 112, 112)

(None, 128, 114, 114)


layer40->layer39




layer41

(keras.layers.normalization.BatchNormalization)

input:

output:

(None, 128, 112, 112)

(None, 128, 112, 112)


layer41->layer40




layer42

(keras.layers.convolutional.Convolution2D)

input:

output:

(None, 64, 114, 114)

(None, 128, 112, 112)


layer42->layer41




layer43

(keras.layers.convolutional.ZeroPadding2D)

input:

output:

(None, 64, 112, 112)

(None, 64, 114, 114)


layer43->layer42




layer44

(keras.layers.convolutional.MaxPooling2D)

input:

output:

(None, 64, 224, 224)

(None, 64, 112, 112)


layer44->layer43




layer45

(keras.layers.normalization.BatchNormalization)

input:

output:

(None, 64, 224, 224)

(None, 64, 224, 224)


layer45->layer44




layer46

(keras.layers.convolutional.Convolution2D)

input:

output:

(None, 64, 226, 226)

(None, 64, 224, 224)


layer46->layer45




layer47

(keras.layers.convolutional.ZeroPadding2D)

input:

output:

(None, 64, 224, 224)

(None, 64, 226, 226)


layer47->layer46




layer48

(keras.layers.normalization.BatchNormalization)

input:

output:

(None, 64, 224, 224)

(None, 64, 224, 224)


layer48->layer47




layer49

(keras.layers.convolutional.Convolution2D)

input:

output:

(None, 6, 226, 226)

(None, 64, 224, 224)


layer49->layer48




layer50

(keras.layers.convolutional.ZeroPadding2D)

input:

output:

(None, 6, 224, 224)

(None, 6, 226, 226)


layer50->layer49




layer51

(keras.layers.core.Layer)

input:

output:

(None, 6, 224, 224)

(None, 6, 224, 224)


layer51->layer50




layer52

(keras.layers.convolutional.MaxPooling2D)

input:

output:

(None, 256, 56, 56)

(None, 256, 28, 28)


layer52->layer26




layer53

(keras.layers.normalization.BatchNormalization)

input:

output:

(None, 256, 56, 56)

(None, 256, 56, 56)


layer53->layer52




layer54

(keras.layers.convolutional.Convolution2D)

input:

output:

(None, 256, 58, 58)

(None, 256, 56, 56)


layer54->layer53




layer55

(keras.layers.convolutional.ZeroPadding2D)

input:

output:

(None, 256, 56, 56)

(None, 256, 58, 58)


layer55->layer54




layer56

(keras.layers.normalization.BatchNormalization)

input:

output:

(None, 256, 56, 56)

(None, 256, 56, 56)


layer56->layer55




layer57

(keras.layers.convolutional.Convolution2D)

input:

output:

(None, 256, 58, 58)

(None, 256, 56, 56)


layer57->layer56




layer58

(keras.layers.convolutional.ZeroPadding2D)

input:

output:

(None, 256, 56, 56)

(None, 256, 58, 58)


layer58->layer57




layer59

(keras.layers.normalization.BatchNormalization)

input:

output:

(None, 256, 56, 56)

(None, 256, 56, 56)


layer59->layer58




layer60

(keras.layers.convolutional.Convolution2D)

input:

output:

(None, 128, 58, 58)

(None, 256, 56, 56)


layer60->layer59




layer61

(keras.layers.convolutional.ZeroPadding2D)

input:

output:

(None, 128, 56, 56)

(None, 128, 58, 58)


layer61->layer60




layer62

(keras.layers.convolutional.MaxPooling2D)

input:

output:

(None, 128, 112, 112)

(None, 128, 56, 56)


layer62->layer61




layer63

(keras.layers.normalization.BatchNormalization)

input:

output:

(None, 128, 112, 112)

(None, 128, 112, 112)


layer63->layer62




layer64

(keras.layers.convolutional.Convolution2D)

input:

output:

(None, 128, 114, 114)

(None, 128, 112, 112)


layer64->layer63




layer65

(keras.layers.convolutional.ZeroPadding2D)

input:

output:

(None, 128, 112, 112)

(None, 128, 114, 114)


layer65->layer64




layer66

(keras.layers.normalization.BatchNormalization)

input:

output:

(None, 128, 112, 112)

(None, 128, 112, 112)


layer66->layer65




layer67

(keras.layers.convolutional.Convolution2D)

input:

output:

(None, 64, 114, 114)

(None, 128, 112, 112)


layer67->layer66




layer68

(keras.layers.convolutional.ZeroPadding2D)

input:

output:

(None, 64, 112, 112)

(None, 64, 114, 114)


layer68->layer67




layer69

(keras.layers.convolutional.MaxPooling2D)

input:

output:

(None, 64, 224, 224)

(None, 64, 112, 112)


layer69->layer68




layer70

(keras.layers.normalization.BatchNormalization)

input:

output:

(None, 64, 224, 224)

(None, 64, 224, 224)


layer70->layer69




layer71

(keras.layers.convolutional.Convolution2D)

input:

output:

(None, 64, 226, 226)

(None, 64, 224, 224)


layer71->layer70




layer72

(keras.layers.convolutional.ZeroPadding2D)

input:

output:

(None, 64, 224, 224)

(None, 64, 226, 226)


layer72->layer71




layer73

(keras.layers.normalization.BatchNormalization)

input:

output:

(None, 64, 224, 224)

(None, 64, 224, 224)


layer73->layer72




layer74

(keras.layers.convolutional.Convolution2D)

input:

output:

(None, 1, 226, 226)

(None, 64, 224, 224)


layer74->layer73




layer75

(keras.layers.convolutional.ZeroPadding2D)

input:

output:

(None, 1, 224, 224)

(None, 1, 226, 226)


layer75->layer74




layer76

(keras.layers.core.Lambda)

input:

output:

(None, 2, 224, 224)

(None, 1, 224, 224)


layer76->layer75




layer77

(keras.layers.core.Layer)

input:

output:

(None, 2, 224, 224)

(None, 2, 224, 224)


layer77->layer76





















In [25]:



    


huge.nodes.layers.keys()



    


















    






---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-25-1c9b0b758727> in <module>()
----> 1 huge.nodes.layers.keys()

AttributeError: 'OrderedDict' object has no attribute 'layers'









    






> <ipython-input-25-1c9b0b758727>(1)<module>()
----> 1 huge.nodes.layers.keys()

ipdb> huge.nodes.keys()
['rgb_conv', 'flow_norm', 'flow_conv', 'shared_layers', 'fc_pslt']
ipdb> huge.nodes['shared_layers']
<keras.models.Sequential object at 0x7f8b2b03be50>
ipdb> q

















In [ ]:

Content source: qxcv/joint-regressor

Similar notebooks:

notebook.community | gallery | about