

In [1]:

    
from lasagne.layers import *
from lasagne.nonlinearities import *
from lasagne import init



In [2]:

    
nn = InputLayer([None,3,100,100])

nn = Conv2DLayer(nn,num_filters=512, filter_size=(3,3),
                 W = init.Constant(0))

nn = Conv2DLayer(nn,num_filters=128,filter_size=(3,3),
                 W = init.Constant(0))

nn = Conv2DLayer(nn,num_filters=32,filter_size=(3,3),
                 W = init.Constant(0))

nn = Pool2DLayer(nn,pool_size=(6,6),mode='max')

nn = Conv2DLayer(nn,num_filters=8,filter_size=(10,10),
                W = init.Normal(std=0.01))

nn = Conv2DLayer(nn,num_filters=8,filter_size=(10,10),
                W = init.Normal(std=0.01))

nn = Pool2DLayer(nn,pool_size=(3,3),mode='max')

nn = DenseLayer(nn,512,nonlinearity=softmax)

nn = DropoutLayer(nn,p=0.5)

nn = DenseLayer(nn,512,nonlinearity=softmax)

nn = DenseLayer(nn,10,nonlinearity=sigmoid)

nn = DropoutLayer(nn,p=0.5)

Book of grudges

zero init for weights will cause symmetry effect
Too many filters for first 3x3 convolution - will lead to enormous matrix while there's just not enough relevant combinations of 3x3 images (overkill).
Usually the further you go, the more filters you need.
large filters (10x10 is generally a bad pactice, and you definitely need more than 10 of them
the second of 10x10 convolution gets 8x6x6 image as input, so it's technically unable to perform such convolution.
Softmax nonlinearity effectively makes only 1 or a few neurons from the entire layer to "fire", rendering 512-neuron layer almost useless. Softmax at the output layer is okay though
Dropout after probability prediciton is just lame. A few random classes get probability of 0, so your probabilities no longer sum to 1 and crossentropy goes -inf.



In [ ]: