From ~/Downloads/github/pylearn2/pylearn2/scripts/tutorials/stacked_autoencoders
In [10]:
target = 'Dog_1'
In [11]:
target2batch_size = {'Dog_1':128}
by Mehdi Mirza
This notebook will show you how to perform layer-wise pre-training using denoising autoencoders (DAEs), and subsequently stack the layers to form a multilayer perceptron (MLP) which can be fine-tuned using supervised training. You can also look at this more detailed tutorial of training DAEs using Theano as well as this tutorial which covers the stacked version.
The methods used here can easily be adapted to other models such as contractive auto-encoders (CAEs) or restricted Boltzmann machines (RBMs) with only small modifications.
In [13]:
layer1_yaml = open('my_dae_l1.yaml', 'r').read()
hyper_params_l1 = {
'batch_size' : target2batch_size.get(target, 128),
'monitoring_batches' : 5,
'nvis' : 3996,
'target' : target,
'nhid' : 1000,
'max_epochs' : 10,
'save_path' : '../data-cache'}
layer1_yaml = layer1_yaml % (hyper_params_l1)
print layer1_yaml
Now we can train the model using the YAML string in the same way as the previous tutorials:
In [14]:
from pylearn2.config import yaml_parse
train = yaml_parse.load(layer1_yaml)
train.main_loop()
The second layer takes the output of the first layer as its input. Hence we must first apply the first layer's transformations to the raw data using datasets.transformer_dataset.TransformerDataset. This class takes two arguments:
raw: the raw datatransformer: a Pylearn2 block that transforms the raw data, which in our case is the dae_l1.pkl file from the previous stepTo train the second layer, we load the YAML file as before and set the hyperparameters before starting the training loop.
In [15]:
layer2_yaml = open('my_dae_l2.yaml', 'r').read()
hyper_params_l2 = {
'target' : target,
'batch_size' : target2batch_size.get(target, 128),
'monitoring_batches' : 5,
'nvis' : hyper_params_l1['nhid'],
'nhid' : 500,
'max_epochs' : 10,
'save_path' : '../data-cache'}
layer2_yaml = layer2_yaml % (hyper_params_l2)
print layer2_yaml
In [16]:
train = yaml_parse.load(layer2_yaml)
train.main_loop()