We'll start off by building a simple autoencoder to compress the MNIST dataset. With autoencoders, we pass input data through an encoder that makes a compressed representation of the input. Then, this representation is passed through a decoder to reconstruct the input data. Generally the encoder and decoder will be built with neural networks, then trained on example data.
In this notebook, we'll be build a simple network architecture for the encoder and decoder. Let's get started by importing our libraries and getting the dataset.
In [1]:
%matplotlib inline
import numpy as np
import tensorflow as tf
import matplotlib.pyplot as plt
In [2]:
from tensorflow.examples.tutorials.mnist import input_data
mnist = input_data.read_data_sets('MNIST_data', validation_size=0)
Successfully downloaded train-images-idx3-ubyte.gz 9912422 bytes.
Extracting MNIST_data\train-images-idx3-ubyte.gz
Successfully downloaded train-labels-idx1-ubyte.gz 28881 bytes.
Extracting MNIST_data\train-labels-idx1-ubyte.gz
Successfully downloaded t10k-images-idx3-ubyte.gz 1648877 bytes.
Extracting MNIST_data\t10k-images-idx3-ubyte.gz
Successfully downloaded t10k-labels-idx1-ubyte.gz 4542 bytes.
Extracting MNIST_data\t10k-labels-idx1-ubyte.gz
Below I'm plotting an example image from the MNIST dataset. These are 28x28 grayscale images of handwritten digits.
In [19]:
img = mnist.train.images[2]
plt.imshow(img.reshape((28, 28)), cmap='Greys_r')
Out[19]:
<matplotlib.image.AxesImage at 0x28fbeff2668>
We'll train an autoencoder with these images by flattening them into 784 length vectors. The images from this dataset are already normalized such that the values are between 0 and 1. Let's start by building basically the simplest autoencoder with a single ReLU hidden layer. This layer will be used as the compressed representation. Then, the encoder is the input layer and the hidden layer. The decoder is the hidden layer and the output layer. Since the images are normalized between 0 and 1, we need to use a sigmoid activation on the output layer to get values matching the input.
Exercise: Build the graph for the autoencoder in the cell below. The input images will be flattened into 784 length vectors. The targets are the same as the inputs. And there should be one hidden layer with a ReLU activation and an output layer with a sigmoid activation. The loss should be calculated with the cross-entropy loss, there is a convenient TensorFlow function for this
tf.nn.sigmoid_cross_entropy_with_logits(documentation). You should note thattf.nn.sigmoid_cross_entropy_with_logitstakes the logits, but to get the reconstructed images you'll need to pass the logits through the sigmoid function.
In [36]:
# Size of the encoding layer (the hidden layer)
encoding_dim = 16 # feel free to change this value
image_dim = 28 * 28
inputs_ = tf.placeholder(tf.float32, [None, image_dim])
targets_ = tf.placeholder(tf.float32, [None, image_dim])
# Output of hidden layer
encoded = tf.contrib.layers.fully_connected(inputs=inputs_, num_outputs=encoding_dim)
#encoded = tf.contrib.layers.dropout(inputs=encoded, keep_prob=0.5)
# Output layer logits
logits = tf.contrib.layers.fully_connected(inputs=encoded, num_outputs=image_dim, activation_fn=None)
# Sigmoid output from logits
decoded = tf.sigmoid(logits)
# Sigmoid cross-entropy loss
loss = tf.nn.sigmoid_cross_entropy_with_logits(labels=targets_, logits=logits)
# Mean of the loss
cost = tf.reduce_mean(loss)
# Adam optimizer
adam = tf.train.AdamOptimizer()
opt = adam.minimize(cost)
In [37]:
# Create the session
sess = tf.Session()
Here I'll write a bit of code to train the network. I'm not too interested in validation here, so I'll just monitor the training loss.
Calling mnist.train.next_batch(batch_size) will return a tuple of (images, labels). We're not concerned with the labels here, we just need the images. Otherwise this is pretty straightfoward training with TensorFlow. We initialize the variables with sess.run(tf.global_variables_initializer()). Then, run the optimizer and get the loss with batch_cost, _ = sess.run([cost, opt], feed_dict=feed).
In [ ]:
epochs = 20
batch_size = 1000
sess.run(tf.global_variables_initializer())
for e in range(epochs):
for ii in range(mnist.train.num_examples//batch_size):
batch = mnist.train.next_batch(batch_size)
feed = {inputs_: batch[0], targets_: batch[0]}
batch_cost, _ = sess.run([cost, opt], feed_dict=feed)
print("Epoch: {}/{}...".format(e+1, epochs),
"Training loss: {:.4f}".format(batch_cost))
Epoch: 1/20... Training loss: 0.6945
Epoch: 1/20... Training loss: 0.6922
Epoch: 1/20... Training loss: 0.6901
Epoch: 1/20... Training loss: 0.6880
Epoch: 1/20... Training loss: 0.6856
Epoch: 1/20... Training loss: 0.6828
Epoch: 1/20... Training loss: 0.6800
Epoch: 1/20... Training loss: 0.6761
Epoch: 1/20... Training loss: 0.6719
Epoch: 1/20... Training loss: 0.6671
Epoch: 1/20... Training loss: 0.6616
Epoch: 1/20... Training loss: 0.6555
Epoch: 1/20... Training loss: 0.6479
Epoch: 1/20... Training loss: 0.6403
Epoch: 1/20... Training loss: 0.6327
Epoch: 1/20... Training loss: 0.6225
Epoch: 1/20... Training loss: 0.6121
Epoch: 1/20... Training loss: 0.6007
Epoch: 1/20... Training loss: 0.5898
Epoch: 1/20... Training loss: 0.5776
Epoch: 1/20... Training loss: 0.5643
Epoch: 1/20... Training loss: 0.5510
Epoch: 1/20... Training loss: 0.5394
Epoch: 1/20... Training loss: 0.5236
Epoch: 1/20... Training loss: 0.5110
Epoch: 1/20... Training loss: 0.4989
Epoch: 1/20... Training loss: 0.4838
Epoch: 1/20... Training loss: 0.4715
Epoch: 1/20... Training loss: 0.4561
Epoch: 1/20... Training loss: 0.4482
Epoch: 1/20... Training loss: 0.4317
Epoch: 1/20... Training loss: 0.4221
Epoch: 1/20... Training loss: 0.4100
Epoch: 1/20... Training loss: 0.3999
Epoch: 1/20... Training loss: 0.3888
Epoch: 1/20... Training loss: 0.3763
Epoch: 1/20... Training loss: 0.3705
Epoch: 1/20... Training loss: 0.3601
Epoch: 1/20... Training loss: 0.3575
Epoch: 1/20... Training loss: 0.3509
Epoch: 1/20... Training loss: 0.3445
Epoch: 1/20... Training loss: 0.3396
Epoch: 1/20... Training loss: 0.3366
Epoch: 1/20... Training loss: 0.3303
Epoch: 1/20... Training loss: 0.3260
Epoch: 1/20... Training loss: 0.3207
Epoch: 1/20... Training loss: 0.3178
Epoch: 1/20... Training loss: 0.3126
Epoch: 1/20... Training loss: 0.3095
Epoch: 1/20... Training loss: 0.3063
Epoch: 1/20... Training loss: 0.3044
Epoch: 1/20... Training loss: 0.3031
Epoch: 1/20... Training loss: 0.3010
Epoch: 1/20... Training loss: 0.2963
Epoch: 1/20... Training loss: 0.2969
Epoch: 1/20... Training loss: 0.3009
Epoch: 1/20... Training loss: 0.2965
Epoch: 1/20... Training loss: 0.2953
Epoch: 1/20... Training loss: 0.2928
Epoch: 1/20... Training loss: 0.2936
Epoch: 2/20... Training loss: 0.2956
Epoch: 2/20... Training loss: 0.2909
Epoch: 2/20... Training loss: 0.2913
Epoch: 2/20... Training loss: 0.2899
Epoch: 2/20... Training loss: 0.2896
Epoch: 2/20... Training loss: 0.2872
Epoch: 2/20... Training loss: 0.2871
Epoch: 2/20... Training loss: 0.2875
Epoch: 2/20... Training loss: 0.2866
Epoch: 2/20... Training loss: 0.2810
Epoch: 2/20... Training loss: 0.2848
Epoch: 2/20... Training loss: 0.2816
Epoch: 2/20... Training loss: 0.2812
Epoch: 2/20... Training loss: 0.2816
Epoch: 2/20... Training loss: 0.2779
Epoch: 2/20... Training loss: 0.2797
Epoch: 2/20... Training loss: 0.2826
Epoch: 2/20... Training loss: 0.2804
Epoch: 2/20... Training loss: 0.2813
Epoch: 2/20... Training loss: 0.2820
Epoch: 2/20... Training loss: 0.2774
Epoch: 2/20... Training loss: 0.2772
Epoch: 2/20... Training loss: 0.2783
Epoch: 2/20... Training loss: 0.2762
Epoch: 2/20... Training loss: 0.2776
Epoch: 2/20... Training loss: 0.2765
Epoch: 2/20... Training loss: 0.2760
Epoch: 2/20... Training loss: 0.2750
Epoch: 2/20... Training loss: 0.2741
Epoch: 2/20... Training loss: 0.2738
Epoch: 2/20... Training loss: 0.2735
Epoch: 2/20... Training loss: 0.2736
Epoch: 2/20... Training loss: 0.2744
Epoch: 2/20... Training loss: 0.2740
Epoch: 2/20... Training loss: 0.2715
Epoch: 2/20... Training loss: 0.2740
Epoch: 2/20... Training loss: 0.2720
Epoch: 2/20... Training loss: 0.2699
Epoch: 2/20... Training loss: 0.2674
Epoch: 2/20... Training loss: 0.2699
Epoch: 2/20... Training loss: 0.2709
Epoch: 2/20... Training loss: 0.2686
Epoch: 2/20... Training loss: 0.2648
Epoch: 2/20... Training loss: 0.2689
Epoch: 2/20... Training loss: 0.2679
Epoch: 2/20... Training loss: 0.2656
Epoch: 2/20... Training loss: 0.2662
Epoch: 2/20... Training loss: 0.2647
Epoch: 2/20... Training loss: 0.2636
Epoch: 2/20... Training loss: 0.2623
Epoch: 2/20... Training loss: 0.2630
Epoch: 2/20... Training loss: 0.2610
Epoch: 2/20... Training loss: 0.2618
Epoch: 2/20... Training loss: 0.2608
Epoch: 2/20... Training loss: 0.2595
Epoch: 2/20... Training loss: 0.2574
Epoch: 2/20... Training loss: 0.2590
Epoch: 2/20... Training loss: 0.2597
Epoch: 2/20... Training loss: 0.2612
Epoch: 2/20... Training loss: 0.2603
Epoch: 3/20... Training loss: 0.2563
Epoch: 3/20... Training loss: 0.2560
Epoch: 3/20... Training loss: 0.2568
Epoch: 3/20... Training loss: 0.2572
Epoch: 3/20... Training loss: 0.2578
Epoch: 3/20... Training loss: 0.2546
Epoch: 3/20... Training loss: 0.2545
Epoch: 3/20... Training loss: 0.2551
Epoch: 3/20... Training loss: 0.2531
Epoch: 3/20... Training loss: 0.2522
Epoch: 3/20... Training loss: 0.2554
Epoch: 3/20... Training loss: 0.2545
Epoch: 3/20... Training loss: 0.2508
Epoch: 3/20... Training loss: 0.2508
Epoch: 3/20... Training loss: 0.2499
Epoch: 3/20... Training loss: 0.2482
Epoch: 3/20... Training loss: 0.2482
Epoch: 3/20... Training loss: 0.2496
Epoch: 3/20... Training loss: 0.2455
Epoch: 3/20... Training loss: 0.2454
Epoch: 3/20... Training loss: 0.2491
Epoch: 3/20... Training loss: 0.2464
Epoch: 3/20... Training loss: 0.2475
Epoch: 3/20... Training loss: 0.2433
Epoch: 3/20... Training loss: 0.2481
Epoch: 3/20... Training loss: 0.2435
Epoch: 3/20... Training loss: 0.2443
Epoch: 3/20... Training loss: 0.2443
Epoch: 3/20... Training loss: 0.2442
Epoch: 3/20... Training loss: 0.2426
Epoch: 3/20... Training loss: 0.2438
Epoch: 3/20... Training loss: 0.2448
Epoch: 3/20... Training loss: 0.2424
Epoch: 3/20... Training loss: 0.2411
Epoch: 3/20... Training loss: 0.2417
Epoch: 3/20... Training loss: 0.2421
Epoch: 3/20... Training loss: 0.2401
Epoch: 3/20... Training loss: 0.2418
Epoch: 3/20... Training loss: 0.2376
Epoch: 3/20... Training loss: 0.2375
Epoch: 3/20... Training loss: 0.2377
Epoch: 3/20... Training loss: 0.2382
Epoch: 3/20... Training loss: 0.2375
Epoch: 3/20... Training loss: 0.2364
Epoch: 3/20... Training loss: 0.2362
Epoch: 3/20... Training loss: 0.2357
Epoch: 3/20... Training loss: 0.2356
Epoch: 3/20... Training loss: 0.2345
Epoch: 3/20... Training loss: 0.2334
Epoch: 3/20... Training loss: 0.2344
Epoch: 3/20... Training loss: 0.2325
Epoch: 3/20... Training loss: 0.2325
Epoch: 3/20... Training loss: 0.2300
Epoch: 3/20... Training loss: 0.2318
Epoch: 3/20... Training loss: 0.2305
Epoch: 3/20... Training loss: 0.2308
Epoch: 3/20... Training loss: 0.2313
Epoch: 3/20... Training loss: 0.2300
Epoch: 3/20... Training loss: 0.2287
Epoch: 3/20... Training loss: 0.2312
Epoch: 4/20... Training loss: 0.2270
Epoch: 4/20... Training loss: 0.2288
Epoch: 4/20... Training loss: 0.2285
Epoch: 4/20... Training loss: 0.2269
Epoch: 4/20... Training loss: 0.2281
Epoch: 4/20... Training loss: 0.2255
Epoch: 4/20... Training loss: 0.2252
Epoch: 4/20... Training loss: 0.2245
Epoch: 4/20... Training loss: 0.2265
Epoch: 4/20... Training loss: 0.2293
Epoch: 4/20... Training loss: 0.2240
Epoch: 4/20... Training loss: 0.2242
Epoch: 4/20... Training loss: 0.2242
Epoch: 4/20... Training loss: 0.2248
Epoch: 4/20... Training loss: 0.2262
Epoch: 4/20... Training loss: 0.2221
Epoch: 4/20... Training loss: 0.2218
Epoch: 4/20... Training loss: 0.2233
Epoch: 4/20... Training loss: 0.2209
Epoch: 4/20... Training loss: 0.2215
Epoch: 4/20... Training loss: 0.2211
Epoch: 4/20... Training loss: 0.2189
Epoch: 4/20... Training loss: 0.2198
Epoch: 4/20... Training loss: 0.2192
Epoch: 4/20... Training loss: 0.2181
Epoch: 4/20... Training loss: 0.2187
Epoch: 4/20... Training loss: 0.2188
Epoch: 4/20... Training loss: 0.2177
Epoch: 4/20... Training loss: 0.2195
Epoch: 4/20... Training loss: 0.2185
Epoch: 4/20... Training loss: 0.2164
Epoch: 4/20... Training loss: 0.2194
Epoch: 4/20... Training loss: 0.2152
Epoch: 4/20... Training loss: 0.2180
Epoch: 4/20... Training loss: 0.2147
Epoch: 4/20... Training loss: 0.2150
Epoch: 4/20... Training loss: 0.2145
Epoch: 4/20... Training loss: 0.2127
Epoch: 4/20... Training loss: 0.2143
Epoch: 4/20... Training loss: 0.2159
Epoch: 4/20... Training loss: 0.2155
Epoch: 4/20... Training loss: 0.2142
Epoch: 4/20... Training loss: 0.2125
Epoch: 4/20... Training loss: 0.2146
Epoch: 4/20... Training loss: 0.2119
Epoch: 4/20... Training loss: 0.2143
Epoch: 4/20... Training loss: 0.2112
Epoch: 4/20... Training loss: 0.2101
Epoch: 4/20... Training loss: 0.2113
Epoch: 4/20... Training loss: 0.2081
Epoch: 4/20... Training loss: 0.2097
Epoch: 4/20... Training loss: 0.2134
Epoch: 4/20... Training loss: 0.2099
Epoch: 4/20... Training loss: 0.2134
Epoch: 4/20... Training loss: 0.2113
Epoch: 4/20... Training loss: 0.2079
Epoch: 4/20... Training loss: 0.2118
Epoch: 4/20... Training loss: 0.2064
Epoch: 4/20... Training loss: 0.2059
Epoch: 4/20... Training loss: 0.2076
Epoch: 5/20... Training loss: 0.2091
Epoch: 5/20... Training loss: 0.2068
Epoch: 5/20... Training loss: 0.2080
Epoch: 5/20... Training loss: 0.2043
Epoch: 5/20... Training loss: 0.2087
Epoch: 5/20... Training loss: 0.2077
Epoch: 5/20... Training loss: 0.2042
Epoch: 5/20... Training loss: 0.2068
Epoch: 5/20... Training loss: 0.2063
Epoch: 5/20... Training loss: 0.2052
Epoch: 5/20... Training loss: 0.2091
Epoch: 5/20... Training loss: 0.2054
Epoch: 5/20... Training loss: 0.2053
Epoch: 5/20... Training loss: 0.2047
Epoch: 5/20... Training loss: 0.2069
Epoch: 5/20... Training loss: 0.2039
Epoch: 5/20... Training loss: 0.2041
Epoch: 5/20... Training loss: 0.2024
Epoch: 5/20... Training loss: 0.2038
Epoch: 5/20... Training loss: 0.2047
Epoch: 5/20... Training loss: 0.2037
Epoch: 5/20... Training loss: 0.2041
Epoch: 5/20... Training loss: 0.2041
Epoch: 5/20... Training loss: 0.2011
Epoch: 5/20... Training loss: 0.1991
Epoch: 5/20... Training loss: 0.2042
Epoch: 5/20... Training loss: 0.2009
Epoch: 5/20... Training loss: 0.2000
Epoch: 5/20... Training loss: 0.2001
Epoch: 5/20... Training loss: 0.2040
Epoch: 5/20... Training loss: 0.2001
Epoch: 5/20... Training loss: 0.2004
Epoch: 5/20... Training loss: 0.1998
Epoch: 5/20... Training loss: 0.1972
Epoch: 5/20... Training loss: 0.2034
Epoch: 5/20... Training loss: 0.1987
Epoch: 5/20... Training loss: 0.2002
Epoch: 5/20... Training loss: 0.2022
Epoch: 5/20... Training loss: 0.1997
Epoch: 5/20... Training loss: 0.1997
Epoch: 5/20... Training loss: 0.1967
Epoch: 5/20... Training loss: 0.1972
Epoch: 5/20... Training loss: 0.1986
Epoch: 5/20... Training loss: 0.1973
Epoch: 5/20... Training loss: 0.1972
Epoch: 5/20... Training loss: 0.1957
Epoch: 5/20... Training loss: 0.1959
Epoch: 5/20... Training loss: 0.1971
Epoch: 5/20... Training loss: 0.1943
Epoch: 5/20... Training loss: 0.1979
Epoch: 5/20... Training loss: 0.1958
Epoch: 5/20... Training loss: 0.1964
Epoch: 5/20... Training loss: 0.1949
Epoch: 5/20... Training loss: 0.1968
Epoch: 5/20... Training loss: 0.1959
Epoch: 5/20... Training loss: 0.1944
Epoch: 5/20... Training loss: 0.1920
Epoch: 5/20... Training loss: 0.1934
Epoch: 5/20... Training loss: 0.1945
Epoch: 5/20... Training loss: 0.1976
Epoch: 6/20... Training loss: 0.1959
Epoch: 6/20... Training loss: 0.1948
Epoch: 6/20... Training loss: 0.1973
Epoch: 6/20... Training loss: 0.1944
Epoch: 6/20... Training loss: 0.1960
Epoch: 6/20... Training loss: 0.1931
Epoch: 6/20... Training loss: 0.1930
Epoch: 6/20... Training loss: 0.1928
Epoch: 6/20... Training loss: 0.1931
Epoch: 6/20... Training loss: 0.1955
Epoch: 6/20... Training loss: 0.1901
Epoch: 6/20... Training loss: 0.1887
Epoch: 6/20... Training loss: 0.1912
Epoch: 6/20... Training loss: 0.1966
Epoch: 6/20... Training loss: 0.1966
Epoch: 6/20... Training loss: 0.1927
Epoch: 6/20... Training loss: 0.1926
Epoch: 6/20... Training loss: 0.1917
Epoch: 6/20... Training loss: 0.1904
Epoch: 6/20... Training loss: 0.1937
Epoch: 6/20... Training loss: 0.1887
Epoch: 6/20... Training loss: 0.1916
Epoch: 6/20... Training loss: 0.1913
Epoch: 6/20... Training loss: 0.1885
Epoch: 6/20... Training loss: 0.1910
Epoch: 6/20... Training loss: 0.1899
Epoch: 6/20... Training loss: 0.1873
Epoch: 6/20... Training loss: 0.1905
Epoch: 6/20... Training loss: 0.1890
Epoch: 6/20... Training loss: 0.1893
Epoch: 6/20... Training loss: 0.1893
Epoch: 6/20... Training loss: 0.1909
Epoch: 6/20... Training loss: 0.1930
Epoch: 6/20... Training loss: 0.1895
Epoch: 6/20... Training loss: 0.1872
Epoch: 6/20... Training loss: 0.1883
Epoch: 6/20... Training loss: 0.1883
Epoch: 6/20... Training loss: 0.1884
Epoch: 6/20... Training loss: 0.1906
Epoch: 6/20... Training loss: 0.1909
Epoch: 6/20... Training loss: 0.1890
Epoch: 6/20... Training loss: 0.1856
Epoch: 6/20... Training loss: 0.1862
Epoch: 6/20... Training loss: 0.1905
Epoch: 6/20... Training loss: 0.1891
Epoch: 6/20... Training loss: 0.1867
Epoch: 6/20... Training loss: 0.1870
Epoch: 6/20... Training loss: 0.1852
Epoch: 6/20... Training loss: 0.1865
Epoch: 6/20... Training loss: 0.1849
Epoch: 6/20... Training loss: 0.1863
Epoch: 6/20... Training loss: 0.1874
Epoch: 6/20... Training loss: 0.1851
Epoch: 6/20... Training loss: 0.1840
Epoch: 6/20... Training loss: 0.1829
Epoch: 6/20... Training loss: 0.1847
Epoch: 6/20... Training loss: 0.1869
Epoch: 6/20... Training loss: 0.1833
Epoch: 6/20... Training loss: 0.1855
Epoch: 6/20... Training loss: 0.1840
Epoch: 7/20... Training loss: 0.1860
Epoch: 7/20... Training loss: 0.1844
Epoch: 7/20... Training loss: 0.1850
Epoch: 7/20... Training loss: 0.1822
Epoch: 7/20... Training loss: 0.1834
Epoch: 7/20... Training loss: 0.1850
Epoch: 7/20... Training loss: 0.1832
Epoch: 7/20... Training loss: 0.1851
Epoch: 7/20... Training loss: 0.1850
Epoch: 7/20... Training loss: 0.1839
Epoch: 7/20... Training loss: 0.1803
Epoch: 7/20... Training loss: 0.1825
Epoch: 7/20... Training loss: 0.1823
Epoch: 7/20... Training loss: 0.1828
Epoch: 7/20... Training loss: 0.1841
Epoch: 7/20... Training loss: 0.1859
Epoch: 7/20... Training loss: 0.1830
Epoch: 7/20... Training loss: 0.1827
Epoch: 7/20... Training loss: 0.1864
Epoch: 7/20... Training loss: 0.1812
Epoch: 7/20... Training loss: 0.1830
Epoch: 7/20... Training loss: 0.1791
Epoch: 7/20... Training loss: 0.1819
Epoch: 7/20... Training loss: 0.1819
Epoch: 7/20... Training loss: 0.1825
Epoch: 7/20... Training loss: 0.1840
Epoch: 7/20... Training loss: 0.1820
Epoch: 7/20... Training loss: 0.1798
Epoch: 7/20... Training loss: 0.1816
Epoch: 7/20... Training loss: 0.1818
Epoch: 7/20... Training loss: 0.1846
Epoch: 7/20... Training loss: 0.1826
Epoch: 7/20... Training loss: 0.1810
Epoch: 7/20... Training loss: 0.1816
Epoch: 7/20... Training loss: 0.1823
Epoch: 7/20... Training loss: 0.1805
Epoch: 7/20... Training loss: 0.1829
Epoch: 7/20... Training loss: 0.1763
Epoch: 7/20... Training loss: 0.1773
Epoch: 7/20... Training loss: 0.1817
Epoch: 7/20... Training loss: 0.1780
Epoch: 7/20... Training loss: 0.1814
Epoch: 7/20... Training loss: 0.1814
Epoch: 7/20... Training loss: 0.1839
Epoch: 7/20... Training loss: 0.1781
Epoch: 7/20... Training loss: 0.1796
Epoch: 7/20... Training loss: 0.1796
Epoch: 7/20... Training loss: 0.1825
Epoch: 7/20... Training loss: 0.1802
Epoch: 7/20... Training loss: 0.1780
Epoch: 7/20... Training loss: 0.1792
Epoch: 7/20... Training loss: 0.1805
Epoch: 7/20... Training loss: 0.1798
Epoch: 7/20... Training loss: 0.1788
Epoch: 7/20... Training loss: 0.1793
Epoch: 7/20... Training loss: 0.1794
Epoch: 7/20... Training loss: 0.1772
Epoch: 7/20... Training loss: 0.1747
Epoch: 7/20... Training loss: 0.1794
Epoch: 7/20... Training loss: 0.1793
Epoch: 8/20... Training loss: 0.1784
Epoch: 8/20... Training loss: 0.1769
Epoch: 8/20... Training loss: 0.1798
Epoch: 8/20... Training loss: 0.1769
Epoch: 8/20... Training loss: 0.1764
Epoch: 8/20... Training loss: 0.1756
Epoch: 8/20... Training loss: 0.1776
Epoch: 8/20... Training loss: 0.1767
Epoch: 8/20... Training loss: 0.1764
Epoch: 8/20... Training loss: 0.1780
Epoch: 8/20... Training loss: 0.1764
Epoch: 8/20... Training loss: 0.1768
Epoch: 8/20... Training loss: 0.1781
Epoch: 8/20... Training loss: 0.1769
Epoch: 8/20... Training loss: 0.1776
Epoch: 8/20... Training loss: 0.1762
Epoch: 8/20... Training loss: 0.1758
Epoch: 8/20... Training loss: 0.1741
Epoch: 8/20... Training loss: 0.1779
Epoch: 8/20... Training loss: 0.1752
Epoch: 8/20... Training loss: 0.1755
Epoch: 8/20... Training loss: 0.1734
Epoch: 8/20... Training loss: 0.1795
Epoch: 8/20... Training loss: 0.1772
Epoch: 8/20... Training loss: 0.1758
Epoch: 8/20... Training loss: 0.1756
Epoch: 8/20... Training loss: 0.1769
Epoch: 8/20... Training loss: 0.1756
Epoch: 8/20... Training loss: 0.1754
Epoch: 8/20... Training loss: 0.1746
Epoch: 8/20... Training loss: 0.1744
Epoch: 8/20... Training loss: 0.1760
Epoch: 8/20... Training loss: 0.1755
Epoch: 8/20... Training loss: 0.1736
Epoch: 8/20... Training loss: 0.1754
Epoch: 8/20... Training loss: 0.1759
Epoch: 8/20... Training loss: 0.1747
Epoch: 8/20... Training loss: 0.1780
Epoch: 8/20... Training loss: 0.1748
Epoch: 8/20... Training loss: 0.1763
Epoch: 8/20... Training loss: 0.1737
Epoch: 8/20... Training loss: 0.1741
Epoch: 8/20... Training loss: 0.1758
Epoch: 8/20... Training loss: 0.1733
Epoch: 8/20... Training loss: 0.1743
Epoch: 8/20... Training loss: 0.1735
Epoch: 8/20... Training loss: 0.1749
Epoch: 8/20... Training loss: 0.1740
Epoch: 8/20... Training loss: 0.1733
Epoch: 8/20... Training loss: 0.1751
Epoch: 8/20... Training loss: 0.1729
Epoch: 8/20... Training loss: 0.1750
Epoch: 8/20... Training loss: 0.1701
Epoch: 8/20... Training loss: 0.1741
Epoch: 8/20... Training loss: 0.1708
Epoch: 8/20... Training loss: 0.1738
Epoch: 8/20... Training loss: 0.1724
Epoch: 8/20... Training loss: 0.1710
Epoch: 8/20... Training loss: 0.1742
Epoch: 8/20... Training loss: 0.1741
Epoch: 9/20... Training loss: 0.1730
Epoch: 9/20... Training loss: 0.1698
Epoch: 9/20... Training loss: 0.1718
Epoch: 9/20... Training loss: 0.1708
Epoch: 9/20... Training loss: 0.1703
Epoch: 9/20... Training loss: 0.1722
Epoch: 9/20... Training loss: 0.1727
Epoch: 9/20... Training loss: 0.1728
Epoch: 9/20... Training loss: 0.1700
Epoch: 9/20... Training loss: 0.1707
Epoch: 9/20... Training loss: 0.1716
Epoch: 9/20... Training loss: 0.1734
Epoch: 9/20... Training loss: 0.1736
Epoch: 9/20... Training loss: 0.1737
Epoch: 9/20... Training loss: 0.1696
Epoch: 9/20... Training loss: 0.1691
Epoch: 9/20... Training loss: 0.1699
Epoch: 9/20... Training loss: 0.1711
Epoch: 9/20... Training loss: 0.1692
Epoch: 9/20... Training loss: 0.1721
Epoch: 9/20... Training loss: 0.1702
Epoch: 9/20... Training loss: 0.1694
Epoch: 9/20... Training loss: 0.1696
Epoch: 9/20... Training loss: 0.1733
Epoch: 9/20... Training loss: 0.1715
Epoch: 9/20... Training loss: 0.1714
Epoch: 9/20... Training loss: 0.1736
Epoch: 9/20... Training loss: 0.1718
Epoch: 9/20... Training loss: 0.1714
Epoch: 9/20... Training loss: 0.1710
Epoch: 9/20... Training loss: 0.1694
Epoch: 9/20... Training loss: 0.1709
Epoch: 9/20... Training loss: 0.1692
Epoch: 9/20... Training loss: 0.1710
Epoch: 9/20... Training loss: 0.1696
Epoch: 9/20... Training loss: 0.1713
Epoch: 9/20... Training loss: 0.1710
Epoch: 9/20... Training loss: 0.1696
Epoch: 9/20... Training loss: 0.1689
Epoch: 9/20... Training loss: 0.1714
Epoch: 9/20... Training loss: 0.1684
Epoch: 9/20... Training loss: 0.1716
Epoch: 9/20... Training loss: 0.1711
Epoch: 9/20... Training loss: 0.1695
Epoch: 9/20... Training loss: 0.1671
Epoch: 9/20... Training loss: 0.1705
Epoch: 9/20... Training loss: 0.1703
Epoch: 9/20... Training loss: 0.1690
Epoch: 9/20... Training loss: 0.1685
Epoch: 9/20... Training loss: 0.1689
Epoch: 9/20... Training loss: 0.1663
Epoch: 9/20... Training loss: 0.1707
Epoch: 9/20... Training loss: 0.1682
Epoch: 9/20... Training loss: 0.1714
Epoch: 9/20... Training loss: 0.1680
Epoch: 9/20... Training loss: 0.1709
Epoch: 9/20... Training loss: 0.1709
Epoch: 9/20... Training loss: 0.1674
Epoch: 9/20... Training loss: 0.1642
Epoch: 9/20... Training loss: 0.1683
Epoch: 10/20... Training loss: 0.1669
Epoch: 10/20... Training loss: 0.1702
Epoch: 10/20... Training loss: 0.1666
Epoch: 10/20... Training loss: 0.1706
Epoch: 10/20... Training loss: 0.1678
Epoch: 10/20... Training loss: 0.1683
Epoch: 10/20... Training loss: 0.1691
Epoch: 10/20... Training loss: 0.1691
Epoch: 10/20... Training loss: 0.1638
Epoch: 10/20... Training loss: 0.1680
Epoch: 10/20... Training loss: 0.1672
Epoch: 10/20... Training loss: 0.1670
Epoch: 10/20... Training loss: 0.1687
Epoch: 10/20... Training loss: 0.1728
Epoch: 10/20... Training loss: 0.1651
Epoch: 10/20... Training loss: 0.1675
Epoch: 10/20... Training loss: 0.1666
Epoch: 10/20... Training loss: 0.1657
Epoch: 10/20... Training loss: 0.1671
Epoch: 10/20... Training loss: 0.1688
Epoch: 10/20... Training loss: 0.1686
Epoch: 10/20... Training loss: 0.1622
Epoch: 10/20... Training loss: 0.1666
Epoch: 10/20... Training loss: 0.1672
Epoch: 10/20... Training loss: 0.1619
Epoch: 10/20... Training loss: 0.1657
Epoch: 10/20... Training loss: 0.1658
Epoch: 10/20... Training loss: 0.1656
Epoch: 10/20... Training loss: 0.1686
Epoch: 10/20... Training loss: 0.1683
Epoch: 10/20... Training loss: 0.1643
Epoch: 10/20... Training loss: 0.1676
Epoch: 10/20... Training loss: 0.1663
Epoch: 10/20... Training loss: 0.1654
Epoch: 10/20... Training loss: 0.1668
Epoch: 10/20... Training loss: 0.1659
Epoch: 10/20... Training loss: 0.1667
Epoch: 10/20... Training loss: 0.1642
Epoch: 10/20... Training loss: 0.1634
Epoch: 10/20... Training loss: 0.1667
Epoch: 10/20... Training loss: 0.1669
Epoch: 10/20... Training loss: 0.1671
Epoch: 10/20... Training loss: 0.1663
Epoch: 10/20... Training loss: 0.1682
Epoch: 10/20... Training loss: 0.1631
Epoch: 10/20... Training loss: 0.1661
Epoch: 10/20... Training loss: 0.1637
Epoch: 10/20... Training loss: 0.1628
Epoch: 10/20... Training loss: 0.1656
Epoch: 10/20... Training loss: 0.1665
Epoch: 10/20... Training loss: 0.1636
Epoch: 10/20... Training loss: 0.1629
Epoch: 10/20... Training loss: 0.1649
Epoch: 10/20... Training loss: 0.1639
Epoch: 10/20... Training loss: 0.1633
Epoch: 10/20... Training loss: 0.1633
Epoch: 10/20... Training loss: 0.1627
Epoch: 10/20... Training loss: 0.1639
Epoch: 10/20... Training loss: 0.1660
Epoch: 10/20... Training loss: 0.1648
Epoch: 11/20... Training loss: 0.1638
Epoch: 11/20... Training loss: 0.1645
Epoch: 11/20... Training loss: 0.1663
Epoch: 11/20... Training loss: 0.1635
Epoch: 11/20... Training loss: 0.1654
Epoch: 11/20... Training loss: 0.1656
Epoch: 11/20... Training loss: 0.1637
Epoch: 11/20... Training loss: 0.1611
Epoch: 11/20... Training loss: 0.1622
Epoch: 11/20... Training loss: 0.1627
Epoch: 11/20... Training loss: 0.1623
Epoch: 11/20... Training loss: 0.1634
Epoch: 11/20... Training loss: 0.1646
Epoch: 11/20... Training loss: 0.1608
Epoch: 11/20... Training loss: 0.1610
Epoch: 11/20... Training loss: 0.1636
Epoch: 11/20... Training loss: 0.1613
Epoch: 11/20... Training loss: 0.1623
Epoch: 11/20... Training loss: 0.1630
Epoch: 11/20... Training loss: 0.1628
Epoch: 11/20... Training loss: 0.1650
Epoch: 11/20... Training loss: 0.1613
Epoch: 11/20... Training loss: 0.1639
Epoch: 11/20... Training loss: 0.1620
Epoch: 11/20... Training loss: 0.1630
Epoch: 11/20... Training loss: 0.1620
Epoch: 11/20... Training loss: 0.1605
Epoch: 11/20... Training loss: 0.1627
Epoch: 11/20... Training loss: 0.1649
Epoch: 11/20... Training loss: 0.1615
Epoch: 11/20... Training loss: 0.1610
Epoch: 11/20... Training loss: 0.1639
Epoch: 11/20... Training loss: 0.1608
Epoch: 11/20... Training loss: 0.1631
Epoch: 11/20... Training loss: 0.1629
Epoch: 11/20... Training loss: 0.1609
Epoch: 11/20... Training loss: 0.1590
Epoch: 11/20... Training loss: 0.1634
Epoch: 11/20... Training loss: 0.1647
Epoch: 11/20... Training loss: 0.1632
Epoch: 11/20... Training loss: 0.1610
Epoch: 11/20... Training loss: 0.1618
Epoch: 11/20... Training loss: 0.1620
Epoch: 11/20... Training loss: 0.1599
Epoch: 11/20... Training loss: 0.1619
Epoch: 11/20... Training loss: 0.1620
Epoch: 11/20... Training loss: 0.1630
Epoch: 11/20... Training loss: 0.1624
Epoch: 11/20... Training loss: 0.1614
Epoch: 11/20... Training loss: 0.1634
Epoch: 11/20... Training loss: 0.1590
Epoch: 11/20... Training loss: 0.1639
Epoch: 11/20... Training loss: 0.1618
Epoch: 11/20... Training loss: 0.1630
Epoch: 11/20... Training loss: 0.1634
Epoch: 11/20... Training loss: 0.1601
Epoch: 11/20... Training loss: 0.1596
Epoch: 11/20... Training loss: 0.1616
Epoch: 11/20... Training loss: 0.1598
Epoch: 11/20... Training loss: 0.1601
Epoch: 12/20... Training loss: 0.1601
Epoch: 12/20... Training loss: 0.1597
Epoch: 12/20... Training loss: 0.1593
Epoch: 12/20... Training loss: 0.1616
Epoch: 12/20... Training loss: 0.1594
Epoch: 12/20... Training loss: 0.1609
Epoch: 12/20... Training loss: 0.1597
Epoch: 12/20... Training loss: 0.1611
Epoch: 12/20... Training loss: 0.1598
Epoch: 12/20... Training loss: 0.1581
Epoch: 12/20... Training loss: 0.1586
Epoch: 12/20... Training loss: 0.1595
Epoch: 12/20... Training loss: 0.1613
Epoch: 12/20... Training loss: 0.1596
Epoch: 12/20... Training loss: 0.1575
Epoch: 12/20... Training loss: 0.1589
Epoch: 12/20... Training loss: 0.1579
Epoch: 12/20... Training loss: 0.1603
Epoch: 12/20... Training loss: 0.1608
Epoch: 12/20... Training loss: 0.1573
Epoch: 12/20... Training loss: 0.1591
Epoch: 12/20... Training loss: 0.1597
Epoch: 12/20... Training loss: 0.1588
Epoch: 12/20... Training loss: 0.1591
Epoch: 12/20... Training loss: 0.1594
Epoch: 12/20... Training loss: 0.1569
Epoch: 12/20... Training loss: 0.1603
Epoch: 12/20... Training loss: 0.1577
Epoch: 12/20... Training loss: 0.1577
Epoch: 12/20... Training loss: 0.1632
Epoch: 12/20... Training loss: 0.1595
Epoch: 12/20... Training loss: 0.1587
Epoch: 12/20... Training loss: 0.1579
Epoch: 12/20... Training loss: 0.1621
Epoch: 12/20... Training loss: 0.1574
Epoch: 12/20... Training loss: 0.1589
Epoch: 12/20... Training loss: 0.1569
Epoch: 12/20... Training loss: 0.1579
Epoch: 12/20... Training loss: 0.1615
Epoch: 12/20... Training loss: 0.1591
Epoch: 12/20... Training loss: 0.1594
Epoch: 12/20... Training loss: 0.1588
Epoch: 12/20... Training loss: 0.1594
Epoch: 12/20... Training loss: 0.1598
Epoch: 12/20... Training loss: 0.1594
Epoch: 12/20... Training loss: 0.1577
Epoch: 12/20... Training loss: 0.1560
Epoch: 12/20... Training loss: 0.1602
Epoch: 12/20... Training loss: 0.1597
Epoch: 12/20... Training loss: 0.1568
Epoch: 12/20... Training loss: 0.1586
Epoch: 12/20... Training loss: 0.1555
Epoch: 12/20... Training loss: 0.1595
Epoch: 12/20... Training loss: 0.1594
Epoch: 12/20... Training loss: 0.1579
Epoch: 12/20... Training loss: 0.1595
Epoch: 12/20... Training loss: 0.1557
Epoch: 12/20... Training loss: 0.1605
Epoch: 12/20... Training loss: 0.1551
Epoch: 12/20... Training loss: 0.1532
Epoch: 13/20... Training loss: 0.1584
Epoch: 13/20... Training loss: 0.1557
Epoch: 13/20... Training loss: 0.1570
Epoch: 13/20... Training loss: 0.1538
Epoch: 13/20... Training loss: 0.1574
Epoch: 13/20... Training loss: 0.1549
Epoch: 13/20... Training loss: 0.1583
Epoch: 13/20... Training loss: 0.1557
Epoch: 13/20... Training loss: 0.1567
Epoch: 13/20... Training loss: 0.1573
Epoch: 13/20... Training loss: 0.1562
Epoch: 13/20... Training loss: 0.1565
Epoch: 13/20... Training loss: 0.1578
Epoch: 13/20... Training loss: 0.1572
Epoch: 13/20... Training loss: 0.1542
Epoch: 13/20... Training loss: 0.1583
Epoch: 13/20... Training loss: 0.1564
Epoch: 13/20... Training loss: 0.1562
Epoch: 13/20... Training loss: 0.1577
Epoch: 13/20... Training loss: 0.1566
Epoch: 13/20... Training loss: 0.1564
Epoch: 13/20... Training loss: 0.1564
Epoch: 13/20... Training loss: 0.1572
Epoch: 13/20... Training loss: 0.1573
Epoch: 13/20... Training loss: 0.1578
Epoch: 13/20... Training loss: 0.1557
Epoch: 13/20... Training loss: 0.1553
Epoch: 13/20... Training loss: 0.1566
Epoch: 13/20... Training loss: 0.1534
Epoch: 13/20... Training loss: 0.1571
Epoch: 13/20... Training loss: 0.1572
Epoch: 13/20... Training loss: 0.1542
Epoch: 13/20... Training loss: 0.1547
Epoch: 13/20... Training loss: 0.1553
Epoch: 13/20... Training loss: 0.1569
Epoch: 13/20... Training loss: 0.1556
Epoch: 13/20... Training loss: 0.1560
Epoch: 13/20... Training loss: 0.1558
Epoch: 13/20... Training loss: 0.1574
Epoch: 13/20... Training loss: 0.1570
Epoch: 13/20... Training loss: 0.1568
Epoch: 13/20... Training loss: 0.1542
Epoch: 13/20... Training loss: 0.1566
Epoch: 13/20... Training loss: 0.1551
Epoch: 13/20... Training loss: 0.1535
Epoch: 13/20... Training loss: 0.1538
Epoch: 13/20... Training loss: 0.1545
Epoch: 13/20... Training loss: 0.1555
Epoch: 13/20... Training loss: 0.1523
Epoch: 13/20... Training loss: 0.1527
Epoch: 13/20... Training loss: 0.1536
Epoch: 13/20... Training loss: 0.1540
Epoch: 13/20... Training loss: 0.1551
Epoch: 13/20... Training loss: 0.1565
Epoch: 13/20... Training loss: 0.1548
Epoch: 13/20... Training loss: 0.1539
Epoch: 13/20... Training loss: 0.1538
Epoch: 13/20... Training loss: 0.1528
Epoch: 13/20... Training loss: 0.1542
Epoch: 13/20... Training loss: 0.1536
Epoch: 14/20... Training loss: 0.1542
Epoch: 14/20... Training loss: 0.1544
Epoch: 14/20... Training loss: 0.1529
Epoch: 14/20... Training loss: 0.1551
Epoch: 14/20... Training loss: 0.1559
Epoch: 14/20... Training loss: 0.1539
Epoch: 14/20... Training loss: 0.1546
Epoch: 14/20... Training loss: 0.1531
Epoch: 14/20... Training loss: 0.1550
Epoch: 14/20... Training loss: 0.1523
Epoch: 14/20... Training loss: 0.1545
Epoch: 14/20... Training loss: 0.1556
Epoch: 14/20... Training loss: 0.1543
Epoch: 14/20... Training loss: 0.1532
Epoch: 14/20... Training loss: 0.1518
Epoch: 14/20... Training loss: 0.1508
Epoch: 14/20... Training loss: 0.1539
Epoch: 14/20... Training loss: 0.1516
Epoch: 14/20... Training loss: 0.1535
Epoch: 14/20... Training loss: 0.1538
Epoch: 14/20... Training loss: 0.1538
Epoch: 14/20... Training loss: 0.1548
Epoch: 14/20... Training loss: 0.1513
Epoch: 14/20... Training loss: 0.1516
Epoch: 14/20... Training loss: 0.1527
Epoch: 14/20... Training loss: 0.1550
Epoch: 14/20... Training loss: 0.1541
Epoch: 14/20... Training loss: 0.1529
Epoch: 14/20... Training loss: 0.1527
Epoch: 14/20... Training loss: 0.1531
Epoch: 14/20... Training loss: 0.1548
Epoch: 14/20... Training loss: 0.1491
Epoch: 14/20... Training loss: 0.1555
Epoch: 14/20... Training loss: 0.1552
Epoch: 14/20... Training loss: 0.1530
Epoch: 14/20... Training loss: 0.1543
Epoch: 14/20... Training loss: 0.1517
Epoch: 14/20... Training loss: 0.1544
Epoch: 14/20... Training loss: 0.1547
Epoch: 14/20... Training loss: 0.1508
Epoch: 14/20... Training loss: 0.1517
Epoch: 14/20... Training loss: 0.1537
Epoch: 14/20... Training loss: 0.1532
Epoch: 14/20... Training loss: 0.1510
Epoch: 14/20... Training loss: 0.1490
Epoch: 14/20... Training loss: 0.1527
Epoch: 14/20... Training loss: 0.1536
Epoch: 14/20... Training loss: 0.1536
Epoch: 14/20... Training loss: 0.1502
Epoch: 14/20... Training loss: 0.1504
Epoch: 14/20... Training loss: 0.1517
Epoch: 14/20... Training loss: 0.1522
Epoch: 14/20... Training loss: 0.1509
Epoch: 14/20... Training loss: 0.1497
Epoch: 14/20... Training loss: 0.1515
Epoch: 14/20... Training loss: 0.1510
Epoch: 14/20... Training loss: 0.1521
Epoch: 14/20... Training loss: 0.1523
Epoch: 14/20... Training loss: 0.1518
Epoch: 14/20... Training loss: 0.1502
Epoch: 15/20... Training loss: 0.1525
Epoch: 15/20... Training loss: 0.1492
Epoch: 15/20... Training loss: 0.1535
Epoch: 15/20... Training loss: 0.1530
Epoch: 15/20... Training loss: 0.1495
Epoch: 15/20... Training loss: 0.1508
Epoch: 15/20... Training loss: 0.1491
Epoch: 15/20... Training loss: 0.1524
Epoch: 15/20... Training loss: 0.1512
Epoch: 15/20... Training loss: 0.1512
Epoch: 15/20... Training loss: 0.1497
Epoch: 15/20... Training loss: 0.1509
Epoch: 15/20... Training loss: 0.1522
Epoch: 15/20... Training loss: 0.1530
Epoch: 15/20... Training loss: 0.1519
Epoch: 15/20... Training loss: 0.1507
Epoch: 15/20... Training loss: 0.1511
Epoch: 15/20... Training loss: 0.1525
Epoch: 15/20... Training loss: 0.1499
Epoch: 15/20... Training loss: 0.1467
Epoch: 15/20... Training loss: 0.1495
Epoch: 15/20... Training loss: 0.1502
Epoch: 15/20... Training loss: 0.1498
Epoch: 15/20... Training loss: 0.1488
Epoch: 15/20... Training loss: 0.1495
Epoch: 15/20... Training loss: 0.1497
Epoch: 15/20... Training loss: 0.1508
Epoch: 15/20... Training loss: 0.1524
Epoch: 15/20... Training loss: 0.1536
Epoch: 15/20... Training loss: 0.1509
Epoch: 15/20... Training loss: 0.1496
Epoch: 15/20... Training loss: 0.1515
Epoch: 15/20... Training loss: 0.1508
Epoch: 15/20... Training loss: 0.1477
Epoch: 15/20... Training loss: 0.1527
Epoch: 15/20... Training loss: 0.1498
Epoch: 15/20... Training loss: 0.1502
Epoch: 15/20... Training loss: 0.1537
Epoch: 15/20... Training loss: 0.1518
Epoch: 15/20... Training loss: 0.1517
Epoch: 15/20... Training loss: 0.1481
Epoch: 15/20... Training loss: 0.1498
Epoch: 15/20... Training loss: 0.1488
Epoch: 15/20... Training loss: 0.1481
Epoch: 15/20... Training loss: 0.1508
Epoch: 15/20... Training loss: 0.1484
Epoch: 15/20... Training loss: 0.1518
Epoch: 15/20... Training loss: 0.1491
Epoch: 15/20... Training loss: 0.1508
Epoch: 15/20... Training loss: 0.1505
Epoch: 15/20... Training loss: 0.1519
Epoch: 15/20... Training loss: 0.1523
Epoch: 15/20... Training loss: 0.1477
Epoch: 15/20... Training loss: 0.1470
Epoch: 15/20... Training loss: 0.1505
Epoch: 15/20... Training loss: 0.1516
Epoch: 15/20... Training loss: 0.1507
Epoch: 15/20... Training loss: 0.1492
Epoch: 15/20... Training loss: 0.1488
Epoch: 15/20... Training loss: 0.1490
Epoch: 16/20... Training loss: 0.1501
Epoch: 16/20... Training loss: 0.1501
Epoch: 16/20... Training loss: 0.1512
Epoch: 16/20... Training loss: 0.1502
Epoch: 16/20... Training loss: 0.1490
Epoch: 16/20... Training loss: 0.1470
Epoch: 16/20... Training loss: 0.1504
Epoch: 16/20... Training loss: 0.1491
Epoch: 16/20... Training loss: 0.1511
Epoch: 16/20... Training loss: 0.1482
Epoch: 16/20... Training loss: 0.1506
Epoch: 16/20... Training loss: 0.1499
Epoch: 16/20... Training loss: 0.1486
Epoch: 16/20... Training loss: 0.1480
Epoch: 16/20... Training loss: 0.1489
Epoch: 16/20... Training loss: 0.1490
Epoch: 16/20... Training loss: 0.1478
Epoch: 16/20... Training loss: 0.1495
Epoch: 16/20... Training loss: 0.1479
Epoch: 16/20... Training loss: 0.1490
Epoch: 16/20... Training loss: 0.1490
Epoch: 16/20... Training loss: 0.1512
Epoch: 16/20... Training loss: 0.1507
Epoch: 16/20... Training loss: 0.1485
Epoch: 16/20... Training loss: 0.1458
Epoch: 16/20... Training loss: 0.1472
Epoch: 16/20... Training loss: 0.1478
Epoch: 16/20... Training loss: 0.1474
Epoch: 16/20... Training loss: 0.1502
Epoch: 16/20... Training loss: 0.1476
Epoch: 16/20... Training loss: 0.1499
Epoch: 16/20... Training loss: 0.1470
Epoch: 16/20... Training loss: 0.1488
Epoch: 16/20... Training loss: 0.1475
Epoch: 16/20... Training loss: 0.1488
Epoch: 16/20... Training loss: 0.1486
Epoch: 16/20... Training loss: 0.1501
Epoch: 16/20... Training loss: 0.1469
Epoch: 16/20... Training loss: 0.1468
Epoch: 16/20... Training loss: 0.1484
Epoch: 16/20... Training loss: 0.1488
Epoch: 16/20... Training loss: 0.1478
Epoch: 16/20... Training loss: 0.1471
Epoch: 16/20... Training loss: 0.1487
Epoch: 16/20... Training loss: 0.1493
Epoch: 16/20... Training loss: 0.1508
Epoch: 16/20... Training loss: 0.1471
Epoch: 16/20... Training loss: 0.1475
Epoch: 16/20... Training loss: 0.1470
Epoch: 16/20... Training loss: 0.1493
Epoch: 16/20... Training loss: 0.1493
Epoch: 16/20... Training loss: 0.1472
Epoch: 16/20... Training loss: 0.1438
Epoch: 16/20... Training loss: 0.1492
Epoch: 16/20... Training loss: 0.1472
Epoch: 16/20... Training loss: 0.1478
Epoch: 16/20... Training loss: 0.1447
Epoch: 16/20... Training loss: 0.1471
Epoch: 16/20... Training loss: 0.1535
Epoch: 16/20... Training loss: 0.1461
Epoch: 17/20... Training loss: 0.1500
Epoch: 17/20... Training loss: 0.1468
Epoch: 17/20... Training loss: 0.1496
Epoch: 17/20... Training loss: 0.1481
Epoch: 17/20... Training loss: 0.1491
Epoch: 17/20... Training loss: 0.1484
Epoch: 17/20... Training loss: 0.1471
Epoch: 17/20... Training loss: 0.1485
Epoch: 17/20... Training loss: 0.1500
Epoch: 17/20... Training loss: 0.1493
Epoch: 17/20... Training loss: 0.1452
Epoch: 17/20... Training loss: 0.1467
Epoch: 17/20... Training loss: 0.1478
Epoch: 17/20... Training loss: 0.1494
Epoch: 17/20... Training loss: 0.1465
Epoch: 17/20... Training loss: 0.1503
Epoch: 17/20... Training loss: 0.1467
Epoch: 17/20... Training loss: 0.1464
Epoch: 17/20... Training loss: 0.1493
Epoch: 17/20... Training loss: 0.1481
Epoch: 17/20... Training loss: 0.1467
Epoch: 17/20... Training loss: 0.1463
Epoch: 17/20... Training loss: 0.1471
Epoch: 17/20... Training loss: 0.1468
Epoch: 17/20... Training loss: 0.1477
Epoch: 17/20... Training loss: 0.1472
Epoch: 17/20... Training loss: 0.1468
Epoch: 17/20... Training loss: 0.1472
Epoch: 17/20... Training loss: 0.1465
Epoch: 17/20... Training loss: 0.1467
Epoch: 17/20... Training loss: 0.1461
Epoch: 17/20... Training loss: 0.1456
Epoch: 17/20... Training loss: 0.1482
Epoch: 17/20... Training loss: 0.1445
Epoch: 17/20... Training loss: 0.1448
Epoch: 17/20... Training loss: 0.1490
Epoch: 17/20... Training loss: 0.1468
Epoch: 17/20... Training loss: 0.1464
Epoch: 17/20... Training loss: 0.1448
Epoch: 17/20... Training loss: 0.1487
Epoch: 17/20... Training loss: 0.1458
Epoch: 17/20... Training loss: 0.1473
Epoch: 17/20... Training loss: 0.1484
Epoch: 17/20... Training loss: 0.1455
Epoch: 17/20... Training loss: 0.1477
Epoch: 17/20... Training loss: 0.1448
Epoch: 17/20... Training loss: 0.1441
Epoch: 17/20... Training loss: 0.1438
Epoch: 17/20... Training loss: 0.1459
Epoch: 17/20... Training loss: 0.1446
Epoch: 17/20... Training loss: 0.1486
Epoch: 17/20... Training loss: 0.1459
Epoch: 17/20... Training loss: 0.1459
Epoch: 17/20... Training loss: 0.1447
Epoch: 17/20... Training loss: 0.1464
Epoch: 17/20... Training loss: 0.1473
Epoch: 17/20... Training loss: 0.1463
Epoch: 17/20... Training loss: 0.1440
Epoch: 17/20... Training loss: 0.1477
Epoch: 17/20... Training loss: 0.1450
Epoch: 18/20... Training loss: 0.1470
Epoch: 18/20... Training loss: 0.1442
Epoch: 18/20... Training loss: 0.1452
Epoch: 18/20... Training loss: 0.1465
Epoch: 18/20... Training loss: 0.1473
Epoch: 18/20... Training loss: 0.1466
Epoch: 18/20... Training loss: 0.1483
Epoch: 18/20... Training loss: 0.1454
Epoch: 18/20... Training loss: 0.1465
Epoch: 18/20... Training loss: 0.1474
Epoch: 18/20... Training loss: 0.1465
Epoch: 18/20... Training loss: 0.1466
Epoch: 18/20... Training loss: 0.1440
Epoch: 18/20... Training loss: 0.1466
Epoch: 18/20... Training loss: 0.1455
Epoch: 18/20... Training loss: 0.1482
Epoch: 18/20... Training loss: 0.1455
Epoch: 18/20... Training loss: 0.1480
Epoch: 18/20... Training loss: 0.1458
Epoch: 18/20... Training loss: 0.1462
Epoch: 18/20... Training loss: 0.1465
Epoch: 18/20... Training loss: 0.1469
Epoch: 18/20... Training loss: 0.1440
Epoch: 18/20... Training loss: 0.1438
Epoch: 18/20... Training loss: 0.1462
Epoch: 18/20... Training loss: 0.1453
Epoch: 18/20... Training loss: 0.1485
Epoch: 18/20... Training loss: 0.1447
Epoch: 18/20... Training loss: 0.1452
Epoch: 18/20... Training loss: 0.1476
Epoch: 18/20... Training loss: 0.1440
Epoch: 18/20... Training loss: 0.1450
Epoch: 18/20... Training loss: 0.1429
Epoch: 18/20... Training loss: 0.1455
Epoch: 18/20... Training loss: 0.1456
Epoch: 18/20... Training loss: 0.1454
Epoch: 18/20... Training loss: 0.1447
Epoch: 18/20... Training loss: 0.1445
Epoch: 18/20... Training loss: 0.1490
Epoch: 18/20... Training loss: 0.1441
Epoch: 18/20... Training loss: 0.1454
Epoch: 18/20... Training loss: 0.1459
Epoch: 18/20... Training loss: 0.1442
Epoch: 18/20... Training loss: 0.1477
Epoch: 18/20... Training loss: 0.1483
Epoch: 18/20... Training loss: 0.1455
Epoch: 18/20... Training loss: 0.1448
Epoch: 18/20... Training loss: 0.1436
Epoch: 18/20... Training loss: 0.1416
Epoch: 18/20... Training loss: 0.1449
Epoch: 18/20... Training loss: 0.1465
Epoch: 18/20... Training loss: 0.1452
Epoch: 18/20... Training loss: 0.1432
Epoch: 18/20... Training loss: 0.1454
Epoch: 18/20... Training loss: 0.1465
Epoch: 18/20... Training loss: 0.1442
Epoch: 18/20... Training loss: 0.1430
Epoch: 18/20... Training loss: 0.1445
Epoch: 18/20... Training loss: 0.1445
Epoch: 18/20... Training loss: 0.1452
Epoch: 19/20... Training loss: 0.1447
Epoch: 19/20... Training loss: 0.1442
Epoch: 19/20... Training loss: 0.1434
Epoch: 19/20... Training loss: 0.1439
Epoch: 19/20... Training loss: 0.1453
Epoch: 19/20... Training loss: 0.1457
Epoch: 19/20... Training loss: 0.1454
Epoch: 19/20... Training loss: 0.1434
Epoch: 19/20... Training loss: 0.1439
Epoch: 19/20... Training loss: 0.1441
Epoch: 19/20... Training loss: 0.1425
Epoch: 19/20... Training loss: 0.1450
Epoch: 19/20... Training loss: 0.1432
Epoch: 19/20... Training loss: 0.1445
Epoch: 19/20... Training loss: 0.1449
Epoch: 19/20... Training loss: 0.1450
Epoch: 19/20... Training loss: 0.1466
Epoch: 19/20... Training loss: 0.1448
Epoch: 19/20... Training loss: 0.1457
Epoch: 19/20... Training loss: 0.1430
Epoch: 19/20... Training loss: 0.1451
Epoch: 19/20... Training loss: 0.1450
Epoch: 19/20... Training loss: 0.1426
Epoch: 19/20... Training loss: 0.1433
Epoch: 19/20... Training loss: 0.1430
Epoch: 19/20... Training loss: 0.1433
Epoch: 19/20... Training loss: 0.1447
Epoch: 19/20... Training loss: 0.1444
Epoch: 19/20... Training loss: 0.1462
Epoch: 19/20... Training loss: 0.1450
Epoch: 19/20... Training loss: 0.1447
Epoch: 19/20... Training loss: 0.1447
Epoch: 19/20... Training loss: 0.1433
Epoch: 19/20... Training loss: 0.1448
Epoch: 19/20... Training loss: 0.1477
Epoch: 19/20... Training loss: 0.1443
Epoch: 19/20... Training loss: 0.1462
Epoch: 19/20... Training loss: 0.1444
Epoch: 19/20... Training loss: 0.1429
Epoch: 19/20... Training loss: 0.1421
Epoch: 19/20... Training loss: 0.1436
Epoch: 19/20... Training loss: 0.1437
Epoch: 19/20... Training loss: 0.1436
Epoch: 19/20... Training loss: 0.1436
Epoch: 19/20... Training loss: 0.1452
Epoch: 19/20... Training loss: 0.1446
Epoch: 19/20... Training loss: 0.1454
Epoch: 19/20... Training loss: 0.1451
Epoch: 19/20... Training loss: 0.1440
Epoch: 19/20... Training loss: 0.1447
Epoch: 19/20... Training loss: 0.1430
Epoch: 19/20... Training loss: 0.1445
Epoch: 19/20... Training loss: 0.1454
Epoch: 19/20... Training loss: 0.1478
Epoch: 19/20... Training loss: 0.1434
Epoch: 19/20... Training loss: 0.1475
Epoch: 19/20... Training loss: 0.1462
Epoch: 19/20... Training loss: 0.1435
Epoch: 19/20... Training loss: 0.1439
Epoch: 19/20... Training loss: 0.1418
Epoch: 20/20... Training loss: 0.1415
Epoch: 20/20... Training loss: 0.1453
Epoch: 20/20... Training loss: 0.1460
Epoch: 20/20... Training loss: 0.1438
Epoch: 20/20... Training loss: 0.1431
Epoch: 20/20... Training loss: 0.1441
Epoch: 20/20... Training loss: 0.1452
Epoch: 20/20... Training loss: 0.1418
Epoch: 20/20... Training loss: 0.1435
Epoch: 20/20... Training loss: 0.1465
Epoch: 20/20... Training loss: 0.1439
Epoch: 20/20... Training loss: 0.1453
Epoch: 20/20... Training loss: 0.1443
Epoch: 20/20... Training loss: 0.1440
Epoch: 20/20... Training loss: 0.1445
Epoch: 20/20... Training loss: 0.1442
Epoch: 20/20... Training loss: 0.1440
Epoch: 20/20... Training loss: 0.1454
Epoch: 20/20... Training loss: 0.1438
Epoch: 20/20... Training loss: 0.1448
Epoch: 20/20... Training loss: 0.1423
Epoch: 20/20... Training loss: 0.1412
Epoch: 20/20... Training loss: 0.1455
Epoch: 20/20... Training loss: 0.1416
Epoch: 20/20... Training loss: 0.1402
Epoch: 20/20... Training loss: 0.1461
Epoch: 20/20... Training loss: 0.1412
Epoch: 20/20... Training loss: 0.1407
Epoch: 20/20... Training loss: 0.1436
Epoch: 20/20... Training loss: 0.1422
Epoch: 20/20... Training loss: 0.1419
Epoch: 20/20... Training loss: 0.1430
Epoch: 20/20... Training loss: 0.1468
Epoch: 20/20... Training loss: 0.1423
Epoch: 20/20... Training loss: 0.1433
Epoch: 20/20... Training loss: 0.1449
Epoch: 20/20... Training loss: 0.1453
Epoch: 20/20... Training loss: 0.1431
Epoch: 20/20... Training loss: 0.1440
Epoch: 20/20... Training loss: 0.1422
Epoch: 20/20... Training loss: 0.1423
Epoch: 20/20... Training loss: 0.1410
Epoch: 20/20... Training loss: 0.1457
Epoch: 20/20... Training loss: 0.1449
Epoch: 20/20... Training loss: 0.1439
Epoch: 20/20... Training loss: 0.1439
Epoch: 20/20... Training loss: 0.1437
Epoch: 20/20... Training loss: 0.1417
Epoch: 20/20... Training loss: 0.1402
Epoch: 20/20... Training loss: 0.1448
Epoch: 20/20... Training loss: 0.1408
Epoch: 20/20... Training loss: 0.1415
Epoch: 20/20... Training loss: 0.1448
Epoch: 20/20... Training loss: 0.1431
Epoch: 20/20... Training loss: 0.1426
Epoch: 20/20... Training loss: 0.1436
Epoch: 20/20... Training loss: 0.1448
Epoch: 20/20... Training loss: 0.1422
Epoch: 20/20... Training loss: 0.1418
Epoch: 20/20... Training loss: 0.1422
In [ ]:
fig, axes = plt.subplots(nrows=2, ncols=10, sharex=True, sharey=True, figsize=(20,4))
in_imgs = mnist.test.images[:10]
reconstructed, compressed = sess.run([decoded, encoded], feed_dict={inputs_: in_imgs})
for images, row in zip([in_imgs, reconstructed], axes):
for img, ax in zip(images, row):
ax.imshow(img.reshape((28, 28)), cmap='Greys_r')
ax.get_xaxis().set_visible(False)
ax.get_yaxis().set_visible(False)
fig.tight_layout(pad=0.1)
In [ ]:
sess.close()
We're dealing with images here, so we can (usually) get better performance using convolution layers. So, next we'll build a better autoencoder with convolutional layers.
In practice, autoencoders aren't actually better at compression compared to typical methods like JPEGs and MP3s. But, they are being used for noise reduction, which you'll also build.
Content source: AlphaGit/deep-learning
Similar notebooks: