Setup Notebook

Ensure plots show up in the notebook itself



In [32]:

    
%matplotlib inline

The entire dogscats/ directory can be downloaded from here. It has been placed in a data/ directory where this notebook is located. Add the entire "data/" directory to the .gitignore file to prevent it from getting uploaded to GitHub.

Add a line to toggle whether the whole dataset will be used or just the sample/ directory for faster prototyping.



In [33]:

    
#path = "data/dogscats/"
path = "data/dogscats/sample/"



In [34]:

    
# Handy trick to get a file link
from IPython.display import FileLink
FileLink('AF.png')









    Out[34]:




Path (AF.png) doesn't exist. It may still be in the process of being generated, or you may have the incorrect path.

LogLoss is known in Keras as binary entropy (or categorical cross-entropy)

submitting 0's or 1's will penalize you a lot

Instead of having absolute values (0 or 1) clip the values so anything less than 0.05 becomes 0.05 and anything above 0.95 becomes 0.95 np.clip(preds[:,1], 0.05, 0.95)

After each epoch, you can save the model weights in case you end up overfitting... then you can go back to a step in the middle vgg.model.save_weights

Halfway through the epochs, change the learning rate from 0.1 to 0.01 vgg.model.optimizer.lr = 0.01

Results get more accurate and confident as you run more epochs



In [35]:

    
import numpy as np
from tensorflow import keras
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense
from tensorflow.keras.optimizers import SGD



In [36]:

    
x = np.random.random((30,2))
y = np.dot(x, [2.,3.]) + 1.



In [37]:

    
x[:5]









    Out[37]:





array([[0.25709044, 0.5276544 ],
       [0.54641298, 0.930478  ],
       [0.77481884, 0.86235503],
       [0.88355727, 0.59494145],
       [0.74826258, 0.8916664 ]])



In [38]:

    
y[:5]









    Out[38]:





array([3.09714407, 4.88425998, 5.13670276, 4.55193891, 5.17152436])



In [39]:

    
lm = Sequential([ Dense(1, input_shape=(2,))])



In [40]:

    
lm.compile(optimizer=SGD(lr=0.1), loss='mse')



In [41]:

    
lm.evaluate(x, y, verbose=0)









    Out[41]:





22.179643630981445



In [42]:

    
lm.fit(x, y, epochs=50, batch_size=10)









    



Train on 30 samples
Epoch 1/50
30/30 [==============================] - 0s 4ms/sample - loss: 12.9626
Epoch 2/50
30/30 [==============================] - 0s 166us/sample - loss: 1.9192
Epoch 3/50
30/30 [==============================] - 0s 132us/sample - loss: 0.6784
Epoch 4/50
30/30 [==============================] - 0s 166us/sample - loss: 0.5003
Epoch 5/50
30/30 [==============================] - 0s 166us/sample - loss: 0.4647
Epoch 6/50
30/30 [==============================] - 0s 132us/sample - loss: 0.4189
Epoch 7/50
30/30 [==============================] - 0s 166us/sample - loss: 0.3870
Epoch 8/50
30/30 [==============================] - 0s 133us/sample - loss: 0.3534
Epoch 9/50
30/30 [==============================] - 0s 166us/sample - loss: 0.3296
Epoch 10/50
30/30 [==============================] - 0s 133us/sample - loss: 0.3137
Epoch 11/50
30/30 [==============================] - 0s 133us/sample - loss: 0.2804
Epoch 12/50
30/30 [==============================] - 0s 167us/sample - loss: 0.2584
Epoch 13/50
30/30 [==============================] - 0s 133us/sample - loss: 0.2422
Epoch 14/50
30/30 [==============================] - 0s 166us/sample - loss: 0.2262
Epoch 15/50
30/30 [==============================] - 0s 166us/sample - loss: 0.2027
Epoch 16/50
30/30 [==============================] - 0s 166us/sample - loss: 0.1896
Epoch 17/50
30/30 [==============================] - 0s 167us/sample - loss: 0.1733
Epoch 18/50
30/30 [==============================] - 0s 99us/sample - loss: 0.1622
Epoch 19/50
30/30 [==============================] - 0s 167us/sample - loss: 0.1524
Epoch 20/50
30/30 [==============================] - 0s 132us/sample - loss: 0.1404
Epoch 21/50
30/30 [==============================] - 0s 166us/sample - loss: 0.1314
Epoch 22/50
30/30 [==============================] - 0s 133us/sample - loss: 0.1210
Epoch 23/50
30/30 [==============================] - 0s 133us/sample - loss: 0.1078
Epoch 24/50
30/30 [==============================] - 0s 166us/sample - loss: 0.0995
Epoch 25/50
30/30 [==============================] - 0s 133us/sample - loss: 0.0975
Epoch 26/50
30/30 [==============================] - 0s 132us/sample - loss: 0.0860
Epoch 27/50
30/30 [==============================] - 0s 166us/sample - loss: 0.0798
Epoch 28/50
30/30 [==============================] - 0s 133us/sample - loss: 0.0731
Epoch 29/50
30/30 [==============================] - 0s 166us/sample - loss: 0.0701
Epoch 30/50
30/30 [==============================] - 0s 134us/sample - loss: 0.0635
Epoch 31/50
30/30 [==============================] - 0s 166us/sample - loss: 0.0582
Epoch 32/50
30/30 [==============================] - 0s 199us/sample - loss: 0.0544
Epoch 33/50
30/30 [==============================] - 0s 132us/sample - loss: 0.0493
Epoch 34/50
30/30 [==============================] - 0s 199us/sample - loss: 0.0465
Epoch 35/50
30/30 [==============================] - 0s 133us/sample - loss: 0.0425
Epoch 36/50
30/30 [==============================] - 0s 132us/sample - loss: 0.0400
Epoch 37/50
30/30 [==============================] - 0s 133us/sample - loss: 0.0363
Epoch 38/50
30/30 [==============================] - 0s 100us/sample - loss: 0.0335
Epoch 39/50
30/30 [==============================] - 0s 200us/sample - loss: 0.0311
Epoch 40/50
30/30 [==============================] - 0s 133us/sample - loss: 0.0296
Epoch 41/50
30/30 [==============================] - 0s 166us/sample - loss: 0.0266
Epoch 42/50
30/30 [==============================] - 0s 166us/sample - loss: 0.0255
Epoch 43/50
30/30 [==============================] - 0s 133us/sample - loss: 0.0228
Epoch 44/50
30/30 [==============================] - 0s 166us/sample - loss: 0.0210
Epoch 45/50
30/30 [==============================] - 0s 100us/sample - loss: 0.0198
Epoch 46/50
30/30 [==============================] - 0s 166us/sample - loss: 0.0185
Epoch 47/50
30/30 [==============================] - 0s 166us/sample - loss: 0.0170
Epoch 48/50
30/30 [==============================] - 0s 133us/sample - loss: 0.0159
Epoch 49/50
30/30 [==============================] - 0s 166us/sample - loss: 0.0151
Epoch 50/50
30/30 [==============================] - 0s 132us/sample - loss: 0.0132






    Out[42]:





<tensorflow.python.keras.callbacks.History at 0x1f3212e7160>



In [43]:

    
lm.evaluate(x, y, verbose=0)









    Out[43]:





0.012495395727455616



In [44]:

    
lm.get_weights()









    Out[44]:





[array([[1.7023187],
        [2.8139684]], dtype=float32), array([1.2720194], dtype=float32)]



In [45]:

    
lm.summary()









    



Model: "sequential_1"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
dense_1 (Dense)              (None, 1)                 3         
=================================================================
Total params: 3
Trainable params: 3
Non-trainable params: 0
_________________________________________________________________



In [ ]: