Excercise 1 Taks 7

Keras and deep dreaming

The deep dreaming script was executed within an anaconda environment with the following packages

  • python 3.5
  • keras 2.0.2
  • tensorflow 1.0
  • pillow 4.0.0

Visualizing image results through parameter changes in deep_dream.py script

Original to be transformed image because my own selfies are simply too ugly and I'm a Star Trek Fan

This Image is part of the Star Trek Discovery Trailer by Netflix and CBS

1. run with original parameters


In [3]:
# First script run with original parameters 
%run deep_dream.py img/std_o.jpg img/std_1.jpg


Using TensorFlow backend.
Model loaded.
/home/henrik/anaconda3/lib/python3.6/site-packages/scipy/ndimage/interpolation.py:568: UserWarning: From scipy 0.13.0, the output shape of zoom() is calculated with round() instead of int() - for these inputs the size of the returned array has changed.
  "the returned array has changed.", UserWarning)
/home/henrik/anaconda3/lib/python3.6/site-packages/scipy/ndimage/interpolation.py:571: RuntimeWarning: invalid value encountered in true_divide
  zoom = (numpy.array(input.shape) - 1) / zoom_div
Processing image shape (236, 420)
..Loss value at 0 : 0.773101
..Loss value at 1 : 0.989443
..Loss value at 2 : 1.36996
..Loss value at 3 : 1.73904
..Loss value at 4 : 2.0989
..Loss value at 5 : 2.46473
..Loss value at 6 : 2.83497
..Loss value at 7 : 3.20704
..Loss value at 8 : 3.53789
..Loss value at 9 : 3.87789
..Loss value at 10 : 4.17761
..Loss value at 11 : 4.54946
..Loss value at 12 : 4.84955
..Loss value at 13 : 5.13712
..Loss value at 14 : 5.45583
..Loss value at 15 : 5.73099
..Loss value at 16 : 5.9894
..Loss value at 17 : 6.26368
..Loss value at 18 : 6.5892
..Loss value at 19 : 6.86595
Processing image shape (331, 589)
..Loss value at 0 : 1.58885
..Loss value at 1 : 2.4101
..Loss value at 2 : 3.0788
..Loss value at 3 : 3.63512
..Loss value at 4 : 4.17944
..Loss value at 5 : 4.66909
..Loss value at 6 : 5.13645
..Loss value at 7 : 5.55448
..Loss value at 8 : 5.94556
..Loss value at 9 : 6.37725
..Loss value at 10 : 6.75097
..Loss value at 11 : 7.12204
..Loss value at 12 : 7.50019
..Loss value at 13 : 7.82671
..Loss value at 14 : 8.1627
..Loss value at 15 : 8.50139
..Loss value at 16 : 8.77757
..Loss value at 17 : 9.09302
..Loss value at 18 : 9.40175
..Loss value at 19 : 9.69899
Processing image shape (464, 825)
..Loss value at 0 : 1.80886
..Loss value at 1 : 2.75668
..Loss value at 2 : 3.5511
..Loss value at 3 : 4.22848
..Loss value at 4 : 4.84531
..Loss value at 5 : 5.4161
..Loss value at 6 : 5.96341
..Loss value at 7 : 6.50308
..Loss value at 8 : 7.03244
..Loss value at 9 : 7.53222
..Loss value at 10 : 8.02889
..Loss value at 11 : 8.60728
..Loss value at 12 : 9.21723
..Loss value at 13 : 9.92354

After execution with original parameter we've got this result :-)

It seems to recognize cats and fishes? -> Lets Play with the parameters and generate some more images

2. Run with double step size (0.02)


In [1]:
%run deep_dream.py img/std_o.jpg img/std_2.png


Using TensorFlow backend.
Model loaded.
/home/henrik/anaconda3/lib/python3.6/site-packages/scipy/ndimage/interpolation.py:568: UserWarning: From scipy 0.13.0, the output shape of zoom() is calculated with round() instead of int() - for these inputs the size of the returned array has changed.
  "the returned array has changed.", UserWarning)
/home/henrik/anaconda3/lib/python3.6/site-packages/scipy/ndimage/interpolation.py:571: RuntimeWarning: invalid value encountered in true_divide
  zoom = (numpy.array(input.shape) - 1) / zoom_div
Processing image shape (236, 420)
..Loss value at 0 : 0.773101
..Loss value at 1 : 0.94976
..Loss value at 2 : 1.34936
..Loss value at 3 : 1.86133
..Loss value at 4 : 2.3787
..Loss value at 5 : 2.85991
..Loss value at 6 : 3.37634
..Loss value at 7 : 3.86549
..Loss value at 8 : 4.38928
..Loss value at 9 : 4.86004
..Loss value at 10 : 5.35465
..Loss value at 11 : 5.85905
..Loss value at 12 : 6.34463
..Loss value at 13 : 6.83067
..Loss value at 14 : 7.28357
..Loss value at 15 : 7.75236
..Loss value at 16 : 8.21696
..Loss value at 17 : 8.67674
..Loss value at 18 : 9.12857
..Loss value at 19 : 9.59682
Processing image shape (331, 589)
..Loss value at 0 : 1.71816
..Loss value at 1 : 2.9377
..Loss value at 2 : 4.07589
..Loss value at 3 : 5.04884
..Loss value at 4 : 6.01486
..Loss value at 5 : 7.03627
..Loss value at 6 : 8.26636
Processing image shape (464, 825)
..Loss value at 0 : 2.42704
..Loss value at 1 : 5.90696

The structures seem to be more coarse than in the original image

3. Run with the following parameters

  • step = 0.02 # Gradient ascent step size
  • num_octave = 4 (changed from 3 to 4) # Number of scales at which to run gradient ascent
  • octave_scale = 1.4 # Size ratio between scales
  • iterations = 20 # Number of ascent steps per scale
  • max_loss = 10.

In [4]:
%run deep_dream.py img/std_o.jpg img/std_3


Model loaded.
/home/henrik/anaconda3/lib/python3.6/site-packages/scipy/ndimage/interpolation.py:568: UserWarning: From scipy 0.13.0, the output shape of zoom() is calculated with round() instead of int() - for these inputs the size of the returned array has changed.
  "the returned array has changed.", UserWarning)
/home/henrik/anaconda3/lib/python3.6/site-packages/scipy/ndimage/interpolation.py:571: RuntimeWarning: invalid value encountered in true_divide
  zoom = (numpy.array(input.shape) - 1) / zoom_div
Processing image shape (236, 420)
..Loss value at 0 : 0.773101
..Loss value at 1 : 0.94976
..Loss value at 2 : 1.34936
..Loss value at 3 : 1.86133
..Loss value at 4 : 2.3787
..Loss value at 5 : 2.85991
..Loss value at 6 : 3.37634
..Loss value at 7 : 3.86549
..Loss value at 8 : 4.38928
..Loss value at 9 : 4.86004
..Loss value at 10 : 5.35465
..Loss value at 11 : 5.85905
..Loss value at 12 : 6.34463
..Loss value at 13 : 6.83067
..Loss value at 14 : 7.28357
..Loss value at 15 : 7.75236
..Loss value at 16 : 8.21696
..Loss value at 17 : 8.67674
..Loss value at 18 : 9.12857
..Loss value at 19 : 9.59682
Processing image shape (331, 589)
..Loss value at 0 : 1.71816
..Loss value at 1 : 2.9377
..Loss value at 2 : 4.07589
..Loss value at 3 : 5.04884
..Loss value at 4 : 6.01486
..Loss value at 5 : 7.03627
..Loss value at 6 : 8.26636
Processing image shape (464, 825)
..Loss value at 0 : 2.42704
..Loss value at 1 : 5.90696

Change of octave does not lead to changed visualization

4. Run with the following parameters

  • step = 0.02 # Gradient ascent step size
  • num_octave = 4 # Number of scales at which to run gradient ascent
  • octave_scale = 2.0 (1.4 -> 2.0) # Size ratio between scales
  • iterations = 30 (20 -> 30 # Number of ascent steps per scale
  • max_loss = 10.

In [5]:
%run deep_dream.py img/std_o.jpg img/std_4


Model loaded.
/home/henrik/anaconda3/lib/python3.6/site-packages/scipy/ndimage/interpolation.py:571: RuntimeWarning: invalid value encountered in true_divide
  zoom = (numpy.array(input.shape) - 1) / zoom_div
Processing image shape (58, 103)
..Loss value at 0 : 0.0
..Loss value at 1 : 0.0
..Loss value at 2 : 0.0
..Loss value at 3 : 0.0
..Loss value at 4 : 0.0
..Loss value at 5 : 0.0
..Loss value at 6 : 0.0
..Loss value at 7 : 0.0
..Loss value at 8 : 0.0
..Loss value at 9 : 0.0
..Loss value at 10 : 0.0
..Loss value at 11 : 0.0
..Loss value at 12 : 0.0
..Loss value at 13 : 0.0
..Loss value at 14 : 0.0
..Loss value at 15 : 0.0
..Loss value at 16 : 0.0
..Loss value at 17 : 0.0
..Loss value at 18 : 0.0
..Loss value at 19 : 0.0
..Loss value at 20 : 0.0
..Loss value at 21 : 0.0
..Loss value at 22 : 0.0
..Loss value at 23 : 0.0
..Loss value at 24 : 0.0
..Loss value at 25 : 0.0
..Loss value at 26 : 0.0
..Loss value at 27 : 0.0
..Loss value at 28 : 0.0
..Loss value at 29 : 0.0
Processing image shape (116, 206)
..Loss value at 0 : 0.198674
..Loss value at 1 : 0.269958
..Loss value at 2 : 0.410425
..Loss value at 3 : 0.630814
..Loss value at 4 : 0.917823
..Loss value at 5 : 1.177
..Loss value at 6 : 1.4362
..Loss value at 7 : 1.7589
..Loss value at 8 : 2.00253
..Loss value at 9 : 2.28321
..Loss value at 10 : 2.53273
..Loss value at 11 : 2.85366
..Loss value at 12 : 3.1513
..Loss value at 13 : 3.28074
..Loss value at 14 : 3.74455
..Loss value at 15 : 4.10269
..Loss value at 16 : 4.36462
..Loss value at 17 : 4.72593
..Loss value at 18 : 5.07547
..Loss value at 19 : 5.57452
..Loss value at 20 : 5.94649
..Loss value at 21 : 6.45474
..Loss value at 22 : 7.0983
..Loss value at 23 : 7.76284
..Loss value at 24 : 8.32685
..Loss value at 25 : 9.10017
..Loss value at 26 : 9.95701
Processing image shape (232, 412)
..Loss value at 0 : 1.23273
..Loss value at 1 : 2.68147
..Loss value at 2 : 6.46694
Processing image shape (464, 825)
..Loss value at 0 : 4.41594

5. Run with the following parameters

  • step = 0.02 # Gradient ascent step size
  • num_octave = 4 # Number of scales at which to run gradient ascent
  • octave_scale = 1.0 # Size ratio between scales
  • iterations = 30 # Number of ascent steps per scale
  • max_loss = 10.

In [6]:
%run deep_dream.py img/std_o.jpg img/std_5


Model loaded.
/home/henrik/anaconda3/lib/python3.6/site-packages/scipy/ndimage/interpolation.py:571: RuntimeWarning: invalid value encountered in true_divide
  zoom = (numpy.array(input.shape) - 1) / zoom_div
Processing image shape (464, 825)
..Loss value at 0 : 0.848442
..Loss value at 1 : 1.02749
..Loss value at 2 : 1.47048
..Loss value at 3 : 2.05187
..Loss value at 4 : 2.69905
..Loss value at 5 : 3.34912
..Loss value at 6 : 3.99514
..Loss value at 7 : 4.65236
..Loss value at 8 : 5.25005
..Loss value at 9 : 5.90118
..Loss value at 10 : 6.55184
..Loss value at 11 : 7.15759
..Loss value at 12 : 7.78682
..Loss value at 13 : 8.40784
..Loss value at 14 : 9.00614
..Loss value at 15 : 9.58601
Processing image shape (464, 825)
Processing image shape (464, 825)
Processing image shape (464, 825)

Octave scale seems to have the most effect on image structure granularity and together with step the most effect on run time behaviour

6. Run with the following parameters

  • step = 0.01 # Gradient ascent step size
  • num_octave = 4 # Number of scales at which to run gradient ascent
  • octave_scale = 0.5 # Size ratio between scales
  • iterations = 30 # Number of ascent steps per scale
  • max_loss = 10.

In [8]:
%run deep_dream.py img/std_o.jpg std_6


Model loaded.
/home/henrik/anaconda3/lib/python3.6/site-packages/scipy/ndimage/interpolation.py:571: RuntimeWarning: invalid value encountered in true_divide
  zoom = (numpy.array(input.shape) - 1) / zoom_div
Processing image shape (3712, 6600)
..Loss value at 0 : 0.622781
..Loss value at 1 : 0.681898
..Loss value at 2 : 0.996126
..Loss value at 3 : 1.43156
..Loss value at 4 : 1.93483
..Loss value at 5 : 2.43186
..Loss value at 6 : 2.92295
..Loss value at 7 : 3.39668
..Loss value at 8 : 3.85623
..Loss value at 9 : 4.31903
..Loss value at 10 : 4.76677
..Loss value at 11 : 5.22494
..Loss value at 12 : 5.67537
..Loss value at 13 : 6.1611
..Loss value at 14 : 6.63339
..Loss value at 15 : 7.17828
..Loss value at 16 : 7.63272
..Loss value at 17 : 8.21257
..Loss value at 18 : 8.68946
..Loss value at 19 : 9.29014
..Loss value at 20 : 9.87115
Processing image shape (1856, 3300)
..Loss value at 0 : 0.915839
..Loss value at 1 : 1.68144
..Loss value at 2 : 2.49308
..Loss value at 3 : 3.14134
..Loss value at 4 : 3.69972
..Loss value at 5 : 4.24104
..Loss value at 6 : 4.74741
..Loss value at 7 : 5.23017
..Loss value at 8 : 5.71133
..Loss value at 9 : 6.17091
..Loss value at 10 : 6.63476
..Loss value at 11 : 7.06975
..Loss value at 12 : 7.5094
..Loss value at 13 : 7.92352
..Loss value at 14 : 8.33478
..Loss value at 15 : 8.74424
..Loss value at 16 : 9.13745
..Loss value at 17 : 9.51827
..Loss value at 18 : 9.89775
Processing image shape (928, 1650)
..Loss value at 0 : 0.816366
..Loss value at 1 : 1.31353
..Loss value at 2 : 1.92044
..Loss value at 3 : 2.51352
..Loss value at 4 : 3.05375
..Loss value at 5 : 3.53912
..Loss value at 6 : 4.03907
..Loss value at 7 : 4.50432
..Loss value at 8 : 4.9034
..Loss value at 9 : 5.38546
..Loss value at 10 : 5.77511
..Loss value at 11 : 6.24854
..Loss value at 12 : 6.64958
..Loss value at 13 : 7.1545
..Loss value at 14 : 7.57403
..Loss value at 15 : 8.06632
..Loss value at 16 : 8.37699
..Loss value at 17 : 8.94505
..Loss value at 18 : 9.41367
..Loss value at 19 : 9.95162
Processing image shape (464, 825)
..Loss value at 0 : 0.879436
..Loss value at 1 : 1.37898
..Loss value at 2 : 1.95472
..Loss value at 3 : 2.50293
..Loss value at 4 : 3.01778
..Loss value at 5 : 3.50392
..Loss value at 6 : 3.95581
..Loss value at 7 : 4.40152
..Loss value at 8 : 4.85826
..Loss value at 9 : 5.26666
..Loss value at 10 : 5.68483
..Loss value at 11 : 6.07954
..Loss value at 12 : 6.45223
..Loss value at 13 : 6.81853
..Loss value at 14 : 7.18966
..Loss value at 15 : 7.54068
..Loss value at 16 : 7.87167
..Loss value at 17 : 8.22661
..Loss value at 18 : 8.53101
..Loss value at 19 : 8.86024
..Loss value at 20 : 9.15279
..Loss value at 21 : 9.44598
..Loss value at 22 : 9.77415

Reducing step and octave_scale leads to fine granular image representation but to very long run times

7. Run with the following parameters

  • step = 0.015 # Gradient ascent step size
  • num_octave = 3 # Number of scales at which to run gradient ascent
  • octave_scale = 1.0 # Size ratio between scales
  • iterations = 15 # Number of ascent steps per scale
  • max_loss = 5.

In [9]:
%run deep_dream.py img/std_o.jpg img/std_7


Model loaded.
/home/henrik/anaconda3/lib/python3.6/site-packages/scipy/ndimage/interpolation.py:571: RuntimeWarning: invalid value encountered in true_divide
  zoom = (numpy.array(input.shape) - 1) / zoom_div
Processing image shape (464, 825)
..Loss value at 0 : 0.848442
..Loss value at 1 : 1.05293
..Loss value at 2 : 1.48599
..Loss value at 3 : 2.0262
..Loss value at 4 : 2.59325
..Loss value at 5 : 3.17196
..Loss value at 6 : 3.74864
..Loss value at 7 : 4.31302
..Loss value at 8 : 4.86509
Processing image shape (464, 825)
Processing image shape (464, 825)

Using a higher octave_scale value together with half the iterations and half max loss keeps fine granularity of the picture with acceptable run time behaviour but also with less artifact building


In [ ]: