Neural Style Transfer



In [1]:

    
from NeuralStyle import NeuralStyleTransfer

Tutorial

Specify the paths for the content and style
Class defaults:
- image_w = 500 width (in pixels) of the generated image
- image_h = 500 height (in pixels) of the generated image
- style_weight=2e5 weight on style
- content_weight=0.001 weight on content. This ratio deviates from Gatys et al. (who used 1e-3 and 1-4 rather than 1e-8)



In [2]:

    
content_path = 'images/tubingen.jpg'
style_path = 'images/starry_night.jpg'
nst = NeuralStyleTransfer(content_path, style_path, image_w=300, image_h=300)

Optimization

There are two optimizers available, L-BFGS and Adam. L-BFGS converges more quickly on lower costs, but 1) requires more memory 2) goes out to the CPU.

Adam has the advantage that is is lower memory and is in Theano so it stays on the GPU, but it can require more/different tuning to achieve comparable results (and often fails to achieve as good results as L-BFGS)

Adam:



In [ ]:

    
nst.fit(iterations=75, save_every_n=10, optimizer='adam')



In [ ]:

    
adam_losses = nst.losses



In [ ]:

    
import matplotlib.pyplot as plt
%matplotlib inline
plt.imshow(nst.final_image())

L-BFGS:



In [ ]:

    
nst_lbfgs = NeuralStyleTransfer(content_path, style_path, image_w=300, image_h=300)
nst_lbfgs.fit(iterations=75, save_every_n=10, optimizer='l-bfgs')



In [ ]:

    
processed = [nst.deprocess(x) for x in nst.xs]
processed_lbfgs = [nst_lbfgs.deprocess(x) for x in nst_lbfgs.xs]


# len(nst.xs)



In [ ]:

    
fig,(ax1,ax2) = plt.subplots(1,2,figsize = (10,5))

ax1.imshow(nst.final_image())
ax1.axis('off')
ax1.set_title('Adam')

ax2.imshow(nst_lbfgs.final_image())
ax2.axis('off')
ax2.set_title('L-BFGS')


plt.tight_layout()
plt.savefig('Adam_vs_LBFGS.png')



In [ ]:

    
%matplotlib inline

plt.plot(nst.losses,label = 'Adam')
plt.plot(nst_lbfgs.losses,label = 'LBFGS')
plt.legend()
plt.xlim(0,75)
plt.ylim(0,200000)

plt.xlabel('Number of Optimization Iterations')
plt.ylabel('Loss')
plt.title('Comparing Optimization Methods Adam vs LBFGS ')
plt.savefig('Optimization Comparisons.png')



In [ ]:

    
import matplotlib.pyplot as plt
%matplotlib inline
plt.imshow(nst.final_image())