This notebook is an attempt to test and prototype using a pretrained tensorflow CNN to do classification. The end goal is to be able to take add the following as an anime-specific feature: a CNN's embeddings of the default image on MyAnimeList for that anime. The idea is that there is certain visual content that goes into a person's enjoyment of an anime: art style, character design, and color scheme, for example. I want that information via a high-level representation of the image in a deep CNN pipeline. This notebook is simply an attempt getting getting the CNN part to work.

The CNN is is downloaded from Illustration2Vec (Saito & Matsui, 2015) and is pretrained on anime images. That means the feature space is uniquely suited to capturing relevant information from anime. However, this was done on Caffe, which I do not have installed. Consequently, I used the caffe-tensorflow tool (https://github.com/ethereon/caffe-tensorflow) to convert this model into a tensorflow model (via an Amazon EC2 instance).

There are examples but no clear tutorials on how to use the caffe-tensorflow tool, so this exploratory notebook is an attempt to get it to work.



In [1]:

    
import numpy as np
import tensorflow as tf
import os.path as osp

Images are 224x224 pixels, with 3 channels. Batch size is 50. This is specified in the caffemodel but not in the tf class (mynet.py)



In [2]:

    
input_size = {50, 3, 224, 224}



In [3]:

    
fake_data = np.random.rand(2, 224, 224, 3)

Now to actually load the model.



In [4]:

    
from mynet import CaffeNet
images = tf.placeholder(tf.float32, [None, 224, 224, 3])

net = CaffeNet({'data':images})



In [5]:

    
sesh = tf.Session()
sesh.run(tf.global_variables_initializer())



In [6]:

    
# Load the data
net.load('mynet.npy', sesh)



In [7]:

    
# Forward pass
output = sesh.run(net.get_output(), feed_dict={images: fake_data})



In [8]:

    
sesh.close()