Using Deep features for image retrieval

Import Graphlabe



In [4]:

    
import graphlab as gl

Laod the CIFAR-10 dataset



In [7]:

    
image_train = gl.SFrame('image_train_data/')

image_train.head()

Train a NN model for retrieving images using deep features



In [5]:

    
knn_model = gl.nearest_neighbors.create(image_train, features=['deep_features'],
                                       label = 'id')









    



PROGRESS: Starting brute force nearest neighbors model training.

Use image retrieval model with deep features to find similar images



In [13]:

    
gl.canvas.set_target('ipynb')
cat = image_train[18:19]
cat['image'].show()



In [8]:

    
knn_model.query(cat)









    



PROGRESS: Starting pairwise querying.
PROGRESS: +--------------+---------+-------------+--------------+
PROGRESS: | Query points | # Pairs | % Complete. | Elapsed Time |
PROGRESS: +--------------+---------+-------------+--------------+
PROGRESS: | 0            | 1       | 0.0498753   | 8.572ms      |
PROGRESS: | Done         |         | 100         | 240.553ms    |
PROGRESS: +--------------+---------+-------------+--------------+






    Out[8]:





    
        query_label
        reference_label
        distance
        rank
    
    
        0
        384
        0.0
        1
    
    
        0
        6910
        36.9403137951
        2
    
    
        0
        39777
        38.4634888975
        3
    
    
        0
        36870
        39.7559623119
        4
    
    
        0
        41734
        39.7866014148
        5
    

[5 rows x 4 columns]



In [11]:

    
def get_images_from_ids(query_result):
    return image_train.filter_by(query_result['reference_label'], 'id')



In [14]:

    
cat_neighbors = get_images_from_ids(knn_model.query(cat))









    



PROGRESS: Starting pairwise querying.
PROGRESS: +--------------+---------+-------------+--------------+
PROGRESS: | Query points | # Pairs | % Complete. | Elapsed Time |
PROGRESS: +--------------+---------+-------------+--------------+
PROGRESS: | 0            | 1       | 0.0498753   | 16.306ms     |
PROGRESS: | Done         |         | 100         | 261.317ms    |
PROGRESS: +--------------+---------+-------------+--------------+



In [19]:

    
cat_neighbors['image'].show()



In [20]:

    
car = image_train[8:9]



In [21]:

    
car['image'].show()

Get similar car



In [22]:

    
get_images_from_ids(knn_model.query(car))['image'].show()









    



PROGRESS: Starting pairwise querying.
PROGRESS: +--------------+---------+-------------+--------------+
PROGRESS: | Query points | # Pairs | % Complete. | Elapsed Time |
PROGRESS: +--------------+---------+-------------+--------------+
PROGRESS: | 0            | 1       | 0.0498753   | 12.773ms     |
PROGRESS: | Done         |         | 100         | 279.012ms    |
PROGRESS: +--------------+---------+-------------+--------------+

Lambda function to show nearest neighbor images



In [23]:

    
show_neighbors = lambda i:get_images_from_ids(knn_model.query(image_train[i:i+1]))['image'].show()



In [26]:

    
show_neighbors(19)









    



PROGRESS: Starting pairwise querying.
PROGRESS: +--------------+---------+-------------+--------------+
PROGRESS: | Query points | # Pairs | % Complete. | Elapsed Time |
PROGRESS: +--------------+---------+-------------+--------------+
PROGRESS: | 0            | 1       | 0.0498753   | 12.833ms     |
PROGRESS: | Done         |         | 100         | 251.998ms    |
PROGRESS: +--------------+---------+-------------+--------------+



In [28]:

    
show_neighbors(1222)









    



PROGRESS: Starting pairwise querying.
PROGRESS: +--------------+---------+-------------+--------------+
PROGRESS: | Query points | # Pairs | % Complete. | Elapsed Time |
PROGRESS: +--------------+---------+-------------+--------------+
PROGRESS: | 0            | 1       | 0.0498753   | 19.876ms     |
PROGRESS: | Done         |         | 100         | 247.165ms    |
PROGRESS: +--------------+---------+-------------+--------------+



In [30]:

    
show_neighbors(2000)









    



PROGRESS: Starting pairwise querying.
PROGRESS: +--------------+---------+-------------+--------------+
PROGRESS: | Query points | # Pairs | % Complete. | Elapsed Time |
PROGRESS: +--------------+---------+-------------+--------------+
PROGRESS: | 0            | 1       | 0.0498753   | 16.089ms     |
PROGRESS: | Done         |         | 100         | 245.259ms    |
PROGRESS: +--------------+---------+-------------+--------------+

Computing summary statistics of the data



In [33]:

    
image_train['label'].sketch_summary()









    Out[33]:





+------------------+-------+----------+
|       item       | value | is exact |
+------------------+-------+----------+
|      Length      |  2005 |   Yes    |
| # Missing Values |   0   |   Yes    |
| # unique values  |   4   |    No    |
+------------------+-------+----------+

Most frequent items:
+-------+------------+-----+-----+------+
| value | automobile | cat | dog | bird |
+-------+------------+-----+-----+------+
| count |    509     | 509 | 509 | 478  |
+-------+------------+-----+-----+------+



In [ ]:

Creating category-specific image retrieval models



In [35]:

    
dog_image_train = image_train[image_train['label']=='dog']



In [69]:

    
cat_image_train = image_train[image_train['label']=='cat']
##cat_image_train.head()



In [70]:

    
auto_image_train = image_train[image_train['label']=='automobile']



In [71]:

    
bird_image_train = image_train[image_train['label']=='bird']

KNN Model creation for each category



In [41]:

    
dog_model = gl.nearest_neighbors.create(dog_image_train,
                                       features = ['deep_features'],
                                        label = 'id')









    



PROGRESS: Starting brute force nearest neighbors model training.



In [43]:

    
image_test = gl.SFrame('image_test_data/')



In [44]:

    
cat_model = gl.nearest_neighbors.create(cat_image_train,
                                       features = ['deep_features'],
                                        label = 'id')









    



PROGRESS: Starting brute force nearest neighbors model training.



In [45]:

    
auto_model = gl.nearest_neighbors.create(auto_image_train,
                                       features = ['deep_features'],
                                        label = 'id')









    



PROGRESS: Starting brute force nearest neighbors model training.



In [46]:

    
bird_model = gl.nearest_neighbors.create(bird_image_train,
                                       features = ['deep_features'],
                                        label = 'id')









    



PROGRESS: Starting brute force nearest neighbors model training.



In [57]:

    
query_result_cat = cat_model.query(image_test[0:1])
cat_image_train.filter_by(query_result_cat['reference_label'], 'id')['image'].show()









    



PROGRESS: Starting pairwise querying.
PROGRESS: +--------------+---------+-------------+--------------+
PROGRESS: | Query points | # Pairs | % Complete. | Elapsed Time |
PROGRESS: +--------------+---------+-------------+--------------+
PROGRESS: | 0            | 1       | 0.0498753   | 12.346ms     |
PROGRESS: | Done         |         | 100         | 214.92ms     |
PROGRESS: +--------------+---------+-------------+--------------+



In [61]:

    
query_result_dog = dog_model.query(image_test[0:1])
dog_image_train.filter_by(query_result_dog['reference_label'], 'id')['label']









    



PROGRESS: Starting pairwise querying.
PROGRESS: +--------------+---------+-------------+--------------+
PROGRESS: | Query points | # Pairs | % Complete. | Elapsed Time |
PROGRESS: +--------------+---------+-------------+--------------+
PROGRESS: | 0            | 1       | 0.196464    | 12.063ms     |
PROGRESS: | Done         |         | 100         | 88.099ms     |
PROGRESS: +--------------+---------+-------------+--------------+






    Out[61]:





dtype: str
Rows: 5
['dog', 'dog', 'dog', 'dog', 'dog']



In [ ]:

A simple example of nearest-neighbors classification



In [63]:

    
## mean to 
query_result_cat['distance'].mean()









    Out[63]:





36.15573070978294



In [76]:

    
query_result_dog['distance'].mean()
query_result_dog









    Out[76]:





    
        query_label
        reference_label
        distance
        rank
    
    
        0
        16976
        37.4642628784
        1
    
    
        0
        13387
        37.5666832169
        2
    
    
        0
        35867
        37.6047267079
        3
    
    
        0
        44603
        37.7065585153
        4
    
    
        0
        6094
        38.5113254907
        5
    

[5 rows x 4 columns]



In [ ]:

Computing NN accuracy using SFrame operations

Split the test dataset into different category



In [65]:

    
cat_test_image = image_test[image_test['label']=='cat']



In [66]:

    
dog_test_image = image_test[image_test['label']=='dog']



In [72]:

    
auto_test_image = image_test[image_test['label']=='automobile']
bird_test_image = image_test[image_test['label']=='bird']

Fin the NN in the training set for every for each category

More KNN model



In [ ]:



In [ ]:

reference_label	distance	rank
384	0.0	1
6910	36.9403137951	2
39777	38.4634888975	3
36870	39.7559623119	4
41734	39.7866014148	5

reference_label	distance	rank
16976	37.4642628784	1
13387	37.5666832169	2
35867	37.6047267079	3
44603	37.7065585153	4
6094	38.5113254907	5