GIFGIF is a project from the MIT media lab that aims at understanding the emotional content of animated GIF images. The project covers 17 emotions, including happiness, fear, amusement, shame, etc. To collect feedback from users, the web site shows two images at a time, and asks feedback as follows.
Which of the left or right image better expresses
[emotion]?
where [emotion] is one of 17 different possibilities.
Therefore, the raw data that is collected consists of outcomes of pairwise comparison between pairs of images.
Just the kind of data that choix is built for!
In this notebook, we will use choix to try making sense of the raw pairwise-comparison data.
In particular, we would like to embed the images on a scale (for a given emotion).
We will use a dump of the raw data available at http://lucas.maystre.ch/gifgif-data. Download and uncompress the dataset (you don't need to download the images).
In [1]:
import choix
import collections
import numpy as np
from IPython.display import Image, display
In [2]:
# Change this with the path to the data on your computer.
PATH_TO_DATA = "/tmp/gifgif/gifgif-dataset-20150121-v1.csv"
We also define a short utility function to display an image based on its identifier.
In [3]:
def show_gif(idx):
template = "http://media.giphy.com/media/{idx}/giphy.gif"
display(Image(url=template.format(idx=idx)))
# A random image.
show_gif("k39w535jFPYrK")
First, we need to transform the raw dataset into a format that choix can process.
Remember that choix encodes pairwise-comparison outcomes as tuples (i, j) (meaning "$i$ won over $j$"), and that items are assumed to be numbered by consecutive integers.
We begin by mapping all distinct images that appear in the dataset to consecutive integers.
In [4]:
# First pass over the data to transform GIFGIF IDs to consecutive integers.
image_ids = set()
with open(PATH_TO_DATA) as f:
next(f) # First line is header.
for line in f:
emotion, left, right, choice = line.strip().split(",")
if len(left) > 0 and len(right) > 0:
# `if` condition eliminates corrupted data.
image_ids.add(left)
image_ids.add(right)
int_to_idx = dict(enumerate(image_ids))
idx_to_int = dict((v, k) for k, v in int_to_idx.items())
n_items = len(idx_to_int)
print("Number of distinct images: {:,}".format(n_items))
Next, we parse the comparisons in the data and convert the image IDs to the corresponding integers. We collect all the comparisons and filter them by emotion.
In [5]:
data = collections.defaultdict(list)
with open(PATH_TO_DATA) as f:
next(f) # First line is header.
for line in f:
emotion, left, right, choice = line.strip().split(",")
if len(left) == 0 or len(right) == 0:
# Datum is corrupted, continue.
continue
# Map ids to integers.
left = idx_to_int[left]
right = idx_to_int[right]
if choice == "left":
# Left image won the comparison.
data[emotion].append((left, right))
if choice == "right":
# Right image won the comparison.
data[emotion].append((right, left))
print("Number of comparisons for each emotion")
for emotion, comps in data.items():
print("{: <14} {: >7,}".format(emotion, len(comps)))
Now, we are ready to fit a Bradley-Terry model to the data, in order to be able to embed the images on a quantitative scale (for a given emotion). In the following, we consider happiness.
In [6]:
# What does the data look like?
data["happiness"][:3]
Out[6]:
In [7]:
%%time
params = choix.opt_pairwise(n_items, data["happiness"])
The parameters induce a ranking over the images. Images ranked at the bottom are consistently found to express less happiness, and vice-versa for images ranked at the top.
In [8]:
ranking = np.argsort(params)
In [9]:
for i in ranking[::-1][:3]:
show_gif(int_to_idx[i])
The top three images that *least express happiness are the following:
In [10]:
for i in ranking[:3]:
show_gif(int_to_idx[i])
In [11]:
rank = 2500
top = ranking[::-1][rank]
show_gif(int_to_idx[top])
bottom = ranking[rank]
show_gif(int_to_idx[bottom])
In [12]:
prob_top_wins, _ = choix.probabilities((top, bottom), params)
print("Prob(user selects top image) = {:.2f}".format(prob_top_wins))