Wikipaintings

This dataset of ~100K paintings and associated genre and style labels (and free-form tags) is scraped from http://wikipaintings.org.


In [1]:
%load_ext autoreload
%autoreload 2
import pandas as pd
import vislab.dataset
import vislab.datasets
import vislab.dataset_viz
import vislab.dataset_stats
import re


/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/configobj.py:145: DeprecationWarning: The compiler package is deprecated and removed in Python 3.x.
  import compiler

Load basic dataset, which is scraped entirely from the search results page and includes the page urls of the artworks.


In [4]:
basic_df = vislab.datasets.wikipaintings.get_basic_df()
basic_df.iloc[:5]


Out[4]:
artist_slug artwork_slug image_id page_url
image_id
nicholas-roerich_buddha-s-test-1927 nicholas-roerich buddha-s-test-1927 nicholas-roerich_buddha-s-test-1927 http://www.wikipaintings.org/en/nicholas-roeri...
sam-francis_2-impressions-swatch-1992 sam-francis 2-impressions-swatch-1992 sam-francis_2-impressions-swatch-1992 http://www.wikipaintings.org/en/sam-francis/2-...
pierre-auguste-renoir_a-young-woman-1875 pierre-auguste-renoir a-young-woman-1875 pierre-auguste-renoir_a-young-woman-1875 http://www.wikipaintings.org/en/pierre-auguste...
sam-francis_abstract-composition-sf-61-1967 sam-francis abstract-composition-sf-61-1967 sam-francis_abstract-composition-sf-61-1967 http://www.wikipaintings.org/en/sam-francis/ab...
jesus-rafael-soto_abstract-composition-with-silver-and-magenta-1970 jesus-rafael-soto abstract-composition-with-silver-and-magenta-1970 jesus-rafael-soto_abstract-composition-with-si... http://www.wikipaintings.org/en/jesus-rafael-s...

5 rows × 4 columns

Load detailed dataset, which is scraped from the individual artwork pages and includes the labels and image urls of the artworks.


In [5]:
df = vislab.datasets.wikipaintings.get_df()
print df.shape
df.iloc[:5]


(101086, 11)
Out[5]:
image_id artist_slug artwork_slug date genre style technique keywords name page_url image_url
paul-reed_12-1964 paul-reed_12-1964 paul-reed 12-1964 1964 abstract painting Abstract Expressionism NaN NaN #12 http://www.wikipaintings.org/en/paul-reed/12-1964 http://uploads5.wikipaintings.org/images/paul-...
john-mccracken_10s john-mccracken_10s john-mccracken 10s NaN installation Light and Space NaN NaN 10S http://www.wikipaintings.org/en/john-mccracken... http://uploads6.wikipaintings.org/images/john-...
nicholas-roerich_buddha-s-test-1927 nicholas-roerich_buddha-s-test-1927 nicholas-roerich buddha-s-test-1927 1927 religious painting Symbolism NaN [seas-and-oceans, Buddhism, Tibetian Mytholo... http://www.wikipaintings.org/en/nicholas-roeri... http://uploads5.wikipaintings.org/images/nicho...
sam-francis_2-impressions-swatch-1992 sam-francis_2-impressions-swatch-1992 sam-francis 2-impressions-swatch-1992 1992 abstract painting Lyrical Abstraction lithograph NaN 2 Impressions: Swatch http://www.wikipaintings.org/en/sam-francis/2-... http://uploads7.wikipaintings.org/images/sam-f...
pierre-auguste-renoir_a-young-woman-1875 pierre-auguste-renoir_a-young-woman-1875 pierre-auguste-renoir a-young-woman-1875 1875 portrait Impressionism NaN NaN A Young Woman http://www.wikipaintings.org/en/pierre-auguste... http://uploads1.wikipaintings.org/images/pierr...

5 rows × 11 columns

Display an image for fun.


In [6]:
from skimage.io import imread, imshow
imshow(imread(df.iloc[30000]['image_url']))


Expand the style label into own DataFrame.


In [16]:
#df.index.name = 'image_id'
#del df['image_id']
#df.to_csv('wikipaintings_oct2013.csv', encoding='utf-8')

In [15]:
df_style = vislab.datasets.wikipaintings.get_style_df(1000)

In [21]:
df_style


Out[21]:
style_Impressionism style_Realism style_Romanticism style_Post-Impressionism style_Expressionism style_Surrealism style_Art_Nouveau_(Modern) style_Symbolism style_Baroque style_Neoclassicism style_Northern_Renaissance style_Nave_Art_(Primitivism) style_Abstract_Expressionism style_Rococo style_Cubism style_Early_Renaissance style_High_Renaissance style_Minimalism style_Color_Field_Painting style_Mannerism_(Late_Renaissance)
paul-reed_12-1964 False False False False False False False False False False False False True False False False False False False False ...
nicholas-roerich_buddha-s-test-1927 False False False False False False False True False False False False False False False False False False False False ...
pierre-auguste-renoir_a-young-woman-1875 True False False False False False False False False False False False False False False False False False False False ...
sam-francis_abstract-composition-sf-61-1967 False False False False False False False False False False False False True False False False False False False False ...
nikolaos-gyzis_after-the-destruction-of-psara-1898 False False True False False False False False False False False False False False False False False False False False ...
alfred-stevens_all-happiness False False True False False False False False False False False False False False False False False False False False ...
john-singleton-copley_ann-tyng-1756 False False False False False False False False False True False False False False False False False False False False ...
john-singleton-copley_anne-erving-mrs-duncan-stewart False False False False False False False False False True False False False False False False False False False False ...
sam-francis_another-disappearance-1963 False False False False False False False False False False False False True False False False False False False False ...
paul-reed_18-l-1965 False False False False False False False False False False False False False False False False False False True False ...
paul-reed_18r-1965 False False False False False False False False False False False False False False False False False False True False ...
john-singleton-copley_eunice-dennie-burr-1760 False False False False False False False False False True False False False False False False False False False False ...
sam-francis_2-pri-rain-1964 False False False False False False False False False False False False True False False False False False False False ...
john-singleton-copley_ezekiel-goldthwait-1771 False False False False False False False False False True False False False False False False False False False False ...
paul-reed_14d-1964 False False False False False False False False False False False False True False False False False False False False ...
paul-reed_20-1962 False False False False False False False False False False False False True False False False False False False False ...
karl-benjamin_20-1977 False False False False False False False False False False False False False False False False False False True False ...
paul-reed_21c-1964 False False False False False False False False False False False False True False False False False False False False ...
cornelis-vreedenburgh_fishermen True False False False False False False False False False False False False False False False False False False False ...
paul-reed_16f-1963 False False False False False False False False False False False False True False False False False False False False ...
paul-reed_28-1963 False False False False False False False False False False False False True False False False False False False False ...
paul-reed_16j-1963 False False False False False False False False False False False False True False False False False False False False ...
paul-reed_29c-1964 False False False False False False False False False False False False True False False False False False False False ...
paul-reed_17-1964 False False False False False False False False False False False False True False False False False False False False ...
john-singleton-copley_george-augustus-eliott-1st-baron-heathfield False False False False False False False False False True False False False False False False False False False False ...
pablo-picasso_tricorne-study-1919 False False False False False False False False False True False False False False False False False False False False ...
perle-fine_cool-series-no-26-first-love-1963 False False False False False False False False False False False False False False False False False False True False ...
maurice-utrillo_belle-gabrielle False False False True False False False False False False False False False False False False False False False False ...
paul-reed_17dr-1965 False False False False False False False False False False False False False False False False False False True False ...
john-singleton-copley_george-iv-1809-1 False False False False False False False False False True False False False False False False False False False False ...
john-singleton-copley_john-spooner-1763 False False False False False False False False False True False False False False False False False False False False ...
john-singleton-copley_thomas-aston-coffin-1758 False False False False False False False False False True False False False False False False False False False False ...
max-liebermann_road-in-dutch-village-1885 True False False False False False False False False False False False False False False False False False False False ...
john-singleton-copley_mr-joseph-barrell-hannah-fitch-1771 False False False False False False False False False True False False False False False False False False False False ...
sam-francis_untitled-sf-358-1993 False False False False False False False False False False False False True False False False False False False False ...
john-singleton-copley_portrait-of-mrs-benjamin-pickman-mary-toppan False False False False False False False False False True False False False False False False False False False False ...
john-singleton-copley_painting-of-samuel-learning-from-eli-1780 False False False False False False False False False True False False False False False False False False False False ...
caspar-david-friedrich_not-detected-8 False False True False False False False False False False False False False False False False False False False False ...
perle-fine_cool-series-no-29-cool-blue-cold-green-1963 False False False False False False False False False False False False False False False False False False True False ...
paul-reed_17f-1963 False False False False False False False False False False False False True False False False False False False False ...
john-singleton-copley_governor-john-wentworth False False False False False False False False False True False False False False False False False False False False ...
john-singleton-copley_mrs-moses-gill-rebecca-boylston-1773 False False False False False False False False False True False False False False False False False False False False ...
paul-cezanne_still-life-with-bottle-and-apple-basket-1894 False False False True False False False False False False False False False False False False False False False False ...
john-singleton-copley_paul-revere-1770 False False False False False False False False False True False False False False False False False False False False ...
john-singleton-copley_henry-addington-first-viscount-sidmouth-1798 False False False False False False False False False True False False False False False False False False False False ...
john-singleton-copley_portrait-of-mr-george-turner False False False False False False False False False True False False False False False False False False False False ...
john-singleton-copley_lucretia-hubbard-towsend-1765 False False False False False False False False False True False False False False False False False False False False ...
nikolaos-gyzis_boy-with-cherries-1888 False True False False False False False False False False False False False False False False False False False False ...
lynd-ward_not-detected-272486 False False False False True False False False False False False False False False False False False False False False ...
utagawa-kunisada_kagamiiwa-hamanosuke-sumo-wrestler False False False False False False False False False False False False False False False False False False False False ...
billy-childish_tea-drinker-high-atlas-2008 False False False False True False False False False False False False False False False False False False False False ...
perle-fine_cool-series-no-36-rough-hewn-1963 False False False False False False False False False False False False False False False False False False True False ...
perle-fine_cool-series-black-over-green-1963 False False False False False False False False False False False False False False False False False False True False ...
nikolaos-gyzis_grandma-and-children-1883 False True False False False False False False False False False False False False False False False False False False ...
john-singleton-copley_mrs-nathaniel-allen-sarah-sargnet False False False False False False False False False True False False False False False False False False False False ...
john-singleton-copley_samuel-barrington False False False False False False False False False True False False False False False False False False False False ...
john-singleton-copley_henry-pelham False False False False False False False False False True False False False False False False False False False False ...
john-singleton-copley_portrait-of-mrs-john-stevens-judith-sargent-later-mr-john-murray-1772 False False False False False False False False False True False False False False False False False False False False ...
lynd-ward_not-detected-272479 False False False False True False False False False False False False False False False False False False False False ...
john-singleton-copley_boy-with-squirrel-henry-pelham-1765 False False False False False False False False False True False False False False False False False False False False ...
... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...

82459 rows × 26 columns

Style, Genre, Technique, and Artist distribution plots.


In [14]:
df['image_url'].iloc[0]


Out[14]:
'http://uploads5.wikipaintings.org/images/paul-reed/12-1964.jpg'

In [7]:
fig = vislab.dataset_viz.plot_column_frequencies(df, 'style', 30)
fig.savefig('/Users/sergeyk/work/aphrodite-writeup/figures/wikipaintings_style.pdf', bbox_inches='tight')
fig = vislab.dataset_viz.plot_column_frequencies(df, 'genre', 20)
fig.savefig('/Users/sergeyk/work/aphrodite-writeup/figures/wikipaintings_genre.pdf', bbox_inches='tight')

fig = vislab.dataset_viz.plot_column_frequencies(df, 'technique', 20)
fig = vislab.dataset_viz.plot_column_frequencies(df, 'artist_slug', 30)
fig = vislab.dataset_viz.plot_column_frequencies(df, 'artist_slug', 60)


Date distribution plot.


In [8]:
import vislab.gg

def plot_date_distribution(df):
    dates = df['date'].dropna().apply(lambda x: int(re.search('(\d+)', x).groups()[0]))
    
    filtered_dates = dates[(dates > 1400) & (dates < 2015)]
    fig = plt.figure(figsize=(12,4))
    ax = fig.add_subplot(111)
    ax.hist(filtered_dates, bins=50)
    ax.set_xlim([1400, 2020])
    ax.set_title('Date Frequency')
    ax.set_xlabel('Year')
    vislab.gg.rstyle(ax)
    return fig
    
fig = plot_date_distribution(df)
fig.savefig('/Users/sergeyk/work/aphrodite-writeup/figures/wikipaintings_date.pdf', bbox_inches='tight')


/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/matplotlib/axes.py:4747: UserWarning: No labeled objects found. Use label='...' kwarg on individual plots.
  warnings.warn("No labeled objects found. "

Prevalence of style given genre and genre given style.


In [12]:
df_m = vislab.dataset_stats.get_joint_occurrence_df(df, 'style', 'genre', top_k=10)
df_m


get_joint_occurrence_df: looking at co-occurrence of style and genre
  dropped 0/46821 rows that don't have both vals in top-10.
Out[12]:
portrait landscape genre painting abstract painting religious painting cityscape sketch and study still life symbolic painting figurative painting
Impressionism 1730 4205 2507 0 18 1463 232 410 5 45
Realism 2526 2631 1524 0 128 392 1161 342 3 6
Romanticism 1487 1124 828 0 403 341 226 31 16 3
Post-Impressionism 765 1595 822 2 13 833 473 562 17 48
Expressionism 1615 560 1036 53 66 297 226 265 111 254
Surrealism 363 391 254 293 133 105 124 131 1481 534
Art Nouveau (Modern) 512 266 593 1 111 178 151 11 81 1
Symbolism 281 1000 386 0 522 137 264 35 411 13
Baroque 898 170 743 1 1003 54 195 119 1 0
Neoclassicism 580 42 203 0 134 351 137 24 3 0

Occurrence of genre conditioned on style.


In [13]:
fig = vislab.dataset_viz.plot_conditional_occurrence(df_m)


Occurrence of style conditioned on genre.


In [14]:
fig = vislab.dataset_viz.plot_conditional_occurrence(df_m.T)