This dataset of ~100K paintings and associated genre and style labels (and free-form tags) is scraped from http://wikipaintings.org.
In [1]:
%load_ext autoreload
%autoreload 2
import pandas as pd
import vislab.dataset
import vislab.datasets
import vislab.dataset_viz
import vislab.dataset_stats
import re
/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/configobj.py:145: DeprecationWarning: The compiler package is deprecated and removed in Python 3.x.
import compiler
Load basic dataset, which is scraped entirely from the search results page and includes the page urls of the artworks.
In [4]:
basic_df = vislab.datasets.wikipaintings.get_basic_df()
basic_df.iloc[:5]
Out[4]:
artist_slug
artwork_slug
image_id
page_url
image_id
nicholas-roerich_buddha-s-test-1927
nicholas-roerich
buddha-s-test-1927
nicholas-roerich_buddha-s-test-1927
http://www.wikipaintings.org/en/nicholas-roeri...
sam-francis_2-impressions-swatch-1992
sam-francis
2-impressions-swatch-1992
sam-francis_2-impressions-swatch-1992
http://www.wikipaintings.org/en/sam-francis/2-...
pierre-auguste-renoir_a-young-woman-1875
pierre-auguste-renoir
a-young-woman-1875
pierre-auguste-renoir_a-young-woman-1875
http://www.wikipaintings.org/en/pierre-auguste...
sam-francis_abstract-composition-sf-61-1967
sam-francis
abstract-composition-sf-61-1967
sam-francis_abstract-composition-sf-61-1967
http://www.wikipaintings.org/en/sam-francis/ab...
jesus-rafael-soto_abstract-composition-with-silver-and-magenta-1970
jesus-rafael-soto
abstract-composition-with-silver-and-magenta-1970
jesus-rafael-soto_abstract-composition-with-si...
http://www.wikipaintings.org/en/jesus-rafael-s...
5 rows × 4 columns
Load detailed dataset, which is scraped from the individual artwork pages and includes the labels and image urls of the artworks.
In [5]:
df = vislab.datasets.wikipaintings.get_df()
print df.shape
df.iloc[:5]
(101086, 11)
Out[5]:
image_id
artist_slug
artwork_slug
date
genre
style
technique
keywords
name
page_url
image_url
paul-reed_12-1964
paul-reed_12-1964
paul-reed
12-1964
1964
abstract painting
Abstract Expressionism
NaN
NaN
#12
http://www.wikipaintings.org/en/paul-reed/12-1964
http://uploads5.wikipaintings.org/images/paul-...
john-mccracken_10s
john-mccracken_10s
john-mccracken
10s
NaN
installation
Light and Space
NaN
NaN
10S
http://www.wikipaintings.org/en/john-mccracken...
http://uploads6.wikipaintings.org/images/john-...
nicholas-roerich_buddha-s-test-1927
nicholas-roerich_buddha-s-test-1927
nicholas-roerich
buddha-s-test-1927
1927
religious painting
Symbolism
NaN
[seas-and-oceans, Buddhism, Tibetian Mytholo...
http://www.wikipaintings.org/en/nicholas-roeri...
http://uploads5.wikipaintings.org/images/nicho...
sam-francis_2-impressions-swatch-1992
sam-francis_2-impressions-swatch-1992
sam-francis
2-impressions-swatch-1992
1992
abstract painting
Lyrical Abstraction
lithograph
NaN
2 Impressions: Swatch
http://www.wikipaintings.org/en/sam-francis/2-...
http://uploads7.wikipaintings.org/images/sam-f...
pierre-auguste-renoir_a-young-woman-1875
pierre-auguste-renoir_a-young-woman-1875
pierre-auguste-renoir
a-young-woman-1875
1875
portrait
Impressionism
NaN
NaN
A Young Woman
http://www.wikipaintings.org/en/pierre-auguste...
http://uploads1.wikipaintings.org/images/pierr...
5 rows × 11 columns
Display an image for fun.
In [6]:
from skimage.io import imread, imshow
imshow(imread(df.iloc[30000]['image_url']))
Expand the style label into own DataFrame.
In [16]:
#df.index.name = 'image_id'
#del df['image_id']
#df.to_csv('wikipaintings_oct2013.csv', encoding='utf-8')
In [15]:
df_style = vislab.datasets.wikipaintings.get_style_df(1000)
In [21]:
df_style
Out[21]:
style_Impressionism
style_Realism
style_Romanticism
style_Post-Impressionism
style_Expressionism
style_Surrealism
style_Art_Nouveau_(Modern)
style_Symbolism
style_Baroque
style_Neoclassicism
style_Northern_Renaissance
style_Nave_Art_(Primitivism)
style_Abstract_Expressionism
style_Rococo
style_Cubism
style_Early_Renaissance
style_High_Renaissance
style_Minimalism
style_Color_Field_Painting
style_Mannerism_(Late_Renaissance)
paul-reed_12-1964
False
False
False
False
False
False
False
False
False
False
False
False
True
False
False
False
False
False
False
False
...
nicholas-roerich_buddha-s-test-1927
False
False
False
False
False
False
False
True
False
False
False
False
False
False
False
False
False
False
False
False
...
pierre-auguste-renoir_a-young-woman-1875
True
False
False
False
False
False
False
False
False
False
False
False
False
False
False
False
False
False
False
False
...
sam-francis_abstract-composition-sf-61-1967
False
False
False
False
False
False
False
False
False
False
False
False
True
False
False
False
False
False
False
False
...
nikolaos-gyzis_after-the-destruction-of-psara-1898
False
False
True
False
False
False
False
False
False
False
False
False
False
False
False
False
False
False
False
False
...
alfred-stevens_all-happiness
False
False
True
False
False
False
False
False
False
False
False
False
False
False
False
False
False
False
False
False
...
john-singleton-copley_ann-tyng-1756
False
False
False
False
False
False
False
False
False
True
False
False
False
False
False
False
False
False
False
False
...
john-singleton-copley_anne-erving-mrs-duncan-stewart
False
False
False
False
False
False
False
False
False
True
False
False
False
False
False
False
False
False
False
False
...
sam-francis_another-disappearance-1963
False
False
False
False
False
False
False
False
False
False
False
False
True
False
False
False
False
False
False
False
...
paul-reed_18-l-1965
False
False
False
False
False
False
False
False
False
False
False
False
False
False
False
False
False
False
True
False
...
paul-reed_18r-1965
False
False
False
False
False
False
False
False
False
False
False
False
False
False
False
False
False
False
True
False
...
john-singleton-copley_eunice-dennie-burr-1760
False
False
False
False
False
False
False
False
False
True
False
False
False
False
False
False
False
False
False
False
...
sam-francis_2-pri-rain-1964
False
False
False
False
False
False
False
False
False
False
False
False
True
False
False
False
False
False
False
False
...
john-singleton-copley_ezekiel-goldthwait-1771
False
False
False
False
False
False
False
False
False
True
False
False
False
False
False
False
False
False
False
False
...
paul-reed_14d-1964
False
False
False
False
False
False
False
False
False
False
False
False
True
False
False
False
False
False
False
False
...
paul-reed_20-1962
False
False
False
False
False
False
False
False
False
False
False
False
True
False
False
False
False
False
False
False
...
karl-benjamin_20-1977
False
False
False
False
False
False
False
False
False
False
False
False
False
False
False
False
False
False
True
False
...
paul-reed_21c-1964
False
False
False
False
False
False
False
False
False
False
False
False
True
False
False
False
False
False
False
False
...
cornelis-vreedenburgh_fishermen
True
False
False
False
False
False
False
False
False
False
False
False
False
False
False
False
False
False
False
False
...
paul-reed_16f-1963
False
False
False
False
False
False
False
False
False
False
False
False
True
False
False
False
False
False
False
False
...
paul-reed_28-1963
False
False
False
False
False
False
False
False
False
False
False
False
True
False
False
False
False
False
False
False
...
paul-reed_16j-1963
False
False
False
False
False
False
False
False
False
False
False
False
True
False
False
False
False
False
False
False
...
paul-reed_29c-1964
False
False
False
False
False
False
False
False
False
False
False
False
True
False
False
False
False
False
False
False
...
paul-reed_17-1964
False
False
False
False
False
False
False
False
False
False
False
False
True
False
False
False
False
False
False
False
...
john-singleton-copley_george-augustus-eliott-1st-baron-heathfield
False
False
False
False
False
False
False
False
False
True
False
False
False
False
False
False
False
False
False
False
...
pablo-picasso_tricorne-study-1919
False
False
False
False
False
False
False
False
False
True
False
False
False
False
False
False
False
False
False
False
...
perle-fine_cool-series-no-26-first-love-1963
False
False
False
False
False
False
False
False
False
False
False
False
False
False
False
False
False
False
True
False
...
maurice-utrillo_belle-gabrielle
False
False
False
True
False
False
False
False
False
False
False
False
False
False
False
False
False
False
False
False
...
paul-reed_17dr-1965
False
False
False
False
False
False
False
False
False
False
False
False
False
False
False
False
False
False
True
False
...
john-singleton-copley_george-iv-1809-1
False
False
False
False
False
False
False
False
False
True
False
False
False
False
False
False
False
False
False
False
...
john-singleton-copley_john-spooner-1763
False
False
False
False
False
False
False
False
False
True
False
False
False
False
False
False
False
False
False
False
...
john-singleton-copley_thomas-aston-coffin-1758
False
False
False
False
False
False
False
False
False
True
False
False
False
False
False
False
False
False
False
False
...
max-liebermann_road-in-dutch-village-1885
True
False
False
False
False
False
False
False
False
False
False
False
False
False
False
False
False
False
False
False
...
john-singleton-copley_mr-joseph-barrell-hannah-fitch-1771
False
False
False
False
False
False
False
False
False
True
False
False
False
False
False
False
False
False
False
False
...
sam-francis_untitled-sf-358-1993
False
False
False
False
False
False
False
False
False
False
False
False
True
False
False
False
False
False
False
False
...
john-singleton-copley_portrait-of-mrs-benjamin-pickman-mary-toppan
False
False
False
False
False
False
False
False
False
True
False
False
False
False
False
False
False
False
False
False
...
john-singleton-copley_painting-of-samuel-learning-from-eli-1780
False
False
False
False
False
False
False
False
False
True
False
False
False
False
False
False
False
False
False
False
...
caspar-david-friedrich_not-detected-8
False
False
True
False
False
False
False
False
False
False
False
False
False
False
False
False
False
False
False
False
...
perle-fine_cool-series-no-29-cool-blue-cold-green-1963
False
False
False
False
False
False
False
False
False
False
False
False
False
False
False
False
False
False
True
False
...
paul-reed_17f-1963
False
False
False
False
False
False
False
False
False
False
False
False
True
False
False
False
False
False
False
False
...
john-singleton-copley_governor-john-wentworth
False
False
False
False
False
False
False
False
False
True
False
False
False
False
False
False
False
False
False
False
...
john-singleton-copley_mrs-moses-gill-rebecca-boylston-1773
False
False
False
False
False
False
False
False
False
True
False
False
False
False
False
False
False
False
False
False
...
paul-cezanne_still-life-with-bottle-and-apple-basket-1894
False
False
False
True
False
False
False
False
False
False
False
False
False
False
False
False
False
False
False
False
...
john-singleton-copley_paul-revere-1770
False
False
False
False
False
False
False
False
False
True
False
False
False
False
False
False
False
False
False
False
...
john-singleton-copley_henry-addington-first-viscount-sidmouth-1798
False
False
False
False
False
False
False
False
False
True
False
False
False
False
False
False
False
False
False
False
...
john-singleton-copley_portrait-of-mr-george-turner
False
False
False
False
False
False
False
False
False
True
False
False
False
False
False
False
False
False
False
False
...
john-singleton-copley_lucretia-hubbard-towsend-1765
False
False
False
False
False
False
False
False
False
True
False
False
False
False
False
False
False
False
False
False
...
nikolaos-gyzis_boy-with-cherries-1888
False
True
False
False
False
False
False
False
False
False
False
False
False
False
False
False
False
False
False
False
...
lynd-ward_not-detected-272486
False
False
False
False
True
False
False
False
False
False
False
False
False
False
False
False
False
False
False
False
...
utagawa-kunisada_kagamiiwa-hamanosuke-sumo-wrestler
False
False
False
False
False
False
False
False
False
False
False
False
False
False
False
False
False
False
False
False
...
billy-childish_tea-drinker-high-atlas-2008
False
False
False
False
True
False
False
False
False
False
False
False
False
False
False
False
False
False
False
False
...
perle-fine_cool-series-no-36-rough-hewn-1963
False
False
False
False
False
False
False
False
False
False
False
False
False
False
False
False
False
False
True
False
...
perle-fine_cool-series-black-over-green-1963
False
False
False
False
False
False
False
False
False
False
False
False
False
False
False
False
False
False
True
False
...
nikolaos-gyzis_grandma-and-children-1883
False
True
False
False
False
False
False
False
False
False
False
False
False
False
False
False
False
False
False
False
...
john-singleton-copley_mrs-nathaniel-allen-sarah-sargnet
False
False
False
False
False
False
False
False
False
True
False
False
False
False
False
False
False
False
False
False
...
john-singleton-copley_samuel-barrington
False
False
False
False
False
False
False
False
False
True
False
False
False
False
False
False
False
False
False
False
...
john-singleton-copley_henry-pelham
False
False
False
False
False
False
False
False
False
True
False
False
False
False
False
False
False
False
False
False
...
john-singleton-copley_portrait-of-mrs-john-stevens-judith-sargent-later-mr-john-murray-1772
False
False
False
False
False
False
False
False
False
True
False
False
False
False
False
False
False
False
False
False
...
lynd-ward_not-detected-272479
False
False
False
False
True
False
False
False
False
False
False
False
False
False
False
False
False
False
False
False
...
john-singleton-copley_boy-with-squirrel-henry-pelham-1765
False
False
False
False
False
False
False
False
False
True
False
False
False
False
False
False
False
False
False
False
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
82459 rows × 26 columns
In [14]:
df['image_url'].iloc[0]
Out[14]:
'http://uploads5.wikipaintings.org/images/paul-reed/12-1964.jpg'
In [7]:
fig = vislab.dataset_viz.plot_column_frequencies(df, 'style', 30)
fig.savefig('/Users/sergeyk/work/aphrodite-writeup/figures/wikipaintings_style.pdf', bbox_inches='tight')
fig = vislab.dataset_viz.plot_column_frequencies(df, 'genre', 20)
fig.savefig('/Users/sergeyk/work/aphrodite-writeup/figures/wikipaintings_genre.pdf', bbox_inches='tight')
fig = vislab.dataset_viz.plot_column_frequencies(df, 'technique', 20)
fig = vislab.dataset_viz.plot_column_frequencies(df, 'artist_slug', 30)
fig = vislab.dataset_viz.plot_column_frequencies(df, 'artist_slug', 60)
In [8]:
import vislab.gg
def plot_date_distribution(df):
dates = df['date'].dropna().apply(lambda x: int(re.search('(\d+)', x).groups()[0]))
filtered_dates = dates[(dates > 1400) & (dates < 2015)]
fig = plt.figure(figsize=(12,4))
ax = fig.add_subplot(111)
ax.hist(filtered_dates, bins=50)
ax.set_xlim([1400, 2020])
ax.set_title('Date Frequency')
ax.set_xlabel('Year')
vislab.gg.rstyle(ax)
return fig
fig = plot_date_distribution(df)
fig.savefig('/Users/sergeyk/work/aphrodite-writeup/figures/wikipaintings_date.pdf', bbox_inches='tight')
/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/matplotlib/axes.py:4747: UserWarning: No labeled objects found. Use label='...' kwarg on individual plots.
warnings.warn("No labeled objects found. "
In [12]:
df_m = vislab.dataset_stats.get_joint_occurrence_df(df, 'style', 'genre', top_k=10)
df_m
get_joint_occurrence_df: looking at co-occurrence of style and genre
dropped 0/46821 rows that don't have both vals in top-10.
Out[12]:
portrait
landscape
genre painting
abstract painting
religious painting
cityscape
sketch and study
still life
symbolic painting
figurative painting
Impressionism
1730
4205
2507
0
18
1463
232
410
5
45
Realism
2526
2631
1524
0
128
392
1161
342
3
6
Romanticism
1487
1124
828
0
403
341
226
31
16
3
Post-Impressionism
765
1595
822
2
13
833
473
562
17
48
Expressionism
1615
560
1036
53
66
297
226
265
111
254
Surrealism
363
391
254
293
133
105
124
131
1481
534
Art Nouveau (Modern)
512
266
593
1
111
178
151
11
81
1
Symbolism
281
1000
386
0
522
137
264
35
411
13
Baroque
898
170
743
1
1003
54
195
119
1
0
Neoclassicism
580
42
203
0
134
351
137
24
3
0
In [13]:
fig = vislab.dataset_viz.plot_conditional_occurrence(df_m)
In [14]:
fig = vislab.dataset_viz.plot_conditional_occurrence(df_m.T)
Content source: Jai-Chaudhary/vislab
Similar notebooks: