In [1]:
import graphlab

In [2]:
song_data = graphlab.SFrame('song_data.gl/')


[INFO] This non-commercial license of GraphLab Create is assigned to satishkt@gmail.comand will expire on September 24, 2016. For commercial licensing options, visit https://dato.com/buy/.

[INFO] Start server at: ipc:///tmp/graphlab_server-19075 - Server binary: /Users/Satish/.graphlab/anaconda/lib/python2.7/site-packages/graphlab/unity_server - Server log: /tmp/graphlab_server_1445800696.log
[INFO] GraphLab Server Version: 1.6.1

In [4]:
song_data.head(5)


Out[4]:
user_id song_id listen_count title artist
b80344d063b5ccb3212f76538
f3d9e43d87dca9e ...
SOAKIMP12A8C130995 1 The Cove Jack Johnson
b80344d063b5ccb3212f76538
f3d9e43d87dca9e ...
SOBBMDR12A8C13253B 2 Entre Dos Aguas Paco De Lucia
b80344d063b5ccb3212f76538
f3d9e43d87dca9e ...
SOBXHDL12A81C204C0 1 Stronger Kanye West
b80344d063b5ccb3212f76538
f3d9e43d87dca9e ...
SOBYHAJ12A6701BF1D 1 Constellations Jack Johnson
b80344d063b5ccb3212f76538
f3d9e43d87dca9e ...
SODACBL12A8C13C273 1 Learn To Fly Foo Fighters
song
The Cove - Jack Johnson
Entre Dos Aguas - Paco De
Lucia ...
Stronger - Kanye West
Constellations - Jack
Johnson ...
Learn To Fly - Foo
Fighters ...
[5 rows x 6 columns]


In [5]:
graphlab.canvas.set_target('ipynb')

In [6]:
song_data['song'].show()



In [7]:
len(song_data)


Out[7]:
1116609

In [8]:
users = song_data['user_id'].unique()

In [9]:
len(users)


Out[9]:
66346

In [10]:
train_data,test_data = song_data.random_split(0.8,seed=0)

Simple Popularity based recommender


In [11]:
popularity_model  = graphlab.popularity_recommender.create(train_data,user_id='user_id',item_id='song')


PROGRESS: Recsys training: model = popularity
PROGRESS: Warning: Ignoring columns song_id, listen_count, title, artist;
PROGRESS:     To use one of these as a target column, set target = <column_name>
PROGRESS:     and use a method that allows the use of a target.
PROGRESS: Preparing data set.
PROGRESS:     Data has 893580 observations with 66085 users and 9952 items.
PROGRESS:     Data prepared in: 2.46863s
PROGRESS: 893580 observations to process; with 9952 unique items.

In [12]:
popularity_model.recommend(users=[users[0]])


Out[12]:
user_id song score rank
c66c10a9567f0d82ff31441a9
fd5063e5cd9dfe8 ...
Sehr kosmisch - Harmonia 4754.0 1
c66c10a9567f0d82ff31441a9
fd5063e5cd9dfe8 ...
Undo - Björk 4227.0 2
c66c10a9567f0d82ff31441a9
fd5063e5cd9dfe8 ...
You're The One - Dwight
Yoakam ...
3781.0 3
c66c10a9567f0d82ff31441a9
fd5063e5cd9dfe8 ...
Dog Days Are Over (Radio
Edit) - Florence + The ...
3633.0 4
c66c10a9567f0d82ff31441a9
fd5063e5cd9dfe8 ...
Revelry - Kings Of Leon 3527.0 5
c66c10a9567f0d82ff31441a9
fd5063e5cd9dfe8 ...
Horn Concerto No. 4 in E
flat K495: II. Romance ...
3161.0 6
c66c10a9567f0d82ff31441a9
fd5063e5cd9dfe8 ...
Secrets - OneRepublic 3148.0 7
c66c10a9567f0d82ff31441a9
fd5063e5cd9dfe8 ...
Fireflies - Charttraxx
Karaoke ...
2532.0 8
c66c10a9567f0d82ff31441a9
fd5063e5cd9dfe8 ...
Tive Sim - Cartola 2521.0 9
c66c10a9567f0d82ff31441a9
fd5063e5cd9dfe8 ...
Drop The World - Lil
Wayne / Eminem ...
2053.0 10
[10 rows x 4 columns]


In [13]:
popularity_model.recommend(users=[users[1]])


Out[13]:
user_id song score rank
279292bb36dbfc7f505e36ebf
038c81eb1d1d63e ...
Sehr kosmisch - Harmonia 4754.0 1
279292bb36dbfc7f505e36ebf
038c81eb1d1d63e ...
Undo - Björk 4227.0 2
279292bb36dbfc7f505e36ebf
038c81eb1d1d63e ...
You're The One - Dwight
Yoakam ...
3781.0 3
279292bb36dbfc7f505e36ebf
038c81eb1d1d63e ...
Dog Days Are Over (Radio
Edit) - Florence + The ...
3633.0 4
279292bb36dbfc7f505e36ebf
038c81eb1d1d63e ...
Revelry - Kings Of Leon 3527.0 5
279292bb36dbfc7f505e36ebf
038c81eb1d1d63e ...
Horn Concerto No. 4 in E
flat K495: II. Romance ...
3161.0 6
279292bb36dbfc7f505e36ebf
038c81eb1d1d63e ...
Secrets - OneRepublic 3148.0 7
279292bb36dbfc7f505e36ebf
038c81eb1d1d63e ...
Hey_ Soul Sister - Train 2538.0 8
279292bb36dbfc7f505e36ebf
038c81eb1d1d63e ...
Fireflies - Charttraxx
Karaoke ...
2532.0 9
279292bb36dbfc7f505e36ebf
038c81eb1d1d63e ...
Tive Sim - Cartola 2521.0 10
[10 rows x 4 columns]

Build a personlized model


In [14]:
personalized_model = graphlab.item_similarity_recommender.create(train_data,user_id='user_id', item_id= 'song')


PROGRESS: Recsys training: model = item_similarity
PROGRESS: Warning: Ignoring columns song_id, listen_count, title, artist;
PROGRESS:     To use one of these as a target column, set target = <column_name>
PROGRESS:     and use a method that allows the use of a target.
PROGRESS: Preparing data set.
PROGRESS:     Data has 893580 observations with 66085 users and 9952 items.
PROGRESS:     Data prepared in: 1.30103s
PROGRESS: Computing item similarity statistics:
PROGRESS: Computing most similar items for 9952 items:
PROGRESS: +-----------------+-----------------+
PROGRESS: | Number of items | Elapsed Time    |
PROGRESS: +-----------------+-----------------+
PROGRESS: | 1000            | 1.63518         |
PROGRESS: | 2000            | 1.69402         |
PROGRESS: | 3000            | 1.7544          |
PROGRESS: | 4000            | 1.81566         |
PROGRESS: | 5000            | 1.87135         |
PROGRESS: | 6000            | 1.92585         |
PROGRESS: | 7000            | 1.9819          |
PROGRESS: | 8000            | 2.0365          |
PROGRESS: | 9000            | 2.10633         |
PROGRESS: +-----------------+-----------------+
PROGRESS: Finished training in 2.70224s

In [15]:
personalized_model.recommend(users=[users[0]])


Out[15]:
user_id song score rank
c66c10a9567f0d82ff31441a9
fd5063e5cd9dfe8 ...
Cuando Pase El Temblor -
Soda Stereo ...
0.0194504525792 1
c66c10a9567f0d82ff31441a9
fd5063e5cd9dfe8 ...
Fireflies - Charttraxx
Karaoke ...
0.0144855208012 2
c66c10a9567f0d82ff31441a9
fd5063e5cd9dfe8 ...
Love Is A Losing Game -
Amy Winehouse ...
0.0142865986808 3
c66c10a9567f0d82ff31441a9
fd5063e5cd9dfe8 ...
Marry Me - Train 0.0141507931471 4
c66c10a9567f0d82ff31441a9
fd5063e5cd9dfe8 ...
Secrets - OneRepublic 0.0135976104935 5
c66c10a9567f0d82ff31441a9
fd5063e5cd9dfe8 ...
Sehr kosmisch - Harmonia 0.0134255592232 6
c66c10a9567f0d82ff31441a9
fd5063e5cd9dfe8 ...
No Dejes Que... -
Caifanes ...
0.0134191754754 7
c66c10a9567f0d82ff31441a9
fd5063e5cd9dfe8 ...
Y solo se me ocurre
amarte (Unplugged) - ...
0.0133210385369 8
c66c10a9567f0d82ff31441a9
fd5063e5cd9dfe8 ...
Te Hacen Falta Vitaminas
- Soda Stereo ...
0.0129302853556 9
c66c10a9567f0d82ff31441a9
fd5063e5cd9dfe8 ...
OMG - Usher featuring
will.i.am ...
0.0127958729689 10
[10 rows x 4 columns]


In [16]:
personalized_model.recommend(users=[users[1]])


Out[16]:
user_id song score rank
279292bb36dbfc7f505e36ebf
038c81eb1d1d63e ...
Riot In Cell Block Number
Nine - Dr Feelgood ...
0.0375 1
279292bb36dbfc7f505e36ebf
038c81eb1d1d63e ...
Sei Lá Mangueira -
Elizeth Cardoso ...
0.0331632653061 2
279292bb36dbfc7f505e36ebf
038c81eb1d1d63e ...
The Stallion - Ween 0.0322580645161 3
279292bb36dbfc7f505e36ebf
038c81eb1d1d63e ...
Rain - Subhumans 0.0314159292035 4
279292bb36dbfc7f505e36ebf
038c81eb1d1d63e ...
West One (Shine On Me) -
The Ruts ...
0.0307080895662 5
279292bb36dbfc7f505e36ebf
038c81eb1d1d63e ...
Back Against The Wall -
Cage The Elephant ...
0.0301204819277 6
279292bb36dbfc7f505e36ebf
038c81eb1d1d63e ...
Life Less Frightening -
Rise Against ...
0.0284431137725 7
279292bb36dbfc7f505e36ebf
038c81eb1d1d63e ...
A Beggar On A Beach Of
Gold - Mike And The ...
0.0230283319543 8
279292bb36dbfc7f505e36ebf
038c81eb1d1d63e ...
Audience Of One - Rise
Against ...
0.0193938442211 9
279292bb36dbfc7f505e36ebf
038c81eb1d1d63e ...
Blame It On The Boogie -
The Jacksons ...
0.0189873417722 10
[10 rows x 4 columns]


In [17]:
personalized_model.get_similar_items(['With Or Without You - U2'])


PROGRESS: Getting similar items completed in 0.008747
Out[17]:
song similar score rank
With Or Without You - U2 I Still Haven't Found
What I'm Looking For ...
0.0428571428571 1
With Or Without You - U2 Hold Me_ Thrill Me_ Kiss
Me_ Kill Me - U2 ...
0.033734939759 2
With Or Without You - U2 Window In The Skies - U2 0.0328358208955 3
With Or Without You - U2 Vertigo - U2 0.0300751879699 4
With Or Without You - U2 Sunday Bloody Sunday - U2 0.0271317829457 5
With Or Without You - U2 Bad - U2 0.0251798561151 6
With Or Without You - U2 A Day Without Me - U2 0.0237154150198 7
With Or Without You - U2 Another Time Another
Place - U2 ...
0.020325203252 8
With Or Without You - U2 Walk On - U2 0.020202020202 9
With Or Without You - U2 Get On Your Boots - U2 0.0196850393701 10
[10 rows x 4 columns]


In [18]:
personalized_model.get_similar_items(['Chan Chan (Live) - Buena Vista Social Club'])


PROGRESS: Getting similar items completed in 0.00296
Out[18]:
song similar score rank
Chan Chan (Live) - Buena
Vista Social Club ...
Murmullo - Buena Vista
Social Club ...
0.188118811881 1
Chan Chan (Live) - Buena
Vista Social Club ...
La Bayamesa - Buena Vista
Social Club ...
0.187192118227 2
Chan Chan (Live) - Buena
Vista Social Club ...
Amor de Loca Juventud -
Buena Vista Social Club ...
0.184834123223 3
Chan Chan (Live) - Buena
Vista Social Club ...
Diferente - Gotan Project 0.0214592274678 4
Chan Chan (Live) - Buena
Vista Social Club ...
Mistica - Orishas 0.0205761316872 5
Chan Chan (Live) - Buena
Vista Social Club ...
Hotel California - Gipsy
Kings ...
0.019305019305 6
Chan Chan (Live) - Buena
Vista Social Club ...
Nací Orishas - Orishas 0.0191570881226 7
Chan Chan (Live) - Buena
Vista Social Club ...
Le Moulin - Yann Tiersen 0.0187969924812 8
Chan Chan (Live) - Buena
Vista Social Club ...
Gitana - Willie Colon 0.0187969924812 9
Chan Chan (Live) - Buena
Vista Social Club ...
Criminal - Gotan Project 0.018779342723 10
[10 rows x 4 columns]


In [19]:
%matplotlib inline

In [20]:
model_performance = graphlab.recommender.util.compare_models(test_data, [popularity_model,personalized_model],user_sample= 0.05)


compare_models: using 2931 users to estimate model performance
PROGRESS: Evaluate model M0
PROGRESS: recommendations finished on 1000/2931 queries. users per second: 10220.6
PROGRESS: recommendations finished on 2000/2931 queries. users per second: 12544.2

Precision and recall summary statistics by cutoff
+--------+-----------------+------------------+
| cutoff |  mean_precision |   mean_recall    |
+--------+-----------------+------------------+
|   1    | 0.0300238826339 | 0.00875167707307 |
|   2    |  0.029341521665 | 0.0168876599915  |
|   3    | 0.0252473558512 | 0.0205396573744  |
|   4    | 0.0224326168543 |  0.023732673464  |
|   5    | 0.0204025929717 | 0.0273456579623  |
|   6    | 0.0193904242011 | 0.0313331371801  |
|   7    | 0.0184724862309 | 0.0349760426654  |
|   8    |  0.01752814739  | 0.0380111742573  |
|   9    | 0.0167178437393 | 0.0404958138454  |
|   10   | 0.0158990105766 | 0.0425634882006  |
+--------+-----------------+------------------+
[10 rows x 3 columns]
[WARNING] Model trained without a target. Skipping RMSE computation.
PROGRESS: Evaluate model M1
PROGRESS: recommendations finished on 1000/2931 queries. users per second: 1271.42
PROGRESS: recommendations finished on 2000/2931 queries. users per second: 1303.45

Precision and recall summary statistics by cutoff
+--------+-----------------+-----------------+
| cutoff |  mean_precision |   mean_recall   |
+--------+-----------------+-----------------+
|   1    |  0.187649266462 | 0.0616257761345 |
|   2    |  0.159843056977 |  0.101001740024 |
|   3    |  0.138178096213 |  0.127534831066 |
|   4    |  0.123080859775 |  0.148361563945 |
|   5    |  0.111497782327 |  0.164334733654 |
|   6    |  0.102979642898 |  0.180691727621 |
|   7    | 0.0956280157918 |  0.195407772788 |
|   8    | 0.0895172296145 |  0.206395223488 |
|   9    | 0.0838545812957 |  0.215965650408 |
|   10   | 0.0797679972706 |  0.227158371785 |
+--------+-----------------+-----------------+
[10 rows x 3 columns]
[WARNING] Model trained without a target. Skipping RMSE computation.


In [22]:
kanye_listeners = song_data[song_data['artist'] =='Kanye West']['user_id'].unique()

In [23]:
len(kanye_listeners)


Out[23]:
2522

In [24]:
def artist_listeners(artist):
    return len(song_data[song_data['artist'] ==artist]['user_id'].unique())

In [25]:
print artist_listeners('Kanye West')


2522

In [26]:
print artist_listeners('Foo Fighters')


2055

In [27]:
print artist_listeners('Taylor Swift')


3246

In [28]:
print artist_listeners('Lady GaGa')


2928

In [38]:
song_data.groupby(key_columns='artist', operations={'total_count': graphlab.aggregate.SUM('listen_count')})


Out[38]:
artist total_count
The Dells 274
Lil Jon / The East Side
Boyz ...
197
Tom Petty And The
Heartbreakers ...
2867
Blackstreet 747
Ratatat 3727
Shotta 82
Airscape 130
Mecano 172
Moimir Papalescu & The
Nihilists ...
177
Brad Paisley 2731
[3375 rows x 2 columns]
Note: Only the head of the SFrame is printed.
You can use print_rows(num_rows=m, num_columns=n) to print more rows and columns.


In [40]:
gr.sort('total_count',ascending=False)


Out[40]:
artist total_count
Kings Of Leon 43218
Dwight Yoakam 40619
Björk 38889
Coldplay 35362
Florence + The Machine 33387
Justin Bieber 29715
Alliance Ethnik 26689
OneRepublic 25754
Train 25402
The Black Keys 22184
[3375 rows x 2 columns]
Note: Only the head of the SFrame is printed.
You can use print_rows(num_rows=m, num_columns=n) to print more rows and columns.


In [49]:
gr.sort('total_count',ascending=True)


Out[49]:
artist total_count
William Tabbert 14
Reel Feelings 24
Beyoncé feat. Bun B and
Slim Thug ...
26
Diplo 30
Boggle Karaoke 30
harvey summers 31
Nâdiya 36
Kanye West / Talib Kweli
/ Q-Tip / Common / ...
38
Aneta Langerova 38
Jody Bernal 38
[3375 rows x 2 columns]
Note: Only the head of the SFrame is printed.
You can use print_rows(num_rows=m, num_columns=n) to print more rows and columns.


In [41]:
subset_test_users = test_data['user_id'].unique()[0:10000]

In [43]:
recommended_songs  = personalized_model.recommend(subset_test_users,k=1)


PROGRESS: recommendations finished on 1000/10000 queries. users per second: 1256.55
PROGRESS: recommendations finished on 2000/10000 queries. users per second: 1296.76
PROGRESS: recommendations finished on 3000/10000 queries. users per second: 1312.02
PROGRESS: recommendations finished on 4000/10000 queries. users per second: 1300.35
PROGRESS: recommendations finished on 5000/10000 queries. users per second: 1309.51
PROGRESS: recommendations finished on 6000/10000 queries. users per second: 1325.81
PROGRESS: recommendations finished on 7000/10000 queries. users per second: 1333.4
PROGRESS: recommendations finished on 8000/10000 queries. users per second: 1335.46
PROGRESS: recommendations finished on 9000/10000 queries. users per second: 1338.97
PROGRESS: recommendations finished on 10000/10000 queries. users per second: 1339.1

In [44]:
recommended_songs


Out[44]:
user_id song score rank
c66c10a9567f0d82ff31441a9
fd5063e5cd9dfe8 ...
Cuando Pase El Temblor -
Soda Stereo ...
0.0194504525792 1
c067c22072a17d33310d7223d
7b79f819e48cf42 ...
Grind With Me (Explicit
Version) - Pretty Ricky ...
0.0459424433009 1
f6c596a519698c97f1591ad89
f540d76f6a04f1a ...
Hey_ Soul Sister - Train 0.0249283462453 1
696787172dd3f5169dc94deef
97e427cee86147d ...
Senza Una Donna (Without
A Woman) - Zucchero / ...
0.0170265780731 1
3a7111f4cdf3c5a85fd4053e3
cc2333562e1e0cb ...
Heartbreak Warfare - John
Mayer ...
0.0320541548711 1
532e98155cbfd1e1a474a28ed
96e59e50f7c5baf ...
Jive Talkin' (Album
Version) - Bee Gees ...
0.0118288659232 1
ee43b175ed753b2e2bce806c9
03d4661ad351a91 ...
Ricordati Di Noi -
Valerio Scanu ...
0.0305171277997 1
e372c27f6cb071518ae500589
ae02c126954c148 ...
Fall Out - The Police 0.0819672131148 1
83b1428917b47a6b130ed471b
09033820be78a8c ...
Clocks - Coldplay 0.0440484388452 1
39487deef9345b1e22881245c
abf4e7c53b6cf6e ...
Black Mirror - Arcade
Fire ...
0.0417819323978 1
[10000 rows x 4 columns]
Note: Only the head of the SFrame is printed.
You can use print_rows(num_rows=m, num_columns=n) to print more rows and columns.


In [47]:
songs_grp = recommended_songs.groupby(key_columns='song', operations={'count': graphlab.aggregate.COUNT()})

In [48]:
songs_grp.sort('count',ascending=False)


Out[48]:
song count
Undo - Björk 430
Secrets - OneRepublic 385
Revelry - Kings Of Leon 232
You're The One - Dwight
Yoakam ...
169
Fireflies - Charttraxx
Karaoke ...
122
Hey_ Soul Sister - Train 105
Horn Concerto No. 4 in E
flat K495: II. Romance ...
99
Sehr kosmisch - Harmonia 74
OMG - Usher featuring
will.i.am ...
59
Dog Days Are Over (Radio
Edit) - Florence + The ...
55
[3135 rows x 2 columns]
Note: Only the head of the SFrame is printed.
You can use print_rows(num_rows=m, num_columns=n) to print more rows and columns.


In [ ]: