In [1]:

    

import pandas as pd
import numpy as np


    

In [5]:

    

review = pd.read_json('../../datasets/yelp_academic_dataset_review.json', lines=True, encoding='utf-8')


    

    





UnicodeDecodeErrorTraceback (most recent call last)
<ipython-input-5-05cc4ebe877a> in <module>()
----> 1 review = pd.read_json('../../datasets/yelp_academic_dataset_review.json', lines=True, encoding='utf-8')

/home/cusp/xz1845/.conda/envs/myPy27/lib/python2.7/site-packages/pandas/io/json.pyc in read_json(path_or_buf, orient, typ, dtype, convert_axes, convert_dates, keep_default_dates, numpy, precise_float, date_unit, encoding, lines)
    273         # commas and put it in a json list to make a valid json object.
    274         lines = list(StringIO(json.strip()))
--> 275         json = u'[' + u','.join(lines) + u']'
    276 
    277     obj = None

UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 173: ordinal not in range(128)

In [6]:

    

pd.read_json?


    

In [7]:

    

# read the entire file into a python array
with open('../../datasets/yelp_academic_dataset_review.json', 'rb') as f:
    data = f.readlines()

# remove the trailing "\n" from each line
data = map(lambda x: x.rstrip(), data)


data_json_str = "[" + ','.join(data) + "]"

# now, load it into pandas
data_df = pd.read_json(data_json_str)


    

In [8]:

    

data_df


    

    
Out[8]:






  

    

      

      
business_id
      
cool
      
date
      
funny
      
review_id
      
stars
      
text
      
type
      
useful
      
user_id
    
  
  

    

      
0
      
2aFiy99vNLklCx3T_tGS9A
      
0
      
2011-10-10
      
0
      
NxL8SIC5yqOdnlXCg18IBg
      
5
      
If you enjoy service by someone who is as comp...
      
review
      
0
      
KpkOkG6RIf4Ra25Lhhxf1A
    
    

      
1
      
2aFiy99vNLklCx3T_tGS9A
      
0
      
2010-12-29
      
0
      
pXbbIgOXvLuTi_SPs1hQEQ
      
5
      
After being on the phone with Verizon Wireless...
      
review
      
1
      
bQ7fQq1otn9hKX-gXRsrgA
    
    

      
2
      
2aFiy99vNLklCx3T_tGS9A
      
0
      
2011-04-29
      
0
      
wslW2Lu4NYylb1jEapAGsw
      
5
      
Great service! Corey is very service oriented....
      
review
      
0
      
r1NUhdNmL6yU9Bn-Yx6FTw
    
    

      
3
      
2LfIuF3_sX6uwe-IR-P0jQ
      
1
      
2014-07-14
      
0
      
GP6YEearUWrzPtQYSF1vVg
      
5
      
Highly recommended. Went in yesterday looking ...
      
review
      
0
      
aW3ix1KNZAvoM8q-WghA3Q
    
    

      
4
      
2LfIuF3_sX6uwe-IR-P0jQ
      
0
      
2014-01-15
      
0
      
25RlYGq2s5qShi-pn3ufVA
      
4
      
I walked in here looking for a specific piece ...
      
review
      
0
      
YOo-Cip8HqvKp_p9nEGphw
    
    

      
5
      
2LfIuF3_sX6uwe-IR-P0jQ
      
1
      
2013-04-28
      
0
      
Uf1Ki1yyH_JDKhLvn2e4FQ
      
5
      
What a great place! Modern on Melrose has amaz...
      
review
      
2
      
bgl3j8yJcRO-00NkUYsXGQ
    
    

      
6
      
2LfIuF3_sX6uwe-IR-P0jQ
      
0
      
2014-10-12
      
0
      
oFmVZh-La7SuvpHrH_Al4Q
      
4
      
A hidden gem! Found a beautiful buffet for a g...
      
review
      
0
      
CWKF9de-nskLYEqDDCfubg
    
    

      
7
      
2LfIuF3_sX6uwe-IR-P0jQ
      
0
      
2012-09-18
      
0
      
bRvdVt88MJ_YMTlLbjDLxQ
      
5
      
This place is a great for those vintage/mid ce...
      
review
      
2
      
GJ7PTY7huYORFKKg3db3Gw
    
    

      
8
      
2LfIuF3_sX6uwe-IR-P0jQ
      
0
      
2015-10-11
      
0
      
zNUSxqflZKgKD1NQH3jdFA
      
5
      
This is the place to go for all your Mid Centu...
      
review
      
0
      
rxqp9eXZj1jYTn0UIsm3Hg
    
    

      
9
      
2LfIuF3_sX6uwe-IR-P0jQ
      
0
      
2015-04-05
      
0
      
LkP1l7sZIwOV6IKNLqQp_A
      
5
      
Great items at a good price.  Helpful, easy to...
      
review
      
0
      
UU0nHQtHPMAfLidk8tOHTg
    
    

      
10
      
2LfIuF3_sX6uwe-IR-P0jQ
      
0
      
2014-07-08
      
0
      
MvvT0BtQH9wq7K-pPgkoEQ
      
1
      
Disappointing.  I've been there twice hoping t...
      
review
      
1
      
A_Hyfk3FcwFVIk1CQC7z7w
    
    

      
11
      
2LfIuF3_sX6uwe-IR-P0jQ
      
1
      
2014-08-23
      
1
      
dJguzIfNKtGkKk8ryOPdfQ
      
5
      
I'm a local Real Estate agent and have fallen ...
      
review
      
1
      
OvD92wp0-uuFoGLBymwfKQ
    
    

      
12
      
2LfIuF3_sX6uwe-IR-P0jQ
      
0
      
2015-01-13
      
0
      
l-yhe0IFJ7WTgEk0qD8yqA
      
4
      
Modern on Melrose is one of our favorite shops...
      
review
      
0
      
5NDk-q5mv8PIDvz83HwMVg
    
    

      
13
      
2LfIuF3_sX6uwe-IR-P0jQ
      
0
      
2014-08-05
      
0
      
cNA3oGTmmeRvt10vrvlJOQ
      
5
      
Walked into the store and stuck up a conversat...
      
review
      
1
      
AziQIgYIAY6uVw1k8sbtTw
    
    

      
14
      
2LfIuF3_sX6uwe-IR-P0jQ
      
1
      
2014-12-22
      
0
      
S_Z5aqlpTVkR9RmYvcPpTQ
      
5
      
What a lovely place. My boyfriend and I went a...
      
review
      
0
      
A65IYKs3FwsyyJH20XOk2w
    
    

      
15
      
2LfIuF3_sX6uwe-IR-P0jQ
      
2
      
2013-11-17
      
3
      
klVqdjXPM2aM8Y0RsDCPoA
      
1
      
Unfortunately, Yelp does not allow negative st...
      
review
      
4
      
wnzfuir72IZFg5RAPOwWCQ
    
    

      
16
      
2LfIuF3_sX6uwe-IR-P0jQ
      
0
      
2013-07-10
      
0
      
Lulhm9mWWcgndcuc_zsgPA
      
4
      
Favorite vintage store I have been in. I have ...
      
review
      
0
      
2N9wrn5A37aOXDrBPIYQdA
    
    

      
17
      
2LfIuF3_sX6uwe-IR-P0jQ
      
0
      
2014-08-23
      
0
      
k4xighfFxKv5H-Eic28MNQ
      
5
      
I stumbled upon Modern on Melrose earlier in t...
      
review
      
0
      
oKZiDLm0D1PIm1-vRtvONA
    
    

      
18
      
0czfEgv9KAD4VlIa7ANPWQ
      
0
      
2010-02-16
      
0
      
U617X1icAyAC5hbjsnhcpA
      
1
      
Overall, I'll never go back. Rewinding to the ...
      
review
      
1
      
XMRkVYpQAZpWOpia5-MoPw
    
    

      
19
      
0czfEgv9KAD4VlIa7ANPWQ
      
1
      
2009-04-10
      
2
      
_a7Zu2ZSEGO4bl2gvu7OtQ
      
5
      
I love Mint, and even though I'm a guy and the...
      
review
      
2
      
jhhHm3Vk9ZlP21WdY_5R0w
    
    

      
20
      
0czfEgv9KAD4VlIa7ANPWQ
      
0
      
2009-01-30
      
0
      
5hU8Uxdem1j_PEr4HFs5HQ
      
5
      
I read the reviewer before me and checked thin...
      
review
      
0
      
9Cmmt_34PxuC84P1AgmetQ
    
    

      
21
      
0czfEgv9KAD4VlIa7ANPWQ
      
4
      
2007-12-15
      
5
      
RAzbLYbgw3tNv8Tu0R7qmQ
      
3
      
Hmmm, what to say about Mint.\n\nOkay, if you ...
      
review
      
5
      
B7ecAeAIrXg7sgmabS38pg
    
    

      
22
      
0czfEgv9KAD4VlIa7ANPWQ
      
2
      
2009-09-07
      
1
      
0-emituHrxFvtYJZVhr2tA
      
3
      
I like this Vintage/New shop.  I think they ne...
      
review
      
3
      
mdVHZbSl97wUUoE6hdxeLg
    
    

      
23
      
0czfEgv9KAD4VlIa7ANPWQ
      
1
      
2009-02-01
      
4
      
dfL5TyVib8uZgxaNMwx7Aw
      
2
      
I remember really enjoying browsing through Mi...
      
review
      
3
      
je5k8a3qIOM0VJE5MaxsfQ
    
    

      
24
      
0czfEgv9KAD4VlIa7ANPWQ
      
0
      
2007-03-04
      
1
      
z5OD5c-nu2v1xwfG0mAIiA
      
4
      
The vintage 1930 beaded earrings are still cal...
      
review
      
2
      
TryfbZaaGtedncOMq9k86g
    
    

      
25
      
0czfEgv9KAD4VlIa7ANPWQ
      
0
      
2010-03-01
      
0
      
fA2NFb69SvDCsB7WSpY2rw
      
2
      
Well.. Well...well..Ive been highly recomended...
      
review
      
2
      
yt1VnahG-MRgU6tswWaUxA
    
    

      
26
      
0czfEgv9KAD4VlIa7ANPWQ
      
1
      
2010-05-14
      
1
      
OiB7X9UxftB0WAWgd2KdJQ
      
3
      
Red's merchandise is amazing. Mint's left a li...
      
review
      
1
      
wJ0mbYw5WoiaKxiNEayZvg
    
    

      
27
      
0czfEgv9KAD4VlIa7ANPWQ
      
0
      
2011-01-14
      
0
      
ezuT8BpvhLITCfbQEWddsw
      
2
      
I have been consistently underwhelmed by this ...
      
review
      
1
      
29rUYREQxeFE6LWGnirgew
    
    

      
28
      
7GI_V9oLCUGdn2ogqB0IBg
      
0
      
2011-10-19
      
1
      
IXQgVQJUQoJ6sKxV7-cREQ
      
5
      
I highly doubt anyone uses Yelp to find a plac...
      
review
      
1
      
zOU38EUHgUtEI4ziZ8DB1w
    
    

      
29
      
7GI_V9oLCUGdn2ogqB0IBg
      
0
      
2014-04-26
      
0
      
ojtDaz6oAxS74VnAAjb4NQ
      
5
      
This place is great!  I first came here to fue...
      
review
      
0
      
hMZ3_yBErMi-pFcquyLJWg
    
    

      
...
      
...
      
...
      
...
      
...
      
...
      
...
      
...
      
...
      
...
      
...
    
    

      
4153120
      
KYasaF1nov1bn7phfSgWeg
      
0
      
2014-09-11
      
0
      
1_sQQl1gkDMMuYAFSWk9xA
      
4
      
#6 is the bomb! I come here a few times a mont...
      
review
      
0
      
_gPCqBwMuXbZrpkZ27vMUQ
    
    

      
4153121
      
KYasaF1nov1bn7phfSgWeg
      
0
      
2015-12-16
      
0
      
SMOObK16fHBGQJ4mKWHe6Q
      
4
      
Great flavored milk tea!  Their boba was reall...
      
review
      
1
      
ZNQgqENlQhmoyqAdrsv1OA
    
    

      
4153122
      
KYasaF1nov1bn7phfSgWeg
      
2
      
2015-12-21
      
1
      
qCEeKzf17x_AsutuEdi_GA
      
4
      
I kept hearing good things about this place an...
      
review
      
2
      
CeQh69cx7UEcAcqM-zFMuw
    
    

      
4153123
      
KYasaF1nov1bn7phfSgWeg
      
0
      
2014-11-18
      
0
      
8h2dVbmdWJRYkBHYKRXmdw
      
4
      
This is a drink spot. I just recently got into...
      
review
      
0
      
jxr8O4H3ivg630bbrix7aQ
    
    

      
4153124
      
KYasaF1nov1bn7phfSgWeg
      
1
      
2015-10-19
      
1
      
82kqlpB4pyxQu3PeNbFrCw
      
5
      
Bambu has now become a huge chain and I'm glad...
      
review
      
1
      
CQZpblGIHeMlD7KqNP7J7Q
    
    

      
4153125
      
KYasaF1nov1bn7phfSgWeg
      
1
      
2014-11-19
      
1
      
csMokBxC3Tx02eAV00j94g
      
5
      
I love that there's a Bambu Dessert here in Ve...
      
review
      
2
      
z4xZolh3zzBmPEW6RuGuJQ
    
    

      
4153126
      
KYasaF1nov1bn7phfSgWeg
      
0
      
2016-07-18
      
0
      
vCY8HeK5vLffGI6aAOS4GQ
      
1
      
price went up, quality went down and quantity ...
      
review
      
0
      
znla6A7Px7BDLOE2KOJ_yQ
    
    

      
4153127
      
KYasaF1nov1bn7phfSgWeg
      
0
      
2016-12-02
      
0
      
7BEeNCcy-AolbiEzlE0pJw
      
2
      
The smoothies and milk tea here are really goo...
      
review
      
0
      
xYpHNJwuXkiG8PB1nMd7Qw
    
    

      
4153128
      
KYasaF1nov1bn7phfSgWeg
      
0
      
2015-07-29
      
0
      
DldwDqTSJ6GorBWQxQ-0hg
      
3
      
Service 1/5\nFood or Drink 4/5\nCleanliness 4/...
      
review
      
0
      
NAXmt_II8Vc_YqxkkSulYA
    
    

      
4153129
      
KYasaF1nov1bn7phfSgWeg
      
0
      
2016-07-27
      
0
      
GTr4zAtCcutvNUHd10elug
      
4
      
Love the #5 here the drink is really good... B...
      
review
      
0
      
EqXooCH8CV3yzATS-mZvaw
    
    

      
4153130
      
KYasaF1nov1bn7phfSgWeg
      
0
      
2014-11-19
      
0
      
AhTBHz8eotgHjLUZtOUcCw
      
5
      
I'm here at least once a week. I love the dese...
      
review
      
0
      
gzaMQDQ5aAf52NgzUnNaCw
    
    

      
4153131
      
KYasaF1nov1bn7phfSgWeg
      
7
      
2014-01-29
      
4
      
6JDC-4qnqErbeA6D05KHfg
      
3
      
The first Bambu store I tried was in San Mateo...
      
review
      
10
      
vxR_YV0atFxIxfOnF9uHjQ
    
    

      
4153132
      
KYasaF1nov1bn7phfSgWeg
      
0
      
2015-01-15
      
0
      
4AQCWalxpWAP4IHqrAge_Q
      
5
      
We've been coming here since it first opened. ...
      
review
      
0
      
l_6G8IPmJmWipXjc3_AlMQ
    
    

      
4153133
      
KYasaF1nov1bn7phfSgWeg
      
1
      
2015-04-25
      
1
      
CynPzCKLryN0x6-5BmG1VQ
      
5
      
Not many places where you'll find good Che (vi...
      
review
      
1
      
Rjp3Wvl4cf-egwdulDH5AQ
    
    

      
4153134
      
KYasaF1nov1bn7phfSgWeg
      
0
      
2016-03-30
      
0
      
-gixaZUDGOyQZZ5e3z6V5g
      
1
      
We order bambu special, # 10 very disappointed...
      
review
      
0
      
ajjIgmA32sgs5GoXKA0qmg
    
    

      
4153135
      
KYasaF1nov1bn7phfSgWeg
      
0
      
2015-01-22
      
0
      
LBARCwOZFF7wez6gO5Bdjg
      
4
      
I went  here a couple days ago an hour or so b...
      
review
      
0
      
mu4XvWvJOb3XpG1C_CHCWA
    
    

      
4153136
      
KYasaF1nov1bn7phfSgWeg
      
0
      
2016-04-16
      
0
      
ktb68mk49Ck16IINby5_7w
      
5
      
I love bambu desserts. My favorite ones are #6...
      
review
      
0
      
E97UxM7LrxcU02fOys77Kg
    
    

      
4153137
      
KYasaF1nov1bn7phfSgWeg
      
0
      
2015-09-24
      
0
      
R_9IfNH7dogmhCL73QwkVw
      
5
      
Ermygerd!!! I absolutely love this place!! My ...
      
review
      
0
      
4j5syz3FV5yPuWRSZA_-rQ
    
    

      
4153138
      
KYasaF1nov1bn7phfSgWeg
      
0
      
2014-08-12
      
0
      
NDY_gk5y4iCFS5Xj1Xh2iw
      
2
      
We are loyal customers to Bambu back in Seattl...
      
review
      
1
      
4jehKOYfQKPeetywYhMz8A
    
    

      
4153139
      
KYasaF1nov1bn7phfSgWeg
      
0
      
2014-11-13
      
1
      
4EMe-XQKhBbW6wmG14sEvQ
      
1
      
The worse avocado smoothly ever.  It doesn't t...
      
review
      
0
      
BHmlldpm2rdBmW7p7EL3zQ
    
    

      
4153140
      
KYasaF1nov1bn7phfSgWeg
      
0
      
2014-01-27
      
0
      
5hxsht_ZoU-CNg8m5dWmUA
      
4
      
Bambu boba has more unique flavors than your a...
      
review
      
0
      
EYcrZxm0A9VKGjR-yizDvw
    
    

      
4153141
      
KYasaF1nov1bn7phfSgWeg
      
0
      
2015-01-03
      
0
      
w9DJNsDP7SzbWiN69aQxcA
      
2
      
Ppl always go here and say it's good, but I've...
      
review
      
0
      
4nWtFVLcbgIjMLXDsslx1A
    
    

      
4153142
      
KYasaF1nov1bn7phfSgWeg
      
3
      
2015-05-22
      
1
      
Qlc4LI3iYCCNCDYydZxAZg
      
4
      
Tried the #6 and the #10, both were great.  Lo...
      
review
      
2
      
kaqlSjOjI8YW7PcLdanK-w
    
    

      
4153143
      
KYasaF1nov1bn7phfSgWeg
      
0
      
2014-02-13
      
0
      
WvWcS6cTlHnOwCxpIXU5jw
      
4
      
Very good! Nice and warm welcome by friendly a...
      
review
      
0
      
nA2sG7_YnN7N-gvn8INPXA
    
    

      
4153144
      
KYasaF1nov1bn7phfSgWeg
      
3
      
2015-04-01
      
1
      
POm7LjLHO7uJP4w31VVw2A
      
4
      
Yes yes YES!!! I was ecstatic when I saw this ...
      
review
      
3
      
0uEqdNTVj-I7zM83bbcgsg
    
    

      
4153145
      
KYasaF1nov1bn7phfSgWeg
      
0
      
2014-11-04
      
0
      
I7VekyujidGgJfzlC3NSig
      
5
      
As many says here as well....I LOVE this place...
      
review
      
0
      
qpActPt-ts9RRiZvn0n2KA
    
    

      
4153146
      
KYasaF1nov1bn7phfSgWeg
      
0
      
2016-11-05
      
0
      
H6gfygxRN9gzI0l9rFlF4w
      
5
      
Found this place on yelp. Really cool place to...
      
review
      
0
      
kSOkDKEdCzDjOVFaqOcTKQ
    
    

      
4153147
      
KYasaF1nov1bn7phfSgWeg
      
0
      
2015-08-15
      
0
      
GXQnlRo41C4WykC_IP2Lbw
      
5
      
Tui was awesome ^^ she made me an amazing jack...
      
review
      
0
      
rWL8bGLRPtYAhX97JnRVXg
    
    

      
4153148
      
KYasaF1nov1bn7phfSgWeg
      
0
      
2016-08-18
      
0
      
tTGXK14g6WKJx78JG3bh_w
      
3
      
Overpriced coffee without the free upgrade at ...
      
review
      
0
      
nS19cEYuD99dvTCYISf0oA
    
    

      
4153149
      
KYasaF1nov1bn7phfSgWeg
      
0
      
2014-05-26
      
0
      
EmuqmSacByt96t8G5GK0KQ
      
4
      
If ur into Asian desserts this is the place to...
      
review
      
0
      
LgpDa5gAoK2q3u4zrjdr-w
    
  

4153150 rows × 10 columns

In [ ]:

    




    

In [15]:

    

df_t = pd.read_pickle('../Anomaly/spectural_cluster.pkl')


    

In [13]:

    

df_t.reset_index(inplace=True)


    

In [16]:

    

df_merged = df_t.merge(data_df, how='left', left_index=True, right_on='business_id')


    

In [21]:

    

df_merged.to_csv('../Anomaly/torronto_chi.csv', index_label=False, encoding='utf-8')


    

In [19]:

    

df_merged.to_csv?


    

In [ ]: