In [13]:
import pandas as pd;
import numpy as np;
In [14]:
regular_tweets = pd.DataFrame(open('dev_test_tweets/nonsarcastic.csv').readlines())
sarcastic_tweets = pd.DataFrame(open('dev_test_tweets/sarcastic.csv').readlines())
In [15]:
# ALL CAPS COUNT
sarcastic_tweets['AllCapsCount'] = [sum([k.isupper() for k in n.split(" ")]) for n in sarcastic_tweets[0]]
regular_tweets['AllCapsCount'] = [sum([k.isupper() for k in n.split(" ")]) for n in regular_tweets[0]]
In [16]:
sarcastic_tweets
Out[16]:
0
AllCapsCount
0
Interesting, I seem to remember one @BarackOba...
2
1
You all should show the #deplorables, #tacobow...
0
2
I thought Trump was talking about the 50's, wi...
2
3
I woke up today in #whitetrashnation. You had ...
3
4
First the #deplorables came for #Chinese busin...
2
5
@jojoh888 They worked hard for their money bu...
0
6
Figuring Out If When His #AltRight and #Deplor...
0
7
The funniest thing about these #deplorables is...
0
8
@JackieTartamell @antcav94 @threelilbirdsss ya...
0
9
Good luck to everyone in America #deplorables ...
0
10
Is this too much? @3WV @tawnehayes #deplorables\n
1
11
This is a dark day for America. Congratulation...
0
12
Ironic that the so called #deplorables aren't ...
0
13
I never thought losing could be so much fun!! ...
1
14
The deplorable faceless trolls are out in forc...
0
15
Thanks Colin for not noticing that I took your...
2
16
Just finished rewatching 5 hours worth of O Ch...
1
17
I'm surprised liberals haven't started crashin...
0
18
Remember when the #TrumpTrain was putting all ...
0
19
This person doesn't want your support; Let him...
0
20
I only voted for this cool sticker #jk #TrumpT...
1
21
Oh look I can't push a button correctly #voter...
1
22
Could Anonymous and Wikileaks be combating the...
0
23
How can you be a president when you don't even...
1
24
I hope it's a landslide. Red across America #T...
1
25
You mean the debates where @HillaryClinton was...
0
26
Like really on top of the stress of pharmacy s...
1
27
Don't you love it when Canadians tell you what...
0
28
I'm wondering if Grahams name is on the Lolita...
0
29
You let the hate win, the racism this is who y...
0
...
...
...
70
#HillaryClinton could still become #POTUS ~ #...
2
71
I guess the airlines will be busy transporting...
4
72
It's NOT a #concession speech! #HillaryClinton...
4
73
All of a sudden, I am supposed to believe that...
2
74
Maybe the rich will be very competent and spur...
1
75
Ah great. The fascism is starting. Loss of cit...
1
76
Remember pics like this Donald doesn't want sh...
1
77
Now that we've addressed our daily #PEOTUS tro...
2
78
#PEOTUS, you won the electoral college vote fa...
1
79
When I see #PEOTUS I read fetus, appropriate a...
3
80
Its freezing and snowing in New York we need ...
1
81
Jackie O and Tacky Hoe #ImWithHer \n
1
82
Dear @ladygaga I'm sure u were waiting to give...
0
83
This is the person you've voted for ! Congratu...
0
84
just gonna let this sit here. congrats america...
1
85
When my coworker greeted me today with "Mornin...
1
86
oh you have no idea what has america come too,...
0
87
I really thought the United States was united ...
1
88
The playlist on @radioeins is really soothing ...
1
89
We had one job, America #LOVETRUMPSHATE\n
1
90
@enews well, if @realDonaldTrump won, isn't th...
0
91
I guess today I'll eat all my feelings. I'm st...
1
92
47% of Americans didn't vote and we got this. ...
0
93
3 more followers lost who next? #Notmypresident\n
0
94
This is awesome! Glad he didn't get caught. He...
0
95
@DRUDGE_REPORT @politico Awesome! I was just w...
3
96
Awesome let's maintain that relationship as we...
0
97
Really? #ObamaDay? What's next #JamesBuchananD...
1
98
@HillaryClinton A promoted tweet this late Hil...
1
99
@JohnKasich... great call. Your write-in won @...
1
100 rows × 2 columns
In [17]:
# '\\ud83d\\ude1c' in set(sarcastic_tweets['0'][11].split(" "))
In [18]:
# master = sarcastic_tweets.append(regular_tweets)
# # master.to_csv('test_tweets_df.csv')
In [39]:
# ToUser
sarcastic_tweets["ToUser"] = sarcastic_tweets[0].map(lambda x: 1 if "@" in x else 0);
regular_tweets["ToUser"] = regular_tweets[0].map(lambda x: 1 if "@" in x else 0);
# Hashtag count
sarcastic_tweets["Hashtags"] = sarcastic_tweets[0].map(lambda x: 1 if "#" in x else 0);
regular_tweets["Hashtags"] = regular_tweets[0].map(lambda x: 1 if "#" in x else 0);
# Labels
sarcastic_tweets['label'] = 'sarcastic'
regular_tweets['label'] = 'genuine'
In [40]:
sarcastic_tweets
Out[40]:
0
AllCapsCount
ToUser
Hashtags
label
0
Interesting, I seem to remember one @BarackOba...
2
1
1
sarcastic
1
You all should show the #deplorables, #tacobow...
0
0
1
sarcastic
2
I thought Trump was talking about the 50's, wi...
2
0
1
sarcastic
3
I woke up today in #whitetrashnation. You had ...
3
0
1
sarcastic
4
First the #deplorables came for #Chinese busin...
2
0
1
sarcastic
5
@jojoh888 They worked hard for their money bu...
0
1
1
sarcastic
6
Figuring Out If When His #AltRight and #Deplor...
0
0
1
sarcastic
7
The funniest thing about these #deplorables is...
0
0
1
sarcastic
8
@JackieTartamell @antcav94 @threelilbirdsss ya...
0
1
1
sarcastic
9
Good luck to everyone in America #deplorables ...
0
0
1
sarcastic
10
Is this too much? @3WV @tawnehayes #deplorables\n
1
1
1
sarcastic
11
This is a dark day for America. Congratulation...
0
0
1
sarcastic
12
Ironic that the so called #deplorables aren't ...
0
0
1
sarcastic
13
I never thought losing could be so much fun!! ...
1
0
1
sarcastic
14
The deplorable faceless trolls are out in forc...
0
0
1
sarcastic
15
Thanks Colin for not noticing that I took your...
2
0
1
sarcastic
16
Just finished rewatching 5 hours worth of O Ch...
1
0
1
sarcastic
17
I'm surprised liberals haven't started crashin...
0
0
1
sarcastic
18
Remember when the #TrumpTrain was putting all ...
0
0
1
sarcastic
19
This person doesn't want your support; Let him...
0
0
1
sarcastic
20
I only voted for this cool sticker #jk #TrumpT...
1
0
1
sarcastic
21
Oh look I can't push a button correctly #voter...
1
0
1
sarcastic
22
Could Anonymous and Wikileaks be combating the...
0
0
1
sarcastic
23
How can you be a president when you don't even...
1
0
1
sarcastic
24
I hope it's a landslide. Red across America #T...
1
0
1
sarcastic
25
You mean the debates where @HillaryClinton was...
0
1
1
sarcastic
26
Like really on top of the stress of pharmacy s...
1
0
1
sarcastic
27
Don't you love it when Canadians tell you what...
0
0
1
sarcastic
28
I'm wondering if Grahams name is on the Lolita...
0
0
1
sarcastic
29
You let the hate win, the racism this is who y...
0
0
1
sarcastic
...
...
...
...
...
...
70
#HillaryClinton could still become #POTUS ~ #...
2
0
1
sarcastic
71
I guess the airlines will be busy transporting...
4
1
1
sarcastic
72
It's NOT a #concession speech! #HillaryClinton...
4
0
1
sarcastic
73
All of a sudden, I am supposed to believe that...
2
0
1
sarcastic
74
Maybe the rich will be very competent and spur...
1
0
1
sarcastic
75
Ah great. The fascism is starting. Loss of cit...
1
0
1
sarcastic
76
Remember pics like this Donald doesn't want sh...
1
0
1
sarcastic
77
Now that we've addressed our daily #PEOTUS tro...
2
0
1
sarcastic
78
#PEOTUS, you won the electoral college vote fa...
1
0
1
sarcastic
79
When I see #PEOTUS I read fetus, appropriate a...
3
0
1
sarcastic
80
Its freezing and snowing in New York we need ...
1
0
1
sarcastic
81
Jackie O and Tacky Hoe #ImWithHer \n
1
0
1
sarcastic
82
Dear @ladygaga I'm sure u were waiting to give...
0
1
1
sarcastic
83
This is the person you've voted for ! Congratu...
0
0
1
sarcastic
84
just gonna let this sit here. congrats america...
1
0
1
sarcastic
85
When my coworker greeted me today with "Mornin...
1
0
1
sarcastic
86
oh you have no idea what has america come too,...
0
0
1
sarcastic
87
I really thought the United States was united ...
1
0
1
sarcastic
88
The playlist on @radioeins is really soothing ...
1
1
1
sarcastic
89
We had one job, America #LOVETRUMPSHATE\n
1
0
1
sarcastic
90
@enews well, if @realDonaldTrump won, isn't th...
0
1
1
sarcastic
91
I guess today I'll eat all my feelings. I'm st...
1
0
1
sarcastic
92
47% of Americans didn't vote and we got this. ...
0
0
1
sarcastic
93
3 more followers lost who next? #Notmypresident\n
0
0
1
sarcastic
94
This is awesome! Glad he didn't get caught. He...
0
0
1
sarcastic
95
@DRUDGE_REPORT @politico Awesome! I was just w...
3
1
1
sarcastic
96
Awesome let's maintain that relationship as we...
0
0
1
sarcastic
97
Really? #ObamaDay? What's next #JamesBuchananD...
1
0
1
sarcastic
98
@HillaryClinton A promoted tweet this late Hil...
1
1
1
sarcastic
99
@JohnKasich... great call. Your write-in won @...
1
1
1
sarcastic
100 rows × 5 columns
In [41]:
master = sarcastic_tweets.append(regular_tweets)
master.to_csv('test_tweets_df.csv')
Content source: thisisbasil/SarcasmDetectionTwitter
Similar notebooks: