In [2]:
install.packages("gender", repos="http://cran.rstudio.com/")
install.packages("genderdata", repos = "http://packages.ropensci.org")
Updating HTML index of packages in '.Library'
Making 'packages.html' ... done
Updating HTML index of packages in '.Library'
Making 'packages.html' ... done
In [20]:
library(gender)
no <- Sys.setlocale(locale="C")
df <- read.csv("input/input.csv")
subdf <- head(df)
print('Column names:')
colnames(subdf)
print('Head:')
head(subdf)
print('Usernames:')
usernames = subdf$User_name
head(usernames)
[1] "Column names:"
- 'Action_id'
- 'TypeAction'
- 'Page_name'
- 'Feed_message'
- 'User_name'
- 'CommentsForItem_message'
- 'Sentiment'
- 'Action_created_time_qvdate'
[1] "Head:"
Action_id TypeAction Page_name Feed_message User_name CommentsForItem_message Sentiment Action_created_time_qvdate 1 Comment/1364017066952510_1364097753611108 Comment Resident Evil [UK ONLY] To celebrate the release of Resident Evil 7, we invite you to feel the fear and dread of the Baker's house in our real life experience! http://RE7experiencelondon.com Ronald Briand Muy buena :) (Y) 0.9880 1/5/2017 2 Comment/10154797365350629_10154797373500629 Comment Bandai Namco We continue our Tales Of look back with Tales of Graces f! We loved Asbel's sense of justice (and awesome moves!) Who was your favorite character? Jon Imboden I may be in the minority here, but I LOVE the battle system in this game. The story gets a bit convoluted, but is very enjoyable overall. 0.9540 12/27/2016 3 Comment/1365217000165850_1365754593445424 Comment Resident Evil Are you afraid of the dark? Because you should be. #RE7 Claudia Alejandra Olgun mira Roberto Zeballos, para probar en tu PEQUE!! :) 0.9527 1/7/2017 4 Comment/1364017066952510_1364061950281355 Comment Resident Evil [UK ONLY] To celebrate the release of Resident Evil 7, we invite you to feel the fear and dread of the Baker's house in our real life experience! http://RE7experiencelondon.com David Lawrence Sarah Amadeus :O 0.9402 1/5/2017 5 Comment/1365217000165850_1225502404193085 Comment Resident Evil Are you afraid of the dark? Because you should be. #RE7 Leandro Guilherme Aeee Paulo Lucas vou te dar de presente no seu niver hahahaha 0.9382 1/8/2017 6 Comment/1363458100341740_1366094460078104 Comment Resident Evil #ThingsIWontApologizeFor: Cheesy one-liners and using up all the magnum ammo. Oliver Yamanaka Tiro da "derrota", de JILL. Muito bom! 0.9373 1/8/2017
[1] "Usernames:"
- Ronald Briand
- Jon Imboden
- Claudia Alejandra Olgu
n
- David Lawrence
- Leandro Guilherme
- Oliver Yamanaka
In [4]:
print('First names:')
first_names <- sapply(strsplit(as.character(usernames)," "), `[`, 1)
head(first_names)
[1] "First names:"
- 'Ronald'
- 'Jon'
- 'Claudia'
- 'David'
- 'Leandro'
- 'Oliver'
In [5]:
# get the gender from the name ( and age?)
# compute the average accuracy ( proportion )
# TODO HSC: how to get the age btw? Social networks?
#install.packages("genderdata", repos = "http://packages.ropensci.org")
print('Genders:')
genders <- gender(first_names)
head(genders)
[1] "Genders:"
name proportion_male proportion_female gender year_min year_max 1 Claudia 0.0051 0.9949 female 1932 2012 2 David 0.9963 0.0037 male 1932 2012 3 Jon 0.9921 0.0079 male 1932 2012 4 Leandro 1 0 male 1932 2012 5 Oliver 0.9956 0.0044 male 1932 2012 6 Ronald 0.9963 0.0037 male 1932 2012
In [7]:
summary(genders)
name proportion_male proportion_female gender
Length:6 Min. :0.0051 Min. :0.000000 Length:6
Class :character 1st Qu.:0.9930 1st Qu.:0.003700 Class :character
Mode :character Median :0.9960 Median :0.004050 Mode :character
Mean :0.8309 Mean :0.169100
3rd Qu.:0.9963 3rd Qu.:0.007025
Max. :1.0000 Max. :0.994900
year_min year_max
Min. :1932 Min. :2012
1st Qu.:1932 1st Qu.:2012
Median :1932 Median :2012
Mean :1932 Mean :2012
3rd Qu.:1932 3rd Qu.:2012
Max. :1932 Max. :2012
In [9]:
all_usernames = df$User_name
all_first_names <- sapply(strsplit(as.character(all_usernames)," "), `[`, 1)
all_genders <- gender(all_first_names)
In [10]:
# now resave the input file, with the new columns
all_first_names <- sapply(strsplit(as.character(df$User_name)," "), `[`, 1)
df$first_name <- tolower(all_first_names)
all_genders <- gender(unique(all_first_names))
all_genders$first_name <- tolower(all_genders$name)
head(all_genders)
summary(all_genders)
name proportion_male proportion_female gender year_min year_max first_name 1 Aadish 1 0 male 1932 2012 aadish 2 Aafreen 0 1 female 1932 2012 aafreen 3 Aakash 1 0 male 1932 2012 aakash 4 Aalia 0 1 female 1932 2012 aalia 5 Aamer 1 0 male 1932 2012 aamer 6 Aaminah 0 1 female 1932 2012 aaminah
Action_id TypeAction Page_name Feed_message User_name CommentsForItem_message Sentiment Action_created_time_qvdate first_name 1 Comment/1364017066952510_1364097753611108 Comment Resident Evil [UK ONLY] To celebrate the release of Resident Evil 7, we invite you to feel the fear and dread of the Baker's house in our real life experience! http://RE7experiencelondon.com Ronald Briand Muy buena :) (Y) 0.9880 1/5/2017 ronald 2 Comment/10154797365350629_10154797373500629 Comment Bandai Namco We continue our Tales Of look back with Tales of Graces f! We loved Asbel's sense of justice (and awesome moves!) Who was your favorite character? Jon Imboden I may be in the minority here, but I LOVE the battle system in this game. The story gets a bit convoluted, but is very enjoyable overall. 0.9540 12/27/2016 jon 3 Comment/1365217000165850_1365754593445424 Comment Resident Evil Are you afraid of the dark? Because you should be. #RE7 Claudia Alejandra Olgun mira Roberto Zeballos, para probar en tu PEQUE!! :) 0.9527 1/7/2017 claudia 4 Comment/1364017066952510_1364061950281355 Comment Resident Evil [UK ONLY] To celebrate the release of Resident Evil 7, we invite you to feel the fear and dread of the Baker's house in our real life experience! http://RE7experiencelondon.com David Lawrence Sarah Amadeus :O 0.9402 1/5/2017 david 5 Comment/1365217000165850_1225502404193085 Comment Resident Evil Are you afraid of the dark? Because you should be. #RE7 Leandro Guilherme Aeee Paulo Lucas vou te dar de presente no seu niver hahahaha 0.9382 1/8/2017 leandro 6 Comment/1363458100341740_1366094460078104 Comment Resident Evil #ThingsIWontApologizeFor: Cheesy one-liners and using up all the magnum ammo. Oliver Yamanaka Tiro da "derrota", de JILL. Muito bom! 0.9373 1/8/2017 oliver
In [36]:
unique(all_genders$gender)
table(all_genders$gender)results <- merge(x = df, y = genders, by = "first_name", all.x = TRUE)
head(results)
head(results[!is.na(results$name),])
first_name Action_id TypeAction Page_name Feed_message User_name CommentsForItem_message Sentiment Action_created_time_qvdate name proportion_male proportion_female gender year_min year_max 1 'regin Like/1351451431565601 Like Bandai Namco Is there even a way to prepare your body for this? We're open to suggestions. #RoadtoBoruto 'regin Iyas - - 1/7/2017 NA NA NA NA NA NA 2 '<93><89><81><82> Like/1840121602944310 Like TEKKEN Dive into the week like Bob dives into his opponent. #MondayMotivation '<93><89><81><82> <93><81><85><80><81><82>' - - 1/9/2017 NA NA NA NA NA NA 3 '<95><81> Like/1790336797885801 Like Resident Evil <80>Day 2 of Resident Evil Christmas! Alexia put her presents under the Christmas tree! <80> '<95><81> <84><84><85>' - - 1/4/2017 NA NA NA NA NA NA 4 ' Like/1127220827394955 Like TEKKEN Need inspiration for your New Year<80><99>s resolutions? Eric Jacobus has exactly what you need. Enjoy! ' <85><8a><85> - - 1/4/2017 NA NA NA NA NA NA 5 - Comment/1194130220666342_1194652720614092 Comment - Ultimate Mortal Kombat 3 #FanArt: #Kitana by Banjamin Jara aka benjawheel. The first art of Kitana in 2017!
http://benjawheel.deviantart.com/
http://www.instagram.com/wheelartstuff/
http://wheelartstuff.tumblr.com - Great art and a great start to 2017! 0.9255 1/6/2017 NA NA NA NA NA NA 6 - Comment/10154779856140629_10154798866910629 Comment Bandai Namco Safety first. Even when you're a living rocket launcher like Mecha Naruto <9f><9a><80> - Worlds best super fan right here 0.8713 12/28/2016 NA NA NA NA NA NA
first_name Action_id TypeAction Page_name Feed_message User_name CommentsForItem_message Sentiment Action_created_time_qvdate name proportion_male proportion_female gender year_min year_max 79 aadish Like/10211262216735703 Like Street Fighter Artgerm starting the year off strong with this amazing Chun-Li fan art! http://bit.ly/2iAJ2qO Aadish Balouria - - 1/7/2017 Aadish 1 0 male 1932 2012 80 aadish Comment/10155001784202147_10155003381432147 Comment Street Fighter Artgerm starting the year off strong with this amazing Chun-Li fan art! http://bit.ly/2iAJ2qO Aadish Balouria Them octiceps femoris though -0.3339 1/7/2017 Aadish 1 0 male 1932 2012 81 aafreen Like/10211099724065980 Like Resident Evil Are you afraid of the dark? Because you should be. #RE7 Aafreen McNamee - - 1/8/2017 Aafreen 0 1 female 1932 2012 83 aakash Like/162748404209324 Like Resident Evil Are you afraid of the dark? Because you should be. #RE7 Aakash Kumar - - 1/8/2017 Aakash 1 0 male 1932 2012 84 aakash Comment/1365217000165850_1232412240181374 Comment Resident Evil Are you afraid of the dark? Because you should be. #RE7 Aakash Dave Munjal Jani 0.0000 1/8/2017 Aakash 1 0 male 1932 2012 85 aakash Comment/1364017066952510_1364253716928845 Comment Resident Evil [UK ONLY] To celebrate the release of Resident Evil 7, we invite you to feel the fear and dread of the Baker's house in our real life experience! http://RE7experiencelondon.com Aakash Pundir Ritwik Sondhi <9f><98> - 1/5/2017 Aakash 1 0 male 1932 2012
In [37]:
# save the results as a new csv
write.csv(results, file = 'output/input_with_gender.csv')
Content source: HelainSchoonjans/fantastic-eureka
Similar notebooks: