In [3]:
import pandas as pd
import cPickle as pickle
import sqlite3
import numpy as np
from gensim.models.doc2vec import LabeledSentence
import cjson

In [4]:
df = pd.DataFrame(np.arange(9).reshape(3,3))

In [5]:
print df.size
print df.shape


9
(3, 3)

In [45]:
pathway = '../../data/labeledRedditComments.p'
path2 = '../../data/RedditMay2015Comments.sqlite'

In [15]:
df = pickle.load(open(pathway, 'rb'))

In [4]:
df.columns


Out[4]:
Index([u'subreddit', u'id', u'name', u'body', u'label'], dtype='object')

In [6]:
numcomments = len(df.index)

In [10]:
df['subreddit'].value_counts()


Out[10]:
worldnews          490354
fatpeoplehate      311183
politics           244927
AskMen             138839
AskWomen           137889
KotakuInAction     128156
TwoXChromosomes    105130
TheRedPill          59145
CoonTown            51979
DebateReligion      41015
islam               25443
history             25242
Judaism              9103
lgbt                 8253
TalesofFatHate       5239
AntiPOZi             4740
blackladies          4396
religion             2623
Trans_fags           2362
WhiteRights          1352
GasTheKikes           919
BodyAcceptance        579
women                 529
TransSpace            472
SlutJustice           309
IslamUnveiled         110
Name: subreddit, dtype: int64

In [11]:
df['label'].value_counts()


Out[11]:
NotHate    1234794
Hate        565494
Name: label, dtype: int64

In [12]:
565494.0/1234794.0


Out[12]:
0.45796626805766794

In [35]:
def reddit_comment_gen(df):
''' Input: a pandas df
    Output: this is a generator that gives the next row in the df'''  

    numrows = df.size
    for row in xrange(numrows):


        # load a comment
        comment = df.iloc[row,:]

        # Load the JSON object
        body = str(comment['body'])
        subreddit = str(comment['subreddit'])

        # Clean and tokenize text
#             body = text_cleaner(body)

        # generate
#             yield LabeledSentence(body,labels=[str(json_object['subreddit'])])
        yield "{}: {}".format(subreddit,body)

In [36]:
mygen = reddit_comment_gen(df)

In [41]:
mygen.next()


Out[41]:
'CoonTown: Someone has deeper internal issues they have with themselves. '

In [46]:
conn = sqlite3.connect(path2)
c = conn.cursor()
c.execute('SELECT subreddit, body FROM MAY2015 LIMIT 5')


Out[46]:
<sqlite3.Cursor at 0x7f51712bf6c0>

In [49]:
val = c.next()

In [53]:
val[1]


Out[53]:
u"gg this one's over. off to watch the NFL draft I guess"

In [54]:
type(c)


Out[54]:
sqlite3.Cursor

In [56]:
for i in c:
    print i


(u'politics', u"Are you really implying we return to those times or anywhere near that political environment?  If so, you won't have much luck selling the American people on that governance concept without ushering in American Revolution 2.0.")
(u'AskReddit', u"No one has a European accent either  because it doesn't exist. There are accents from Europe but not a European accent.")
(u'AskReddit', u'That the kid "..reminds me of Kevin."   so sad :-(')

In [84]:
def sql_gen(c):
''' Input: sqlite3 cursor to a sqlite3 database
    Output: this is a generator that gives the next query result from c'''  

    for comment in c:
        try:
            subreddit = str(comment[0])
            body = str(comment[1])
            yield "{}: {}".format(subreddit,body)
        except:
            yield None

# need except statement because sometimes the body or subreddit can't be converted to a str'
#         yield comment

In [90]:
conn = sqlite3.connect(path2)
c = conn.cursor()
c.execute('SELECT subreddit, body FROM MAY2015')
mygen = sql_gen(c)


Exception RuntimeError: 'generator ignored GeneratorExit' in <generator object sql_gen at 0x7f514691adc0> ignored

In [93]:
print mygen.next()


politics: Are you really implying we return to those times or anywhere near that political environment?  If so, you won't have much luck selling the American people on that governance concept without ushering in American Revolution 2.0.

In [95]:
for i in xrange(100):
    print mygen.next()
    print ""


gaming: Sweet , great Idea there as well 

BabyBumps: Thanks for the heads up! Good to know!

trees: Bruh

IAmA: VERY looking forward to *Pollo Empire* once Better Call Saul finishes it's sixth season!

nfl: Can we sticky these main 4 threads? 

WTF: what does it look like now?

Frugal: Right, the tens of thousands of people who installed their own all killed themselves.

Electricity is only dangerous if you're stupid and don't follow the electrical code.

Its very simple stuff that you can pick up if you're handy and can read.

Do have a licensed electrician install the crossover switch as most municipalities require that.


CoDCompetitive: Some hands don't physically allow you to claw 

AskReddit: Go back to Route 21, dude.

WTF: I'm trying to eat a fucking fajita here dammit

lgbt: Thank you for your advice. I'll try to remember that this all takes time-even though it's frustrating :)

trees: Legalize medical marijuana in 2001 too! Just goes to show how forward thinking this guy is.

AskReddit: They're oh-so-bravely standing up to the massive circlejerk... only to be showered in upvotes, and probably gold.

leagueoflegends: Honestly I don't have as much of a problem paying for Chroma's when they are for the base texture and not for a skin. The "Blucian" as I'll call it for now, is one I want to get, and will probably purchase at some point and feel the price is, while above what I would say is worth the value of it, what I will pay for this **one** in particular.

However I do think it's worth looking into other ways of obtaining the Chroma's outside of RP purchases.

WouldYouFuckMyWife: i love your ass also i can see why he does to

circlejerk: [deleted]

pcmasterrace: Yeah, but [60% boards,](http://i.imgur.com/gKoR38B.png) and the like, have layouts that are of normal size, just very minimal in all other aspects. What I mean is, all the keys are about the same size, and about the same distance from each other as larger, more regular keyboards; they just have fewer keys and lower profile casings etc.

This keyboard looks like all the keys are smaller, and not spaced out to compensate for the size, meaning the whole layout is very compressed. Must be intended for mobile/portable use.

EnoughLibertarianSpam: Sounds to me like unprofitable businesses are finding anything to blame but themselves for their failures.  Nah, can't be.  People who run businesses are all brilliant captains of industry incapable of failure, right?

AskReddit: I think the whole "American politics are crazy" thing came from the fact that they are relatively transparent and that we treat them half seriously and half as entertainment value and then export them to the rest of the world for the same purpose.  If you have a modicum of understanding of how the American system works and who some of the major personalities are, you can enjoy shows like the Daily Show.  I'm sure satirical political programming exists in other countries, but we've mastered the genre.  Plus its like sports-- we can banter about how bad our team's draft picks were and how congress/the president/the democrats/the republicans are taking us to hell in a hand basket over a few beers.

because when I read about some of the political sleaze that goes on in other countries that I never hear about, I'm fascinated and amazed.

AskReddit: Fuck pound coins... thick and heavy fuckers with almost no use...

beerporn: discontinued...for a good reason .   .   .   oops

nfl: Woo. 2015 NFL Draft Draft, here we go.

FancyFollicles: Rainbow is just too much fun to not do at least once.

Koyoteelaughter: We do like them !!!

AskReddit: Someone was telling me a long story about an elderly man they knew who had recently gotten remarried after his wife died.  I was barely listening and said something like "when did she die?", to which they replied "2 years ago", to which I replied.. "oh that's cute".  

Then realized I meant it was cute he got remarried or something.  Whatev I'm an ass and I don't listen.  

Smite: That means each of those teams will play two games back-to-back. This isn't a best of series like they normally do, it's seeing which team can gather the most wins throughout the tournament. At least, that's what I've gathered about it.

Fireteams: freelancer i0wa 31 warlock


civ: I'm pretty sure that's the peace music.

todayilearned: This shouldn't be fucking with me as much as it is

xboxone: I'd bet ui gets faster, probably refreshed look (probably nothing major) and new functionality (streaming, app store, background apps, etc)

LifeProTips: Good thing you didn't get pulled over by a Cop, who would have taken that cash off your hands.

PhotoshopRequest: Thank you kind strangers

CircleGrump: priv8

OpTicGaming: If he's first to die, he'll act like the round is already over. 

Reds: Nope. 

TheRedPill: Because they, in fact, we the British, are cowards who shy away from real, painful issues like Rotherham, and squabble over non-issues like this. 

GamerGhazi: Yes! He is the most ethical and not racist of all GGers!

AskReddit: Yes, that's what's going on.  "Fatter" means something different to a starving child than it does to a 300lb adult.

KotakuInAction: &gt;a woman's life was torn apart

Dude, she has fame, a successful career, a rich boyfriend, and legions of devoted supporters. Her life was torn apart? She's living the American Dream.

linuxadmin: You would do mdadm --detail /dev/sda[x], on the partition, not the drive.  List the output from the partitions of each md.  Use pastebin if you need to.

List what drives you have, which are the existing ones, and which is the replacement.  List the output of fdisk -l.



SquaredCircle: In commenting on the botch, you botched your words. Beautiful.

hookah: Why choose to stop glazing all the way through?

nfl: THANK YOU!  Everywhere else on that site I tried needed me to login to something and I wasn't having any of it.

TotalDramaRoleplay: Not Heather's fault! Dakota fail...

trees: [A$AP Rocky - Purple Swag (Jim-E Stack Remix)](https://www.youtube.com/watch?v=RGwbF8rCX6s) 

Not a top five list, but I really think this one's worth a listen.

Minecraft: Here is a better idea of the changes I would like to see made:
http://www.docdroid.net/yz50/a-list-of-tweaks.pdf.html


Once I have some free time, I'm going to try working these into a mod.

AskReddit: [deleted]

whatisthisthing: Difficult to tell from the potato but maybe a [silverfish](http://en.wikipedia.org/wiki/Silverfish)

hiphopheads: Blue Dream &amp; Lean 2- Juicy J

If You're Reading This It's Too Late- Drake

Sorry 4 the Wait 2- Lil Wayne

AskReddit: "Japanese slut double penetrated by horny black guys"......

Am I one of the black guys or am I the Asian chick? It could mean the difference between getting my ass rammed by two ginormous dongs and having rough sex with a cute girl.

AskReddit: "That's great and all, but what about my anecdotal evidence?"

"Same industry" means nothing. I know a lot of people in my "industry" that make way more and way less for no particular reason other than different companies pay different amounts.

jakeandamir: I'm not wearing the - oh... dude, what the f- Chill dude!

hockey: That's not silence. That's the sound of thousands of clenched assholes. 

WTF: Idiot, tattoos are permanent. 

Minecraft: I experience this too, I think it's just a bug you have to get used to.

I've tried everything as far as encasing them entirely in obsidian and they still escape. It seems to occur when you leave the area and then return to it or close the game and reload while near it.

todayilearned: /r/DickPics4Freedom [NSFW]

twitchplayspokemon: You double posted.

AskReddit: The implications of that varies between cultures. Don't be racist.

WTF: He'd make a great extra in The Walking Dead.

IAmA: I think only once actually.

electronic_cigarette: How much does a mod this unnecessarily large and gorgeous set you back if you don't mind me asking?

AdviceAnimals: I don't know. Where?

soccer: Was that close to ozil leaving when he said it? I'm not aware of the situation with ozil's transfer aside from Real Madrid fans being really upset at him leaving but i did not follow the team/la liga itself much at that time.

nba: What did the 5 fingers say to the face?

#Pau!!!

AskReddit: [deleted]

CanadianForces: What was the process like for you to join the boarding party?

malefashion: Happy to help. Do note that I didn't get to wear it as much as I wanted, so the leather MIGHT get softer after breaking it in more. But I'll have to wait for better weather to bring it out.

amiibo: I Don't Really Know Why... But Why Not? :)

legaladvice: *I am a bot whose sole purpose is to improve the timeliness and accuracy of responses in this subreddit.*

---
**It appears you forgot to include your location in the title or body of your post.**

**Please update the original post to include this information.**

---
[Report Inaccuracies Here](http://www.reddit.com/r/LocationBot/) | [Author](http://reddit.com/u/ianp)


---
Original Post:

**Boss required me to sign a document waiving my right take a lunch/breaks. Help /r/legaladvice!**

&gt; I work in the food service industry in Seattle, and have been with my current company for about 5 years. Back in December, every member of our customer service staff (it's a small corporate chain, less than 200 employees) was required to sign [this document](http://imgur.com/iwa6IEQ), or else have all of our shifts cut to 4 hours each in length or less.  I work for minimum wage, and have never been offered or granted a raise by this company when I've asked.  I'm not passionate about my currently line of work, but my personal sense of justice feels violated to the point of needing to put my foot down.
&gt; 
&gt; I know what you're thinking: "Get a new job!" And yes, I'm in the process of looking for another job using other skills. A few of my coworkers are like family to me.  While I might be able to leave the situation, they aren't as fortunate, and that frustration bugs me to no end. The management at this company has stepped over the line of reason, and I'm not sure what legal recourse to utilize, or where to start.
&gt; 
&gt; -I know that their encouraging me to sign this document is illegal.
&gt; -I also know that I would lose my job if my employer read this post (as such, I've removed the header from the document)
&gt; 
&gt; What do I do /r/legaladvice!?  Halp!

None

drunk: "Why don't you just eat it you weirdo?"
-My girlfriend

politics: Seems morally questionable to me, but if that's what you support? I would disagree but to each their own.

Guildwars2: So it's literally bots buying it?

AskReddit: You're just trying to get to the front page.  I see through your facade!!!

pokemonconspiracies: Let's face it, we also couldn't imagine small animals beating each other to death with fire, but then it happened when Pokemon released and we see it just fine. 

DotA2: Would you like more salt on that rant ?

TrueDoTA2: 4 defend 1 where the 1 is the weakest hero and not worth ganking. He may offer xp but he will yield such low gold. I think it is a legitimate strat in itself. 

DebateAnAtheist: Faith is not based on logic or reason, and therefore, logic and reason are of little import to the faithful, and of little use in defeating faith.

A theist could answer any and all of these questions with a blithe "Isn't God amazing?  He can do ANYTHING!!".

None

bodybuilding: Just wait till you hop on the juice. Will never feel the same.

TrollXSupport: [deleted]

Smite: Huh, I never knew about the double space trick.  
Thanks!

funny: Is this from a show or....?

streetwear: Looks like something that could be in vintage versace.

rangers:  try to get loud please

Tinder: I've got two tickets to taradise!

MLPLounge: [](/rocklobster) what would you like in return?

LucidDreaming: But if the alarm doesn't wake you up, how would you tap? If you're a deep sleeper, you could just set a loud, long alarm.

Planetside: i dont know, things get added into COMING SOON a week in advance anyway, and are in the files long before.

^^i ^^just ^^really ^^am ^^lonley ^^without ^^anyone ^^else ^^to ^^be ^^tron ^^magrider ^^with

technology: I live in a rural area and have been sold 'high speed' internet before by a representative by phone.  I received all the account information, dsl router, and it was totally legitimate.  Of course, did not have dsl available in my area and I had to return all the equipment they shipped to me in a timely manner.  
We really need more people holding these guys accountable for this kind of shit.

nfl: In a show of friendship, I'm gonna join the Jets fans in booing their draft pick

pics: Try Dominion and Betrayal at House on the Hill (terrible name, I know). They are not difficult in any sense, but great boardgames!

PurplePillDebate: Well, I do see where you're coming from, but since the system is impossible, I"m going to stick with democracy:)

civbattleroyale: On an unrelated note, does anyone notice how close Poland resembles the Third Reich at its maximum expansion?

nfl: Whatchu doing here Hawkeye?

teslore: The Dragonborn has the problem of being a PC, a character which is undefined apart from their destiny (i.e. the Main Quest).

What could be said, however, is that the Dragonborn does face their destiny. They are not ignorant or avoiding of their purpose, and are up to the challenge.

Tgirls: It looks like anime?

pics: It's like the living people are zombies and the dead one is a survivor.

chicagobulls: I truly believe with this combination, we only lose to ourselves. No team can compete consistently if we keep this effort up.

IAmA: Yeah. People try to make a defending argument, but they always end up sounding like poetic, pretentious bullshit. They jump though hoops to try and find ways to make it true, but it's just not.


In [ ]: