In [72]:

    

import numpy as np, pandas as pd, unicodedata, pickle, re


    

In [73]:

    

txtfilename = "quotes.txt"
txtfile = open(txtfilename)


    

In [74]:

    

r = re.compile("(.*?)(http(s?)://.+)?(\s\w{3}\s\d{2},\s\d{4})$")
def clean(s):
    m = r.match(s)
    return m.group(1)


    

In [75]:

    

clean('"qsdfmlqsmdflkj qsmdlkfj mqsdlfkj " - Qqsdf qsdf ht qsdf Dec 15, 2015')


    

    
Out[75]:





'"qsdfmlqsmdflkj qsmdlkfj mqsdlfkj " - Qqsdf qsdf ht qsdf'

In [76]:

    

def char2idx(c):
    return ord(c)
def toascii(s):
    return unicodedata.normalize("NFKD", s).encode("ascii", "ignore")
def tostring(lc):
    return "".join(map(chr, lc))


    

In [77]:

    

with open("quotes.clean.txt","w") as outf:
    for line in txtfile:
        try: 
            outf.write(clean(line)+"\n")
        except AttributeError:
            print line


    

In [78]:

    

txtfile = open("quotes.clean.txt")

datamat = []
for line in txtfile:
    idxs = map(char2idx, toascii(line.decode("utf8")))
    datamat.append(idxs)


    

In [79]:

    

df = pd.DataFrame(datamat)


    

In [80]:

    

data = df.fillna(0).values


    

In [81]:

    

savefile = "quotes.np.pkl"
pickle.dump(data, open(savefile, "w"))


    

In [82]:

    

sdata = pickle.load(open(savefile))


    

In [83]:

    

l = 100
li = 0
for q in sdata:
    print tostring(map(int, q))
    li += 1
    if li > l:
        break


    

    




"He who must travel happily must travel light." - Antoine de Saint-Exupery

"Your time is limited, so don't waste it living someone else's life." - Steve Jobs

"Do you want to know who you are? Don't ask. Act! Action will delineate and define you." - Thomas Jefferson

"To fear love is to fear life, and those who fear life are already 3-parts dead." - Bertrand Russell

"Every victory is only the price of admission to a more difficult problem - Henry Kissinger

"Every worthy act is difficult. Ascent is always difficult. Descent is easy and often slippery." - Mahatma Gandhi

"Everyone sees what you appear to be, few experience what you really are." - Niccol? Machiavelli

"We must begin thinking like a river if we are to leave a legacy of beauty and life for future generations." - David Brower

"If there is no struggle, there is no progress." - Frederick Douglass

"Memories mean more to me than dresses." - Anne Frank

"It is the trouble that never comes that causes the loss of sleep." - Charles Austin Bates

"True happiness comes from the joy of deeds well done, the zest of creating things new." - Antoine de Saint-Exupery

"Imagine your ideal future. Visualize yourself as if your life were perfect in every respect." - Brian Tracy

"Some cause happiness wherever they go; others whenever they go." - Oscar Wilde

"Each success only buys an admission ticket to a more difficult problem." - Henry Kissinger

"Real growth comes through intense, difficult, and challenging situations."  Kamal Ravikant

"Vision is the art of seeing the invisible." - Jonathan Swift

"There are just some kind of menwhore so busy worrying about the next world theyve never learned to live in this one." - Harper Lee

"Laziness may appear attractive, but work gives satisfaction." - Anne Frank

"Commit to living your truth."  @kamalravikant

"No one has ever become poor by giving." - Anne Frank

"Age is foolish and forgetful when it underestimates youth." - J. K. Rowling

"Love and you shall be loved." - Ralph Waldo Emerson

"I never panic when I get lost. I just change where it is I want to go." - Rita Rudner

"The good news is that once the spotlight shines from within yourself, there is no going back."  @kamalravikant

"The only way to survive with grace is to forgive." - Lesley Garner

"In true dialogue, both sides are willing to change." - Thich Nhat Hanh

"Do what you feel in your heart to be right  for youll be criticized anyway." - Eleanor Roosevelt

"Any fool can fight a winning battle, but it needs character to fight a losing one, and that should inspire us." - W..B. Yeats

"One can choose to go back toward safety or forward toward growth." - Abraham Maslow

"Trust men and they will be true to you; treat them greatly and they will show themselves great." - Ralph Waldo Emerson

"Turn your wounds into wisdom." - Oprah Winfrey

"A stumbling block to the pessimist is a stepping-stone to the optimist." - Eleanor Roosevelt

"To produce we must be able to make endless mistakes." - Lesley Garner

"Few things help an individual more than to place responsibility upon him, and to let him know that you trust him." - Booker T. Washington

"Walking is man's best medicine." - Hippocrates

"Don't learn to do, but learn in doing." - Samuel Butler

"To conquer fear is the beginning of wisdom." - Bertrand Russell

"Experience is merely the name men gave to their mistakes." - Oscar Wilde

"If you deliberately set out to be less than you are capable, you'll be unhappy for the rest of your life." - Abraham Maslow

"You are never too old to chase your dreams." - Diana Nyad

"If the only tool you have is a hammer, all your problems begin to look like nails." - Abraham Maslow

"Sharing tales of those we've lost is how we keep from really losing them." - Mitch Albom

"Look at how a single candle can both defy and define the darkness." - Anne Frank

"Our vision is more obstructed by what we think we know than by our lack of knowledge." - Kristen Stendahl

"Never judge a book by its movie." - J.W. Eagan

"He who is not everyday conquering some fear has not learned the secret of life." - Ralph Waldo Emerson

"Gratitude is always a good place to start from in the middle of turmoil."  Leslie Garner"

"People generally see what they look for, and hear what they listen for."- Harper Lee

"Belief consists in accepting the affirmations of the soul; unbelief, in denying them." - Ralph Waldo Emerson

"Be content with what you have; rejoice in the way things are. When you realize there is nothing lacking."- Lao Tzu

"As we are liberated from our fear, our presence automatically liberates others."" - Marianne Williamson

"Change your opinions, keep to your principles; change your leaves, keep intact your roots." - Victor Hugo

"If you begin to practice forgiveness , it has the power of the drip of water that wears away the stone."  Lesley Garner

"To handle yourself, use your head; to handle others, use your heart." - Eleanor Roosevelt

"Because you are alive, everything is possible." - Th?ch Nhat Hanh

"Don't be afraid to be unique or speak your mind, because that's what makes you different from everyone else." - Dave Thomas

"If something else matters to you, then do that."  @kamalravikant

"If something else matters to you, then do that." @kamalravikant 

"Every morning we are born again. What we do today is what matters most." - Buddha

"Education is not the filling of a pail, but the lighting of a fire." - W.B. Yeats

"Prefer to be defeated in the presence of the wise than to excel among fools." - Dogen

"Nurture your mind with great thoughts, for you will never go any higher than you think." - Benjamin Disraeli

"The greatest good you can do for another is not just to share your riches but to reveal to him his own." - Benjamin Disraeli

"What is my life if I am no longer useful to others." - Goethe

"There is nothing impossible to him who will try." - Alexander the Great

"In the midst of movement and chaos, keep stillness inside of you." - Deepak Chopra

Learn from yesterday, live for today, hope for tomorrow. - Unknown Source

Big fleas have little fleas, Upon their backs to bite 'em, And little fleas have lesser fleas, and so, ad infinitum. - Augustus De Morgan

Sometimes Ive believed as many as six impossible things before breakfast.  Lewis Carroll

"Nothing is more intolerable than to have to admit to yourself your own errors." - Ludwig van Beethoven

"Happiness depends upon ourselves." - Aristotle

"When one must, one can." - Charlotte Whitton

"Never give up on what you really want to do. The person with big dreams is more powerful than one with all the facts." - Albert Einstein

"Great things are not done by impulse, but by a series of small things brought together." - Vincent van Gogh

"Victory is sweetest when you've known defeat." - Malcolm Forbes

"I failed my way to success." - Thomas Edison

"No one succeeds without effort. Those who succeed owe their success to perseverance." - Ramana Maharshi

"It is hard to fail, but it is worse never to have tried to succeed." - Theodore Roosevelt

"A single conversation across the table with a wise person is worth a month's study of books." - Chinese Proverb

"The average teacher explains complexity; the gifted teacher reveals simplicity." - Robert Brault

"Braille is knowledge, and knowledge is power." - Louis Braille

Nothing is built on stone; all is built on sand, but we must build as if the sand were stone. - Jorge Luis Borges

Things are never so bad they can't be made worse. - Humphrey Bogart

It is easier to forgive an enemy than to forgive a friend. - William Blake

"If the doors of perception were cleansed everything would appear to man as it is, infinite. - William Blake

"Think in the morning. Act in the noon. Eat in the evening. Sleep in the night." - William Blake

"He who asks is a fool for five minutes, but he who does not ask remains a fool forever." - Chinese Proverb

"I am the wisest man alive, for I know one thing, and that is that I know nothing." - Plato

"Courage is knowing what not to fear. - Playto

"Kind words can be short and easy to speak, but their echoes are truly endless." - Mother Teresa

"Differences are not intended to separate, to alienate. We are different precisely in order to realize our need of one another." - D Tutu

"Don't raise your voice, improve your argument." - Desmond Tutu

"I have the simplest tastes. I am always satisfied with the best." - Oscar Wilde

"Do your little bit of good where you are; it's those little bits of good put together that overwhelm the world." - Desmond Tutu

"Be not afraid of greatness. Some are born great, some achieve greatness, and others have greatness thrust upon them." - William Shakespeare

"The fool doth think he is wise, but the wise man knows himself to be a fool." - William Shakespeare

"Do not look for happiness outside yourself. The awakened seek happiness inside." - Peter Deunov

"The true value of a human being can be found in the degree to which he has attained liberation from the self." - Albert Einstein

"There is creative reading as well as creative writing." - Ralph Waldo Emerson

"Education is the kindling of a flame, not the filling of a vessel." - Socrates

In [ ]: