In [1]:

    

from keras.models import Sequential
from keras.layers.core import Dense, Activation, Dropout
from keras.layers.recurrent import LSTM
import numpy as np


    

    




Using Theano backend.
WARNING (theano.sandbox.cuda): The cuda backend is deprecated and will be removed in the next release (v0.10).  Please switch to the gpuarray backend. You can get more information about how to switch at this URL:
 https://github.com/Theano/Theano/wiki/Converting-to-the-new-gpu-back-end%28gpuarray%29

Using gpu device 0: Quadro M6000 (CNMeM is enabled with initial size: 95.0% of memory, cuDNN not available)

In [2]:

    

filename = "trump_speech.txt"
raw_text = open(filename).read()
raw_text = raw_text.lower()
chars = set(raw_text)

char_indices = dict((c, i) for i, c in enumerate(chars))
indices_char = dict((i, c) for i, c in enumerate(chars))


maxlen = 100
step = 1
sentences = []
next_chars = []
for i in range(0, len(raw_text) - maxlen, step):
    sentences.append(raw_text[i: i + maxlen])
    next_chars.append(raw_text[i + maxlen])
print('nb sequences:', len(sentences))



X = np.zeros((len(sentences), maxlen, len(chars)), dtype=np.bool)
y = np.zeros((len(sentences), len(chars)), dtype=np.bool)
for i, sentence in enumerate(sentences):
    for t, char in enumerate(sentence):
        X[i, t, char_indices[char]] = 1
    y[i, char_indices[next_chars[i]]] = 1


    

    




('nb sequences:', 916951)

In [ ]:

    




    

In [8]:

    

model = Sequential()
model.add(LSTM(512,  input_shape=(100,69), return_sequences=True))
model.add(Dropout(0.20))

# model.add(LSTM(512, return_sequences=True))
# model.add(Dropout(0.20))

# model.add(LSTM(512, return_sequences=True))
# model.add(Dropout(0.20))

model.add(LSTM(256, return_sequences=False))
model.add(Dropout(0.20))

model.add(Dense(len(chars)))
model.add(Activation('softmax'))

# compile or load weights then compile depending


    

In [13]:

    

# "This above all: to thine own self be true". - Hamlet (Act I, Scene III).
def generatetext(i):
    seed_text = "Yet Hillary Clinton, for months, and despite so many attacks, repeatedly refused to even say the words radical Islam until I challenged her yesterday.".lower()
    generated = '' + seed_text[-100:]
    print(i,"Seed",seed_text)
    for iteration in range(300):
            # create x vector from seed to predict off of
            x = np.zeros((1, 100, len(chars)))
            for t, char in enumerate(seed_text[-100:]):
                x[0, t, char_indices[char]] = 1.

            preds = model.predict(x, verbose=0)[0]
            next_index = np.argmax(preds)
            next_char = indices_char[next_index]

            generated += next_char
            seed_text = seed_text[1:] + next_char
#             print seed_text,next_char
    print('follow up with: ' + generated)


    

In [14]:

    

model.compile(loss='categorical_crossentropy', optimizer='rmsprop',metrics=['accuracy'])


    

In [15]:

    

# model.load_weights('One_go_larger_network_0.h5')


    

In [17]:

    

epochs = 30
for i in range(12,epochs):
    model.fit(X,y, batch_size=1000,nb_epoch=1)
    model.save_weights('trump_layers'+str(i)+'.h5')
    generatetext(i)


    

    




Epoch 1/1
916951/916951 [==============================] - 1124s - loss: 1.0908 - acc: 0.6644  
(12, 'Seed', 'yet hillary clinton, for months, and despite so many attacks, repeatedly refused to even say the words radical islam until i challenged her yesterday.')
follow up with: ny attacks, repeatedly refused to even say the words radical islam until i challenged her yesterday. the world that we have to do it and the world that we have to do it. i don’t know what they don’t want to be a lot of money. i don’t know what they don’t want to be a lot of money. i don’t know what they don’t want to be a lot of money. i don’t know what they don’t want to be a lot 
Epoch 1/1
916951/916951 [==============================] - 1125s - loss: 1.0740 - acc: 0.6690  
(13, 'Seed', 'yet hillary clinton, for months, and despite so many attacks, repeatedly refused to even say the words radical islam until i challenged her yesterday.')
follow up with: ny attacks, repeatedly refused to even say the words radical islam until i challenged her yesterday. it’s a great company that we have to do it. we have to do it. we have to do it. i want to be a lot of money to do it. i want to be a lot of money to do it. i want to be a lot of money to do it. i want to be a lot of money to do it. i want to be a lot of money to do it. i want to be a lot of money
Epoch 1/1
916951/916951 [==============================] - 1125s - loss: 1.0593 - acc: 0.6729  
(14, 'Seed', 'yet hillary clinton, for months, and despite so many attacks, repeatedly refused to even say the words radical islam until i challenged her yesterday.')
follow up with: ny attacks, repeatedly refused to even say the words radical islam until i challenged her yesterday. the worst thing is the only one of the problem is the way it was a great person and the press wants to be a lot of money. i want to be a lot of money. i don’t know what they’re going to be a lot of money. i don’t know what they’re going to be able to do it. i don’t know what they’re goi
Epoch 1/1
916951/916951 [==============================] - 1124s - loss: 1.0447 - acc: 0.6769  
(15, 'Seed', 'yet hillary clinton, for months, and despite so many attacks, repeatedly refused to even say the words radical islam until i challenged her yesterday.')
follow up with: ny attacks, repeatedly refused to even say the words radical islam until i challenged her yesterday. i will never ever be the worst thing in the world. i was a great people and i was a great people and i was a great people and i was a great people and the press was a great people and i was a great people and i was a great people and i was a great people and i was a great people and the press was a
Epoch 1/1
916951/916951 [==============================] - 1125s - loss: 1.0345 - acc: 0.6800  
(16, 'Seed', 'yet hillary clinton, for months, and despite so many attacks, repeatedly refused to even say the words radical islam until i challenged her yesterday.')
follow up with: ny attacks, repeatedly refused to even say the words radical islam until i challenged her yesterday. i was a great deal and i said, "what do you want to do it?" i said, "what do you want to do it?" i said, "what do you want to do it?" i said, "what do you want to do it?" i said, "what do you want to do it?" i said, "what do you want to do it?" i said, "what do you want to do it?" i said, "what do 
Epoch 1/1
916951/916951 [==============================] - 1125s - loss: 1.0209 - acc: 0.6835  
(17, 'Seed', 'yet hillary clinton, for months, and despite so many attacks, repeatedly refused to even say the words radical islam until i challenged her yesterday.')
follow up with: ny attacks, repeatedly refused to even say the words radical islam until i challenged her yesterday. they want to be a lot of the states and they don’t know what they’re going to be a lot of the people of the polls. we’re going to be so good to me and they’re going to be a lot of the people of the people of the polls. we’re going to be a lot of the people of the polls. we’re going to b
Epoch 1/1
916951/916951 [==============================] - 1125s - loss: 1.0098 - acc: 0.6864  
(18, 'Seed', 'yet hillary clinton, for months, and despite so many attacks, repeatedly refused to even say the words radical islam until i challenged her yesterday.')
follow up with: ny attacks, repeatedly refused to even say the words radical islam until i challenged her yesterday. the world was a great people. i don’t know what the world is a great people. i don’t know what the people that are the same thing that we have to be a lot of money. we have to be a lot of money. we have to be a lot of money. we have to be a lot of money. we have to be a lot of money. we have to
Epoch 1/1
916951/916951 [==============================] - 1125s - loss: 1.0002 - acc: 0.6886  
(19, 'Seed', 'yet hillary clinton, for months, and despite so many attacks, repeatedly refused to even say the words radical islam until i challenged her yesterday.')
follow up with: ny attacks, repeatedly refused to even say the words radical islam until i challenged her yesterday. the whole thing we have the world is a great company and they want to do it. i want to do it. i want to do it. i want to do it. i want to do it. i want to do it. i want to do it. i want to do it. i want to do it. i want to do it. i want to do it. i want to do it. i want to do it. i want to do it. i
Epoch 1/1
916951/916951 [==============================] - 1124s - loss: 0.9911 - acc: 0.6915  
(20, 'Seed', 'yet hillary clinton, for months, and despite so many attacks, repeatedly refused to even say the words radical islam until i challenged her yesterday.')
follow up with: ny attacks, repeatedly refused to even say the words radical islam until i challenged her yesterday. the world was a great company that we have to be a lot of money. we have to be a lot of money. we have to be a lot of money. we have to be a lot of money. we have to be a lot of money. we have to be a lot of money. we have to be a lot of money. we have to be a lot of money. we have to be a lot of m
Epoch 1/1
916951/916951 [==============================] - 1125s - loss: 0.9823 - acc: 0.6939  
(21, 'Seed', 'yet hillary clinton, for months, and despite so many attacks, repeatedly refused to even say the words radical islam until i challenged her yesterday.')
follow up with: ny attacks, repeatedly refused to even say the words radical islam until i challenged her yesterday. the world is a great people and the press where i was a great job. i want to be a lot of money. i want to take care of our vets. we have to be a lot of money. we have to be a lot of money. we have to be a lot of money. we have to be a lot of money. we have to be a lot of money. we have to be a lot 
Epoch 1/1
916951/916951 [==============================] - 1125s - loss: 0.9728 - acc: 0.6964  
(22, 'Seed', 'yet hillary clinton, for months, and despite so many attacks, repeatedly refused to even say the words radical islam until i challenged her yesterday.')
follow up with: ny attacks, repeatedly refused to even say the words radical islam until i challenged her yesterday. it’s a great company. i was a great deal for the world trade deals. i was a problem. i was a problem. i was a problem. i was a problem. i was a problem. i was a problem. i was a friend of mine who was a great deal for the american people and the world will be a lot of things that are the worst th
Epoch 1/1
916951/916951 [==============================] - 1125s - loss: 0.9664 - acc: 0.6983  
(23, 'Seed', 'yet hillary clinton, for months, and despite so many attacks, repeatedly refused to even say the words radical islam until i challenged her yesterday.')
follow up with: ny attacks, repeatedly refused to even say the words radical islam until i challenged her yesterday. i will tell you that i was a great people. i was the one that was a great deal with the statement that i said the state of the state of the state is a great deal for the state of the state of the states and the state of the states are the second amendment that we have to be a lot of them back. they
Epoch 1/1
916951/916951 [==============================] - 1125s - loss: 0.9593 - acc: 0.7003  
(24, 'Seed', 'yet hillary clinton, for months, and despite so many attacks, repeatedly refused to even say the words radical islam until i challenged her yesterday.')
follow up with: ny attacks, repeatedly refused to even say the words radical islam until i challenged her yesterday. the worst thing that i was a great company and i was a great company and i was a great company and i was a great company and i was a great company and i was a great company and i was a great company and i was a great company and i was a great company and i was a great company and i was a great comp
Epoch 1/1
916951/916951 [==============================] - 1125s - loss: 0.9521 - acc: 0.7024  
(25, 'Seed', 'yet hillary clinton, for months, and despite so many attacks, repeatedly refused to even say the words radical islam until i challenged her yesterday.')
follow up with: ny attacks, repeatedly refused to even say the words radical islam until i challenged her yesterday. it will be a major that we have to be a man of the world trade deals. we have to stop it. we have to be smart. we have to do it. we have to be smart. we have to do it. we have to be smart. we have to do it. we have to be smart. we have to do it. we have to be smart. we have to do it. we have to be 
Epoch 1/1
916951/916951 [==============================] - 1125s - loss: 0.9441 - acc: 0.7045  
(26, 'Seed', 'yet hillary clinton, for months, and despite so many attacks, repeatedly refused to even say the words radical islam until i challenged her yesterday.')
follow up with: ny attacks, repeatedly refused to even say the words radical islam until i challenged her yesterday. the worst thing that i was a statement and i said, "what do you think? i don’t know what the hell he does that?" i said, "what do you think? i don’t know what the hell he does that?" i said, "what do you think? i don’t know what the hell he does that?" i said, "what do you think? i don’t kn
Epoch 1/1
916951/916951 [==============================] - 1126s - loss: 0.9373 - acc: 0.7061  
(27, 'Seed', 'yet hillary clinton, for months, and despite so many attacks, repeatedly refused to even say the words radical islam until i challenged her yesterday.')
follow up with: ny attacks, repeatedly refused to even say the words radical islam until i challenged her yesterday. the world was a friend of mine who was a total disaster. i was a tough country. i was a tough country. i was a tough country. i was the one that had a great deal with the world that i was the worst thing in the world, and i was a tough country. i was a tough country. i was the one that had a great 
Epoch 1/1
916951/916951 [==============================] - 1125s - loss: 0.9309 - acc: 0.7077  
(28, 'Seed', 'yet hillary clinton, for months, and despite so many attacks, repeatedly refused to even say the words radical islam until i challenged her yesterday.')
follow up with: ny attacks, repeatedly refused to even say the words radical islam until i challenged her yesterday. the world is a statement that was a total disaster. i was a great company and i said, "what are you going to go to the country and they do it?" i said, "what do you think? i don’t know what the hell is it was a tough country. i don’t know what they do is that i don’t know what they do is that
Epoch 1/1
916951/916951 [==============================] - 1125s - loss: 0.9255 - acc: 0.7093  
(29, 'Seed', 'yet hillary clinton, for months, and despite so many attacks, repeatedly refused to even say the words radical islam until i challenged her yesterday.')
follow up with: ny attacks, repeatedly refused to even say the words radical islam until i challenged her yesterday. the world is a great company. i was a tough company. i was a tough company. i was a tough country. i was a great person. i was a tough country. i was a tough country. i was a tough country. i was a tough country. i was a great person. i was a tough country. i was a tough country. i was a tough coun

In [ ]:

    

generatetext(i)


    

In [ ]: