This problem is from the last assignment of the Udacity Deep Learning course. The idea is to build a sequence to sequence model using LSTMs that will convert sequences of words of the form:
the quick brown fox
to this form:
eht kciuq nworb xof
i.e., the characters of each word are reversed. This is a similar (although much simplified) scenario to machine translation where input words are in one language and output words are in another. However, the creation of training data has been simplified with this approach.
One caveat with this approach is that we cannot make it a word based seq2seq model, since there is a 1 to 1 correspondence between the two "languages". Instead, we will create a character based seq2seq model so the model cannot depend on any regularity.
The file name for the notebook is a reference to The Shining in case you were wondering.
In [1]:
from __future__ import division, print_function
from keras.layers.core import Activation, Dense, RepeatVector
from keras.layers.recurrent import LSTM
from keras.layers.wrappers import TimeDistributed
from keras.models import Sequential
from sklearn.cross_validation import train_test_split
import nltk
import numpy as np
Using Theano backend.
In [2]:
char_vocab = set(" ")
sentences = []
fin = open("../data/alice_in_wonderland.txt", "rb")
for line in fin:
line = line.strip()
if len(line) == 0:
continue
for sentence in nltk.sent_tokenize(line):
words = []
for word in nltk.word_tokenize(sentence):
word = word.lower()
words.append(word)
for c in word:
char_vocab.add(c)
sentences.append(words)
fin.close()
vocab_size = len(char_vocab)
print("vocab size: %d" % (vocab_size))
vocab size: 45
In [3]:
def reverse_words(words):
reversed_words = []
for w in words:
reversed_words.append("".join(reversed([c for c in w])))
return reversed_words
nb_words_in_seq = 4
input_texts = []
output_texts = []
for sentence in sentences:
ngrams = nltk.ngrams(sentence, nb_words_in_seq)
for ngram in ngrams:
input_texts.append(" ".join(ngram))
output_texts.append(" ".join(reverse_words(ngram)))
maxlen = max([len(x) for x in input_texts])
print("maximum length of sequence: %d chars" % (maxlen))
maximum length of sequence: 36 chars
In [4]:
char2idx = dict((c, i) for i, c in enumerate(char_vocab))
idx2char = {v:k for k, v in char2idx.items()}
In [5]:
X = np.zeros((len(input_texts), maxlen, vocab_size), dtype=np.bool)
Y = np.zeros((len(output_texts), maxlen, vocab_size), dtype=np.bool)
for i, input_text in enumerate(input_texts):
input_text = input_text.ljust(maxlen)
for j, ch in enumerate([c for c in input_text]):
X[i, j, char2idx[ch]] = 1
for i, output_text in enumerate(output_texts):
output_text = output_text.ljust(maxlen)
for j, ch in enumerate([c for c in output_text]):
Y[i, j, char2idx[ch]] = 1
In [6]:
Xtrain, Xtest, Ytrain, Ytest = train_test_split(X, Y, test_size=0.3, random_state=0)
print(Xtrain.shape, Xtest.shape, Ytrain.shape, Ytest.shape)
(16621, 36, 45) (7124, 36, 45) (16621, 36, 45) (7124, 36, 45)
In [7]:
model = Sequential()
model.add(LSTM(512, input_shape=(maxlen, vocab_size), return_sequences=False))
model.add(RepeatVector(maxlen))
model.add(LSTM(512, return_sequences=True))
model.add(TimeDistributed(Dense(vocab_size)))
model.add(Activation("softmax"))
model.compile(loss="categorical_crossentropy", optimizer="adam", metrics=["accuracy"])
In [8]:
def decode_text(probas):
text_seq = []
for i in range(probas.shape[0]):
idx = np.argmax(probas[i])
text_seq.append(idx2char[idx])
return "".join(text_seq).strip()
def cosine_sim(y_test, y_pred):
ytest_flat = np.ravel(y_test)
ypred_flat = np.ravel(y_pred)
cosim = np.dot(ytest_flat, ypred_flat) / (np.linalg.norm(ytest_flat, 2) *
np.linalg.norm(ypred_flat, 2))
return cosim
for iteration in range(51):
print("=" * 50)
print("Iteration-#: %d" % (iteration))
model.fit(Xtrain, Ytrain, batch_size=128, nb_epoch=1,
verbose=0, validation_data=(Xtest, Ytest))
avg_cosim = 0
for i in range(10):
test_idx = np.random.randint(Xtest.shape[0])
x_test = np.array([Xtest[test_idx, :, :]])
y_test = np.array([Ytest[test_idx, :, :]])
y_pred = model.predict([x_test], verbose=0)
cosim = cosine_sim(y_test, y_pred)
xtest_text = decode_text(x_test[0])
ytest_text = decode_text(y_test[0])
ypred_text = decode_text(y_pred[0])
print("input: [%s], expected: [%s], got: [%s], similarity: %.3f" %
(xtest_text, ytest_text, ypred_text, cosim))
avg_cosim += cosim
avg_cosim /= 10
print("Average cosine similarity between label and prediction: %.3f" % (avg_cosim))
==================================================
Iteration-#: 0
input: [think nothing of tumbling], expected: [kniht gnihton fo gnilbmut], got: [eee eeeeee], similarity: 0.577
input: [all , ' he], expected: [lla , ' eh], got: [], similarity: 0.867
input: [, ' alice cautiously], expected: [, ' ecila ylsuoituac], got: [eee], similarity: 0.682
input: [laughing : and when], expected: [gnihgual : dna nehw], got: [ee], similarity: 0.704
input: [with a growl ,], expected: [htiw a lworg ,], got: [], similarity: 0.801
input: [were nine o'clock in], expected: [erew enin kcolc'o ni], got: [ee], similarity: 0.690
input: [-- '' take care], expected: [-- '' ekat erac], got: [e], similarity: 0.780
input: [nearly getting up and], expected: [ylraen gnitteg pu dna], got: [ee], similarity: 0.669
input: [, '' but it], expected: [, '' tub ti], got: [], similarity: 0.841
input: [`` too far ,], expected: [`` oot raf ,], got: [], similarity: 0.833
Average cosine similarity between label and prediction: 0.744
==================================================
Iteration-#: 1
input: [mary ann , what], expected: [yram nna , tahw], got: [eaaaaa aaaaa], similarity: 0.792
input: [asked another of the], expected: [deksa rehtona fo eht], got: [ee eeeeeeeeee], similarity: 0.706
input: [again ( or grunted], expected: [niaga ( ro detnurg], got: [nn rr], similarity: 0.740
input: [, all except the], expected: [, lla tpecxe eht], got: [e eeeee], similarity: 0.778
input: [long claws and a], expected: [gnol swalc dna a], got: [daaa], similarity: 0.778
input: [if it had lost], expected: [fi ti dah tsol], got: [t tt], similarity: 0.803
input: [he wore his crown], expected: [eh erow sih nworc], got: [eeee], similarity: 0.757
input: [in the sun .], expected: [ni eht nus .], got: [e], similarity: 0.837
input: [a hot tureen !], expected: [a toh neerut !], got: [ee eeee], similarity: 0.804
input: [like a candle .], expected: [ekil a eldnac .], got: [e], similarity: 0.789
Average cosine similarity between label and prediction: 0.778
==================================================
Iteration-#: 2
input: [word , i will], expected: [drow , i lliw], got: [ooo www], similarity: 0.845
input: [cat , ' said], expected: [tac , ' dias], got: [tii ,,, aa], similarity: 0.859
input: [them , ' thought], expected: [meht , ' thguoht], got: [thhh euu], similarity: 0.786
input: [the great hall ,], expected: [eht taerg llah ,], got: [eht tttt aa], similarity: 0.806
input: ['' but it all], expected: ['' tub ti lla], got: [, tt llt], similarity: 0.831
input: [the creatures argue .], expected: [eht serutaerc eugra .], got: [ehhtttrrrrrrrrrrrrp], similarity: 0.718
input: [found her way into], expected: [dnuof reh yaw otni], got: [eoo www], similarity: 0.740
input: [the dormouse , '], expected: [eht esuomrod , '], got: [ehh eeeoooo], similarity: 0.821
input: [you could draw treacle], expected: [uoy dluoc ward elcaert], got: [dooo eeeeaa], similarity: 0.672
input: [the name 'w .], expected: [eht eman w' .], got: [ehhh aaa], similarity: 0.835
Average cosine similarity between label and prediction: 0.791
==================================================
Iteration-#: 3
input: [for a conversation .], expected: [rof a noitasrevnoc .], got: [f nnnnncccc], similarity: 0.709
input: [i never heard it], expected: [i reven draeh ti], got: [ev eeeeeeee], similarity: 0.787
input: [long and a sad], expected: [gnol dna a das], got: [dni daa], similarity: 0.831
input: [things are worse than], expected: [sgniht era esrow naht], got: [enis nsss eee], similarity: 0.679
input: [* * * *], expected: [* * * *], got: [o], similarity: 0.913
input: [they were in the], expected: [yeht erew ni eht], got: [eee eee ew], similarity: 0.816
input: [with some surprise that], expected: [htiw emos esirprus taht], got: [sh ssssssrrrrrrrrrrr], similarity: 0.633
input: [piece out of his], expected: [eceip tuo fo sih], got: [eeeppp o ii], similarity: 0.810
input: [over with fright .], expected: [revo htiw thgirf .], got: [rrr hhiiih], similarity: 0.757
input: [; 'but little girls], expected: [; tub' elttil slrig], got: [, tttttllllllllli], similarity: 0.740
Average cosine similarity between label and prediction: 0.767
==================================================
Iteration-#: 4
input: [n't be particular --], expected: [t'n eb ralucitrap --], got: [t' tt eeaaaaaaa---], similarity: 0.754
input: [the flurry of the], expected: [eht yrrulf fo eht], got: [ehh luuof f hhh], similarity: 0.829
input: [but then she remembered], expected: [tub neht ehs derebmemer], got: [too eh eeeeeeeeeeeeee], similarity: 0.718
input: ['i can see you], expected: [i' nac ees uoy], got: ['' naa' ee oo], similarity: 0.856
input: [as i do ,], expected: [sa i od ,], got: [i i o], similarity: 0.884
input: [soo -- oop !], expected: [oos -- poo !], got: [ooo ooo o -], similarity: 0.874
input: [not talk ! '], expected: [ton klat ! '], got: [to tta !!], similarity: 0.868
input: [' said the sage], expected: [' dias eht egas], got: [' diaaseea eaa], similarity: 0.885
input: [cakes , and was], expected: [sekac , dna saw], got: [ecccc , naa aa], similarity: 0.852
input: [and the baby --], expected: [dna eht ybab --], got: [dna ehbbbbbb---], similarity: 0.842
Average cosine similarity between label and prediction: 0.836
==================================================
Iteration-#: 5
input: [, and waving their], expected: [, dna gnivaw rieht], got: [, dna gnii iih], similarity: 0.851
input: [' said alice .], expected: [' dias ecila .], got: [' dias eciia .], similarity: 0.948
input: [set to work at], expected: [tes ot krow ta], got: [ss too ooo aa], similarity: 0.833
input: [the window . '], expected: [eht wodniw . '], got: [eht nnwwwww '], similarity: 0.853
input: ['i 'm getting tired], expected: [i' m' gnitteg derit], got: ['' i' giiigggggiiit], similarity: 0.788
input: [, and would only], expected: [, dna dluow ylno], got: [, dna dooo loo], similarity: 0.865
input: [through the air !], expected: [hguorht eht ria !], got: [ghuohh thh aia], similarity: 0.801
input: [you were all in], expected: [uoy erew lla ni], got: [yoy ellw ii ii], similarity: 0.869
input: [thought about it ,], expected: [thguoht tuoba ti ,], got: [tuuohhtttttb ,], similarity: 0.792
input: [alive with the strange], expected: [evila htiw eht egnarts], got: [eeiil eht eht nnnaaa], similarity: 0.702
Average cosine similarity between label and prediction: 0.830
==================================================
Iteration-#: 6
input: [went on , 'that], expected: [tnew no , taht'], got: [ennn nn , ttaa], similarity: 0.873
input: [too glad to do], expected: [oot dalg ot od], got: [oog daal oo oo], similarity: 0.888
input: [the less there is], expected: [eht ssel ereht si], got: [eht eeet eeee si], similarity: 0.867
input: [, said the dormouse], expected: [, dias eht esuomrod], got: [, dias eht eeemoom], similarity: 0.887
input: [' said the cat], expected: [' dias eht tac], got: [' dias eht aa], similarity: 0.957
input: [some of them hit], expected: [emos fo meht tih], got: [emom fo eht iit], similarity: 0.875
input: [nearly as she could], expected: [ylraen sa ehs dluoc], got: [deaaaa sa ehh dlooc], similarity: 0.849
input: [said the caterpillar .], expected: [dias eht rallipretac .], got: [dias eht ellllrrrrac .], similarity: 0.859
input: [me ' were beautifully], expected: [em ' erew yllufituaeb], got: [e ee eet llllttteee], similarity: 0.726
input: [, she was appealed], expected: [, ehs saw delaeppa], got: [, ehs daa neppppp], similarity: 0.871
Average cosine similarity between label and prediction: 0.865
==================================================
Iteration-#: 7
input: [as the jury had], expected: [sa eht yruj dah], got: [sa eht yuuu aa], similarity: 0.911
input: [spectacles and looked anxiously], expected: [selcatceps dna dekool ylsuoixna], got: [eeellleeec no llllllllllooo], similarity: 0.502
input: [the immediate adoption of], expected: [eht etaidemmi noitpoda fo], got: [eht eeiemme tiiiiirr o], similarity: 0.658
input: [i 'll set dinah], expected: [i ll' tes hanid], got: [i ll' eiit tiit], similarity: 0.850
input: [shaking him and punching], expected: [gnikahs mih dna gnihcnup], got: [gnikhhs nii gnnnnnuuu], similarity: 0.736
input: [alice soon came to], expected: [ecila noos emac ot], got: [ecill ecoc eoo tt], similarity: 0.872
input: [to alice to find], expected: [ot ecila ot dnif], got: [ot ecill ti dnno], similarity: 0.890
input: [waited patiently until it], expected: [detiaw yltneitap litnu ti], got: [ylitaa ynnttrrp eiiiitiii], similarity: 0.656
input: [great hurry to change], expected: [taerg yrruh ot egnahc], got: [rrrrrr ruur o niiaa], similarity: 0.706
input: [in their proper places], expected: [ni rieht reporp secalp], got: [ti eettt rrppppppppppp], similarity: 0.749
Average cosine similarity between label and prediction: 0.753
==================================================
Iteration-#: 8
input: [tunnel for some way], expected: [lennut rof emos yaw], got: [tnnnne foo yee aaw], similarity: 0.827
input: [, perhaps your feelings], expected: [, spahrep ruoy sgnileef], got: [a suurrrr oo niiieef], similarity: 0.685
input: [on both sides of], expected: [no htob sedis fo], got: [oo enht ssis fo], similarity: 0.841
input: [, as usual ,], expected: [, sa lausu ,], got: [, sa ssaaa ,], similarity: 0.922
input: ['s head with great], expected: [s' daeh htiw taerg], got: ['' deah thtt grarr], similarity: 0.815
input: ['they ca n't have], expected: [yeht' ac t'n evah], got: [yeht' t' aa eeeht], similarity: 0.833
input: [, i think --], expected: [, i kniht --], got: [, ni kiiit -], similarity: 0.858
input: [ann , and be], expected: [nna , dna eb], got: [nna , dna ee], similarity: 0.969
input: [thought was that it], expected: [thguoht saw taht ti], got: [hhuuoht sa taat tii], similarity: 0.809
input: [all pardoned . '], expected: [lla denodrap . '], got: [daa dneeeee .''], similarity: 0.854
Average cosine similarity between label and prediction: 0.841
==================================================
Iteration-#: 9
input: [the goldfish kept running], expected: [eht hsifdlog tpek gninnur], got: [eht gnilllol hhh nninnnn], similarity: 0.730
input: [as well as she], expected: [sa llew sa ehs], got: [sa llaw sa ehs], similarity: 0.975
input: [because they would not], expected: [esuaceb yeht dluow ton], got: [eltuaab eehh dooo oo], similarity: 0.737
input: [pencil that squeaked .], expected: [licnep taht dekaeuqs .], got: [ecippp taht deeepppp .], similarity: 0.773
input: [worth while finishing the], expected: [htrow elihw gnihsinif eht], got: [eehhww niiw ggniinii eht], similarity: 0.665
input: [, ' said the], expected: [, ' dias eht], got: [, ' dias eht], similarity: 0.998
input: [up again , dear], expected: [pu niaga , raed], got: [p nnaaa , eead], similarity: 0.878
input: [as much as serpents], expected: [sa hcum sa stnepres], got: [sa scum sa ssssseec], similarity: 0.838
input: [the rabbit 's --], expected: [eht tibbar s' --], got: [eht ttitaa t- s'], similarity: 0.868
input: [the patriotic archbishop of], expected: [eht citoirtap pohsibhcra fo], got: [eht giittppp iiiihhhh ooo], similarity: 0.615
Average cosine similarity between label and prediction: 0.808
==================================================
Iteration-#: 10
input: [made her feel very], expected: [edam reh leef yrev], got: [deam eee eee ylee], similarity: 0.843
input: [the same words as], expected: [eht emas sdrow sa], got: [eht eeam ssra sa], similarity: 0.910
input: [queen 's ears --], expected: [neeuq s' srae --], got: [eeeuq -' saaa --], similarity: 0.915
input: [there 's no pleasing], expected: [ereht s' on gnisaelp], got: [eeeh' s' oo gnisaees], similarity: 0.888
input: [like to go nearer], expected: [ekil ot og reraen], got: [ekil ot ot rrrrrg], similarity: 0.897
input: [, finding that nothing], expected: [, gnidnif taht gnihton], got: [, dnidaaa gnih nihttt], similarity: 0.806
input: [the sort of thing], expected: [eht tros fo gniht], got: [eht tsos fo gniht], similarity: 0.958
input: [you , will you], expected: [uoy , lliw uoy], got: [uoy , lliw uoy], similarity: 0.904
input: [the dormouse , '], expected: [eht esuomrod , '], got: [eht ssommrrd , '], similarity: 0.931
input: [, and was surprised], expected: [, dna saw desirprus], got: [, dna saw ssisssrrus], similarity: 0.864
Average cosine similarity between label and prediction: 0.892
==================================================
Iteration-#: 11
input: [, as it left], expected: [, sa ti tfel], got: [, sa eatt ttl], similarity: 0.876
input: [way to fly up], expected: [yaw ot ylf pu], got: [law o uuy uu], similarity: 0.805
input: [i have dropped them], expected: [i evah deppord meht], got: [i eeahd depppp hht], similarity: 0.798
input: [the rabbit 's --], expected: [eht tibbar s' --], got: [eht saibaa -- -'], similarity: 0.862
input: [said very politely ,], expected: [dias yrev yletilop ,], got: [diassyyev yleiiiit ,], similarity: 0.878
input: [after her as she], expected: [retfa reh sa ehs], got: [retff eeh ss sss], similarity: 0.888
input: [she was looking about], expected: [ehs saw gnikool tuoba], got: [shs saw gnikooo oooa], similarity: 0.882
input: ['do n't be angry], expected: [od' t'n eb yrgna], got: [ddn't'' ebb yggbb], similarity: 0.790
input: [continued the hatter ,], expected: [deunitnoc eht rettah ,], got: [deencccc eht reatataa ,], similarity: 0.688
input: [and looked at it], expected: [dna dekool ta ti], got: [dna kkoll t aa], similarity: 0.877
Average cosine similarity between label and prediction: 0.834
==================================================
Iteration-#: 12
input: [sure she 's the], expected: [erus ehs s' eht], got: [esuu ees o' eht], similarity: 0.943
input: [, and he checked], expected: [, dna eh dekcehc], got: [, dna eh eekcchc], similarity: 0.861
input: [hedgehog , which seemed], expected: [gohegdeh , hcihw demees], got: [geghhrhh , ehhs eeeeees], similarity: 0.699
input: [said to herself ,], expected: [dias ot flesreh ,], got: [dias ot flesehh ,], similarity: 0.958
input: [it , ' she], expected: [ti , ' ehs], got: [ti , ' ehs], similarity: 0.999
input: [they got settled down], expected: [yeht tog delttes nwod], got: [thht tog dettttt dwod], similarity: 0.856
input: [window , and on], expected: [wodniw , dna no], got: [wwwoww , dna oo], similarity: 0.900
input: [and she hurried out], expected: [dna ehs deirruh tuo], got: [dna ehs deurruu too], similarity: 0.931
input: [alice watched the white], expected: [ecila dehctaw eht etihw], got: [ecila dettaaw eht ehiht], similarity: 0.899
input: [' said the cook], expected: [' dias eht kooc], got: [' dias eht kooo], similarity: 0.990
Average cosine similarity between label and prediction: 0.904
==================================================
Iteration-#: 13
input: [, 'to speak to], expected: [, ot' kaeps ot], got: [, ot' eeas ot], similarity: 0.906
input: [just been reading about], expected: [tsuj neeb gnidaer tuoba], got: [tsut eeee dnnnaa tooaa], similarity: 0.790
input: ['what sort of people], expected: [tahw' tros fo elpoep], got: [tahw' trrt ot elppop], similarity: 0.867
input: [too far off to], expected: [oot raf ffo ot], got: [oot faf too ot], similarity: 0.923
input: [myself , i 'm], expected: [flesym , i m'], got: [ylesem , i m'], similarity: 0.931
input: [race was over .], expected: [ecar saw revo .], got: [rrac saw rrov .], similarity: 0.889
input: [other curious creatures .], expected: [rehto suoiruc serutaerc .], got: [ehhht ssorrrc surrcacc], similarity: 0.705
input: [dinn -- ' she], expected: [nnid -- ' ehs], got: [nnnd - '' eh], similarity: 0.924
input: [all round the hall], expected: [lla dnuor eht llah], got: [laa dluol eht llah], similarity: 0.952
input: [, ' said the], expected: [, ' dias eht], got: [, ' dias eht], similarity: 1.000
Average cosine similarity between label and prediction: 0.889
==================================================
Iteration-#: 14
input: [for your walk !], expected: [rof ruoy klaw !], got: [rof roow klaw !], similarity: 0.944
input: [he was n't going], expected: [eh saw t'n gniog], got: [eh saw t'n gniog], similarity: 0.989
input: [then came a little], expected: [neht emac a elttil], got: [eeht ecac a elttil], similarity: 0.948
input: [an offended tone ,], expected: [na dedneffo enot ,], got: [na deednfff eett ,], similarity: 0.884
input: [alice guessed in a], expected: [ecila desseug ni a], got: [ecila dessees ni a], similarity: 0.943
input: ['three inches is such], expected: [eerht' sehcni si hcus], got: [eeet'' eciin ih scuc], similarity: 0.760
input: [and what does it], expected: [dna tahw seod ti], got: [dna dahw ssoo ti], similarity: 0.943
input: [give all else for], expected: [evig lla esle rof], got: [evig lla llel fo], similarity: 0.881
input: [and she began thinking], expected: [dna ehs nageb gnikniht], got: [dna ehs nneeb gnihnnhh], similarity: 0.905
input: [me , please ,], expected: [em , esaelp ,], got: [em , eepeep ,], similarity: 0.932
Average cosine similarity between label and prediction: 0.913
==================================================
Iteration-#: 15
input: [a pair of gloves], expected: [a riap fo sevolg], got: [a taap fo sevlol], similarity: 0.903
input: [, turning to alice], expected: [, gninrut ot ecila], got: [, gnittrt ot ecila], similarity: 0.946
input: [, and found that], expected: [, dna dnuof taht], got: [, dna dnuof taht], similarity: 0.985
input: [hare , who had], expected: [erah , ohw dah], got: [rrah , hhh dah], similarity: 0.953
input: [are they made of], expected: [era yeht edam fo], got: [yre eeet eeam fo], similarity: 0.898
input: [dreadfully savage ! '], expected: [yllufdaerd egavas ! '], got: [ylllfreer ddaaaa ,], similarity: 0.730
input: [herself up closer to], expected: [flesreh pu resolc ot], got: [sleseeh fs esolss oo], similarity: 0.818
input: [lessons , and began], expected: [snossel , dna nageb], got: [sssssss , dna nnabb], similarity: 0.864
input: [to get dry again], expected: [ot teg yrd niaga], got: [ot teg tag niagg], similarity: 0.923
input: [if she was going], expected: [fi ehs saw gniog], got: [fi ehs saw gniog], similarity: 0.985
Average cosine similarity between label and prediction: 0.901
==================================================
Iteration-#: 16
input: [, 'as i mentioned], expected: [, sa' i denoitnem], got: [, saa ii denttnio], similarity: 0.804
input: [say pig , or], expected: [yas gip , ro], got: [yas pip , ro], similarity: 0.964
input: [of great dismay ,], expected: [fo taerg yamsid ,], got: [fo taarr yyamam ,], similarity: 0.894
input: [were shaped like ears], expected: [erew depahs ekil srae], got: [ereh deklas eees erpp], similarity: 0.786
input: [was as steady as], expected: [saw sa ydaets sa], got: [saw sa deeeas sa], similarity: 0.938
input: [panther were sharing a], expected: [rehtnap erew gnirahs a], got: [retttaa rrhh gniaaaa a], similarity: 0.824
input: [a mournful tone ,], expected: [a lufnruom enot ,], got: [a luuuuuo ett], similarity: 0.797
input: [, i 'm not], expected: [, i m' ton], got: [, i m' t'n], similarity: 0.983
input: [what the moral of], expected: [tahw eht larom fo], got: [tahw eht erorf fo], similarity: 0.933
input: [beginning to end ,], expected: [gninnigeb ot dne ,], got: [gnitteee tt dnd ,], similarity: 0.893
Average cosine similarity between label and prediction: 0.882
==================================================
Iteration-#: 17
input: [will you , old], expected: [lliw uoy , dlo], got: [lliw uoy , dod], similarity: 0.964
input: [but i must be], expected: [tub i tsum eb], got: [tub t tsu ees], similarity: 0.907
input: [if he had a], expected: [fi eh dah a], got: [fi eh dah a], similarity: 0.996
input: [like it put more], expected: [ekil ti tup erom], got: [ekil ti ttp errt], similarity: 0.947
input: [-- '' oh ,], expected: [-- '' ho ,], got: [-' '' , o], similarity: 0.879
input: [, turning to the], expected: [, gninrut ot eht], got: [, gninruu ot eht], similarity: 0.966
input: [it woke up again], expected: [ti ekow pu niaga], got: [oi ewow ab niaaa], similarity: 0.912
input: [you were me ?], expected: [uoy erew em ?], got: [uoy eeew em ?], similarity: 0.975
input: [alice found at first], expected: [ecila dnuof ta tsrif], got: [ecila dnuff fa steff], similarity: 0.912
input: [herself from being run], expected: [flesreh morf gnieb nur], got: [slerreh foof gnibb nnn], similarity: 0.813
Average cosine similarity between label and prediction: 0.927
==================================================
Iteration-#: 18
input: [tells the day of], expected: [sllet eht yad fo], got: [llttt shs yad fo], similarity: 0.893
input: [' said the hatter], expected: [' dias eht rettah], got: [' dias eht rettah], similarity: 0.996
input: [their arguments to her], expected: [rieht stnemugra ot reh], got: [eeeht seggggrrr ot eet], similarity: 0.798
input: [her sister sat still], expected: [reh retsis tas llits], got: [rht sseiis sss lltlt], similarity: 0.785
input: [he went mad ,], expected: [eh tnew dam ,], got: [eh neew mam ,], similarity: 0.958
input: [which gave the pigeon], expected: [hcihw evag eht noegip], got: [hiihw evah eht nnitog], similarity: 0.873
input: [whether the blows hurt], expected: [rehtehw eht swolb truh], got: [rehhthw ehs ssuhw ruow], similarity: 0.799
input: [, something comes at], expected: [, gnihtemos semoc ta], got: [, gnihtooos emes s], similarity: 0.816
input: [her sister sat still], expected: [reh retsis tas llits], got: [rht sseiis sss lltlt], similarity: 0.785
input: [went on , 'and], expected: [tnew no , dna'], got: [nnew no , dnas], similarity: 0.961
Average cosine similarity between label and prediction: 0.866
==================================================
Iteration-#: 19
input: [said the cat .], expected: [dias eht tac .], got: [dias eht tac .], similarity: 0.999
input: [gryphon never learnt it], expected: [nohpyrg reven tnrael ti], got: [goipyrg reeee neeea ti], similarity: 0.827
input: [alice , 'how am], expected: [ecila , woh' ma], got: [ecila , wow' aa], similarity: 0.965
input: [it was the first], expected: [ti saw eht tsrif], got: [ti saw eht ttrif], similarity: 0.975
input: [to turn into a], expected: [ot nrut otni a], got: [ot trrt ttniaa], similarity: 0.898
input: [and two , as], expected: [dna owt , sa], got: [dna ttw , a], similarity: 0.908
input: [pray , what is], expected: [yarp , tahw si], got: [yaap , tahw si], similarity: 0.971
input: [fly up into a], expected: [ylf pu otni a], got: [ylf pi ton ia], similarity: 0.903
input: [jury all looked puzzled], expected: [yruj lla dekool delzzup], got: [yrac dla dlkool dlzzup], similarity: 0.836
input: [choking of the suppressed], expected: [gnikohc fo eht desserppus], got: [gnikoow fo eht sseepppuus], similarity: 0.820
Average cosine similarity between label and prediction: 0.910
==================================================
Iteration-#: 20
input: [trees had a door], expected: [seert dah a rood], got: [ereht das a roo], similarity: 0.918
input: [soup of the evening], expected: [puos fo eht gnineve], got: [pupp fo eht gninee], similarity: 0.914
input: [in the grass ,], expected: [ni eht ssarg ,], got: [ni eht ssgrg ,], similarity: 0.980
input: [been that , '], expected: [neeb taht , '], got: [neeb taht , '], similarity: 0.999
input: [: it just grazed], expected: [: ti tsuj dezarg], got: [: ti tsuj degrr], similarity: 0.943
input: [examining the roses .], expected: [gninimaxe eht sesor .], got: [gninninem eet sseeg .], similarity: 0.827
input: [gone far before they], expected: [enog raf erofeb yeht], got: [eoog rof reffeb yeht], similarity: 0.906
input: [carrier , ' she], expected: [reirrac , ' ehs], got: [rrrrrac , ' ehs], similarity: 0.952
input: [always to have lessons], expected: [syawla ot evah snossel], got: [saaaaa ot eveh snosoel], similarity: 0.854
input: [n't much care where], expected: [t'n hcum erac erehw], got: [tcc hcum ecac eeehw], similarity: 0.936
Average cosine similarity between label and prediction: 0.923
==================================================
Iteration-#: 21
input: [at the bottom of], expected: [ta eht mottob fo], got: [ta eht eootob fo], similarity: 0.957
input: [was very fond of], expected: [saw yrev dnof fo], got: [saw yrev dnuf fo], similarity: 0.991
input: [understand why it is], expected: [dnatsrednu yhw ti si], got: [dnneeddnus yahw ii s], similarity: 0.703
input: [his shining tail ,], expected: [sih gninihs liat ,], got: [sih gnihnhs eiil ,], similarity: 0.927
input: [come on ! '], expected: [emoc no ! '], got: [emoc no ! '], similarity: 0.992
input: [' ( 'i have], expected: [' ( i' evah], got: [' i m' evah], similarity: 0.963
input: [larger , it must], expected: [regral , ti tsum], got: [reraag , ti sum], similarity: 0.924
input: [her voice sounded hoarse], expected: [reh eciov dednuos esraoh], got: [eeh ncioo deuoooo sessah], similarity: 0.785
input: [fast asleep , and], expected: [tsaf peelsa , dna], got: [tsof eeesep , dna], similarity: 0.916
input: [to have changed since], expected: [ot evah degnahc ecnis], got: [ot eva devvaahh ecinn], similarity: 0.773
Average cosine similarity between label and prediction: 0.893
==================================================
Iteration-#: 22
input: [, she went on], expected: [, ehs tnew no], got: [, ehs tnew no], similarity: 0.998
input: [; but when the], expected: [; tub nehw eht], got: [; tub neht eht], similarity: 0.979
input: [she had never seen], expected: [ehs dah reven nees], got: [ehs dah reven sses], similarity: 0.959
input: ['of course not ,], expected: [fo' esruoc ton ,], got: [fo' esruoc ton ,], similarity: 0.994
input: [table , half hoping], expected: [elbat , flah gnipoh], got: [elaat , llah gnihoh], similarity: 0.932
input: [by her sister on], expected: [yb reh retsis no], got: [yb ehs srtsis oo], similarity: 0.882
input: [little pattering of footsteps], expected: [elttil gnirettap fo spetstoof], got: [eltta dittprrrp ooofssssffoo], similarity: 0.671
input: [on it ( as], expected: [no ti ( sa], got: [no ti s sa], similarity: 0.981
input: [felt quite relieved to], expected: [tlef etiuq deveiler ot], got: [tlel etiuq denieeee ot], similarity: 0.913
input: ['off with her head], expected: [ffo' htiw reh daeh], got: [ffo' htiw reh daeh], similarity: 0.992
Average cosine similarity between label and prediction: 0.930
==================================================
Iteration-#: 23
input: [beautiful , beauti --], expected: [lufituaeb , ituaeb --], got: [tufuffaa , tiiae -], similarity: 0.718
input: [edge of her skirt], expected: [egde fo reh triks], got: [deeg fo reh ttiis], similarity: 0.849
input: [too glad to do], expected: [oot dalg ot od], got: [oot dlal ot od], similarity: 0.936
input: ['who are you ?], expected: [ohw' era uoy ?], got: [ohw'ere uuoy ?], similarity: 0.904
input: [such a thing as], expected: [hcus a gniht sa], got: [hcus a gnnht sa], similarity: 0.974
input: [, ' said alice], expected: [, ' dias ecila], got: [, ' dias ecila], similarity: 1.000
input: [, so suddenly that], expected: [, os ylneddus taht], got: [, os dnnndsus ttht], similarity: 0.910
input: [the trumpet , and], expected: [eht tepmurt , dna], got: [eht tetrrut , dna], similarity: 0.938
input: [saying , in a], expected: [gniyas , ni a], got: [gniyas , iiaa], similarity: 0.944
input: [and no room at], expected: [dna on moor ta], got: [dna oo ooom ta], similarity: 0.945
Average cosine similarity between label and prediction: 0.912
==================================================
Iteration-#: 24
input: [the parchment scroll ,], expected: [eht tnemhcrap llorcs ,], got: [eht nnntcceec llssss ,], similarity: 0.813
input: [dig of her sharp], expected: [gid fo reh prahs], got: [gig fo reh harhs], similarity: 0.912
input: [is thirteen , and], expected: [si neetriht , dna], got: [si reettttt , dna], similarity: 0.907
input: ['i only wish it], expected: [i' ylno hsiw ti], got: [i' ylno hihw ti], similarity: 0.966
input: [the way out of], expected: [eht yaw tuo fo], got: [eht yaw tuo fo], similarity: 0.995
input: [downwards , and the], expected: [sdrawnwod , dna eht], got: [srwwwwwww , dna eht], similarity: 0.745
input: [this a very difficult], expected: [siht a yrev tluciffid], got: [tiht a yrev tliifffdd], similarity: 0.887
input: [could n't have done], expected: [dluoc t'n evah enod], got: [dluoc t'n evah eeod], similarity: 0.985
input: [' ( she might], expected: [' ( ehs thgim], got: [' ' ehs thgim], similarity: 0.969
input: [, i only wish], expected: [, i ylno hsiw], got: [, i ylnl hsiw], similarity: 0.976
Average cosine similarity between label and prediction: 0.916
==================================================
Iteration-#: 25
input: [the officer could get], expected: [eht reciffo dluoc teg], got: [eht rrecoff dluoc tet], similarity: 0.869
input: [we need n't try], expected: [ew deen t'n yrt], got: [ew deee t'n yet], similarity: 0.949
input: [her toes when they], expected: [reh seot nehw yeht], got: [eet seos neht yeht], similarity: 0.914
input: [and as for the], expected: [dna sa rof eht], got: [dna sa rof eht], similarity: 0.996
input: [of the baby ?], expected: [fo eht ybab ?], got: [fo eht yaab ?], similarity: 0.988
input: [then stop . '], expected: [neht pots . '], got: [neht toht . '], similarity: 0.953
input: [, ' said the], expected: [, ' dias eht], got: [, ' dias eht], similarity: 1.000
input: [in a mournful tone], expected: [ni a lufnruom enot], got: [ni a ruuuuuoo nnn], similarity: 0.861
input: [my kitchen at all], expected: [ym nehctik ta lla], got: [ym eehtint ta lla], similarity: 0.909
input: [mock turtle went on], expected: [kcom eltrut tnew no], got: [kcom eltruuttnew no], similarity: 0.975
Average cosine similarity between label and prediction: 0.941
==================================================
Iteration-#: 26
input: [are gone from this], expected: [era enog morf siht], got: [era rnor roof hiht], similarity: 0.904
input: [on the spot .], expected: [no eht tops .], got: [no eht toos .], similarity: 0.969
input: [there she saw maps], expected: [ereht ehs was spam], got: [ereht ehs sas saas], similarity: 0.926
input: [' said alice ,], expected: [' dias ecila ,], got: [' dias ecila ,], similarity: 1.000
input: [said alice , always], expected: [dias ecila , syawla], got: [dias ecila , ssaaaa], similarity: 0.935
input: [looking for it ,], expected: [gnikool rof ti ,], got: [gnikool roof t ,,], similarity: 0.923
input: [the queen , and], expected: [eht neeuq , dna], got: [eht neeuq , dna], similarity: 1.000
input: [a good way off], expected: [a doog yaw ffo], got: [a wood faa fff], similarity: 0.924
input: [little pebbles came rattling], expected: [elttil selbbep emac gnilttar], got: [elttib elbleep slaa gnitttra], similarity: 0.766
input: [it , and yet], expected: [ti , dna tey], got: [ti , dna tey], similarity: 0.993
Average cosine similarity between label and prediction: 0.934
==================================================
Iteration-#: 27
input: ['who are you ?], expected: [ohw' era uoy ?], got: [hhw' era uoy ?], similarity: 0.970
input: [taken his watch out], expected: [nekat sih hctaw tuo], got: [nenet saw hciww tuo], similarity: 0.902
input: [shepherd boy -- and], expected: [drehpehs yob -- dna], got: [dehehehh yoy -- dna], similarity: 0.863
input: [-- or if you], expected: [-- ro fi uoy], got: [-- r- f uoo], similarity: 0.929
input: [might just as well], expected: [thgim tsuj sa llew], got: [thgim tsum sa llew], similarity: 0.985
input: [saying to herself ,], expected: [gniyas ot flesreh ,], got: [gnisas ot flesreh ,], similarity: 0.966
input: [going to happen next], expected: [gniog ot neppah txen], got: [gnioo ot nappah nnen], similarity: 0.918
input: [heard the rabbit just], expected: [draeh eht tibbar tsuj], got: [deaeh eht ttibar ttub], similarity: 0.903
input: [it does n't matter], expected: [ti seod t'n rettam], got: [ti ssod t'n rettaa], similarity: 0.957
input: [in reply ( it], expected: [ni ylper ( ti], got: [ni yppep ` ti], similarity: 0.953
Average cosine similarity between label and prediction: 0.935
==================================================
Iteration-#: 28
input: [will you , wo], expected: [lliw uoy , ow], got: [lliw uoy , ow], similarity: 0.999
input: [the mock turtle in], expected: [eht kcom eltrut ni], got: [eht kcom eltrut ni], similarity: 0.998
input: [up in spite of], expected: [pu ni etips fo], got: [pu ni etisi fo], similarity: 0.935
input: [; if i 'm], expected: [; fi i m'], got: [; fi i m'], similarity: 0.996
input: [hungry , in which], expected: [yrgnuh , ni hcihw], got: [ylguhh , ii hcihw], similarity: 0.934
input: [is , you see], expected: [si , uoy ees], got: [si , uoy ees], similarity: 0.999
input: [: ' '' miss], expected: [: ' '' ssim], got: [: ' '' sshs], similarity: 0.959
input: [any more ! '], expected: [yna erom ! '], got: [ynm erom ! '], similarity: 0.978
input: [air of great relief], expected: [ria fo taerg feiler], got: [rir ro taerr eeelel], similarity: 0.879
input: [should it ? '], expected: [dluohs ti ? '], got: [dluohs ti ? '], similarity: 0.993
Average cosine similarity between label and prediction: 0.967
==================================================
Iteration-#: 29
input: [at all , '], expected: [ta lla , '], got: [ta lla , '], similarity: 1.000
input: [beginning , ' the], expected: [gninnigeb , ' eht], got: [gnineege , ' ehtt], similarity: 0.796
input: [the duchess , who], expected: [eht ssehcud , ohw], got: [eht sseccud , whh], similarity: 0.948
input: ['' be what you], expected: ['' eb tahw uoy], got: ['' eb tahw uoy], similarity: 0.988
input: [every now and then], expected: [yreve won dna neht], got: [yrvev woy dna neht], similarity: 0.939
input: [magpie began wrapping itself], expected: [eipgam nageb gnipparw flesti], got: [eeiea naaep gnipaar lliies], similarity: 0.695
input: [, at any rate], expected: [, ta yna etar], got: [, ta yna etar], similarity: 0.997
input: [you our cat dinah], expected: [uoy ruo tac hanid], got: [uoy uoy dac haiad], similarity: 0.876
input: [sister was reading ,], expected: [retsis saw gnidaer ,], got: [setsis srw gnidaea ,], similarity: 0.901
input: [screamed the gryphon .], expected: [demaercs eht nohpyrg .], got: [demaacs reh nohpprg .], similarity: 0.639
Average cosine similarity between label and prediction: 0.878
==================================================
Iteration-#: 30
input: [, and then nodded], expected: [, dna neht deddon], got: [, dna neht dednoh], similarity: 0.956
input: [and a scroll of], expected: [dna a llorcs fo], got: [dna a llrocc fo], similarity: 0.943
input: [said a sleepy voice], expected: [dias a ypeels eciov], got: [dias a yyevss eciov], similarity: 0.943
input: [the march hare .], expected: [eht hcram erah .], got: [eht hcram erah .], similarity: 1.000
input: [puppy ; whereupon the], expected: [yppup ; nopuerehw eht], got: [ppppp ; roohheuq' eht], similarity: 0.823
input: [about for it ,], expected: [tuoba rof ti ,], got: [tuoba tof ti ,], similarity: 0.987
input: [i 'll come up], expected: [i ll' emoc pu], got: [i ll' ecoc ro], similarity: 0.957
input: [to say anything .], expected: [ot yas gnihtyna .], got: [ot yas nnihtyyn .], similarity: 0.956
input: ['re nervous or not], expected: [er' suovren ro ton], got: [re' suorrrc uo ton], similarity: 0.898
input: [coming to look for], expected: [gnimoc ot kool rof], got: [gnioos ta kool fff], similarity: 0.883
Average cosine similarity between label and prediction: 0.935
==================================================
Iteration-#: 31
input: [' but they began], expected: [' tub yeht nageb], got: [' tub yeht nageb], similarity: 0.983
input: [see you 're trying], expected: [ees uoy er' gniyrt], got: [ess uoy rr' gniyrt], similarity: 0.957
input: [that they were filled], expected: [taht yeht erew dellif], got: [taht yeht erew dlliiw], similarity: 0.936
input: [folded her hands ,], expected: [dedlof reh sdnah ,], got: [dedlof reh sdnah ,], similarity: 0.981
input: [hard as it could], expected: [drah sa ti dluoc], got: [daah sa t dluoc], similarity: 0.953
input: [it only grinned a], expected: [ti ylno dennirg a], got: [ti ylno dninrr], similarity: 0.922
input: [said , by way], expected: [dias , yb yaw], got: [dias , yb yaw], similarity: 0.996
input: [were lying on their], expected: [erew gniyl no rieht], got: [erew gnilb no rieht], similarity: 0.948
input: [it belongs to a], expected: [ti sgnoleb ot a], got: [ti snollel ot a], similarity: 0.912
input: [to your places !], expected: [ot ruoy secalp !], got: [ot uuoy ecallp !], similarity: 0.936
Average cosine similarity between label and prediction: 0.952
==================================================
Iteration-#: 32
input: ['boots and shoes under], expected: [stoob' dna seohs rednu], got: [sooob' dna eeshs dedoo], similarity: 0.867
input: ['you mean you ca], expected: [uoy' naem uoy ac], got: [uoy' naen uoy ua], similarity: 0.948
input: [it , ' added], expected: [ti , ' dedda], got: [ti , ' dedda], similarity: 1.000
input: [and their slates and], expected: [dna rieht setals dna], got: [dna sreht seeals nna], similarity: 0.909
input: [the small ones choked], expected: [eht llams seno dekohc], got: [eht slaam elol dekoos], similarity: 0.822
input: ['we went to school], expected: [ew' tnew ot loohcs], got: [ew' tnew ot loohcs], similarity: 0.987
input: [me hear the name], expected: [em raeh eht eman], got: [em maeh eht eman], similarity: 0.976
input: [' -- change lobsters], expected: [' -- egnahc sretsbol], got: [' -- ecnnah ssehteeo], similarity: 0.825
input: [at the bottom of], expected: [ta eht mottob fo], got: [ta eht mottob fo], similarity: 0.987
input: [, the lizard )], expected: [, eht drazil )], got: [, eht drriag ,], similarity: 0.897
Average cosine similarity between label and prediction: 0.922
==================================================
Iteration-#: 33
input: [a thousand times as], expected: [a dnasuoht semit sa], got: [a dnaatto ssat ss], similarity: 0.772
input: [been the right size], expected: [neeb eht thgir ezis], got: [neeb eht thgib ezis], similarity: 0.981
input: [she added aloud .], expected: [ehs dedda duola .], got: [ehs dedda lluoa .], similarity: 0.923
input: [laughed so much at], expected: [dehgual os hcum ta], got: [dedalad os hcc ta], similarity: 0.802
input: [under the circumstances .], expected: [rednu eht secnatsmucric .], got: [deduo eht scnacccccccc], similarity: 0.697
input: [first ; then followed], expected: [tsrif ; neht dewollof], got: [tsrff ; teht dellllff], similarity: 0.870
input: [i shall see it], expected: [i llahs ees ti], got: [i llahs ees ti], similarity: 1.000
input: [, perhaps your feelings], expected: [, spahrep ruoy sgnileef], got: [, sparrrp roof gniseeef], similarity: 0.795
input: [peeping anxiously into her], expected: [gnipeep ylsuoixna otni reh], got: [niipaep ylsiiiaaa ttoi eht], similarity: 0.750
input: [moment alice appeared ,], expected: [tnemom ecila deraeppa ,], got: [nnmmom eiila reaappaa ,], similarity: 0.884
Average cosine similarity between label and prediction: 0.847
==================================================
Iteration-#: 34
input: [they passed too close], expected: [yeht dessap oot esolc], got: [yeht sesaep ott esolc], similarity: 0.937
input: [look at it !], expected: [kool ta ti !], got: [kool ta ti !], similarity: 0.996
input: [another moment it was], expected: [rehtona tnemom ti saw], got: [rehtona tnemon ti saw], similarity: 0.975
input: [came first ; then], expected: [emac tsrif ; neht], got: [emac tsrff ; neet], similarity: 0.972
input: [would n't keep appearing], expected: [dluow t'n peek gniraeppa], got: [dluow t'n eeet niiaappee], similarity: 0.852
input: [duchess was very ugly], expected: [ssehcud saw yrev ylgu], got: [ssehsud saw yrev yluy], similarity: 0.948
input: [enough to get through], expected: [hguone ot teg hguorht], got: [tggooe ot teg hggooht], similarity: 0.922
input: [and found herself lying], expected: [dna dnuof flesreh gniyl], got: [dna dnuofffllesreh nnil], similarity: 0.785
input: [in a mournful tone], expected: [ni a lufnruom enot], got: [ni a luuuuroo etnt], similarity: 0.855
input: [be almost out of], expected: [eb tsomla tuo fo], got: [eb tsolla tuo fo], similarity: 0.961
Average cosine similarity between label and prediction: 0.920
==================================================
Iteration-#: 35
input: [in a wondering tone], expected: [ni a gnirednow enot], got: [ni a gnirwdrow enot], similarity: 0.945
input: [creature when i get], expected: [erutaerc nehw i teg], got: [errcceec nehw n teg], similarity: 0.877
input: [except the lizard ,], expected: [tpecxe eht drazil ,], got: [tcccep eht drraac ,], similarity: 0.849
input: [whistle to it ;], expected: [eltsihw ot ti ;], got: [elttiiw ot ti ;], similarity: 0.942
input: [give all else for], expected: [evig lla esle rof], got: [evig lll leel rof], similarity: 0.929
input: [must remember , '], expected: [tsum rebmemer , '], got: [tsum eebmemmr , '], similarity: 0.941
input: [, carried on the], expected: [, deirrac no eht], got: [, deirrrc no eht], similarity: 0.981
input: [are gone , if], expected: [era enog , fi], got: [era enog , fi], similarity: 0.991
input: [first witness was the], expected: [tsrif ssentiw saw eht], got: [tteif ssenniw saw eht], similarity: 0.945
input: [, turning to alice], expected: [, gninrut ot ecila], got: [, gninrut ot ecila], similarity: 0.992
Average cosine similarity between label and prediction: 0.939
==================================================
Iteration-#: 36
input: [say anything about it], expected: [yas gnihtyna tuoba ti], got: [yaw gnihtyna tuoba ti], similarity: 0.972
input: [down the middle ,], expected: [nwod eht elddim ,], got: [nwod eht elddim ,], similarity: 0.984
input: [window , and one], expected: [wodniw , dna eno], got: [wwwoid , dna eno], similarity: 0.927
input: [the fan she was], expected: [eht naf ehs saw], got: [eht nna ehh saw], similarity: 0.941
input: [because she was exactly], expected: [esuaceb ehs saw yltcaxe], got: [esuaceb ehs saw lltaexa], similarity: 0.933
input: [it can talk :], expected: [ti nac klat :], got: [ti kna klat :], similarity: 0.944
input: [their slates , and], expected: [rieht setals , dna], got: [reeht setall , dna], similarity: 0.968
input: [all its feet at], expected: [lla sti teef ta], got: [lla tti teef ta], similarity: 0.960
input: [in your knocking ,], expected: [ni ruoy gnikconk ,], got: [ni yooo gnikconc ,], similarity: 0.929
input: [the dodo in an], expected: [eht odod ni na], got: [eht oodd ni na], similarity: 0.976
Average cosine similarity between label and prediction: 0.953
==================================================
Iteration-#: 37
input: [on the stairs .], expected: [no eht sriats .], got: [no eht sirats .], similarity: 0.955
input: ['yes , ' said], expected: [sey' , ' dias], got: [sey' , ' dias], similarity: 0.999
input: [diamonds , and walked], expected: [sdnomaid , dna deklaw], got: [ssasssid , dna dllle], similarity: 0.817
input: [after them ! '], expected: [retfa meht ! '], got: [retfa meht !!'], similarity: 0.968
input: [said to herself 'now], expected: [dias ot flesreh won'], got: [dias ot feesrrh wnw], similarity: 0.910
input: [the baby at her], expected: [eht ybab ta reh], got: [eht yabb ya reh], similarity: 0.948
input: [alice , whose thoughts], expected: [ecila , esohw sthguoht], got: [ecila , ehoh' thghohes], similarity: 0.815
input: [this short speech ,], expected: [siht trohs hceeps ,], got: [siht tseht hcccps ,], similarity: 0.907
input: [this a very curious], expected: [siht a yrev suoiruc], got: [siht , rrvv suoirec], similarity: 0.933
input: [alice , 'how am], expected: [ecila , woh' ma], got: [ecila , oo'' mm], similarity: 0.937
Average cosine similarity between label and prediction: 0.919
==================================================
Iteration-#: 38
input: [not remember ever having], expected: [ton rebmemer reve gnivah], got: [too eemmemem rvvv giivah], similarity: 0.858
input: [better now -- but], expected: [retteb won -- tub], got: [retteb woo -- tub], similarity: 0.987
input: [the queen , and], expected: [eht neeuq , dna], got: [eht neeuq , dna], similarity: 1.000
input: [the duchess said to], expected: [eht ssehcud dias ot], got: [eht ssehcud dias ot], similarity: 0.993
input: [had this fit --], expected: [dah siht tif --], got: [dah tiht tfi --], similarity: 0.951
input: [was delighted to find], expected: [saw dethgiled ot dnif], got: [saw detigglee ot dnid], similarity: 0.923
input: [it up into a], expected: [ti pu otni a], got: [ti ni otni a], similarity: 0.964
input: ['only , as it], expected: [ylno' , sa ti], got: [ylno' , sa ti], similarity: 0.997
input: [and looked at it], expected: [dna dekool ta ti], got: [dna dekool ta ti], similarity: 1.000
input: [-- the rabbit 's], expected: [-- eht tibbar s'], got: [-- eht tibbar s'], similarity: 0.989
Average cosine similarity between label and prediction: 0.966
==================================================
Iteration-#: 39
input: [, of course ,], expected: [, fo esruoc ,], got: [, fo esruoc ,], similarity: 0.999
input: [next thing was to], expected: [txen gniht saw ot], got: [txen giiit saw ot], similarity: 0.960
input: [, i 'm afraid], expected: [, i m' diarfa], got: [, i mi diarfa], similarity: 0.976
input: [meekly : 'i 'm], expected: [ylkeem : i' m'], got: [yleeem : i' mm], similarity: 0.961
input: [it , ' said], expected: [ti , ' dias], got: [ti , ' dias], similarity: 1.000
input: [and she , oh], expected: [dna ehs , ho], got: [dna ehs , oh], similarity: 0.953
input: [duchess ; 'and that], expected: [ssehcud ; dna' taht], got: [ssehcud ; dna' taht], similarity: 0.988
input: [to write with one], expected: [ot etirw htiw eno], got: [ot etirw htiw eno], similarity: 0.993
input: [said in a low], expected: [dias ni a wol], got: [dias ni a wol], similarity: 1.000
input: [world am i ?], expected: [dlrow ma i ?], got: [dlrow mi ? '], similarity: 0.924
Average cosine similarity between label and prediction: 0.975
==================================================
Iteration-#: 40
input: [how neatly spread his], expected: [woh yltaen daerps sih], got: [noh yltnan deprus sih], similarity: 0.859
input: [herself up closer to], expected: [flesreh pu resolc ot], got: [flesreh fu eeollp ot], similarity: 0.899
input: [e -- e --], expected: [e -- e --], got: [e -- e---], similarity: 0.974
input: [twice she had peeped], expected: [eciwt ehs dah depeep], got: [ecitw ehs dah depeppp], similarity: 0.931
input: [wood , ' continued], expected: [doow , ' deunitnoc], got: [dood , ' dennitnoc], similarity: 0.950
input: [are you fond --], expected: [era uoy dnof --], got: [era roy dnoo --], similarity: 0.966
input: ['i do n't see], expected: [i' od t'n ees], got: [i' od t'n ees], similarity: 1.000
input: [i ca n't understand], expected: [i ac t'n dnatsrednu], got: [i t' t'n dnasseennn], similarity: 0.858
input: [as she said this], expected: [sa ehs dias siht], got: [sa ehs dias siht], similarity: 1.000
input: [top with its arms], expected: [pot htiw sti smra], got: [oot htiw sii srra], similarity: 0.954
Average cosine similarity between label and prediction: 0.939
==================================================
Iteration-#: 41
input: [march hare said in], expected: [hcram erah dias ni], got: [hcram erah dias ni], similarity: 0.998
input: [, as the large], expected: [, sa eht egral], got: [, sa eht egral], similarity: 0.998
input: [i meant , '], expected: [i tnaem , '], got: [i taeem , '], similarity: 0.960
input: ['i wo n't have], expected: [i' ow t'n evah], got: [i' ow t'n evah], similarity: 0.995
input: [far off ) .], expected: [raf ffo ) .], got: [faf ffo . )], similarity: 0.935
input: [in fact , we], expected: [ni tcaf , ew], got: [ni tcaf , ew], similarity: 0.999
input: [see what was the], expected: [ees tahw saw eht], got: [ees tahw taw ehs], similarity: 0.960
input: [, it must be], expected: [, ti tsum eb], got: [, ti tsum eb], similarity: 1.000
input: [the turtles all advance], expected: [eht seltrut lla ecnavda], got: [eht lletrrt dla eenavaa], similarity: 0.835
input: [last it unfolded its], expected: [tsal ti dedlofnu sti], got: [tsal ti dednnnoo tii], similarity: 0.845
Average cosine similarity between label and prediction: 0.952
==================================================
Iteration-#: 42
input: [-- and the twinkling], expected: [-- dna eht gnilkniwt], got: [-- dna eht gnikniiww], similarity: 0.909
input: [could n't have done], expected: [dluoc t'n evah enod], got: [dluoc t'n evah enod], similarity: 0.995
input: [it woke up again], expected: [ti ekow pu niaga], got: [oi ekow pu niaga], similarity: 0.977
input: [which the wretched hatter], expected: [hcihw eht dehcterw rettah], got: [hcihw eht tetcttre ettah], similarity: 0.902
input: [ought to have been], expected: [thguo ot evah neeb], got: [thgoo ot evah nevb], similarity: 0.965
input: [by a row of], expected: [yb a wor fo], got: [yb a wor fo], similarity: 0.990
input: [and the two creatures], expected: [dna eht owt serutaerc], got: [dna eht ott serttaarr], similarity: 0.935
input: [in the wood ,], expected: [ni eht doow ,], got: [ni eht doow ,], similarity: 0.997
input: [came the king and], expected: [emac eht gnik dna], got: [emac eht gnik dna], similarity: 0.997
input: [but the wise little], expected: [tub eht esiw elttil], got: [tub eht esiw elttil], similarity: 0.995
Average cosine similarity between label and prediction: 0.966
==================================================
Iteration-#: 43
input: [began to get rather], expected: [nageb ot teg rehtar], got: [gngeb ot tag rehtar], similarity: 0.959
input: [, dear ! ''], expected: [, raed ! ''], got: [, raed ! '], similarity: 0.983
input: [round and look up], expected: [dnuor dna kool pu], got: [dnuor dna kool pu], similarity: 0.999
input: [the caterpillar angrily ,], expected: [eht rallipretac ylirgna ,], got: [eht rallipretcc ylirana ,], similarity: 0.932
input: [king repeated angrily ,], expected: [gnik detaeper ylirgna ,], got: [gnik detnaper yliagaa ,], similarity: 0.909
input: [' said alice .], expected: [' dias ecila .], got: [' dias ecila .], similarity: 1.000
input: [ought to have been], expected: [thguo ot evah neeb], got: [thguo ot evah nevb], similarity: 0.983
input: [begged the mouse to], expected: [deggeb eht esuom ot], got: [deggeb eht esuod ot], similarity: 0.975
input: [a pause : 'the], expected: [a esuap : eht'], got: [a esuup , ehtw], similarity: 0.964
input: [-- and they do], expected: [-- dna yeht od], got: [-- dna yeht od], similarity: 1.000
Average cosine similarity between label and prediction: 0.970
==================================================
Iteration-#: 44
input: [key was too small], expected: [yek saw oot llams], got: [yek kaw oss llams], similarity: 0.941
input: [use in the trial], expected: [esu ni eht lairt], got: [ess ni eht llart], similarity: 0.936
input: [she found her head], expected: [ehs dnuof reh daeh], got: [ehs dnuof ehh daeh], similarity: 0.955
input: [hookah into its mouth], expected: [hakooh otni sti htuom], got: [hoohoh ttiw ton htuom], similarity: 0.790
input: [rabbit blew three blasts], expected: [tibbar welb eerht stsalb], got: [tibbab lleb ereht ssrblr], similarity: 0.805
input: [garden with one eye], expected: [nedrag htiw eno eye], got: [dedrah htiw eno eee], similarity: 0.923
input: [lessons to learn !], expected: [snossel ot nrael !], got: [snossel ot nrael !], similarity: 0.975
input: [signifies much , '], expected: [seifingis hcum , '], got: [ssehisii' hcuj ,], similarity: 0.865
input: [and such things that], expected: [dna hcus sgniht taht], got: [dna hcnh snniht taht], similarity: 0.935
input: [can i have done], expected: [nac i evah enod], got: [nac a evah enod], similarity: 0.971
Average cosine similarity between label and prediction: 0.909
==================================================
Iteration-#: 45
input: [do it again and], expected: [od ti niaga dna], got: [od ti niaga dna], similarity: 0.989
input: [in my own tears], expected: [ni ym nwo sraet], got: [ni nm won sraet], similarity: 0.892
input: [, ' the king], expected: [, ' eht gnik], got: [, ' eht gnik], similarity: 1.000
input: [thought this a very], expected: [thguoht siht a yrev], got: [thguoht tiht i yrev], similarity: 0.966
input: [thought it over a], expected: [thguoht ti revo a], got: [thguoht ti revo a], similarity: 0.999
input: [of it altogether ;], expected: [fo ti rehtegotla ;], got: [fo ti rehteggttl ;], similarity: 0.921
input: ['ve seen hatters before], expected: [ev' nees srettah erofeb], got: [ee' neve seetteh erooeb], similarity: 0.908
input: [you , and no], expected: [uoy , dna on], got: [uoy , dna no], similarity: 0.951
input: ['' but it all], expected: ['' tub ti lla], got: ['' tub ti lla], similarity: 0.996
input: [march hare was said], expected: [hcram erah saw dias], got: [hcram erah saw sias], similarity: 0.973
Average cosine similarity between label and prediction: 0.960
==================================================
Iteration-#: 46
input: [fly up into a], expected: [ylf pu otni a], got: [yll uo otni a], similarity: 0.927
input: [she tipped over the], expected: [ehs deppit revo eht], got: [ehs eeppis revo eht], similarity: 0.954
input: [quite a new idea], expected: [etiuq a wen aedi], got: [etiuq a wnn aiew], similarity: 0.899
input: [, who seemed to], expected: [, ohw demees ot], got: [, ohw demees ot], similarity: 1.000
input: [fear of their hearing], expected: [raef fo rieht gniraeh], got: [raef fo rieht gniraeh], similarity: 0.996
input: [cheshire cats always grinned], expected: [erihsehc stac syawla dennirg], got: [sgihsehw yaww yywwla nnnnreg], similarity: 0.704
input: [, could not ,], expected: [, dluoc ton ,], got: [, dluoc ton ,], similarity: 1.000
input: [was mystery , '], expected: [saw yretsym , '], got: [saw yretsum , '], similarity: 0.980
input: [the trial 's begun], expected: [eht lairt s' nugeb], got: [eht llirt s' nngeb], similarity: 0.928
input: [it was not easy], expected: [ti saw ton ysae], got: [ti saw ton yyee], similarity: 0.963
Average cosine similarity between label and prediction: 0.935
==================================================
Iteration-#: 47
input: [of hearts , he], expected: [fo straeh , eh], got: [fo straeh , eh], similarity: 0.993
input: [from here ? '], expected: [morf ereh ? '], got: [morf ereh ? '], similarity: 0.989
input: [knowledge of history ,], expected: [egdelwonk fo yrotsih ,], got: [dedewwonl fti rrttoh ,], similarity: 0.788
input: [sneezing all at once], expected: [gnizeens lla ta ecno], got: [gnileeno lla ta ecol], similarity: 0.931
input: [the panther were sharing], expected: [eht rehtnap erew gnirahs], got: [eht rehttap eraw gnisess], similarity: 0.932
input: [could see her after], expected: [dluoc ees reh retfa], got: [dluoc ees reh ettff], similarity: 0.951
input: [, it 's got], expected: [, ti s' tog], got: [, ti s' tog], similarity: 0.994
input: [that she hardly knew], expected: [taht ehs yldrah wenk], got: [taht ehs dllrah knew], similarity: 0.874
input: [duchess 's cook .], expected: [ssehcud s' kooc .], got: [ssehcud sas ooc .], similarity: 0.942
input: [milk-jug into his plate], expected: [guj-klim otni sih etalp], got: [guo--yuid noii a ppal], similarity: 0.647
Average cosine similarity between label and prediction: 0.904
==================================================
Iteration-#: 48
input: [shut again , and], expected: [tuhs niaga , dna], got: [sahs niaga , dna], similarity: 0.974
input: [way forwards each time], expected: [yaw sdrawrof hcae emit], got: [yaw ddwwwrrf miaa emiht], similarity: 0.769
input: [the duchess , the], expected: [eht ssehcud , eht], got: [eht ssehcud , eht], similarity: 1.000
input: [-- oop of the], expected: [-- poo fo eht], got: [-- poo fo eht], similarity: 0.999
input: [goose , with the], expected: [esoog , htiw eht], got: [esoog , hiiw eht], similarity: 0.991
input: [away from her as], expected: [yawa morf reh sa], got: [yaaa rrf rrh aa], similarity: 0.809
input: [tiny white kid gloves], expected: [ynit etihw dik sevolg], got: [yia eiiw diw sevgel], similarity: 0.612
input: [and she drew herself], expected: [dna ehs werd flesreh], got: [dna eht nrrd flesreh], similarity: 0.937
input: [alice 's side as], expected: [ecila s' edis sa], got: [ecila s' sesss a], similarity: 0.899
input: [from him to you], expected: [morf mih ot uoy], got: [mof mih oi ooy], similarity: 0.760
Average cosine similarity between label and prediction: 0.875
==================================================
Iteration-#: 49
input: [it was not easy], expected: [ti saw ton ysae], got: [ti saw oon yaee], similarity: 0.934
input: [, they began moving], expected: [, yeht nageb gnivom], got: [, yeht nageb gninam], similarity: 0.950
input: [tiptoe , and peeped], expected: [eotpit , dna depeep], got: [etttip , dna depepp], similarity: 0.921
input: [escape ; so she], expected: [epacse ; os ehs], got: [esaeee ; os ehs], similarity: 0.948
input: [song , she kept], expected: [gnos , ehs tpek], got: [gnok , ehs tkep], similarity: 0.953
input: [but to her great], expected: [tub ot reh taerg], got: [tub ot reh taerr], similarity: 0.986
input: [kettle had been broken], expected: [elttek dah neeb nekorb], got: [eetteb dah neeb nnoobb], similarity: 0.875
input: [, ' said alice], expected: [, ' dias ecila], got: [, ' dias ecila], similarity: 1.000
input: ['well , perhaps you], expected: [llew' , spahrep uoy], got: [llew' , ssairep poy], similarity: 0.940
input: [of mine , the], expected: [fo enim , eht], got: [fo emim , eht], similarity: 0.972
Average cosine similarity between label and prediction: 0.948
==================================================
Iteration-#: 50
input: [up as the other], expected: [pu sa eht rehto], got: [pu sa eht rehto], similarity: 0.999
input: [sitting sad and lonely], expected: [gnittis das dna ylenol], got: [gnitsis das dna ylenol], similarity: 0.964
input: [they in the prisoner], expected: [yeht ni eht renosirp], got: [yeht ni eht renosrip], similarity: 0.963
input: [reality -- the grass], expected: [ytilaer -- eht ssarg], got: [ttalaer -- eht ssarg], similarity: 0.916
input: [the mock turtle drew], expected: [eht kcom eltrut werd], got: [eht kcom eltrut need], similarity: 0.959
input: [but all he said], expected: [tub lla eh dias], got: [tub lla eh dias], similarity: 1.000
input: [had no very clear], expected: [dah on yrev raelc], got: [dah od yrev rraec], similarity: 0.914
input: [of meaning in it], expected: [fo gninaem ni ti], got: [fo gninaem ni ti], similarity: 0.998
input: [alice as it spoke], expected: [ecila sa ti ekops], got: [ecila sa ti ekops], similarity: 0.994
input: [said the dodo solemnly], expected: [dias eht odod ylnmelos], got: [dias eht odod ylnmllos], similarity: 0.978
Average cosine similarity between label and prediction: 0.969
Final average accuracy (computed as cosine similarity between expected and generated sequence over a random sample of 10 sequences) is 0.97, compared to about 0.74 initially.
In [ ]:
Content source: sujitpal/intro-dl-talk-code
Similar notebooks: