In [1]:

    

from nltk.book import *


    

    




*** Introductory Examples for the NLTK Book ***
Loading text1, ..., text9 and sent1, ..., sent9
Type the name of the text or sentence to view it.
Type: 'texts()' or 'sents()' to list the materials.
text1: Moby Dick by Herman Melville 1851
text2: Sense and Sensibility by Jane Austen 1811
text3: The Book of Genesis
text4: Inaugural Address Corpus
text5: Chat Corpus
text6: Monty Python and the Holy Grail
text7: Wall Street Journal
text8: Personals Corpus
text9: The Man Who Was Thursday by G . K . Chesterton 1908

In [21]:

    

#dir(text7)


    

In [22]:

    

#help(text7)


    

In [7]:

    

text7.concordance('inflation')


    

    




Displaying 6 of 6 matches:
tterns , but were n't adjusted for inflation . Kenneth Mayland , economist for 
conomic slowdown that *T*-49 eases inflation without *-1 leading to a recession
lion *U* . After *-1 adjusting for inflation , the Commerce Department said 0 c
 ' report also added evidence that inflation is under control . For the fifth c
4 to fall *T*-3 or during times of inflation . Teikoku Oil , also stimulated * 
 sense of job security , and a low inflation rate has kept the purchasing power

In [12]:

    

text3.concordance('love', width = 140)


    

    




Displaying 4 of 4 matches:
d , and take me some venison ; And make me savoury meat , such as I love , and bring it to me , that I may eat ; that my soul may bless thee
ats ; and I will make them savoury meat for thy father , such as he love And thou shalt bring it to thy father , that he may eat , and that 
ears for Rachel ; and they seemed unto him but a few days , for the love he had to her . And Jacob said unto Laban , Give me my wife , for m
LORD hath looked upon my affliction ; now therefore my husband will love me . And she conceived again , and bare a son ; and said , Because 

In [13]:

    

text3.concordance('lived')


    

    




Displaying 25 of 38 matches:
ay when they were created . And Adam lived an hundred and thirty years , and be
ughters : And all the days that Adam lived were nine hundred and thirty yea and
nd thirty yea and he died . And Seth lived an hundred and five years , and bega
ve years , and begat Enos : And Seth lived after he begat Enos eight hundred an
welve years : and he died . And Enos lived ninety years , and begat Cainan : An
 years , and begat Cainan : And Enos lived after he begat Cainan eight hundred 
ive years : and he died . And Cainan lived seventy years and begat Mahalaleel :
rs and begat Mahalaleel : And Cainan lived after he begat Mahalaleel eight hund
years : and he died . And Mahalaleel lived sixty and five years , and begat Jar
s , and begat Jared : And Mahalaleel lived after he begat Jared eight hundred a
and five yea and he died . And Jared lived an hundred sixty and two years , and
o years , and he begat Eno And Jared lived after he begat Enoch eight hundred y
 and two yea and he died . And Enoch lived sixty and five years , and begat Met
 ; for God took him . And Methuselah lived an hundred eighty and seven years , 
 , and begat Lamech . And Methuselah lived after he begat Lamech seven hundred 
nd nine yea and he died . And Lamech lived an hundred eighty and two years , an
ch the LORD hath cursed . And Lamech lived after he begat Noah five hundred nin
naan shall be his servant . And Noah lived after the flood three hundred and fi
xad two years after the flo And Shem lived after he begat Arphaxad five hundred
at sons and daughters . And Arphaxad lived five and thirty years , and begat Sa
ars , and begat Salah : And Arphaxad lived after he begat Salah four hundred an
begat sons and daughters . And Salah lived thirty years , and begat Eber : And 
y years , and begat Eber : And Salah lived after he begat Eber four hundred and
 begat sons and daughters . And Eber lived four and thirty years , and begat Pe
y years , and begat Peleg : And Eber lived after he begat Peleg four hundred an

In [15]:

    

text4.concordance('liberty', width = 150)


    

    




Displaying 25 of 116 matches:
itself has ordained ; and since the preservation of the sacred fire of liberty and the destiny of the republican model of government are justly consi
on defense , promote the general welfare , and secure the blessings of liberty . The public disquisitions , discussions , and deliberations issued in
 same virtues and animated with the same ardent patriotism and love of liberty to independence and peace , to increasing wealth and unexampled prospe
in of Justice , and the Protector in all ages of the world of virtuous liberty , continue His blessing upon this nation and its Government and give i
restore to social intercourse that harmony and affection without which liberty and even life itself are but dreary things . And let us reflect that ,
f infuriated man , seeking through blood and slaughter his long - lost liberty , it was not wonderful that the agitation of the billows should reach 
 retrace our steps and to regain the road which alone leads to peace , liberty , and safety . I repair , then , fellow - citizens , to the post you h
with the faculties and the rights of men , breathing an ardent love of liberty and independence , and occupying a country which left them no desire b
ties ; and no other definite line can be drawn between the inestimable liberty of the press and its demoralizing licentiousness . If there be still i
ly to the public good , that peace be cultivated , civil and religious liberty unassailed , law and order preserved , equality of rights maintained ,
the firmest bulwark of republics -- that without standing armies their liberty can never be in danger , nor with large ones safe ; to promote by auth
f science and the diffusion of information as the best aliment to true liberty ; to carry on the benevolent plans which have been so meritoriously ap
oduce a like accord in all questions touching , however remotely , the liberty , prosperity and happiness of our country will always be the object of
on defense , promote the general welfare , and secure the blessings of liberty to the people of this Union in their successive generations . Since th
er physical nature has been extended by the invention of our artists . Liberty and law have marched hand in hand . All the purposes of human associat
lity , the common defense , the general welfare , and the blessings of liberty -- all have been promoted by the Government under which we have lived 
l ; as long as it secures to us the rights of person and of property , liberty of conscience and of the press , it will be worth defending ; and so l
continued efforts so to administer the Government as to preserve their liberty and promote their happiness . So many events have occurred within the 
 link together the various parts ." Without union our independence and liberty would never have been achieved ; without union they never can be maint
eir victorious legions becoming our lawgivers and judges . The loss of liberty , of all good government , of peace , plenty , and happiness , must in
nd that in entering into society " individuals must give up a share of liberty to preserve the rest ," it will be my desire so to discharge my duties
timate union effected , domestic tranquillity preserved , and personal liberty secured to the citizen . As was to be expected , however , from the de
ll I consent to serve a second term . But if there is danger to public liberty from the acknowledged defects of the Constitution in the want of limit
ended and on the preservation of which they relied as the last hope of liberty . Without denying that the result to which they looked with so much ap
" the freedom of the press is the great bulwark of civil and religious liberty " is one of the most precious legacies which they have left us . We ha

In [16]:

    

text1.similar('monstrous')


    

    




christian vexatious mystifying curious determined abundant untoward
exasperate fearless subtly part few mouldy lamentable delightfully
uncommon careful reliable maddens singular

In [25]:

    

text3.similar('value')


    

    




No matches

In [24]:

    

text1.similar('value')


    

    




bow call cry grandmother expression gush boots possibilities butts
soles nearest eye faces marvels compasses halls leaves smell ears
baling

In [26]:

    

text7.similar('value')


    

    




amount deaths rate department philosophy text pretext beheading church
language sound status bottom society exclusion sale inverse risk brunt
unit

In [27]:

    

text2.common_contexts(['monstrous', 'very'])


    

    




be_glad is_pretty a_pretty am_glad a_lucky

In [30]:

    

text3.common_contexts(['God', 'men'])


    

    




of_and with_and the_which the_of

In [31]:

    

help(text3.common_contexts)


    

    




Help on method common_contexts in module nltk.text:

common_contexts(words, num=20) method of nltk.text.Text instance
    Find contexts where the specified words appear; list
    most frequent common contexts first.
    
    :param word: The word used to seed the similarity search
    :type word: str
    :param num: The number of words to generate (default=20)
    :type num: int
    :seealso: ContextIndex.common_contexts()

In [38]:

    

text3.dispersion_plot(['God', 'woman', 'women', 'man', 'men', 'person', 'people', 'servant'])


    

In [44]:

    

#tokens vs distinct word types (words including punctuation)
print('len(text) = {},\nlen(set(text3)) = {}'.format(len(text3), len(set(text3))))


    

    




len(text) = 44764,
len(set(text3)) = 2789

In [ ]:

    

#lexical richness
lexical_richness = len(set(text3))/len(text3)


    

In [45]:

    

text3.count('God')


    

    
Out[45]:





231

In [51]:

    

#percent of text taken up by this word
def text_percent(text, word):
    return 100*text.count(word)/len(text)

text_percent(text3, 'God')


    

    
Out[51]:





0.5160396747386292

In [52]:

    

text_percent(text5, 'lol')


    

    
Out[52]:





1.5640968673628082

In [54]:

    

text_percent(text5, 'haha')


    

    
Out[54]:





0.09775605421017551

In [57]:

    

text_percent(text5, 'hi')


    

    
Out[57]:





1.213063763608087

In [ ]:

    




    

In [ ]:

    




    

In [ ]:

    




    

In [ ]:

    




    

In [ ]:

    




    

In [ ]:

    




    

In [ ]:

    




    

In [ ]: