Create a variable phrase containing a list of words. Experiment with the operations described in this chapter, including addition, multiplication, indexing, slicing, and sorting.
In [10]:
wphrase = ['I','love','NLP']
print wphrase
wphrase = wphrase + ['and','ML'] #addition
print wphrase
print wphrase*2 #multiply
print wphrase[2] #indexing
print wphrase[1:3] #slicing
print sorted(wphrase) #sorting
Use the corpus module to explore austen-persuasion.txt. How many word tokens does this book have? How many word types?
In [14]:
from nltk.corpus import gutenberg
persuasion = gutenberg.words('austen-persuasion.txt')
print 'There are %d tokens and %d word types'%(len(persuasion),len(set(persuasion)))
Use the Brown corpus reader nltk.corpus.brown.words() or the Web text corpus reader nltk.corpus.webtext.words() to access some sample text in two different genres.
In [15]:
from nltk.corpus import brown
brown.categories()
Out[15]:
In [17]:
print 'words in Romance and humor genres',
brown.words(categories=['romance','humor'])[:10]
Out[17]:
Read in the texts of the State of the Union addresses, using the state_union corpus reader. Count occurrences of men, women, and people in each document. What has happened to the usage of these words over time?
In [28]:
%matplotlib inline
from nltk.corpus import state_union
cfd = nltk.ConditionalFreqDist(
(target, fileid[:4])
for fileid in state_union.fileids()
for w in state_union.words(fileid)
for target in ['men', 'women','people']
if w.lower().startswith(target))
cfd.plot()
Investigate the holonym-meronym relations for some nouns. Remember that there are three kinds of holonym-meronym relation, so you need to use: member_meronyms(), part_meronyms(), substance_meronyms(), member_holonyms(), part_holonyms(), and substance_holonyms().
In [38]:
from nltk.corpus import wordnet as wn
print wn.synset('tree.n.01').member_meronyms()
print wn.synset('tree.n.01').part_meronyms()
print wn.synset('tree.n.01').substance_meronyms()
print wn.synset('tree.n.01').member_holonyms()
print wn.synset('tree.n.01').part_holonyms()
print wn.synset('tree.n.01').substance_holonyms()
In the discussion of comparative wordlists, we created an object called translate which you could look up using words in both German and Italian in order to get corresponding words in English. What problem might arise with this approach? Can you suggest a way to avoid this problem?
According to Strunk and White's Elements of Style, the word however, used at the start of a sentence, means "in whatever way" or "to whatever extent", and not "nevertheless". They give this example of correct usage: However you advise him, he will probably do as he thinks best. (http://www.bartleby.com/141/strunk3.html) Use the concordance tool to study actual usage of this word in the various texts we have been considering. See also the LanguageLog posting "Fossilized prejudices about 'however'" at http://itre.cis.upenn.edu/~myl/languagelog/archives/001913.html
In [40]:
nltk.Text(persuasion).concordance("however")