cfgen demos

Import package



In [1]:

    
from cfgen import GrammarModel

# can turn these off:
%load_ext autoreload
%autoreload 2
# %autoreload 0









    



/Users/william/python_files/cfgen/cfgen/cfgen.py:40: UserWarning: Could not import language-check or LanguageTool. Rule-based post-processing will not be applied to generated text.
  warnings.warn('Could not import language-check or LanguageTool. Rule-based' \

Initialize model



In [2]:

    
CORPUS_PATH = './resources/corpora/frankenstein.txt'
MODEL_ORDER = 2
my_model = GrammarModel(CORPUS_PATH, MODEL_ORDER)

Make a random sentence with fixed grammar



In [3]:

    
from random import seed
seed(5)

simple_sentence = 'The cow jumped over the moon, and the little dog laughed.'
for ii in range(3):
    print(my_model.make_sentence(fixed_grammar=True, sample_sentence=simple_sentence))









    



Building the Grammar Model
Time: (3.00)s

and the old woman prepared of the spot . , but every i judged on the coffin .
the great road discovered that a prison , or these country seized of the landscape .
all man had , and the fair expressed .

Compare to a CFG without Markov



In [4]:

    
from random import seed
seed(5)

simple_sentence = 'The cow jumped over the moon, and the little dog laughed.'

for ii in range(3):
    print(my_model.make_sentence(fixed_grammar=True, sample_sentence=simple_sentence, do_markov=False))









    



and the mad land was . , the night put
an winter enjoyed , and the dread became .
the heart returned in these country , the difficult kindness taken in the prospect

Base style on a random sentence from the input text, but don't use Markov chain to select words



In [5]:

    
from random import seed
seed(5)

for ii in range(2):
    print(my_model.make_sentence(do_markov=False))









    



play me closer more , own .
rendered upon the moment present affection , such cheerful that the letter wretched dear as the agony , wonderful continued to i in the prospect to which dimming expressed only taught a time .

Parse a random sentence from the input text, and use Markov chain to select words



In [9]:

    
from random import seed
seed(5)

for ii in range(2):
    print(my_model.make_sentence(do_markov=True))









    



play me closer more , own .
rendered upon the dusky present affection , such cheerful that the captive possessed dear as the trial , wonderful continued to read with the mighty to which i had no taught a time .

Make a sentence using a vanilla Markov chain



In [12]:

    
for ii in range(2):
    print(my_model.make_sentence_markov(10))









    



the blast tore along like a child. dear mountains ! my father,
untaught peasant beheld the elements of which they dressed; and these fits

DEV

Need to put these into setup.py:

import nltk
nltk.download('punkt')
nltk.download('averaged_perceptron_tagger')

Select a random sentence from the corpus

Issue: rare symbols that appear and cause problems when random sentences are used



In [ ]: