Encoder-Decoder Analysis

In [1]:

    

report_file = '/Users/bking/IdeaProjects/LanguageModelRNN/reports/encdec_250_800_dr.json'
log_file = '/Users/bking/IdeaProjects/LanguageModelRNN/logs/encdec_250_800_dr_logs.json'

import json
import matplotlib.pyplot as plt
with open(report_file) as f:
    report = json.loads(f.read())
with open(log_file) as f:
    logs = json.loads(f.read())
print'Encoder: \n\n', report['architecture']['encoder']
print'Decoder: \n\n', report['architecture']['decoder']


    

    




Encoder: 

nn.Sequential {
  [input -> (1) -> (2) -> (3) -> output]
  (1): nn.LookupTable
  (2): nn.LSTM(250 -> 800)
  (3): nn.Dropout(0.500000)
}
Decoder: 

nn.gModule

In [2]:

    

print('Train Perplexity: ', report['train_perplexity'])
print('Valid Perplexity: ', report['valid_perplexity'])
print('Test Perplexity: ', report['test_perplexity'])


    

    




('Train Perplexity: ', 5.5018499958222)
('Valid Perplexity: ', 27.312297195403)
('Test Perplexity: ', 25.383458037513)

In [3]:

    

%matplotlib inline
for k in logs.keys():
    plt.plot(logs[k][0], logs[k][1], label=str(k) + ' (train)')
    plt.plot(logs[k][0], logs[k][2], label=str(k) + ' (valid)')
plt.title('Loss v. Epoch')
plt.xlabel('Epoch')
plt.ylabel('Loss')
plt.legend()
plt.show()


    

In [4]:

    

%matplotlib inline
for k in logs.keys():
    plt.plot(logs[k][0], logs[k][3], label=str(k) + ' (train)')
    plt.plot(logs[k][0], logs[k][4], label=str(k) + ' (valid)')
plt.title('Perplexity v. Epoch')
plt.xlabel('Epoch')
plt.ylabel('Perplexity')
plt.legend()
plt.show()


    

In [5]:

    

def print_sample(sample):
    enc_input = ' '.join([w for w in sample['encoder_input'].split(' ') if w != '<pad>'])
    gold = ' '.join([w for w in sample['gold'].split(' ') if w != '<mask>'])
    print('Input: '+ enc_input + '\n')
    print('Gend: ' + sample['generated'] + '\n')
    print('True: ' + gold + '\n')
    print('\n')


    

In [6]:

    

for sample in report['train_samples']:
    print_sample(sample)


    

    




Input:  churro waffles <begin_ingredients> 1.5 cups almond meal / flour <ing> ½ teaspoon baking soda <ing> ½ teaspoon cinnamon <ing> pinch of salt <ing> ⅓ cup canned coconut milk <ing> 2 eggs , whisked <ing> 1 tablespoon honey <ing> 1 teaspoon vanilla extract <ing> 3 tablespoons grass fed butter , melted ( or coconut oil if you don’t do butter ) <ing> ¼ - 1 / 3 cup coconut sugar <end>

Gend:  <beg> preheat the oven to 350 degrees . <step> in a large bowl , combine the flour , baking powder , salt , sugar , cinnamon , nutmeg , and cloves . <step> add the milk , vanilla , and milk . stir until the mixture is smooth . <step> pour the mixture into the prepared

True:  place almond flour , baking soda , cinnamon and salt in a bowl and whisk together . <step> then add coconut milk , eggs , honey , and vanilla extract and mix well . <step> place mixture into hot waffle iron and cook until cooked through . i used a non stick waffle iron but you may want to grease it just in case . <step> melt butter in a wide bowl and place coconut sugar in another wide bowl . <step> after waffles are cooked through , place each other into the melted butter , soaking it on both sides , then place them into the coconut sugar to coat . <step> then eat ! <end>



Input:  rice pulao | veggie biryani <begin_ingredients> basmati rice - 1 cup <ing> potato - 1 no ( diced in to pieces ) <ing> tomato - 1 no ( chopped ) <ing> green peas - 2 tbsp . <ing> green chillies ( finely chopped ) - 1 no <ing> jeera <ing> fresh lime juice - 2 tbsp . <ing> coriander leaves ( finely chopped ) - 1 tbsp . <ing> vegetable oil - 1 tbsp . <ing> water - 2 1 / 2 cup <ing> red chilli powder <ing> salt to taste <ing> coriander powder <ing> garam masala powder <ing> turmeric powder ( optional , i did n't put in my recipe , if you want you can put in it ) <end>

Gend:  <beg> 1 . heat the oil in a pan and add the onion . cook for 2 minutes . add the garlic and cook for another minute . <step> 2 . add the garlic and ginger and cook for another minute . <step> 2 . add the tomatoes , salt and pepper . cook for another

True:  1 . clean the rice , wash it and after wash the rice with water , soak it in water for 1 / 2 an hr . <step> 2 . peel the potato and onion . cut all vegetables like below in the picture or as per your like . <step> 3 . after 1 / 2 an hour , drain the water from the rice and discard the water . <step> 4 . take a non - stick kadai and place it on medium flame , pour oil and when it is hot , add jeera . <step> 5 . first fry potatoes with green chillies , then add onions , saute for few min till onion becomes transparent . add green peas and potatoes . <step> 6 . add all indian spices - salt to taste , coriander powder , red chilli powder , garam masala powder . when this tadka is prepared . <step> 7 . add soaked rice and water and let it boil . give first boil on a high flame and then cooked the rice on a medium flame for 10 mins . <step> 8 . once the rice will be semi - cooked , add lemon juice and coriander leaves , mix it well . <step> 9 . cover the kadai and let it cook until water will get evaporated . you can sprinkle coriander leaves before serving . <step> 10 . serve hot - hot rice pulao with dahi , raita , pickle or any curry vegetables . <end>



Input:  grilled tandoori lamb <begin_ingredients> 2 tablespoons cumin seeds <ing> 2 teaspoons fennel seeds <ing> 2 teaspoons cardamom seeds <ing> 1 / 2 teaspoon whole black peppercorns <ing> 1 1 / 2 teaspoons dried crushed red pepper <ing> 6 tablespoons olive oil <ing> 2 tablespoons minced fresh ginger <ing> 4 garlic cloves , pressed <ing> 1 5 3 / 4 - pound leg of lamb , boned and butterflied ( about 4 pounds boned ) <end>

Gend:  <beg> combine first 6 ingredients in a large zip - top plastic bag . add steak to bag ; seal . marinate in refrigerator 30 minutes , turning occasionally . <step> preheat grill to medium - high heat . <step> remove salmon from marinade ; discard marinade . <step> grill shrimp , turning once , until

True:  spray grill rack with nonstick spray ; prepare barbecue ( medium - high heat ) . grill lamb 25 minutes for medium - rare ( internal temperature at the thickest part of meat should be 125f ° ) . let lamb rest 5 minutes before slicing . <end>

In [7]:

    

for sample in report['valid_samples']:
    print_sample(sample)


    

    




Input:  lemon syllabub <begin_ingredients> 1+ ½ cup heavy cream <ing> ¼ cup sugar <ing> 2 tablespoons apple juice <ing> juice of one lemon <ing> zest of one lemon <ing> ½ teaspoon vanilla <end>

Gend:  <beg> in a small bowl , combine the cream cheese , sugar , and salt . <step> whisk together the egg yolks , and the remaining ingredients . <step> pour the mixture into the prepared pan and bake for 35 - 40 minutes , or until the top is golden brown . <step> serve warm with

True:  zest and juice lemon . place all ingredients ( including lemon juice and zest ) in large mixing bowl . beat with an electric mixer , stopping to scrape down sides and stir from the bottom once , until thick and slightly stiff . <step> chill until ready to serve - serve same day . <end>



Input:  potato pancake <begin_ingredients> 5 teaspoons unsalted butter <ing> 4 cups frozen hash browns ( 1 pound ) , thawed <ing> coarse salt and ground pepper <ing> coarse salt and ground pepper <end>

Gend:  <beg> in a large bowl , combine the cream cheese , sour cream , mayonnaise , garlic , parsley , salt , pepper , and cayenne pepper . stir until well combined . <step> place the mixture in a large bowl and cover with plastic wrap . refrigerate for at least 2 hours . <step> preheat

True:  in an 8 - inch nonstick skillet , melt 2 1 / 2 teaspoons butter over medium - high . add hash browns and press evenly into skillet with a rubber spatula . season with salt and pepper . cook , pressing on potatoes and tucking in edge occasionally with spatula , until crisp and deep golden brown , 10 to 15 minutes . ( adjust heat if necessary to prevent cake from over - browning . ) flip pancake and repeat , adding 2 1 / 2 teaspoons butter to skillet . to serve , cut pancake into quarters . <end>



Input:  french fry stuffed chili enchiladas <begin_ingredients> 16 flour tortillas <ing> 32 ounces seasoned french fries <ing> 24 ounces wolf brand chili <ing> 2 cups shredded cheese <end>

Gend:  <beg> preheat oven to 350 degrees . <step> in a large bowl , combine the flour , salt , pepper , and garlic powder . <step> mix together the flour , salt , pepper , and garlic powder . <step> in a large bowl , mix together the flour , baking powder , salt and pepper

True:  preheat oven and bake french fries as directed on bag . <step> take your flour tortillas and heat them on flat iron skillet until warm . <step> stuff your tortillas with french fries , wrap them up and place them in baking dish . <step> pour chili over top of enchiladas and cover with cheese . <step> bake at 425 until cheese is melted and chili is warm and enjoy ! <end>

In [8]:

    

for sample in report['test_samples']:
    print_sample(sample)


    

    




Input:  vegan celery root mashed potatoes <begin_ingredients> 1 celery root , diced <ing> 4 small or 2 large idaho potatoes , peeled and diced <ing> 1 / 4 cup olive oil , plus more to taste <ing> kosher salt <ing> chopped fresh parsley , for serving <end>

Gend:  <beg> in a large bowl , combine the cream cheese , sour cream , mayonnaise , garlic , salt , pepper and lemon juice . <step> mix well and set aside . <step> in a large bowl , combine the tomatoes , tomatoes , olives , olives , olives , onion , parsley , salt and

True:  1 . put the celery root and potatoes in a saucepan . cover with 1 inch of water . bring to a boil and then simmer until tender , about 20 minutes depending on the size of the dice . <step> 2 . drain the water from the celery root and potatoes . add the olive oil and use a potato masher until you reach your desired chunkiness level . season with salt and add more olive oil as desired . <step> 3 . transfer to a bowl , drizzle a little extra olive oil on top , and sprinkle with some parsley . <end>



Input:  chocolate mousse , french style <begin_ingredients> 11 2 / 3 oz ( 330 g ) bittersweet ( 60 - 68 % ) chocolate , finely chopped <ing> 2 / 3 cup ( 150 ml ) whipping cream <ing> 3 large egg yolks <ing> 6 large egg whites <ing> 1 / 4 cup ( 50 g ) sugar <end>

Gend:  <beg> preheat oven to 350 degrees . <step> in a large bowl , combine the flour , sugar , baking powder , baking soda , and salt . <step> in a medium bowl , whisk together the eggs , milk , vanilla , and vanilla . <step> pour the wet ingredients into the dry ingredients and

True:  place chocolate in a heatproof bowl and melt over a pot of simmering water . remove from heat and set aside . bring cream to a boil in a heavy saucepan . remove from heat . pour about one - third into the chocolate and whisk quickly to combine . pour in another third of the cream and whisk to combine . pour in the remaining cream and whisk to combine . the mixture should be very smooth and shiny and feel slightly elastic like thick pudding - this is the sign of a proper emulsion . add in the egg yolks and beat to combine . combine egg whites and sugar in a stand mixer and whisk with the whisk attachment until soft peaks form . when the chocolate mixture has cooled to 113 - 122 degrees f , fold in a quarter of the whipped egg whites to lighten . add the rest of the egg whites and fold to combine . divide mousse among dessert glasses and chill for 12 hours . serve within 24 hours because of the raw egg yolks . <end>



Input:  roasted red pepper and white bean dip <begin_ingredients> 1 can ( about 15 ounces ) cannellini beans <ing> 12 ounces roasted red peppers <ing> ¼ cup basil ( loosely packed ) <ing> 1 garlic clove , crushed <ing> ½ cup shredded mozzarella cheese <ing> alfredo or marinara pasta chips , for serving ( recommended ) <end>

Gend:  <beg> preheat oven to 350 degrees . <step> in a large bowl , combine the flour , salt , pepper , and garlic powder . <step> mix together the flour , salt , pepper , and garlic powder . <step> in a large bowl , mix together the flour , baking powder , salt and pepper

True:  heat your oven to 350ºf . <step> add the beans , red peppers , basil , and garlic to your blender . blend on high for 8 - 10 seconds , or until smooth . pour into an oven - safe ramekin and top with a thin layer of cheese . <step> bake for 15 - 20 minutes , or until the cheese bubbles and begins to brown . <step> serve with pasta chips . <end>

In [9]:

    

print 'Overall Score: ', report['bleu']['score'], '\n'
print '1-gram Score: ', report['bleu']['components']['1']
print '2-gram Score: ', report['bleu']['components']['2']
print '3-gram Score: ', report['bleu']['components']['3']
print '4-gram Score: ', report['bleu']['components']['4']


    

    




Overall Score:  0.05 

1-gram Score:  39.1
2-gram Score:  8.7
3-gram Score:  1.2
4-gram Score:  0.4