Encoder-Decoder Analysis

Model Architecture


In [1]:
report_file = 'reports/encdec_200_512_norem.json'
log_file = 'logs/encdec_200_512_norem_logs.json'

import json
import matplotlib.pyplot as plt
with open(report_file) as f:
    report = json.loads(f.read())
with open(log_file) as f:
    logs = json.loads(f.read())
print'Encoder: \n\n', report['architecture']['encoder']
print'Decoder: \n\n', report['architecture']['decoder']


Encoder: 

nn.Sequential {
  [input -> (1) -> (2) -> output]
  (1): nn.LookupTable
  (2): nn.LSTM(200 -> 512)
}
Decoder: 

nn.gModule

Perplexity on Each Dataset


In [2]:
print('Train Perplexity: ', report['train_perplexity'])
print('Valid Perplexity: ', report['valid_perplexity'])
print('Test Perplexity: ', report['test_perplexity'])


('Train Perplexity: ', 9.0876709171962)
('Valid Perplexity: ', 25.55261758451)
('Test Perplexity: ', 24.14035993694)

Loss vs. Epoch


In [3]:
%matplotlib inline
for k in logs.keys():
    plt.plot(logs[k][0], logs[k][1], label=str(k) + ' (train)')
    plt.plot(logs[k][0], logs[k][2], label=str(k) + ' (valid)')
plt.title('Loss v. Epoch')
plt.xlabel('Epoch')
plt.ylabel('Loss')
plt.legend()
plt.show()


Perplexity vs. Epoch


In [4]:
%matplotlib inline
for k in logs.keys():
    plt.plot(logs[k][0], logs[k][3], label=str(k) + ' (train)')
    plt.plot(logs[k][0], logs[k][4], label=str(k) + ' (valid)')
plt.title('Perplexity v. Epoch')
plt.xlabel('Epoch')
plt.ylabel('Perplexity')
plt.legend()
plt.show()


Generations


In [5]:
def print_sample(sample):
    enc_input = ' '.join([w for w in sample['encoder_input'].split(' ') if w != '<pad>'])
    gold = ' '.join([w for w in sample['gold'].split(' ') if w != '<mask>'])
    print('Input: '+ enc_input + '\n')
    print('Gend: ' + sample['generated'] + '\n')
    print('True: ' + gold + '\n')
    print('\n')

In [6]:
for sample in report['train_samples']:
    print_sample(sample)


Input:  thai - style chicken coleslaw <begin_ingredients> 2 small shallots , finely chopped <ing> 1 red chili , fresh , seeded and finely chopped <ing> 2 tablespoons honey <ing> 1 / 4 cup rice vinegar <ing> 1 lime , juice of <ing> 3 tablespoons fish sauce <ing> 3 tablespoons soy sauce <ing> 1 tablespoon chives , finely chopped fresh chives <ing> 2 chicken fillets , cooked and shredded <ing> 1 / 2 a medium cabbage , finely shredded <ing> 2 carrots , medium size , cut into very thin strips <ing> 6 mint leaves , finely chopped <ing> 1 / 4 cup coriander , lightly chopped <ing> 1 / 4 cup peanuts , coarsely chopped <ing> 3 - 4 slices pink pickled ginger ( to garnish ) <end>

Gend:  <beg> 1 . in a large bowl , combine the rice , rice , rice vinegar , rice wine , and rice . stir . add the rice and stir . cover and cook on low for 8 to 10 hours

True:  place the chicken in a large serving bowl . <step> combine the shallots , chilli , honey , rice vinegar , fish sauce , lime juice , soy sauce and chives . pour the mixture over the shredded chicken . keep mix in fridge until ready to serve . <step> when ready to serve add the cabbage , carrot , mint leaves , coriander and peanuts to the chicken mix . <step> toss lightly until well combined and finish with a garnish of pink pickled ginger . <step> serve the salad well chilled . <end>



Input:  perfect vanilla bean rice pudding <begin_ingredients> 1 / 2 vanilla bean <ing> 1 / 2 cup ( approximately ) plain , cooked rice <ing> 2 cups milk ( pref . low fat ) <ing> 1 / 2 cup heavy cream <ing> 1 large egg <ing> 1 / 3 cup sugar <end>

Gend:  <beg> 1 . in a large bowl , stir together the flour , baking powder , sugar , salt , and baking soda . add the eggs , milk , vanilla , and oil . beat until smooth . <step> 2

True:  take a medium saucepan . split the vanilla bean in half and scrape out the seeds . combine vanilla bean seeds with rice , milk and cream and place in the saucepan . bring milk mixture to a simmer over low heat , stirring occasionally , and cook until the liquid reduces by at least half , about 30 minutes . the mixture should be roughly 2 / 3 rice and 1 / 3 liquid . <step> in a medium bowl , whisk together the egg and sugar until smooth . pour in a small amount ( approximately 1 / 2 cup ) of the milk from the rice while whisking constantly . stirring with a whisk , stream the sugar mixture back into the large pot . cook over medium - low heat until the pudding bubbles and thickens enough to coat the back of a spoon . remove from heat and transfer to a separate bowl to cool . <step> store , covered , in the refrigerator . <end>



Input:  sweet & spicy chinese chicken <begin_ingredients> 2 bunches green onions <ing> 1 / 4 cup olive oil <ing> 1 tablespoon minced garlic <ing> 12 oz pineapple juice <ing> 1 / 4 cup chili garlic sauce <ing> 1 / 4 cup cider vinegar <ing> 1 / 4 cup brown sugar <ing> 2 tablespoons soy sauce <ing> 3 - 4 pounds boneless , skinless chicken <end>

Gend:  <beg> mix all ingredients together and pour over chicken . <step> cover and cook on low for 8 hours . <step> remove chicken from slow cooker and shred with a fork . <step> place ribs on grill and cook for 5

True:  instructions chop green onion ; set aside . in large bowl , whisk together olive oil , garlic , pineapple juice , chili sauce , vinegar , brown sugar , green onions and soy sauce . divide chicken into two gallon size freezer bags ( be sure to label bags first ! ) divide sauce into bags over top of chicken . freeze until needed . thaw ; pour contents into slow cooker . cook on low for 4 - 5 hours or high for 2 - 3 hours , or place in casserole dish and bake at 400 degrees for 30 - 40 minutes . serve over cooked rice . <end>



Input:  creamy chipotle shrimp <begin_ingredients> 1 lb raw shrimp , peeled & devined <ing> 1 cup diced yellow onion <ing> 1 1 / 2 cups canned corn <ing> 1 / 4 cup fat free cream cheese ( i used philadelphia ) <ing> 2 tbsps chipotle peppers in adobo ( i used la costena ) <ing> 1 / 4 cup whole wheat panko crumbs ( i used ian 's ) <ing> 1 / 3 cup shredded reduced fat cheddar cheese ( i used sargento ) <ing> 1 1 / 2 tsps extra virgin olive oil <ing> 1 / 2 tsp lime juice <ing> 2 tsps cilantro <ing> 1 tsp sea salt & coarse black peppercorn - approx . <end>

Gend:  <beg> 1 . combine all ingredients in a food processor and pulse until well combined . <step> 2 . heat oil in a large skillet over medium heat . add garlic and cook until fragrant , about 1 minute . add

True:  1 . in food processor puree chipotle peppers until smooth . transfer 2 tbsps puree into small bowl , then combine with cream cheese and lime juice , stirring thoroughly . next , add shrimp to large bowl before seasoning with salt and pepper . pour over with sauce , then allow to marinate in refrigerator for at least 1 hour . <step> 2 . before cooking , in small bowl combine panko crumbs , cheddar cheese , cilantro , and 1 / 2 tsp evoo . set aside . <step> 3 . in oven - proof skillet sauté onions in remaining tsp evoo over medium - high heat until soft and translucent . next stir in corn , sautéing for several minutes before adding in shrimp . cook until shrimp just begins to turn pink on one side , then remove from heat . <step> 4 . evenly sprinkle panko mixture atop skillet , then bake at 350f for 12 - 15 minutes until crispy and browned . immediately plate and serve . <end>




In [ ]: