Seq2Seq Recurrent Neural Networks in Python

A demo of grapheme to phoneme with LSTMs

The applications and code shown in this presentation are from https://github.com/cmusphinx/g2p-seq2seq

Intuition behind recurrent neural networks (RNNs)

Traditional Neural Networks do not persist information

• When we think, we do not read every word independently; their meaning is gleaned from the context of the words that have come before them

RNNs attempt to solve this issue

• Neural networks with loops, so information can persist from cell to cell. (image courtesy of Christopher Olah)

Many types and applications of them

• These slides won't go over how they work or what types there are, but there is ample literature out there to explain.
• Understanding LSTM networks - Christopher Olah
• The Unreasonable Effectiveness of Recurrent Neural Networks - Andrej Karpathy
• A Tutorial on Deep Learning Part 2: Autoencoders, Convolutional Neural Networks and Recurrent Neural Networks - Quoc Le

Seq2Seq Encoder-Decoder

Core question - can we condition generated words on some input to get some meaningful response?

• Yes! Encoder-decoders are used for machine translation, conversation generation, and image captioning.

Grapheme to Phoneme

Given a word, how do we pronounce it?

• Grapheme refers to the orthographic spelling, phoneme is the sounds
• for example: green is mapped to G R IY N

Let's use RNNs to generate a phonemes from graphemes!

• this is super useful --- imagine a machine being able to pronounce novel words that it has never seen before.

Demo