Deep Learning

Representation learning

Traditional approach: hand-crafted features + train a classifier

New approach: feaure learning (trainable features) + a trainable classifier

Deep archictecture:

• hierarchical structure of features
• more than one stage of non-linear feature extraction

[low level features] -> [med-level features] -> [high-level features] --> classifier

Trainable feature hierarchies:

• A hierarchy of trianbale feature transforms
  • Each module transfors its inpit representation into a higher-level one
  • High-level features are more global and more invariant
  • Low level features are shared among categories
• Deep learning goal:
  • Make all modules trainable and get them to learn appropriate representations

Brain: single learning algorithm

• Auditory cortex can learn how to see

Supervised learning

• Feed-forward networks
• Backpropagation
• Optimization
• Convolutional neural network
• Efficient convolutions
• Sequence modeling (RNNs, LSTMs, GRU