Deep neural networks I

Software for deep learning

Deep learning is about composing differentiable functions into more complex functions, represented as a computation graph, and then using automatic differentiation ("autograd") to compute gradients, which we can pass to an optimizer, to fit the function to data. This is sometimes called "differentiable programming".

There are several libraries that can execute such computation graphs on hardware accelerators, such as GPUs. (Some libraries also support distributed computation, but we will not need use this feature in this book.) We list a few popular libraries below.

Multilayer perceptrons (MLPs)

In this section, we give some examples of MLPs using TF2. (* denotes official TF tutorials, which are not part of the pyprobml repo.)

• Auto-MPG regression *
• Tabular medical data classification *
• FashionMNIST image classification *
• IMDB movie review sentiment classification using pre-trained word embeddings from TF-hub *
• IMDB movie review sentiment classification using keras pre-processed data *

Convolutional neural networks (CNNs)

In this section, we give some examples of CNNs using TF2. (* denotes official TF tutorials, which are not part of the pyprobml repo.)

Image classification

• Dogs vs cats image classification *
• Transfer learning of a mobilenet model (trained on Imagenet) from keras to dogs vs cats classification *
• Transfer learning of a mobilenet model (trained on Imagenet) from TF-hub to flower classification *

Other vision tasks

• Image segmentation using Oxford-IIT pets dataset *
• Object detection using TF-hub *
• More advanced object detection *