Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements; and to You under the Apache License, Version 2.0.

A Tutorial of SINGA V1

Welcome to this tutorial for Apache Incubator-singa. Please install conda and SINGA before running the notebooks.

Outline

• Introduction and installation
• SINGA classes
  • Core classes - Tensor and Device
  • Model classes - Layer, Loss, Metric, FeedForwardNet, Optimzier, Initializer
• Simple models
  • Linear Regression
  • Multi-layer Perceptron
• Deep learning models
  • Convolutional Neural Network (CNN)
  • Recurrent Neural Networks (RNN)
  • Restricted Boltzmann Machine (RBM)
• Distributed training (WIP)
• Rafiki (WIP)

Introduction

Training and Inference of deep learning models

Training objective

$\min_{\theta} \sum_{<x, y>\in D}L(f(x|\theta), y)$

Training procedure

1. randomly initialize $\theta$
2. get a mini-batch of data $\{<x, y>\}$ and compute $\frac{\partial L}{\partial \theta}$
3. $\theta \leftarrow \theta - \alpha \frac{\partial L}{\partial \theta}$
4. repeat 2-3 until converge, i.e., $||\frac{\partial L}{\partial \theta}|| < \epsilon$

Back-propagation for gradient computation

Popular deep learning models

SINGA V1

Software Stack

Progress

Schedule

Next: SINGA core classes