This is a class for engineers who want to apply machine learning/deep learning to real world problems.
The material assumes no prior experience with machine learning or deep learning and tries to have as little of a math prerequisite as possible. Unlike other introductory courses, it covers the real-world pitfalls like messy data preparation, comparing models to baselines and deploying models. All of the material uses real-world data sets and real wolrd applications and best-in-class libraries. I'm passionate about teaching this material becuase I think machine learning is incredibly powerful and it still isn't used as much as it could be - and I believe the limiting factor is a lack of people who know how to really make it work.
Personally, I love machine learning theory as well, but this course covers much more practice than theory because there are many excellent resources online to learn the theory.
I strongly believe in learning by doing and learning by experimentation and I hope putting this course in a Jupyter notebook encourages students to explore the material and ask questions. It should be possible to work through each notebook in under 30 minutes and if you get through all the notebooks you will learn a lot.
I cover everything in python because python is by far the most popular language for deep learning. If you haven't done any python but have done a lot of programming, you shouldn't find python too hard to pick up. If you haven't done any programming before, I might recommend checking out Learn Python the Hard Way (https://learnpythonthehardway.org/) or many of the online courses, or if you feel ambitious, you can try to just jump right in.
If you want to jump right in and skip this section - go for it! It might make more sense once you have finished the courses.
Most of the real world applications of Machine Learning are instances of supervised machine learning. In supervised machine learning you take training data which includes inputs and outputs and you build a model that predicts outputs from new inputs.
You have probably done a linear regression in your life. We don't always think of that as machine learning but it's definitely a simple type of machine learning.
Let's do a simple example where we do a regression to predict regional house prices based on the size of houses.
In [1]:
import numpy as np
from sklearn.linear_model import LinearRegression
from sklearn import datasets
import matplotlib.pyplot as plt # matplotlib is a graphing library
%matplotlib inline
# Load the boston housing price dataset
# Dataset of house prices by area
boston_houses = datasets.load_boston()
# load the average number of rooms for the first 20 records
avg_rooms = boston_houses.data[:20, np.newaxis, 5]
# load the price for the first 20 records
price = boston_houses.target[:20]
# build a linear regression model
model = LinearRegression()
model.fit(avg_rooms, price)
# predict the best fit line
preds = model.predict(avg_rooms)
# graph the data
plt.scatter(avg_rooms, price, color='black')
plt.plot(avg_rooms, preds)
Out[1]:
In [2]:
from sklearn.gaussian_process import GaussianProcessRegressor
from sklearn.gaussian_process.kernels import RBF, ConstantKernel as C, WhiteKernel
# initializa a model
model = GaussianProcessRegressor(
kernel=C(1.0, (1e-3, 1e3)) * RBF(1.0, (1e-1, 1e1)) + WhiteKernel(1.0),
normalize_y=True)
# fit the model
model.fit(avg_rooms, price)
# pick points to sample
num_samples = 100
X = np.linspace(min(avg_rooms), max(avg_rooms), num_samples).reshape(num_samples,1)
preds = model.predict(X)
plt.scatter(avg_rooms, price, color='black')
plt.plot(X, preds)
Out[2]:
In [3]:
from sklearn.gaussian_process import GaussianProcessRegressor
from sklearn.gaussian_process.kernels import RBF, ConstantKernel as C
# initialize a model
model = GaussianProcessRegressor(
kernel=C(1.0, (1e-3, 1e3)) * RBF(10, (1e-2, 1e2)),
normalize_y=True)
# fit the model
model.fit(avg_rooms, price)
X = np.linspace(min(avg_rooms), max(avg_rooms), 1000).reshape(1000,1)
preds = model.predict(X)
plt.scatter(avg_rooms, price, color='black')
plt.plot(X, preds)
Out[3]:
This line doesn't seem as good, but it does pass through all the points. We would call this model "overfitting".
This data has only one number as input, typically we might have thousands or millions of inputs. This data also has a number as an output. When we output a number we call the modeling problem a regression. When we output a category, we call the modeling problem classification. Classification and regression might seem like different problems but they are very similar.
One of the most popular machine learning websites, Kaggle, did a survey in 2017 asking data scients which alogrithms they used. Logistic Regression, a slight variant on the linear regression above was actually the most frequently used machine learning technique.
Deep Learning is the most exciting type of Machine Learning right now. It tends to be useful when you have really large data sets and you're willing to trade compute complexity for accuracy. As computers get faster and data sets get larger, deep learning becomes more and more important.
Deep Learning typically involves using neural networks which is a class of models. The models tend to take a longer time to train and are trickier to deploy but fundamentally deep learning is not very different from machine learning. It tends to be especially useful in vision and speech, although these days people have found ways to apply it to nearly every problem.
I typically start teaching the fundamentals of machine learning without deep learning because the models are a little faster to run and there is a lot of overlap. But I try to get to deep learning quickly because I know that's what students are excited about and that's where most of the best models are these days.
Francois Chollet, the author of the Keras deep learning library put it well:
The ML research community has long been driven by the need to publish, which results in a stark, sometimes ridiculous bias towards complexity. Remember to ask: "can we do this with k-means and logistic regression?"
In [ ]: