First Steps in Supervised Learning

This is the moment you've been waiting for, isn't it?

We have covered all the bases - we have a functioning Python environment, we have OpenCV installed, and we know how to handle data in Python. Now it's time to build our first machine learning system! And what better way to start off than to focus on one of the most common and successful types of machine learning: supervised learning?

From the previous chapter, we already know that supervised learning is all about learning regularities in some training data by using the labels that come with it so that we can predict the labels of some new, never-seen-before test data. In this chapter, we want to dig a little deeper, and learn how to turn our theoretical knowledge into something practical.

Along the way, we want to address the following questions:

What's the difference between classification and regression, and when do I use which?
What is a $k$-nearest neighbor ($k$-NN) classifier, and how do I implement one in OpenCV?
How do I use logistic regression for classification, and why is it named so confusingly?
How do I build a linear regression model in OpenCV, and how does it differ from Lasso and ridge regression?

Outline

Let's jump right in!

The book provides an overview of common supervised learnig methods, and features a detailed treatment of common machine learning workflows. Below is a summary of these topics. For more information, please refer to the book.

Supervised learning in OpenCV

OpenCV provides a pretty straightforward interface for all its statistical learning models, which includes all supervised learning models. In OpenCV, every machine learning model derives from the cv::ml::StatModel base class. This is fancy talk for saying that if we want to be a machine learning model in OpenCV, we have to provide all the functionality that StatModel tells us to. This includes a method to train the model (called train) and a method to measure the performance of the model (called calcError).

Thanks to this organization of the software, setting up a machine learning model in OpenCV always follows the same logic:

Initialization: We call the model by name to create an empty instance of the model.
Set parameters: If the model needs some parameters, we can set them via setter methods, which can be different for every model. For example, in order for a $k$-NN algorithm to work, we need to specify its open parameter, $k$ (as we will find out later).
Train the model: Every model must provide a method called train, used to fit the model to some data.
Predict new labels: Every model must provide a method called predict, used to predict the labels of new data.
Score the model: Every model must provide a method called calcError, used to measure performance. This calculation might be different for every model.