CSAL4243: Introduction to Machine Learning

Muhammad Mudassir Khan (mudasssir.khan@ucp.edu.pk)

Lecture 6: Polynomial Regression and Intro to Classification

Overview

• Linear Regression with multiple variables
• Resources
• Credits

Linear Regression with multiple variables

Model $h(x) = \theta_0 + \theta_1 x_1 + \theta_2 x_2 .... + \theta_n x_n$

Psuedocode

Linear Regressio with Gradient Descent

• Load training data into X_train and y_train
• [Optionally] normalize features X_train using $x^i = \frac{x^i - \mu^i}{\rho^i}$ where $\mu^i$ is mean and $\rho^i$ is standard deviation of feature $i$
• Initialize hyperparameters
  • iterations
  • learning rate $\alpha$
• Initialize $\theta_s$
• At each iteration
  • Compute cost using $J(\theta) = \frac{1}{2m}\sum_{i=1}^{m} (h(x^i) - y^i)^2$ where $h(x) = \theta_0 + \theta_1 x_1 + \theta_2 x_2 .... + \theta_n x_n$
  • Update $\theta_s$ using $\begin{align*} \; \; & \theta_j := \theta_j - \alpha \frac{1}{m} \sum\limits_{i=1}^{m} (h_\theta(x_{i}) - y_{i}) \cdot x^j_{i} \; & & \text{for j := 0...n} \end{align*}$
  • [Optionally] Break if cost $J(\theta)$ does not change.

Polynomial Regression

Example

Classification vs Regression

Classification Examples

1. There will be rain tomorrow or not
2. Email is spam or not
3. Bank transaction is fraud or not
4. Image is cat / dog

K - Nearest Neighbor Classifier

Nearest Neighbor Classifier

Resources

Course website: https://w4zir.github.io/ml17s/

Course resources

Credits

Raschka, Sebastian. Python machine learning. Birmingham, UK: Packt Publishing, 2015. Print.

Andrew Ng, Machine Learning, Coursera

Lucas Shen

David Kaleko