CSAL4243: Introduction to Machine Learning

Muhammad Mudassir Khan (mudasssir.khan@ucp.edu.pk)

Lecture 6: Polynomial Regression and Intro to Classification

Overview




Linear Regression with multiple variables


Model $h(x) = \theta_0 + \theta_1 x_1 + \theta_2 x_2 .... + \theta_n x_n$


Psuedocode

Linear Regressio with Gradient Descent

  • Load training data into X_train and y_train
  • [Optionally] normalize features X_train using $x^i = \frac{x^i - \mu^i}{\rho^i}$ where $\mu^i$ is mean and $\rho^i$ is standard deviation of feature $i$
  • Initialize hyperparameters
    • iterations
    • learning rate $\alpha$
  • Initialize $\theta_s$
  • At each iteration
    • Compute cost using $J(\theta) = \frac{1}{2m}\sum_{i=1}^{m} (h(x^i) - y^i)^2$ where $h(x) = \theta_0 + \theta_1 x_1 + \theta_2 x_2 .... + \theta_n x_n$
    • Update $\theta_s$ using $\begin{align*} \; \; & \theta_j := \theta_j - \alpha \frac{1}{m} \sum\limits_{i=1}^{m} (h_\theta(x_{i}) - y_{i}) \cdot x^j_{i} \; & & \text{for j := 0...n} \end{align*}$
    • [Optionally] Break if cost $J(\theta)$ does not change.


Polynomial Regression


Example


Classification vs Regression


Classification Examples

  1. There will be rain tomorrow or not
  2. Email is spam or not
  3. Bank transaction is fraud or not
  4. Image is cat / dog



K - Nearest Neighbor Classifier



Nearest Neighbor Classifier

Credits

Raschka, Sebastian. Python machine learning. Birmingham, UK: Packt Publishing, 2015. Print.

Andrew Ng, Machine Learning, Coursera

Lucas Shen

David Kaleko