CSAL4243: Introduction to Machine Learning

Muhammad Mudassir Khan (mudasssir.khan@ucp.edu.pk)

Lecture 3: Gradient Descent

Overview

What is Machine Learning?

Machine Learning is making computers/machcines learn from data
Learning improve over time with more data

The three different types of machine learning

Supervised Learning

Machine Learning pipeline

x is called input variables or input features.
y is called output or target variable. Also sometimes known as label.
h is called hypothesis or model.
pair (x⁽ⁱ⁾,y⁽ⁱ⁾) is called a sample or training example
dataset of all training examples is called training set.
m is the number of samples in a dataset.
n is the number of features in a dataset excluding label.

Goal of Machine Learning algorithm

How well the algorithm will perform on unseen data.
Also called generalization.

Linear Regression with one variable

Model Representation

Model is represented by h_$\theta$(x) or simply h(x)
For Linear regression with one input variable h(x) = $\theta$₀ + $\theta$₁x

$\theta$₀ and $\theta$₁ are called weights or parameters.
Need to find $\theta$₀ and $\theta$₁ that maximizes the performance of model.

Cost Function

Let $\hat{y}$ = h(x) = $\theta$₀ + $\theta$₁x

Error in single sample (x,y) = $\hat{y}$ - y = h(x) - y

Cummulative error of all m samples = $\sum_{i=1}^{m} (h(x^i) - y^i)^2$

Finally mean error or cost function = J($\theta$) = $\frac{1}{2m}\sum_{i=1}^{m} (h(x^i) - y^i)^2$

Gradient Descent

Gradient Descent Equation

$\alpha$ is called learning rate that control the step size.

Derivative Part

Learning Rate $\alpha$

Convergence of Gradient Descent

Gradient Descent for Linear Regression

Cost function:

J($\theta$) = $\frac{1}{2m}\sum_{i=1}^{m} (h(x^i) - y^i)^2$

Gradient descent equation:

$\theta_j := \theta_j - \alpha \frac{\partial}{\partial \theta_j} J(\theta_0, \theta_1)$

Replacing J($\theta$) for each j

\begin{align*} \text{repeat until convergence: } \lbrace & \newline \theta_0 := & \theta_0 - \alpha \frac{1}{m} \sum\limits_{i=1}^{m}(h_\theta(x_{i}) - y_{i}) \newline \theta_1 := & \theta_1 - \alpha \frac{1}{m} \sum\limits_{i=1}^{m}\left((h_\theta(x_{i}) - y_{i}) x_{i}\right) \newline \rbrace& \end{align*}

Deriving $\frac{\partial}{\partial \theta_j} J(\theta_0, \theta_1)$

Price of the house yet again?

Read data



In [1]:

    
%matplotlib inline
import pandas as pd
import numpy as np
from sklearn import linear_model
import matplotlib.pyplot as plt

# read data in pandas frame
dataframe = pd.read_csv('datasets/house_dataset1.csv')

# assign x and y
X = np.array(dataframe[['Size']])
y = np.array(dataframe[['Price']])

m = y.size # number of training examples
#Insert the usual column of 1's into the "X" matrix
X = np.insert(X,0,1,axis=1)



In [2]:

    
# check data by printing first few rows
dataframe.head()

Plot data



In [3]:

    
#visualize results
plt.scatter(X[:,1], y)
plt.show()

Initialize Hyper Parameters



In [4]:

    
iterations = 1500
alpha = 0.000000001

Model/Hypothesis Function



In [5]:

    
def h(theta,X): #Linear hypothesis function
    return np.dot(X,theta)

Cost Function



In [6]:

    
def computeCost(mytheta,X,y): #Cost function
    """
    theta_start is an n- dimensional vector of initial theta guess
    X is matrix with n- columns and m- rows
    y is a matrix with m- rows and 1 column
    """
    #note to self: *.shape is (rows, columns)
    return float((1./(2*m)) * np.dot((h(mytheta,X)-y).T,(h(mytheta,X)-y)))

#Test that running computeCost with 0's as theta returns 65591548106.45744:
initial_theta = np.zeros((X.shape[1],1)) #(theta is a vector with n rows and 1 columns (if X has n features) )
print (computeCost(initial_theta,X,y))









    



65591548106.45744

Gradient Descent Function



In [7]:

    
#Actual gradient descent minimizing routine
def descendGradient(X, theta_start = np.zeros(2)):
    """
    theta_start is an n- dimensional vector of initial theta guess
    X is matrix with n- columns and m- rows
    """
    theta = theta_start
    jvec = [] #Used to plot cost as function of iteration
    thetahistory = [] #Used to visualize the minimization path later on
    for meaninglessvariable in range(iterations):
        tmptheta = theta
        # append for plotting
        jvec.append(computeCost(theta,X,y))
        thetahistory.append(list(theta[:,0]))
        #Simultaneously updating theta values
        for j in range(len(tmptheta)):
            tmptheta[j] = theta[j] - (alpha/m)*np.sum((h(theta,X) - y)*np.array(X[:,j]).reshape(m,1))
        theta = tmptheta
    return theta, thetahistory, jvec

Run Gradient Descent



In [8]:

    
#Actually run gradient descent to get the best-fit theta values
initial_theta = np.zeros((X.shape[1],1))
theta, thetahistory, jvec = descendGradient(X,initial_theta)



In [12]:

    
thetahistory









    Out[12]:





[[0.0, 0.0],
 [0.00034041265957446807, 0.76420912751043235],
 [0.00067929638024090519, 1.5248869674413239],
 [0.0010166582269802802, 2.2820498373050588],
 [0.0013525052321274205, 3.0357139792134187],
 [0.0016868443955218642, 3.7858955602259972],
 [0.0020196826846580161, 4.532610672697003],
 [0.002351027034834609, 5.275875334620463],
 [0.0026808843493034747, 6.0157054899738283],
 [0.0030092614994176284, 6.7521170090599911],
 [0.0033361653247786661, 7.4851256888477264],
 [0.0036616026333834853, 8.2147472533105539],
 [0.0039855802017703234, 8.9409973537640397],
 [0.0043081047751641231, 9.6638915692015352],
 [0.0046291830676212275, 10.383445406628367],
 [0.0049488217621734042, 11.099674301394481],
 [0.0052670275109712051, 11.812593617525557],
 [0.0055838069354266634, 12.522218648052576],
 [0.0058991666263553313, 13.228564615339883],
 [0.0062131131441176615, 13.931646671411722],
 [0.0065256530187597361, 14.631479898277268],
 [0.0068367927501533426, 15.328079308254161],
 [0.0071465388081354058, 16.021459844290526],
 [0.0074548976326467746, 16.71163638028553],
 [0.0077618756338703665, 17.398623721408441],
 [0.008067479192368672, 18.082436604416223],
 [0.0083717146592206301, 18.763089697969651],
 [0.0086745883561578623, 19.440597602947978],
 [0.0089761065757002871, 20.114974852762138],
 [0.0092762755812910973, 20.786235913666509],
 [0.0095751016074311218, 21.454395185069234],
 [0.0098725908598125637, 22.119466999841105],
 [0.01016874951545212, 22.781465624623021],
 [0.010463583722823485, 23.440405260132025],
 [0.010757099601989248, 24.096300041465931],
 [0.011049303244732167, 24.749164038406537],
 [0.011340200714685854, 25.399011255721444],
 [0.011629798047464845, 26.045855633464463],
 [0.01191810125079407, 26.689711047274667],
 [0.012205116304637727, 27.330591308674023],
 [0.012490849161327562, 27.968510165363682],
 [0.012775305745690553, 28.603481301518869],
 [0.013058491955176004, 29.235518338082443],
 [0.013340413659982054, 29.864634833057067],
 [0.013621076703181597, 30.490844281796061],
 [0.013900486900847631, 31.114160117292879],
 [0.01417865004217801, 31.734595710469275],
 [0.01445557188961964, 32.35216437046212],
 [0.014731258178992084, 32.966879344908904],
 [0.015005714619610602, 33.578753820231917],
 [0.015278946894408632, 34.187800921921102],
 [0.015550960660059683, 34.794033714815633],
 [0.015821761547098689, 35.397465203384151],
 [0.016091355160042786, 35.998108332003746],
 [0.01635974707751154, 36.595975985237615],
 [0.016626942852346605, 37.191080988111466],
 [0.01689294801173085, 37.783436106388621],
 [0.017157768057306923, 38.373054046843862],
 [0.017421408465295267, 38.959947457536003],
 [0.017683874686611582, 39.544128928079218],
 [0.017945172146983768, 40.125610989913085],
 [0.018205306247068308, 40.704406116571434],
 [0.018464282362566118, 41.280526723949876],
 [0.01872210584433787, 41.85398517057218],
 [0.018978782018518759, 42.424793757855362],
 [0.019234316186632773, 42.992964730373558],
 [0.019488713625706405, 43.5585102761207],
 [0.019741979588381857, 44.121442526771965],
 [0.019994119303029721, 44.681773557943998],
 [0.02024513797386112, 45.239515389453963],
 [0.020495040781039361, 45.794679985577389],
 [0.020743832880791048, 46.347279255304798],
 [0.020991519405516686, 46.897325052597189],
 [0.021238105463900789, 47.444829176640305],
 [0.021483596141021452, 47.989803372097754],
 [0.02172799649845945, 48.532259329362944],
 [0.0219713115744068, 49.072208684809851],
 [0.022213546383774849, 49.609663021042635],
 [0.022454705918301843, 50.144633867144101],
 [0.022694795146660004, 50.677132698923018],
 [0.02293381901456212, 51.207170939160292],
 [0.023171782444867638, 51.734759957853989],
 [0.023408690337688262, 52.25991107246324],
 [0.023644547570493064, 52.782635548151013],
 [0.02387935899821312, 53.302944598025782],
 [0.024113129453345643, 53.82084938338204],
 [0.024345863746057659, 54.33636101393973],
 [0.024577566664289181, 54.849490548082564],
 [0.024808242973855928, 55.360248993095247],
 [0.025037897418551541, 55.868647305399584],
 [0.02526653472024936, 56.374696390789524],
 [0.025494159579003705, 56.878407104665079],
 [0.025720776673150696, 57.379790252265217],
 [0.025946390659408621, 57.87885658889963],
 [0.026171006172977816, 58.375616820179452],
 [0.026394627827640106, 58.870081602246906],
 [0.026617260215857766, 59.362261542003893],
 [0.026838907908872042, 59.852167197339526],
 [0.027059575456801214, 60.339809077356612],
 [0.02727926738873819, 60.825197642597082],
 [0.027497988212847674, 61.308343305266391],
 [0.027715742416462862, 61.789256429456849],
 [0.027932534466181699, 62.267947331369975],
 [0.028148368807962711, 62.74442627953777],
 [0.028363249867220351, 63.218703495043009],
 [0.028577182048919954, 63.690789151738471],
 [0.028790169737672212, 64.160693376465218],
 [0.029002217297827239, 64.628426249269779],
 [0.029213329073568186, 65.093997803620439],
 [0.02942350938900443, 65.557418026622415],
 [0.029632762548264337, 66.018696859232122],
 [0.029841092835587588, 66.477844196470429],
 [0.030048504515417081, 66.934869887634875],
 [0.030255001832490403, 67.389783736510992],
 [0.0304605890119309, 67.842595501582608],
 [0.030665270259338295, 68.293314896241142],
 [0.030869049760878911, 68.741951588994013],
 [0.031071931683375471, 69.188515203672011],
 [0.03127392017439648, 69.633015319635746],
 [0.031475019362345195, 70.075461471981143],
 [0.031675233356548191, 70.515863151743986],
 [0.031874566247343493, 70.954229806103513],
 [0.032073022106168338, 71.390570838585049],
 [0.03227060498564651, 71.824895609261759],
 [0.032467318919675262, 72.257213434955403],
 [0.032663167923511864, 72.687533589436214],
 [0.032858155993859724, 73.115865303621817],
 [0.033052287108954116, 73.542217765775234],
 [0.033245565228647543, 73.966600121702029],
 [0.033437994294494663, 74.389021474946446],
 [0.033629578229836837, 74.809490886986737],
 [0.033820320939886302, 75.228017377429495],
 [0.03401022631180993, 75.644609924203166],
 [0.034199298214812622, 76.059277463750647],
 [0.034387540500220318, 76.472028891220944],
 [0.03457495700156258, 76.882873060660032],
 [0.034761551534654857, 77.291818785200761],
 [0.034947327897680326, 77.698874837251893],
 [0.035132289871271372, 78.10404994868631],
 [0.035316441218590677, 78.507352811028312],
 [0.035499785685411969, 78.908792075640065],
 [0.03568232700020034, 79.308376353907178],
 [0.035864068874192252, 79.706114217423433],
 [0.036045015001475146, 80.102014198174643],
 [0.03622516905906667, 80.496084788721703],
 [0.036404534706993563, 80.888334442382742],
 [0.036583115588370171, 81.27877157341446],
 [0.036760915329476604, 81.667404557192626],
 [0.036937937539836499, 82.054241730391752],
 [0.037114185812294469, 82.43929139116392],
 [0.037289663723093176, 82.822561799316787],
 [0.037464374831950038, 83.20406117649074],
 [0.037638322682133581, 83.583797706335318],
 [0.037811510800539468, 83.961779534684723],
 [0.037983942697766136, 84.338014769732553],
 [0.038155621868190118, 84.712511482205755],
 [0.038326551790040991, 85.085277705537763],
 [0.038496735925475992, 85.456321436040767],
 [0.038666177720654282, 85.825650633077316],
 [0.038834880605810883, 86.193273219231003],
 [0.039002847995330242, 86.559197080476451],
 [0.039170083287819492, 86.923430066348445],
 [0.039336589866181347, 87.285979990110349],
 [0.039502371097686673, 87.646854628921687],
 [0.03966743033404671, 88.006061724004979],
 [0.039831770911484987, 88.36360898081179],
 [0.039995396150808882, 88.719504069188048],
 [0.040158309357480847, 89.073754623538534],
 [0.040320513821689337, 89.426368242990691],
 [0.040482012818419358, 89.777352491557608],
 [0.040642809607522749, 90.126714898300293],
 [0.040802907433788088, 90.474462957489166],
 [0.04096230952701032, 90.820604128764813],
 [0.041121019102060008, 91.165145837298041],
 [0.041279039358952337, 91.50809547394914],
 [0.041436373482915713, 91.849460395426405],
 [0.04159302464446013, 92.189247924443961],
 [0.041748995999445163, 92.527465349878881],
 [0.041904290689147659, 92.864119926927458],
 [0.042058911840329137, 93.199218877260932],
 [0.04221286256530285, 93.532769389180345],
 [0.042366145962000556, 93.864778617770753],
 [0.042518765114038959, 94.195253685054723],
 [0.042670723090785873, 94.524201680145083],
 [0.042822022947426056, 94.851629659397034],
 [0.042972667725026746, 95.177544646559468],
 [0.043122660450602902, 95.501953632925677],
 [0.04327200413718213, 95.824863577483299],
 [0.043420701783869316, 96.14628140706364],
 [0.043568756375910976, 96.466214016490184],
 [0.043716170884759271, 96.784668268726591],
 [0.043862948268135762, 97.101650995023832],
 [0.044009091470094852, 97.417168995066774],
 [0.044154603421086951, 97.731229037120059],
 [0.044299487038021314, 98.043837858173234],
 [0.044443745224328639, 98.355002164085306],
 [0.044587380870023334, 98.6647286297286],
 [0.044730396851765511, 98.973023899131917],
 [0.044872796032922706, 99.279894585623069],
 [0.045014581263631288, 99.585347271970747],
 [0.045155755380857601, 99.88938851052572],
 [0.045296321208458827, 100.19202482336141],
 [0.045436281557243566, 100.49326270241376],
 [0.045575639225032105, 100.79310860962055],
 [0.045714396996716471, 101.09156897705996],
 [0.045852557644320141, 101.38865020708862],
 [0.045990123927057522, 101.68435867247882],
 [0.046127098591393134, 101.97870071655535],
 [0.046263484371100519, 102.2716826533315],
 [0.046399283987320905, 102.56331076764451],
 [0.046534500148621552, 102.85359131529042],
 [0.046669135551053877, 103.14253052315821],
 [0.046803192878211261, 103.43013458936345],
 [0.046936674801286639, 103.71640968338116],
 [0.047069583979129803, 104.00136194617825],
 [0.04720192305830441, 104.28499749034521],
 [0.047333694673144779, 104.5673224002272],
 [0.047464901445812398, 104.84834273205462],
 [0.047595545986352163, 105.12806451407299],
 [0.047725630892748373, 105.4064937466723],
 [0.047855158750980467, 105.68363640251567],
 [0.047984132135078468, 105.95949842666754],
 [0.048112553607178241, 106.23408573672113],
 [0.048240425717576425, 106.50740422292543],
 [0.048367751004785145, 106.77945974831154],
 [0.048494531995586475, 107.05025814881841],
 [0.048620771205086628, 107.31980523341808],
 [0.048746471136769928, 107.58810678424025],
 [0.048871634282552485, 107.85516855669636],
 [0.048996263122835677, 108.12099627960295],
 [0.049120360126559334, 108.3855956553047],
 [0.049243927751254725, 108.6489723597966],
 [0.049366968443097257, 108.91113204284584],
 [0.049489484636958951, 109.1720803281129],
 [0.049611478756460679, 109.43182281327225],
 [0.049732953214024142, 109.6903650701324],
 [0.049853910410923644, 109.94771264475544],
 [0.049974352737337573, 110.20387105757597],
 [0.050094282572399687, 110.45884580351957],
 [0.050213702284250157, 110.71264235212065],
 [0.050332614230086357, 110.9652661476398],
 [0.050451020756213424, 111.21672260918051],
 [0.050568924198094604, 111.46701713080549],
 [0.050686326880401339, 111.71615508165236],
 [0.050803231117063141, 111.96414180604881],
 [0.050919639211317223, 112.21098262362726],
 [0.051035553455757908, 112.45668282943895],
 [0.051150976132385804, 112.70124769406753],
 [0.051265909512656777, 112.94468246374215],
 [0.051380355857530639, 113.18699236044999],
 [0.05149431741751969, 113.42818258204824],
 [0.051607796432736962, 113.66825830237562],
 [0.051720795132944294, 113.90722467136339],
 [0.051833315737600145, 114.14508681514583],
 [0.051945360455907218, 114.38184983617013],
 [0.052056931486859838, 114.61751881330592],
 [0.052168031019291142, 114.85209880195418],
 [0.052278661231920015, 115.0855948341557],
 [0.052388824293397834, 115.31801191869901],
 [0.052498522362354985, 115.54935504122784],
 [0.052607757587447172, 115.77962916434807],
 [0.052716532107401501, 116.00883922773414],
 [0.052824848051062363, 116.23699014823511],
 [0.052932707537437111, 116.46408681998],
 [0.053040112675741505, 116.6901341144829],
 [0.053147065565444941, 116.9151368807474],
 [0.053253568296315529, 117.13909994537062],
 [0.053359622948464872, 117.36202811264674],
 [0.053465231592392727, 117.58392616467009],
 [0.053570396289031402, 117.8047988614377],
 [0.053675119089789967, 118.02465094095142],
 [0.053779402036598263, 118.24348711931955],
 [0.053883247161950691, 118.46131209085802],
 [0.053986656488949833, 118.67813052819108],
 [0.054089632031349838, 118.89394708235152],
 [0.054192175793599619, 119.10876638288048],
 [0.054294289770885844, 119.32259303792669],
 [0.054395975949175758, 119.53543163434544],
 [0.054497236305259766, 119.74728673779683],
 [0.054598072806793838, 119.9581628928438],
 [0.054698487412341727, 120.1680646230496],
 [0.054798482071416972, 120.37699643107483],
 [0.054898058724524725, 120.584962798774],
 [0.054997219303203379, 120.7919681872917],
 [0.055095965730066004, 120.99801703715828],
 [0.055194299918841568, 121.20311376838512],
 [0.055292223774416019, 121.40726278055945],
 [0.055389739192873121, 121.61046845293869],
 [0.05548684806153515, 121.81273514454443],
 [0.055583552259003369, 122.01406719425592],
 [0.055679853655198321, 122.21446892090313],
 [0.055775754111399949, 122.41394462335946],
 [0.05587125548028752, 122.61249858063384],
 [0.055966359605979364, 122.81013505196263],
 [0.056061068324072448, 123.00685827690094],
 [0.056155383461681731, 123.20267247541356],
 [0.056249306837479369, 123.39758184796553],
 [0.056342840261733723, 123.59159057561217],
 [0.056435985536348203, 123.78470282008887],
 [0.056528744454899908, 123.97692272390026],
 [0.056621118802678103, 124.16825441040916],
 [0.056713110356722524, 124.35870198392497],
 [0.056804720885861491, 124.54826952979175],
 [0.05689595215074985, 124.73696111447583],
 [0.056986805903906743, 124.92478078565306],
 [0.057077283889753201, 125.11173257229562],
 [0.057167387844649573, 125.29782048475847],
 [0.057257119496932761, 125.48304851486537],
 [0.057346480566953302, 125.66742063599449],
 [0.057435472767112271, 125.85094080316364],
 [0.05752409780189801, 126.03361295311517],
 [0.057612357367922695, 126.21544100440035],
 [0.057700253153958728, 126.39642885746346],
 [0.057787786840974964, 126.57658039472545],
 [0.057874960102172773, 126.75589948066724],
 [0.057961774603021939, 126.93438996191259],
 [0.058048232001296364, 127.11205566731063],
 [0.058134333947109651, 127.28890040801801],
 [0.058220082082950493, 127.46492797758061],
 [0.058305478043717905, 127.64014215201499],
 [0.058390523456756292, 127.81454668988928],
 [0.058475219941890373, 127.98814533240393],
 [0.05855956911145991, 128.16094180347187],
 [0.058643572570354296, 128.33293980979843],
 [0.058727231916047, 128.50414304096088],
 [0.058810548738629821, 128.67455516948752],
 [0.058893524620846988, 128.84417985093646],
 [0.058976161138129139, 129.01302072397411],
 [0.059058459858627087, 129.18108141045317],
 [0.059140422343245486, 129.34836551549037],
 [0.059222050145676282, 129.51487662754371],
 [0.059303344812432081, 129.68061831848956],
 [0.059384307882879288, 129.8455941436992],
 [0.059464940889271144, 130.00980764211511],
 [0.059545245356780602, 130.17326233632687],
 [0.059625222803533023, 130.33596173264678],
 [0.059704874740638778, 130.49790932118501],
 [0.05978420267222561, 130.65910857592448],
 [0.059863208095470952, 130.8195629547954],
 [0.05994189250063401, 130.9792758997495],
 [0.060020257371087751, 131.13825083683369],
 [0.060098304183350701, 131.29649117626377],
 [0.060176034407118641, 131.45400031249744],
 [0.060253449505296125, 131.61078162430712],
 [0.060330550934027852, 131.76683847485253],
 [0.060407340142729908, 131.92217421175275],
 [0.060483818574120878, 132.07679216715803],
 [0.06055998766425276, 132.23069565782131],
 [0.060635848842541785, 132.38388798516934],
 [0.060711403531799095, 132.53637243537349],
 [0.060786653148261247, 132.68815227942031],
 [0.060861599101620582, 132.83923077318158],
 [0.060936242795055495, 132.98961115748429],
 [0.061010585625260516, 133.13929665818003],
 [0.061084628982476269, 133.28829048621427],
 [0.061158374250519308, 133.43659583769519],
 [0.061231822806811782, 133.58421589396235],
 [0.061304976022411001, 133.73115382165477],
 [0.061377835262038834, 133.87741277277897],
 [0.061450401884111001, 134.02299588477655],
 [0.06152267724076619, 134.16790628059152],
 [0.061594662677895075, 134.31214706873723],
 [0.061666359535169191, 134.45572134336311],
 [0.061737769146069661, 134.59863218432099],
 [0.061808892837915803, 134.74088265723125],
 [0.061879731931893618, 134.88247581354852],
 [0.061950287743084108, 135.02341469062713],
 [0.062020561580491497, 135.1637023117863],
 [0.062090554747071321, 135.30334168637495],
 [0.062160268539758354, 135.44233580983629],
 [0.062229704249494457, 135.58068766377212],
 [0.062298863161256247, 135.71840021600667],
 [0.06236774655408267, 135.85547642065035],
 [0.062436355701102446, 135.99191921816313],
 [0.062504691869561355, 136.12773153541755],
 [0.062572756320849465, 136.26291628576158],
 [0.062640550310528134, 136.39747636908103],
 [0.062708075088357004, 136.53141467186185],
 [0.062775331898320755, 136.66473406725197],
 [0.062842321978655816, 136.79743741512297],
 [0.062909046561876916, 136.92952756213143],
 [0.062975506874803563, 137.06100734178],
 [0.063041704138586313, 137.19187957447815],
 [0.063107639568732971, 137.32214706760269],
 [0.063173314375134682, 137.451812615558],
 [0.063238729762091869, 137.58087899983599],
 [0.063303886928340103, 137.70934898907569],
 [0.063368787067075771, 137.83722533912277],
 [0.063433431365981702, 137.96451079308849],
 [0.063497821007252611, 138.09120808140869],
 [0.063561957167620517, 138.21731992190229],
 [0.063625841018379911, 138.3428490198296],
 [0.063689473725412948, 138.46779806795038],
 [0.063752856449214426, 138.59216974658156],
 [0.063815990344916668, 138.71596672365479],
 [0.063878876562314321, 138.83919165477357],
 [0.063941516245889021, 138.96184718327032],
 [0.064003910534833952, 139.08393594026307],
 [0.064066060563078231, 139.20546054471183],
 [0.064127967459311305, 139.32642360347484],
 [0.064189632347007128, 139.44682771136448],
 [0.064251056344448235, 139.56667545120291],
 [0.064312240564749781, 139.68596939387749],
 [0.064373186115883385, 139.80471209839587],
 [0.064433894100700911, 139.92290611194102],
 [0.064494365616958121, 140.04055396992572],
 [0.064554601757338223, 140.15765819604704],
 [0.06461460360947531, 140.27422130234046],
 [0.064674372255977691, 140.39024578923375],
 [0.064733908774451121, 140.50573414560057],
 [0.064793214237521907, 140.62068884881396],
 [0.064852289712859931, 140.73511236479933],
 [0.064911136263201538, 140.84900714808754],
 [0.064969754946372332, 140.96237564186737],
 [0.065028146815309898, 141.07522027803807],
 [0.065086312918086336, 141.1875434772615],
 [0.06514425429793079, 141.29934764901395],
 [0.065201971993251812, 141.41063519163794],
 [0.065259467037659627, 141.52140849239368],
 [0.065316740459988309, 141.63166992751016],
 [0.065373793284317874, 141.74142186223625],
 [0.065430626529996219, 141.85066665089138],
 [0.065487241211660982, 141.95940663691604],
 [0.065543638339261343, 142.06764415292207],
 [0.065599818918079672, 142.17538152074263],
 [0.065655783948753071, 142.28262105148215],
 [0.065711534427294882, 142.38936504556574],
 [0.065767071345116015, 142.49561579278867],
 [0.065822395689046229, 142.60137557236541],
 [0.065877508441355315, 142.70664665297855],
 [0.065932410579774128, 142.81143129282745],
 [0.06598710307751561, 142.91573173967669],
 [0.066041586903295632, 143.01955023090431],
 [0.066095863021353785, 143.12288899354982],
 [0.066149932391474056, 143.22575024436188],
 [0.066203795969005441, 143.32813618984591],
 [0.066257454704882424, 143.43004902631145],
 [0.06631090954564535, 143.53149093991928],
 [0.066364161433460772, 143.63246410672821],
 [0.06641721130614163, 143.73297069274184],
 [0.066470060097167397, 143.8330128539551],
 [0.066522708735704081, 143.9325927364003],
 [0.066575158146624153, 144.03171247619335],
 [0.066627409250526418, 144.13037419957951],
 [0.066679462963755709, 144.22858002297897],
 [0.066731320198422608, 144.3263320530323],
 [0.066782981862422972, 144.42363238664564],
 [0.066834448859457413, 144.52048311103562],
 [0.066885722089050692, 144.61688630377421],
 [0.066936802446571006, 144.71284403283326],
 [0.066987690823249194, 144.80835835662884],
 [0.067038388106197874, 144.90343132406539],
 [0.06708889517843046, 144.99806497457976],
 [0.067139212918880087, 145.09226133818481],
 [0.067189342202418484, 145.18602243551308],
 [0.067239283899874755, 145.27935027786006],
 [0.067289038878054028, 145.3722468672274],
 [0.067338607999756064, 145.46471419636583],
 [0.067387992123793788, 145.55675424881787],
 [0.067437192105011645, 145.64836899896042],
 [0.067486208794304017, 145.7395604120471],
 [0.067535043038633433, 145.83033044425039],
 [0.06758369568104873, 145.92068104270368],
 [0.067632167560703171, 146.01061414554286],
 [0.06768045951287241, 146.10013168194811],
 [0.067728572368972437, 146.18923557218514],
 [0.067776506956577401, 146.27792772764639],
 [0.067824264099437356, 146.36621005089208],
 [0.067871844617495958, 146.45408443569104],
 [0.067919249326908016, 146.54155276706126],
 [0.06796647904005701, 146.62861692131037],
 [0.068013534565572561, 146.7152787660759],
 [0.068060416708347712, 146.80154016036531],
 [0.068107126269556242, 146.8874029545959],
 [0.068153664046669823, 146.97286899063448],
 [0.068200030833475159, 147.05794010183689],
 [0.068246227420090955, 147.14261811308731],
 [0.068292254592984949, 147.22690484083748],
 [0.068338113134990705, 147.31080209314553],
 [0.068383803825324455, 147.39431166971491],
 [0.068429327439601784, 147.47743536193292],
 [0.068474684749854298, 147.56017495290911],
 [0.068519876524546164, 147.64253221751358],
 [0.068564903528590598, 147.72450892241505],
 [0.068609766523366283, 147.80610682611876],
 [0.068654466266733674, 147.88732767900419],
 [0.06869900351305129, 147.96817322336253],
 [0.068743379013191871, 148.04864519343417],
 [0.068787593514558515, 148.12874531544588],
 [0.068831647761100664, 148.20847530764777],
 [0.068875542493330111, 148.2878368803502],
 [0.06891927844833684, 148.36683173596049],
 [0.068962856359804883, 148.44546156901936],
 [0.069006276958028023, 148.52372806623737],
 [0.06904954096992548, 148.60163290653105],
 [0.069092649119057492, 148.67917776105892],
 [0.069135602125640849, 148.75636429325735],
 [0.069178400706564322, 148.83319415887624],
 [0.069221045575404067, 148.90966900601452],
 [0.069263537442438905, 148.98579047515554],
 [0.06930587701466559, 149.06156019920223],
 [0.069348064995813932, 149.13697980351216],
 [0.069390102086361954, 149.21205090593236],
 [0.069431988983550866, 149.28677511683404],
 [0.069473726381400022, 149.36115403914715],
 [0.069515314970721845, 149.43518926839479],
 [0.069556755439136628, 149.50888239272737],
 [0.069598048471087262, 149.58223499295673],
 [0.069639194747853947, 149.65524864259001],
 [0.069680194947568788, 149.72792490786347],
 [0.069721049745230371, 149.80026534777596],
 [0.069761759812718219, 149.87227151412256],
 [0.069802325818807187, 149.94394495152767],
 [0.069842748429181839, 150.01528719747824],
 [0.069883028306450726, 150.08629978235678],
 [0.069923166110160584, 150.15698422947412],
 [0.069963162496810477, 150.2273420551021],
 [0.070003018119865923, 150.29737476850616],
 [0.070042733629772855, 150.3670838719776],
 [0.07008230967397161, 150.43647086086594],
 [0.070121746896910808, 150.5055372236109],
 [0.070161045940061173, 150.57428444177435],
 [0.070200207441929299, 150.64271399007211],
 [0.070239232038071336, 150.71082733640557],
 [0.070278120361106644, 150.77862594189321],
 [0.070316873040731337, 150.84611126090186],
 [0.070355490703731802, 150.91328474107803],
 [0.070393973973998158, 150.98014782337884],
 [0.070432323472537639, 151.04670194210297],
 [0.070470539817487868, 151.11294852492148],
 [0.070508623624130198, 151.17888899290833],
 [0.070546575504902828, 151.24452476057098],
 [0.070584396069413996, 151.30985723588066],
 [0.070622085924455041, 151.37488782030258],
 [0.070659645674013408, 151.43961790882602],
 [0.070697075919285618, 151.50404888999424],
 [0.070734377258690154, 151.56818214593426],
 [0.070771550287880319, 151.63201905238651],
 [0.070808595599756982, 151.69556097873433],
 [0.070845513784481312, 151.7588092880334],
 [0.070882305429487452, 151.82176533704092],
 [0.070918971119495092, 151.88443047624469],
 [0.070955511436522026, 151.94680604989219],
 [0.070991926959896659, 152.00889339601929],
 [0.071028218266270385, 152.07069384647906],
 [0.071064385929629992, 152.13220872697028],
 [0.071100430521309974, 152.19343935706587],
 [0.071136352610004783, 152.25438705024123],
 [0.071172152761781016, 152.31505311390239],
 [0.071207831540089569, 152.37543884941408],
 [0.071243389505777729, 152.43554555212762],
 [0.071278827217101201, 152.49537451140873],
 [0.071314145229736059, 152.55492701066518],
 [0.071349344096790709, 152.6142043273743],
 [0.071384424368817714, 152.67320773311042],
 [0.071419386593825629, 152.73193849357213],
 [0.071454231317290756, 152.79039786860943],
 [0.071488959082168846, 152.84858711225075],
 [0.07152357042890671, 152.90650747272986],
 [0.071558065895453868, 152.96416019251262],
 [0.071592446017274047, 153.02154650832369],
 [0.071626711327356693, 153.07866765117302],
 [0.071660862356228386, 153.13552484638222],
 [0.071694899631964223, 153.19211931361093],
 [0.071728823680199172, 153.24845226688294],
 [0.071762635024139293, 153.30452491461219],
 [0.071796334184573027, 153.36033845962876],
 [0.071829921679882308, 153.41589409920468],
 [0.071863398026053737, 153.47119302507951],
 [0.071896763736689615, 153.52623642348601],
 [0.071930019323018957, 153.58102547517555],
 [0.071963165293908476, 153.63556135544343],
 [0.071996202155873504, 153.68984523415409],
 [0.072029130413088838, 153.74387827576624],
 [0.072061950567399558, 153.79766163935778],
 [0.07209466311833182, 153.85119647865076],
 [0.07212726856310353, 153.90448394203599],
 [0.072159767396635044, 153.9575251725978],
 [0.072192160111559753, 154.01032130813849],
 [0.072224447198234676, 154.06287348120276],
 [0.072256629144750992, 154.11518281910199],
 [0.072288706436944447, 154.16725044393846],
 [0.072320679558405862, 154.21907747262941],
 [0.072352548990491428, 154.27066501693093],
 [0.072384315212333086, 154.3220141834619],
 [0.072415978700848788, 154.37312607372772],
 [0.072447539930752702, 154.42400178414383],
 [0.07247899937456545, 154.47464240605939],
 [0.07251035750262419, 154.52504902578056],
 [0.07254161478309272, 154.57522272459391],
 [0.072572771681971557, 154.62516457878948],
 [0.072603828663107869, 154.67487565968401],
 [0.0726347861882055, 154.72435703364383],
 [0.072665644716834801, 154.77360976210778],
 [0.072696404706442549, 154.82263490160994],
 [0.07272706661236171, 154.87143350380234],
 [0.072757630887821251, 154.92000661547743],
 [0.072788097983955816, 154.96835527859071],
 [0.072818468349815452, 155.01648053028288],
 [0.072848742432375199, 155.06438340290222],
 [0.072878920676544723, 155.1120649240267],
 [0.072909003525177804, 155.15952611648601],
 [0.072938991419081875, 155.20676799838347],
 [0.072968884797027483, 155.25379158311796],
 [0.072998684095757674, 155.30059787940559],
 [0.073028389749997369, 155.34718789130133],
 [0.073058002192462709, 155.39356261822061],
 [0.07308752185387031, 155.43972305496072],
 [0.073116949162946532, 155.48567019172214],
 [0.073146284546436652, 155.53140501412977],
 [0.073175528429114012, 155.57692850325412],
 [0.073204681233789162, 155.62224163563235],
 [0.073233743381318911, 155.66734538328916],
 [0.073262715290615343, 155.71224071375769],
 [0.073291597378654844, 155.75692859010027],
 [0.07332039006048699, 155.80140997092909],
 [0.073349093749243491, 155.84568581042672],
 [0.073377708856147028, 155.88975705836663],
 [0.073406235790520122, 155.93362466013352],
 [0.073434674959793847, 155.97728955674364],
 [0.073463026769516632, 156.02075268486493],
 [0.073491291623362903, 156.06401497683717],
 [0.073519469923141789, 156.1070773606919],
 [0.073547562068805708, 156.14994076017243],
 [0.073575568458458987, 156.19260609475356],
 [0.073603489488366355, 156.23507427966135],
 [0.07363132555296148, 156.27734622589276],
 [0.073659077044855401, 156.31942284023518],
 [0.073686744354844985, 156.36130502528587],
 [0.073714327871921267, 156.40299367947136],
 [0.07374182798327783, 156.44448969706667],
 [0.07376924507431909, 156.48579396821449],
 [0.073796579528668582, 156.52690737894437],
 [0.073823831728177178, 156.56783081119164],
 [0.073851002052931256, 156.6085651428163],
 [0.073878090881260905, 156.64911124762196],
 [0.073905098589748003, 156.68946999537451],
 [0.07393202555323429, 156.72964225182076],
 [0.073958872144829435, 156.76962887870707],
 [0.073985638735919013, 156.8094307337978],
 [0.074012325696172512, 156.84904867089372],
 [0.074038933393551221, 156.88848353985034],
 [0.074065462194316131, 156.92773618659609],
 [0.074091912463035814, 156.96680745315055],
 [0.074118284562594228, 157.00569817764242],
 [0.074144578854198492, 157.04440919432756],
 [0.07417079569738666, 157.08294133360684],
 [0.074196935450035403, 157.12129542204403],
 [0.074222998468367704, 157.15947228238346],
 [0.074248985106960513, 157.19747273356768],
 [0.074274895718752315, 157.23529759075504],
 [0.074300730655050745, 157.27294766533717],
 [0.074326490265540085, 157.31042376495643],
 [0.074352174898288784, 157.34772669352313],
 [0.074377784899756924, 157.38485725123289],
 [0.074403320614803647, 157.42181623458373],
 [0.074428782386694578, 157.45860443639319],
 [0.074454170557109137, 157.49522264581532],
 [0.074479485466147924, 157.53167164835764],
 [0.074504727452339964, 157.56795222589795],
 [0.074529896852650004, 157.6040651567011],
 [0.074554994002485747, 157.64001121543572],
 [0.074580019235705008, 157.67579117319082],
 [0.074604972884622922, 157.71140579749235],
 [0.074629855280019017, 157.74685585231961],
 [0.074654666751144363, 157.78214209812171],
 [0.074679407625728625, 157.81726529183379],
 [0.07470407822998705, 157.8522261868934],
 [0.074728678888627526, 157.88702553325649],
 [0.074753209924857497, 157.92166407741365],
 [0.07477767166039094, 157.95614256240603],
 [0.074802064415455227, 157.99046172784131],
 [0.074826388508798011, 158.02462230990955],
 [0.074850644257694054, 158.05862504139898],
 [0.074874831977952069, 158.09247065171178],
 [0.07489895198392145, 158.12615986687968],
 [0.074923004588499043, 158.15969340957946],
 [0.074946990103135838, 158.1930719991486],
 [0.074970908837843658, 158.22629635160064],
 [0.074994761101201818, 158.25936717964046],
 [0.075018547200363744, 158.29228519267974],
 [0.075042267441063543, 158.32505109685201],
 [0.075065922127622572, 158.3576655950279],
 [0.075089511562955971, 158.39012938683021],
 [0.075113036048579176, 158.42244316864884],
 [0.075136495884614352, 158.45460763365585],
 [0.075159891369796838, 158.4866234718202],
 [0.075183222801481583, 158.51849136992266],
 [0.075206490475649493, 158.55021201157047],
 [0.075229694686913817, 158.58178607721206],
 [0.07525283572852641, 158.61321424415158],
 [0.075275913892384083, 158.64449718656348],
 [0.075298929469034825, 158.67563557550699],
 [0.07532188274768406, 158.70663007894044],
 [0.075344774016200836, 158.73748136173563],
 [0.075367603561124, 158.76819008569214],
 [0.075390371667668363, 158.79875690955143],
 [0.075413078619730775, 158.82918248901106],
 [0.075435724699896275, 158.8594674767387],
 [0.075458310189444114, 158.88961252238613],
 [0.075480835368353788, 158.91961827260323],
 [0.075503300515311053, 158.94948537105179],
 [0.075525705907713903, 158.97921445841936],
 [0.075548051821678536, 159.00880617243297],
 [0.075570338532045242, 159.03826114787279],
 [0.075592566312384341, 159.06758001658579],
 [0.075614735435001995, 159.09676340749931],
 [0.075636846170946112, 159.12581194663449],
 [0.075658898790012113, 159.15472625711968],
 [0.075680893560748752, 159.18350695920395],
 [0.075702830750463856, 159.21215467027022],
 [0.075724710625230077, 159.24067000484862],
 [0.07574653344989056, 159.26905357462962],
 [0.075768299488064669, 159.29730598847718],
 [0.07579000900215363, 159.32542785244178],
 [0.075811662253346151, 159.35341976977344],
 [0.075833259501624034, 159.38128234093469],
 [0.075854801005767733, 159.40901616361339],
 [0.075876287023361938, 159.43662183273557],
 [0.075897717810801055, 159.46409994047823],
 [0.075919093623294756, 159.49145107628203],
 [0.075940414714873436, 159.51867582686387],
 [0.075961681338393622, 159.54577477622954],
 [0.07598289374554347, 159.57274850568623],
 [0.076004052186848095, 159.59959759385507],
 [0.076025156911674996, 159.62632261668335],
 [0.076046208168239371, 159.65292414745713],
 [0.076067206203609461, 159.67940275681332],
 [0.076088151263711851, 159.70575901275203],
 [0.076109043593336742, 159.73199348064875],
 [0.076129883436143186, 159.75810672326645],
 [0.076150671034664316, 159.78409930076765],
 [0.076171406630312591, 159.80997177072646],
 [0.076192090463384904, 159.83572468814054],
 [0.076212722773067815, 159.861358605443],
 [0.0762333037974426, 159.88687407251422],
 [0.076253833773490431, 159.91227163669367],
 [0.07627431293709741, 159.93755184279166],
 [0.076294741523059664, 159.96271523310105],
 [0.076315119765088354, 159.98776234740882],
 [0.076335447895814687, 160.01269372300766],
 [0.07635572614679495, 160.03750989470757],
 [0.076375954748515412, 160.06221139484725],
 [0.07639613393039732, 160.08679875330554],
 [0.076416263920801808, 160.1112724975128],
 [0.076436344947034773, 160.13563315246225],
 [0.076456377235351786, 160.15988124072115],
 [0.076476361010962904, 160.18401728244208],
 [0.076496296498037547, 160.20804179537413],
 [0.076516183919709282, 160.23195529487387],
 [0.076536023498080599, 160.25575829391653],
 [0.076555815454227696, 160.27945130310698],
 [0.076575560008205201, 160.30303483069065],
 [0.076595257379050929, 160.32650938256444],
 [0.076614907784790534, 160.3498754622876],
 [0.076634511442442224, 160.3731335710925],
 [0.0766540685680214, 160.39628420789538],
 [0.076673579376545273, 160.41932786930712],
 [0.076693044082037504, 160.44226504964382],
 [0.076712462897532793, 160.46509624093738],
 [0.076731836035081435, 160.48782193294616],
 [0.076751163705753883, 160.5104426131654],
 [0.076770446119645244, 160.53295876683768],
 [0.076789683485879834, 160.55537087696339],
 [0.076808876012615629, 160.57767942431104],
 [0.076828023907048737, 160.59988488742758],
 [0.076847127375417851, 160.62198774264866],
 [0.076866186623008662, 160.6439884641089],
 [0.076885201854158286, 160.66588752375199],
 [0.07690417327225961, 160.68768539134084],
 [0.076923101079765674, 160.70938253446766],
 [0.076941985478194044, 160.73097941856403],
 [0.076960826668131074, 160.7524765069108],
 [0.076979624849236247, 160.77387426064809],
 [0.076998380220246465, 160.79517313878515],
 [0.077017092978980295, 160.81637359821025],
 [0.077035763322342202, 160.83747609370045],
 [0.077054391446326809, 160.85848107793134],
 [0.077072977546023061, 160.8793890014868],
 [0.07709152181561843, 160.90020031286858],
 [0.077110024448403092, 160.92091545850604],
 [0.077128485636774047, 160.94153488276561],
 [0.077146905572239258, 160.96205902796041],
 [0.07716528444542177, 160.98248833435966],
 [0.07718362244606379, 161.00282324019821],
 [0.077201919763030757, 161.02306418168587],
 [0.077220176584315389, 161.04321159301679],
 [0.07723839309704171, 161.06326590637883],
 [0.07725656948746909, 161.08322755196272],
 [0.077274705940996224, 161.10309695797139],
 [0.077292802642165093, 161.12287455062906],
 [0.077310859774664953, 161.1425607541905],
 [0.077328877521336256, 161.16215599095],
 [0.077346856064174588, 161.18166068125058],
 [0.077364795584334553, 161.20107524349282],
 [0.077382696262133677, 161.22040009414405],
 [0.077400558277056247, 161.23963564774706],
 [0.077418381807757203, 161.2587823169292],
 [0.077436167032065933, 161.27784051241107],
 [0.077453914126990125, 161.29681064301545],
 [0.077471623268719536, 161.31569311567594],
 [0.077489294632629754, 161.33448833544585],
 [0.077506928393286018, 161.35319670550675],
 [0.077524524724446917, 161.37181862717719],
 [0.077542083799068115, 161.3903544999213],
 [0.07755960578930611, 161.40880472135731],
 [0.077577090866521856, 161.42716968726617],
 [0.077594539201284493, 161.44544979159997],
 [0.077611950963374976, 161.46364542649042],
 [0.07762932632178976, 161.48175698225722],
 [0.077646665444744356, 161.49978484741649],
 [0.077663968499677008, 161.51772940868909],
 [0.077681235653252231, 161.53559105100888],
 [0.077698467071364435, 161.55337015753102],
 [0.077715662919141434, 161.57106710964018],
 [0.077732823360948028, 161.58868228695869],
 [0.077749948560389512, 161.60621606735469],
 [0.07776703868031519, 161.62366882695031],
 [0.077784093882821856, 161.64104094012964],
 [0.077801114329257293, 161.65833277954681],
 [0.077818100180223698, 161.67554471613397],
 [0.077835051595581184, 161.69267711910925],
 [0.077851968734451146, 161.7097303559847],
 [0.077868851755219737, 161.72670479257414],
 [0.077885700815541192, 161.74360079300101],
 [0.077902516072341291, 161.76041871970622],
 [0.077919297681820654, 161.77715893345592],
 [0.077936045799458148, 161.79382179334917],
 [0.077952760580014185, 161.8104076568257],
 [0.077969442177534043, 161.82691687967358],
 [0.077986090745351205, 161.84334981603683],
 [0.078002706436090613, 161.85970681842306],
 [0.078019289401671951, 161.87598823771094],
 [0.078035839793312911, 161.89219442315786],
 [0.078052357761532423, 161.90832572240728],
 [0.078068843456153905, 161.92438248149631],
 [0.078085297026308467, 161.94036504486303],
 [0.07810171862043809, 161.95627375535395],
 [0.078118108386298846, 161.97210895423132],
 [0.078134466470964045, 161.98787098118052],
 [0.078150793020827405, 162.00356017431727],
 [0.078167088181606156, 162.01917687019491],
 [0.078183352098344219, 162.03472140381163],
 [0.07819958491541526, 162.0501941086176],
 [0.078215786776525853, 162.06559531652226],
 [0.078231957824718507, 162.08092535790129],
 [0.078248098202374758, 162.09618456160376],
 [0.078264208051218212, 162.11137325495918],
 [0.078280287512317609, 162.12649176378454],
 [0.078296336726089821, 162.14154041239127],
 [0.078312355832302893, 162.15651952359221],
 [0.078328344970078997, 162.17142941870856],
 [0.078344304277897459, 162.18627041757668],
 [0.078360233893597711, 162.20104283855511],
 [0.078376133954382268, 162.21574699853122],
 [0.078392004596819623, 162.23038321292816],
 [0.07840784595684723, 162.2449517957115],
 [0.078423658169774396, 162.25945305939609],
 [0.078439441370285176, 162.27388731505263],
 [0.07845519569244129, 162.2882548723145],
 [0.078470921269684954, 162.3025560393842],
 [0.078486618234841793, 162.31679112304019],
 [0.078502286720123646, 162.33096042864327],
 [0.078517926857131437, 162.34506426014326],
 [0.078533538776857981, 162.35910292008549],
 [0.078549122609690797, 162.37307670961724],
 [0.078564678485414888, 162.38698592849423],
 [0.078580206533215569, 162.40083087508711],
 [0.078595706881681188, 162.41461184638777],
 [0.078611179658805916, 162.42832913801573],
 [0.07862662499199248, 162.44198304422454],
 [0.078642043008054913, 162.45557385790804],
 [0.078657433833221241, 162.46910187060664],
 [0.078672797593136209, 162.48256737251361],
 [0.078688134412864, 162.49597065248128],
 [0.07870344441689088, 162.50931199802719],
 [0.078718727729127899, 162.5225916953404],
 [0.078733984472913507, 162.53581002928743],
 [0.078749214771016249, 162.54896728341859],
 [0.07876441874563736, 162.56206373997387],
 [0.078779596518413431, 162.57509967988909],
 [0.078794748210418958, 162.58807538280192],
 [0.078809873942168984, 162.60099112705788],
 [0.078824973833621678, 162.61384718971627],
 [0.078840048004180888, 162.62664384655616],
 [0.078855096572698721, 162.63938137208231],
 [0.078870119657478102, 162.65206003953099],
 [0.078885117376275279, 162.66468012087594],
 [0.078900089846302368, 162.67724188683414],
 [0.078915037184229875, 162.68974560687158],
 [0.078929959506189196, 162.70219154920918],
 [0.078944856927775087, 162.71457998082835],
 [0.078959729564048162, 162.72691116747691],
 [0.078974577529537363, 162.73918537367467],
 [0.078989400938242416, 162.75140286271915],
 [0.07900419990363626, 162.76356389669118],
 [0.079018974538667519, 162.77566873646057],
 [0.079033724955762871, 162.7877176416917],
 [0.079048451266829495, 162.79971087084911],
 [0.079063153583257481, 162.81164868120297],
 [0.079077832015922181, 162.82353132883466],
 [0.079092486675186632, 162.83535906864225],
 [0.079107117670903879, 162.84713215434599],
 [0.079121725112419369, 162.85885083849368],
 [0.079136309108573277, 162.8705153724662],
 [0.079150869767702844, 162.88212600648279],
 [0.079165407197644713, 162.89368298960648],
 [0.079179921505737202, 162.90518656974942],
 [0.079194412798822666, 162.91663699367822],
 [0.079208881183249757, 162.92803450701919],
 [0.079223326764875687, 162.93937935426365],
 [0.079237749649068531, 162.9506717787732],
 [0.079252149940709476, 162.96191202278484],
 [0.079266527744195056, 162.97310032741626],
 [0.079280883163439425, 162.98423693267102],
 [0.079295216301876567, 162.9953220774436],
 [0.079309527262462501, 163.00635599952463],
 [0.079323816147677501, 163.01733893560595],
 [0.079338083059528303, 163.02827112128566],
 [0.079352328099550284, 163.03915279107318],
 [0.079366551368809657, 163.04998417839437],
 [0.079380752967905602, 163.06076551559642],
 [0.07939493299697245, 163.07149703395285],
 [0.079409091555681849, 163.08217896366858],
 [0.079423228743244853, 163.09281153388474],
 [0.07943734465841408, 163.10339497268365],
 [0.07945143939948586, 163.11392950709367],
 [0.079465513064302273, 163.12441536309416],
 [0.079479565750253306, 163.1348527656202],
 [0.079493597554278914, 163.14524193856749],
 [0.079507608572871119, 163.15558310479719],
 [0.079521598902076052, 163.16587648614063],
 [0.079535568637496057, 163.17612230340407],
 [0.079549517874291681, 163.18632077637349],
 [0.07956344670718378, 163.19647212381926],
 [0.079577355230455504, 163.20657656350087],
 [0.07959124353795434, 163.21663431217155],
 [0.079605111723094119, 163.22664558558296],
 [0.079618959878857018, 163.23661059848979],
 [0.079632788097795559, 163.2465295646544],
 [0.079646596472034606, 163.25640269685138],
 [0.07966038509327332, 163.26623020687214],
 [0.079674154052787133, 163.27601230552941],
 [0.079687903441429717, 163.28574920266183],
 [0.079701633349634898, 163.29544110713837],
 [0.079715343867418659, 163.30508822686286],
 [0.079729035084381006, 163.31469076877843],
 [0.079742707089707932, 163.32424893887199],
 [0.079756359972173313, 163.33376294217857],
 [0.0797699938201408, 163.34323298278582],
 [0.079783608721565757, 163.35265926383829],
 [0.079797204763997098, 163.36204198754189],
 [0.079810782034579197, 163.37138135516807],
 [0.079824340620053752, 163.38067756705834],
 [0.079837880606761627, 163.3899308226284],
 [0.079851402080644743, 163.39914132037251],
 [0.079864905127247865, 163.4083092578677],
 [0.079878389831720492, 163.41743483177805],
 [0.07989185627881866, 163.42651823785889],
 [0.079905304552906758, 163.43555967096097],
 [0.079918734737959324, 163.44455932503467],
 [0.079932146917562896, 163.45351739313421],
 [0.079945541174917742, 163.46243406742164],
 [0.079958917592839698, 163.47130953917113],
 [0.079972276253761895, 163.480143998773],
 [0.079985617239736587, 163.48893763573778],
 [0.079998940632436844, 163.49769063870031],
 [0.080012246513158372, 163.50640319542379],
 [0.080025534962821193, 163.51507549280379],
 [0.080038806061971418, 163.52370771687228],
 [0.08005205989078297, 163.53230005280156],
 [0.080065296529059318, 163.54085268490834],
 [0.080078516056235144, 163.54936579665761],
 [0.080091718551378088, 163.55783957066657],
 [0.080104904093190432, 163.5662741887086],
 [0.080118072760010789, 163.57466983171713],
 [0.080131224629815784, 163.58302667978953],
 [0.080144359780221705, 163.59134491219095],
 [0.080157478288486211, 163.5996247073582],
 [0.080170580231509955, 163.60786624290358],
 [0.080183665685838248, 163.61606969561865],
 [0.0801967347276627, 163.62423524147803],
 [0.080209787432822838, 163.63236305564325],
 [0.080222823876807764, 163.64045331246638],
 [0.080235844134757747, 163.64850618549383],
 [0.080248848281465848, 163.65652184747015],
 [0.080261836391379529, 163.66450047034164],
 [0.080274808538602221, 163.67244222526],
 [0.08028776479689495, 163.68034728258615],
 [0.080300705239677903, 163.68821581189377],
 [0.080313629940032008, 163.69604798197295],
 [0.080326538970700481, 163.70384396083386],
 [0.08033943240409043, 163.7116039157103],
 [0.080352310312274355, 163.71932801306329],
 [0.080365172766991733, 163.7270164185847],
 [0.080378019839650552, 163.73466929720072],
 [0.080390851601328819, 163.74228681307551],
 [0.08040366812277612, 163.74986912961458],
 [0.080416469474415095, 163.75741640946845],
 [0.080429255726342991, 163.76492881453595],
 ...]

Plot Convergence



In [9]:

    
plt.plot(jvec)
plt.title("Convergence of Cost Function")
plt.xlabel("Iteration number")
plt.ylabel("Cost function")
plt.show()

Predict output using trained model



In [10]:

    
hx = h(theta, X)

Plot Results



In [11]:

    
plt.scatter(X[:,1], y)
plt.plot(X[:,1], hx)
plt.show()

Resources

Course website: https://w4zir.github.io/ml17s/

Course resources

Credits

Raschka, Sebastian. Python machine learning. Birmingham, UK: Packt Publishing, 2015. Print.

Andrew Ng, Machine Learning, Coursera

Lucas Shen

David Kaleko