Sveučilište u Zagrebu
Fakultet elektrotehnike i računarstva
http://www.fer.unizg.hr/predmet/su
Ak. god. 2015./2016.
(c) 2015 Jan Šnajder
Verzija: 0.1 (2015-11-11)
In [188]:
import scipy as sp
import scipy.stats as stats
import matplotlib.pyplot as plt
import pandas as pd
%pylab inline
In [189]:
def h(x, w): return sp.dot(x, w)
In [190]:
def plot_decision_boundary(h, boundary=0, margins=None):
x = linspace(-10, 10)
y = linspace(-10, 10)
X1, X2 = np.meshgrid(x, y)
XX = sp.dstack((sp.ones((50, 50)), X1, X2))
plt.contour(X1, X2, h(XX), linecolor='red', levels=[boundary])
if margins!=None:
CS = plt.contour(X1, X2, h(XX), colors=['gray', 'gray'], levels=[margins[0],margins[1]])
plt.clabel(CS, fontsize=9, inline=1)
In [191]:
w = [-2, 1, -0.5]
plot_decision_boundary(lambda x : h(x, w), margins=(-1,1))
In [112]:
w = [2, -1, 0.5]
plot_decision_boundary(lambda x : h(x, w), margins=(-1,1))
In [115]:
w = [2, -1.5, 0.3]
plot_decision_boundary(lambda x : h(x, w), margins=(-1,1))
$\Rightarrow$ Linearna granica u ulaznom prostoru (premda je $f$ nelinearna)
$\Rightarrow$ Model je nelinearan u parametrima (jer je $f$ nelinearna)
In [32]:
xs = sp.linspace(-5,5, 100)
plt.plot(xs, xs);
In [33]:
def sigm(x): return 1 / (1 + sp.exp(-x))
plt.plot(xs, sigm(xs));
In [34]:
plt.plot(xs, sign(xs));
In [192]:
w = [-4, 2, -1]
plot_decision_boundary(lambda x : h(x, w), margins=(-1,1))
In [193]:
plot_decision_boundary(lambda x : sigm(sp.dot(x, w)), boundary=0.5, margins=(0.1,0.9))
Kao i kod regresije, možemo koristiti preslikavanje $\boldsymbol{\phi}:\mathbb{R}^n\to\mathbb{R}^m$ iz ulaznog prostora u prostor značajki:
$$ h(\mathbf{x}) = f\big(\mathbf{w}^\intercal\mathbf{\phi}(\mathbf{x})\big) $$$\Rightarrow$ Linearna granica u prostoru značajki
$\Rightarrow$ Nelinearna granica u ulaznom prostoru
$\Rightarrow$ Model je nelinearan u parametrima (jer je $f$ nelinearna)
In [194]:
def h2(x, w):
x2 = sp.dstack((x, x[:,:,1]*x[:,:,2], x[:,:,1]**2, x[:,:,2]**2))
return sp.dot(x2, w)
In [121]:
w = [-0.05 , -0.15 , -0.5 , 0.15 , -0.08 , 0.05]
plot_decision_boundary(lambda x : h2(x, w), margins=[-1, 1])
$\Rightarrow$ udaljenost točke $\mathbf{x}$ od ravnine je $d=h(\mathbf{x})/\|\mathbf{w}\|$
In [122]:
w = sp.array([-4, 2, -1])
X = sp.array([[1, 5, -1],
[1, -5, 5]])
plot_decision_boundary(lambda x : h(x, w), margins=(-1, 1))
plt.scatter(X[:,1],X[:,2]);
In [78]:
h(X[0], w)
Out[78]:
In [82]:
h(X[1], w)
Out[82]:
In [176]:
sp.linalg.norm(w[1:])
Out[176]:
In [195]:
def distance(x,w): return sp.dot(x, w) / sp.linalg.norm(w[1:])
In [196]:
distance(X[0], w)
Out[196]:
In [180]:
distance(X[1], w)
Out[180]:
In [181]:
w2 = w/10.0
plot_decision_boundary(lambda x : h(x, w2), margins=(-1,1))
plt.scatter(X[:,1],X[:,2]);
In [182]:
h(X[0], w2)
Out[182]:
In [183]:
h(X[1], w2)
Out[183]:
In [185]:
sp.linalg.norm(w2[1:])
Out[185]:
In [186]:
distance(X[0], w2)
Out[186]:
In [187]:
distance(X[1], w2)
Out[187]:
In [197]:
h1 = lambda x: h(x, [0, 2, 1])
plot_decision_boundary(h1)
h2 = lambda x: h(x, [-0.2, 0.7, -0.8])
plot_decision_boundary(h2)
h3 = lambda x: h(x, [-1.5, 0.1, 0.5])
plot_decision_boundary(h3)
plt.scatter(X[:,1],X[:,2]);
In [152]:
print h1(X[0]), h2(X[0]), h3(X[0])
In [153]:
print h1(X[1]), h2(X[1]), h3(X[1])
In [198]:
def ovr(x): return sp.argmax([h1(x), h2(x), h3(x)])
In [199]:
ovr(X[0])
Out[199]:
In [200]:
ovr(X[1])
Out[200]:
In [157]:
x = linspace(-10, 10)
y = linspace(-10, 10)
X1, X2 = np.meshgrid(x, y)
XX = sp.dstack((sp.ones((50, 50)), X1, X2))
In [166]:
n, m, _ = shape(XX)
YY = sp.zeros((n,m))
for i in range(0,n):
for j in range(0,m):
YY[i,j] = ovr(XX[i, j])
In [175]:
plt.contourf(X1, X2, YY);
Model (OVR): $$ h(\mathbf{x}) = \mathrm{argmax}_j\ \mathbf{w}_j\boldsymbol\phi(\mathbf{x}) $$
Primjer ($K=3$)