Naive Bayes
Given a class variable $y$ and a dependent feature vector $x_1$ through $x_n$, Bayes' theorem states the following relationship.
$$P(y|x_1,\ldots,x_n)=\frac{P(y)P(x_1,\ldots,x_n | y)}{P(x_1,\ldots,x_n)}$$
Using the naive independence assumption that
$$P(x_i|y,x_1,\ldots,x_{i-1},x_{i+1},\ldots,x_n)=P(x_i|y)$$
for all $i$ this relationship is simplified to
$$P(y|x_1,\ldots,x_n)=\frac{P(y)\prod_{i=1}^{n}P(x_i|y)}{P(x_1,\ldots,x_n)}$$
Since $P(x_1,\ldots,x_n)$ is constant given the input, we can use the following classification rule:
$$P(y|x_1,\ldots,x_n) \propto P(y)\prod_{i=1}^{n}P(x_i|y)$$
$\Rightarrow$
$$\hat{y}=arg \overset{max}{y} P(y)\prod_{i=1}^{n}P(x_i|y)$$
In [2]:
from sklearn import datasets
iris = datasets.load_iris()
from sklearn.naive_bayes import GaussianNB
gnb=GaussianNB()
y_pred=gnb.fit(iris.data, iris.target).predict(iris.data)
In [4]:
print('Number of mislabeled points out of the a total %d points: %d'
% (iris.data.shape[0], (iris.target!=y_pred).sum()))
In [24]:
import numpy as np
x_1=np.array([1,1,1,1,1,
2,2,2,2,2,
3,3,3,3,3])
x_2=np.array(['S','M','M','S','S',
'S','M','M','L','L',
'L','M','M','L','L'])
X=np.vstack((x_1,x_2)).T
y=np.array([-1,-1,1,1,-1,
-1,-1,1,1,1,
1, 1, 1,1,-1])
from sklearn.naive_bayes import MultinomialNB
clf=MultinomialNB()
clf.fit(X,y)
In [ ]: