Naive Bayes

• Idea here is to ask the classifier to give us a best guess about the class and assign a probability estimate to that best guess
• Pros are that it works with a small amount of data and is able to handle multiple classes however it is quite sensitive to how the input data is prepared
• Naive Bayes is actually a subset of Bayesian decision theory...

Bayesian Decision Theory
Let's say we have an equation for the probability of a piece of data belonging to Class 1: p1(x,y) and an equation for the data belonging to Class 2: p2(x,y). To classify a new measurement with features (x,y), we can use the below rules:

• If p1(x,y) > p2(x,y), then the class is 1
• If p2(x,y) > p1(x,y), then the class is 2

That’s Bayesian decision theory in a nutshell: choosing the decision with the highest probability

Classifying with Conditional Probabilities
The two rules above don't tell the whole story however, the more precise version is:

• If p1($c_{1}$|x,y) > p2($c_{2}$|x,y), then the class is 1
• If p1($c_{2}$|x,y) > p2($c_{1}$|x,y), then the class is 2

And we can use Bayes' rule to determine p($c_{i}$|x,y)