Naive Bayes Classifier

The classifier that determines the probability of whether an observation or an object belongs to one of the classes. It is assumed that the probability is independent from being affected by various attributes (assumption about conditional independence of classes) that greatly simplifies the associated calculations. Thus, this method is named naive (simple) Bayes classifier.

The naive Bayes classifier considers the object X as belonging to the class C_i if and only if the following condition is satisfied: P(C_i|X)>P(c_j|X),

where:

P(C_i|X). The posterior probability that the X object belongs to the C_i class.
P(C_j|X). The posterior probability that the X object belongs to the C_j random class different from C_i.

In other words, the posterior probability that the object belongs to the C_i class is greater than the posterior probability that the object belongs to any other class.

The posterior probability is a conditional probability of an random variable that is assigned after the relevant evidence is taken into account. In other words, this is the probability of an event A provided that there is an event B.

Theoretically, the error in the Bayes classifier is comparable to errors in other types of classifiers such as neural networks and decision trees. However, in practice, its use may be limited since the assumption about the conditional independency of classes is not always met.