Bayes Classifier

Tags

text-processing

A probabilistic approach to sentiment-analysis based on Bayes rule.

For this system we store our text as feature vectors and calculate the probability of each vector having a given sentiment, classifying the sentiment of the text to the most likely sentiment considered.

We see page 7, define: \(s^{*} = argmax_{s_i} p(s_i|T)\) for \(s_{i} \in \{negative, positive, neural\}\)

Where the:

• Left hand side is the sentiment of the text \(T\)
• Right hand side calculates the probability of the text being one of the given sentiments.

We can rephrase this equation with Bayes rule disregarding the denominator because \(p(T)\) is independent of \( s_i \): \[ p(s_i | T) = \frac{p(T|s_i)p(s_i)}{p(T)} \]

We calculate the [see page 9, prior-proability] for the sentiment classes by simply calculating the frequency of each sentiment value in a training dataset. To calculate the correlation of each feature term (t) with a sentiment \(S_i\) we simply [see page 17, consider] the frequency of that term in positive and negative texts.

Note: Prior probability is the safest decision when no other information is given.

Given both these probabilities we can [see page 11, calculate] the likelihood of a text having a given sentiment by: calculating the cumulative product of:

• the probability of any text having that sentiment, and
• of each of the terms in the text having that sentiment

See an [see page 37, example] using Laplace Smoothing.