- 2018-09-10 23:50
*views 25*- Machine learning

A simple example

Let's take a look at how this example works in practice . Suppose we are building a classifier , Explain whether the text involves sports or not . Our training set has 5 A sentence ：

Text

Category

A great game（ A great game ）

Sports（ athletic sports ）

The election was over（ Election is over ）

Not sports（ It's not sports ）

Very clean match（ No inside game ）

Sports（ athletic sports ）

A clean but forgettable game

（ A game that can't be forgotten ）

Sports（ athletic sports ）

It was a close election

（ It was an election that was equally close ）

Not sports（ It's not sports ）

Because naive Bayes Bayesian is a probability classifier , We want to calculate sentences “A very close

game” It's the probability of sports and it's not a sport . In Mathematics , What we want is P（Sports | a very close

game） The category of this sentence is the probability of sports .

But how do we calculate these probabilities ?

Characteristic Engineering

When creating a machine learning model , The first thing we need to do is to decide what to use as a feature . for example , If we classify health , The characteristic is probably a person's height , weight , Gender, etc . We'll exclude useless things for the model , Such as a person's name or favorite color .

under these circumstances , We don't even have digital features . We only have words . We need to somehow convert this text to a number that can be calculated .

So what should we do ? It is usually used with word frequency . in other words , We ignore the word order and the construction of sentences , Treat each file as a word bank . Our characteristic will be the count of these words . Although it seems to be too simplistic , But it's amazing .

Bayes theorem

Bayesian theorem in using conditional probability （ If we do it here ） It's useful , Because it provides us with a way to reverse them ：P（A|B）=P(B|A)×P(A)/P(B). In our case , We have P（sports

| a very close game）, So using this theorem, we can reverse the conditional probability ：

Because for our classifier , We just tried to figure out which category had a greater probability , We can discard divisor , It's just a comparison

So that's a better understanding , Because we can actually calculate these probabilities ! Just count the sentences “A very close game” How many times “Sports”

Training focus for , Divide it by the total , You can get it P（a very close game | Sports）.

There is a problem , But our training focus didn't show up “A very close

game”, So this probability is zero . Unless every sentence we want to classify is in our training set , Otherwise, the model won't be useful .

Being Naive

We assume that every word in a sentence has nothing to do with other words . That means we don't look at the whole sentence anymore , It's a single word . We'll P(A very close game) finish writing sth. ：P(a very

close game)=P(a)×P(very)×P(close)×P(game)

This assumption is very strong , But it's very useful . This enables the entire model to handle a small amount of data or data that could be labeled incorrectly . Next, apply it to what we said before ：

P（a very close

game|Sports)=P(a|Sports)×P(very|Sports)×P(close|Sports)×P(game|Sports)

Now? , All of our words actually appeared several times in our training set , We can calculate it !

Calculation probability

The process of calculating probability is actually just counting in our training set .

first , We calculate the prior probability of each category ： For a given sentence in the training set ,P（ athletic sports ） The probability of ⅗. then ,P（ Non Sports ） yes ⅖. then , In calculation P（game |

Sports） namely “game” How many times did it appear sports Sample of , And divide by sports Total number of （11）. therefore ,P(game|Sports)=2/11.

however , We have a problem ：“close” It doesn't appear in any sports In the sample ! That means P（close | Sports）=

0. It's quite inconvenient , Because we're going to multiply it by other probabilities , So we'll get it in the end P(a|Sports)×P(very|Sports)×0×P(game|Sports) be equal to 0. And what we do is not give us any information at all , So we have to find a way .

What should we do ? By using a method known as Laplace smoothing ： We add for each count 1, So it won't be zero . To balance this , We add the possible words to the divisor , So this part will never be greater than 1. In our case , If possible, it is [

“a” ,“great” ,“very” ,“over” ,'it' ,'but' ,'game' ,'election' ,'close' ,'clean'

,'the' ,'was' ,'forgettable' ,'match' ] .

Because the number of possible words is 14, We get it with Laplace smoothing . The results are as follows ：

Now we just double all the probability , See who's bigger ：

perfect ! Our classifier gives “A very close game” yes Sport class .

Advanced technology

There are many things that can be done to improve this basic model . These techniques can make naive Bayes comparable to more advanced methods .

*

Removing

stopwords（ Delete Inactive words ）. These common words , No categories will be added really , for example , One , capable , There are other , Always wait . So for our purpose , The election will be over , A very close game will be a very close match .

*

Lemmatizing words（ Word variation restore ）. This is a combination of different words . So the election , general election , Being elected, etc. will be grouped together , More appearance of the same word .

*

Using n-grams（ Use instance ）. We can calculate some common examples , as “ No inside game ” and “ A close election ”. Not just a word , Calculation of a word .

*

use TF-IDF. Not just counting frequency , We can do something higher

Technology

- Java296 blogs
- Python265 blogs
- Vue125 blogs
- C Language122 blogs
- Algorithm108 blogs
- MySQL96 blogs
- Flow Chart84 blogs
- JavaScript79 blogs
- More...

©2020-2024 ioDraw All rights reserved,
Privacy Policy