Probabilistic Classifiers -- Generative Naive Bayes Announcements - - PowerPoint PPT Presentation

▶
probabilistic classifiers generative
SMART_READER_LITE
LIVE PREVIEW

Probabilistic Classifiers -- Generative Naive Bayes Announcements - - PowerPoint PPT Presentation

Nov 17, 2023 185 likes •385 views

Probabilistic Classifiers -- Generative Naive Bayes Announcements Math for Visual Computing Course Agenda: First classifier using our knowledge of probabilities Quick overview of Linear Algebra Least Square

slide-1
SLIDE 1

Probabilistic Classifiers -- Generative

Naive Bayes

slide-2
SLIDE 2

Announcements

  • Math for Visual Computing Course
  • Agenda:

○ First classifier using our knowledge of probabilities ○ Quick overview of Linear Algebra ○ Least Square Regression

slide-3
SLIDE 3
slide-4
SLIDE 4
slide-5
SLIDE 5

Definition

Classifier outputs a probability distribution over the class labels: X → P(y|x)

slide-6
SLIDE 6

Types of Classifiers

  • Generative: learns to generate the examples

○ Learn P(x|y) and P(y) from the training data and then apply Bayes rule to find P(y|x)

slide-7
SLIDE 7

Types of Classifiers

  • Conditional Classifiers: Model conditional distribution

P(y|x) directly. ○ Example: logistic regression classifier ○ Neural Networks.

slide-8
SLIDE 8

Naive Bayes Classifier: A Generative Classifer

Each attribute is conditionally independent given the class label Formula:

slide-9
SLIDE 9

Notion of conditional independence

slide-10
SLIDE 10
slide-11
SLIDE 11

Training a Naïve Bayesian Classifier

  • Given training data:
  • Estimating P(y): A multinomial distribution.
  • Estimating
  • ○

If j-th attribute is categorical: P(xj|y) is estimated as the relative freq of samples having value di as j-th attribute in class y ○ If j-th attribute is continuous: P(xj|y) is estimated through a continuous density function: eg. Gaussian density function

  • Computationally easy in both cases
slide-12
SLIDE 12

Play-tennis example: estimating P(xi|C)

P(true|n) = 3/5 P(true|p) = 3/9 P(false|n) = 2/5 P(false|p) = 6/9 P(high|n) = 4/5 P(high|p) = 3/9 P(normal|n) = 2/5 P(normal|p) = 6/9 P(hot|n) = 2/5 P(hot|p) = 2/9 P(mild|n) = 2/5 P(mild|p) = 4/9 P(cool|n) = 1/5 P(cool|p) = 3/9 P(rain|n) = 2/5 P(rain|p) = 3/9 P(overcast|n) = 0 P(overcast|p) = 4/9 P(sunny|n) = 3/5 P(sunny|p) = 2/9 windy humidity temperature

  • utlook

P(n) = 5/14 P(p) = 9/14

slide-13
SLIDE 13

Naive Bayesian Classifier (II)

  • Given a training set, we can compute the probabilities
slide-14
SLIDE 14

Play-tennis example: classifying X

  • An unseen sample X = <rain, hot, high, false>
  • P(X|p)·P(p) =

P(rain|p)·P(hot|p)·P(high|p)·P(false|p)·P(p)

  • = 3/9·2/9·3/9·6/9·9/14 = 0.010582
  • P(X|n)·P(n)

=P(rain|n)·P(hot|n)·P(high|n)·P(false|n)·P(n)

  • = 2/5·2/5·4/5·2/5·5/14 = 0.018286
  • Sample X is classified in class n (don’t play)
slide-15
SLIDE 15

Example: MNIST digit classification

Jupyter Notebook: http://d2l.ai/chapter_crashcourse/naive-bayes.html

slide-16
SLIDE 16

Naive Bayes for Spam Filtering.

slide-17
SLIDE 17
slide-18
SLIDE 18
slide-19
SLIDE 19

Summary of Naïve Bayes Classifier

  • Creates a distribution over class label given x P(y|x) by applying Bayes rule.

○ Requires estimating P(x|y) for each class y and P(y)

  • Estimates P(x|y) by assuming that each attributes of x are conditionally

independent given the class label

○ Very easy computationally.

  • Many applications in spite of simplistic assumption.
  • Can we do better?