probabilistic classifiers generative
play

Probabilistic Classifiers -- Generative Naive Bayes Announcements - PowerPoint PPT Presentation

Nov 17, 2023 •185 likes •384 views

Probabilistic Classifiers -- Generative Naive Bayes Announcements Math for Visual Computing Course Agenda: First classifier using our knowledge of probabilities Quick overview of Linear Algebra Least Square

  1. Probabilistic Classifiers -- Generative Naive Bayes

  2. Announcements ● Math for Visual Computing Course ● Agenda: ○ First classifier using our knowledge of probabilities ○ Quick overview of Linear Algebra ○ Least Square Regression

  5. Definition Classifier outputs a probability distribution over the class labels: X → P(y|x)

  6. Types of Classifiers ● Generative: learns to generate the examples ○ Learn P(x|y) and P(y) from the training data and then apply Bayes rule to find P(y|x)

  7. Types of Classifiers ● Conditional Classifiers: Model conditional distribution P(y|x) directly. ○ Example: logistic regression classifier ○ Neural Networks.

  8. Naive Bayes Classifier: A Generative Classifer Each attribute � is conditionally independent given the class label Formula:

  9. Notion of conditional independence

  11. Training a Naïve Bayesian Classifier ● Given training data: ● Estimating P(y): A multinomial distribution. ● Estimating � ○ If j-th attribute is categorical: P(xj|y) is estimated as the relative freq of samples having value di as j-th attribute in class y ○ If j-th attribute is continuous: P(xj|y) is estimated through a continuous density function: eg. Gaussian density function ● Computationally easy in both cases

  12. Play-tennis example: estimating P(x i |C) outlook P(sunny|p) = 2/9 P(sunny|n) = 3/5 P(overcast|p) = 4/9 P(overcast|n) = 0 P(rain|p) = 3/9 P(rain|n) = 2/5 temperature P(hot|p) = 2/9 P(hot|n) = 2/5 P(mild|p) = 4/9 P(mild|n) = 2/5 P(cool|p) = 3/9 P(cool|n) = 1/5 humidity P(high|p) = 3/9 P(high|n) = 4/5 P(normal|p) = 6/9 P(normal|n) = 2/5 P(p) = 9/14 windy P(n) = 5/14 P(true|p) = 3/9 P(true|n) = 3/5 P(false|p) = 6/9 P(false|n) = 2/5

  13. Naive Bayesian Classifier (II) ● Given a training set, we can compute the probabilities

  14. Play-tennis example: classifying X ● An unseen sample X = <rain, hot, high, false> ● P(X|p)·P(p) = P(rain|p)·P(hot|p)·P(high|p)·P(false|p)·P(p) = 3/9·2/9·3/9·6/9·9/14 = 0.010582 ● ● P(X|n)·P(n) =P(rain|n)·P(hot|n)·P(high|n)·P(false|n)·P(n) = 2/5·2/5·4/5·2/5·5/14 = 0.018286 ● ● Sample X is classified in class n (don’t play)

  15. Example: MNIST digit classification Jupyter Notebook: http://d2l.ai/chapter_crashcourse/naive-bayes.html

  16. Naive Bayes for Spam Filtering.

  19. Summary of Naïve Bayes Classifier ● Creates a distribution over class label given x P(y|x) by applying Bayes rule. ○ Requires estimating P(x|y) for each class y and P(y) ● Estimates P(x|y) by assuming that each attributes of x are conditionally independent given the class label ○ Very easy computationally. ● Many applications in spite of simplistic assumption. ● Can we do better?

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Logistic Regression, Generative and Discriminative Classifiers Recommended reading: Ng and

Logistic Regression, Generative and Discriminative Classifiers Recommended reading: Ng and

Logistic Regression, Generative and Discriminative Classifiers Recommended reading: Ng and Jordan paper On Discriminative vs. Generative classifiers: A comparison of logistic regression and nave Bayes, A. Ng and M. Jordan, NIPS

392 views • 28 slides
Data Warehousing and Machine Learning Probabilistic Classifiers Thomas D. Nielsen Aalborg

Data Warehousing and Machine Learning Probabilistic Classifiers Thomas D. Nielsen Aalborg

Data Warehousing and Machine Learning Probabilistic Classifiers Thomas D. Nielsen Aalborg University Department of Computer Science Spring 2008 DWML Spring 2008 1 / 34 Probabilistic Classifiers Conditional class probabilities Id. Savings

761 views • 59 slides
Modals, conditionals, and probabilistic generative models Topic 2: Indicative conditionals

Modals, conditionals, and probabilistic generative models Topic 2: Indicative conditionals

Modals, conditionals, and probabilistic generative models Topic 2: Indicative conditionals Separating semantics from reasoning Dan Lassiter, Stanford Linguistics Universit de Paris VII, 2/12/19 Overall plan 1. probability, generative

997 views • 52 slides
Augmented Statistical Models: Exploiting Generative Models in Discriminative Classifiers Martin

Augmented Statistical Models: Exploiting Generative Models in Discriminative Classifiers Martin

Augmented Statistical Models: Exploiting Generative Models in Discriminative Classifiers Martin Layton &amp; Mark Gales 9 December 2005 Cambridge University Engineering Department NIPS 2005 Augmented Statistical Models: Exploiting Generative

726 views • 25 slides
Probabilistic Models of Cognition: Generative models Table of Contents Chapter

Probabilistic Models of Cognition: Generative models Table of Contents Chapter

Probabilistic Models of Cognition: Generative models Table of Contents Chapter Content Generative Model Example: Plinko Machine Working Model can be used captures some for simulation structure of the world in useful way

825 views • 78 slides
Adversarial BoltzmannMachines Belief Nets Networks Variational NADE

Adversarial BoltzmannMachines Belief Nets Networks Variational NADE

Unsupervised Learning Non-probabilistic Models Probabilistic Sparse Coding (Generative) Models Autoencoders Others (e.g. k-means) Tractable Models Non-Tractable Models Generative Fully observed Adversarial

815 views • 58 slides
Modals, conditionals, and probabilistic generative models Topic 1: intro to probability &amp;

Modals, conditionals, and probabilistic generative models Topic 1: intro to probability &amp;

Modals, conditionals, and probabilistic generative models Topic 1: intro to probability &amp; generative models; a bit on modality Dan Lassiter, Stanford Linguistics Universit de Paris VII, 25/11/19 4 lectures: The plan 1. probability,

694 views • 56 slides
A Tutorial on Deep Probabilistic Generative Models Ryan P. Adams Princeton University Machine

A Tutorial on Deep Probabilistic Generative Models Ryan P. Adams Princeton University Machine

A Tutorial on Deep Probabilistic Generative Models Ryan P. Adams Princeton University Machine Learning Summer School Buenos Aires, Argentina June 2018 lips.cs.princeton.edu @ryan_p_adams Tutorial Outline What is generative modeling?

2.33k views • 156 slides
Generative Models and Nave Bayes Ke Chen Reading: [14.3, EA], [3.5, KPM], [1.5.4, CMB]

Generative Models and Nave Bayes Ke Chen Reading: [14.3, EA], [3.5, KPM], [1.5.4, CMB]

Generative Models and Nave Bayes Ke Chen Reading: [14.3, EA], [3.5, KPM], [1.5.4, CMB] COMP24111 Machine Learning Outline Background and Probability Basics Probabilistic Classification Principle Probabilistic discriminative

330 views • 18 slides
CSE 571 A simple algorithm for learning robust Probabilistic Robotics classifiers Freund

CSE 571 A simple algorithm for learning robust Probabilistic Robotics classifiers Freund

11/13/2007 Why boosting? CSE 571 A simple algorithm for learning robust Probabilistic Robotics classifiers Freund &amp; Shapire, 1995 Friedman, Hastie, Tibshhirani, 1998 Boosting Provides efficient algorithm for sparse feature

739 views • 8 slides
Nonlinear Classifiers II 2 Nonlinear Classifiers: Introduction Classifiers Supervised

Nonlinear Classifiers II 2 Nonlinear Classifiers: Introduction Classifiers Supervised

1 Nonlinear Classifiers II 2 Nonlinear Classifiers: Introduction Classifiers Supervised Classifiers XOR problem Linear Classifiers Perceptron Least Squares Methods Linear Support Vector Machine Nonlinear

355 views • 20 slides
Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling

Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling

Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling Jiajun Wu* Chengkai Zhang* Tianfan Xue Bill Freeman Josh Tenenbaum NIPS 2016 (* indicates equal contribution) Outline Synthesizing 3D shapes

967 views • 65 slides
Outline CS 188: Artificial Intelligence Generative vs. Discriminative Binary Linear

Outline CS 188: Artificial Intelligence Generative vs. Discriminative Binary Linear

Outline CS 188: Artificial Intelligence Generative vs. Discriminative Binary Linear Classifiers Perceptron Lecture 21: Perceptrons Multi-class Linear Classifiers Multi-class Perceptron Fixing the Perceptron:

182 views • 7 slides
Making Generative Classifiers Robust to Selection Bias Andrew Smith Charles Elkan November

Making Generative Classifiers Robust to Selection Bias Andrew Smith Charles Elkan November

Making Generative Classifiers Robust to Selection Bias Andrew Smith Charles Elkan November 30th, 2007 Outline What is selection bias? Types of selection bias. Overcoming learnable bias with weighting. Overcoming bias with

736 views • 59 slides
Classification with generative models 2 DSE 210 Classification with parametrized models

Classification with generative models 2 DSE 210 Classification with parametrized models

Classification with generative models 2 DSE 210 Classification with parametrized models Classifiers with a fixed number of parameters can represent a limited set of functions. Learning a model is about picking a good approximation. Typically the

427 views • 39 slides
Probabilistic Foundations of Statistical Network Analysis Chapter 4: Generative models Harry

Probabilistic Foundations of Statistical Network Analysis Chapter 4: Generative models Harry

Probabilistic Foundations of Statistical Network Analysis Chapter 4: Generative models Harry Crane Based on Chapter 4 of Probabilistic Foundations of Statistical Network Analysis Book website: http://www.harrycrane.com/networks.html Harry Crane

289 views • 13 slides
Applications November 20, 2008 CS 486/686 University of Waterloo Outline Alchemy

Applications November 20, 2008 CS 486/686 University of Waterloo Outline Alchemy

Applications November 20, 2008 CS 486/686 University of Waterloo Outline Alchemy applications Readings: Marc Summer and Pedro Domingos (2007), The Alchemy Tutorial , Department of Computer Science and Engineering, University of

324 views • 13 slides
Nave Bayes CMSC 470 Marine Carpuat Slides credit: Dan Jurafsky &amp; James Martin, Jacob

Nave Bayes CMSC 470 Marine Carpuat Slides credit: Dan Jurafsky &amp; James Martin, Jacob

Classification, Linear Models, Nave Bayes CMSC 470 Marine Carpuat Slides credit: Dan Jurafsky &amp; James Martin, Jacob Eisenstein Today Text classification problems and their evaluation Linear classifiers Features &amp;

460 views • 30 slides
Multimodal Abstractive Summarization for How2 Videos ACL19 Shru* Palaskar Jindrich

Multimodal Abstractive Summarization for How2 Videos ACL19 Shru* Palaskar Jindrich

Multimodal Abstractive Summarization for How2 Videos ACL19 Shru* Palaskar Jindrich Libovick Spandana Gella Florian Metze School of Computer Science, Carnegie Mellon University Faculty of Mathema*cs and Physics, Charles University

280 views • 27 slides
Speaker and Emotion Recognition of TV-Series Data Using Multimodal and Multitask Deep Learning

Speaker and Emotion Recognition of TV-Series Data Using Multimodal and Multitask Deep Learning

Speaker and Emotion Recognition of TV-Series Data Using Multimodal and Multitask Deep Learning Sashi Novitasari 1 , Quoc Truong Do 1 , Sakriani Saktj 1,3 , Dessi Lestari 2 , Satoshi Nakamura 1,3 1 Graduate School of Informatjon Science, Nara

395 views • 16 slides
Sifting through images with Multinomial Relevance Feedback Dorota G lowacka, Alan Medlar and

Sifting through images with Multinomial Relevance Feedback Dorota G lowacka, Alan Medlar and

Sifting through images with Multinomial Relevance Feedback Dorota G lowacka, Alan Medlar and John Shawe-Taylor Univeristy College London December 8, 2010 Dorota G lowacka, Alan Medlar and John Shawe-Taylor Sifting through images with

659 views • 17 slides
Visualization DOs &amp; DONTs Duen Horng (Polo) Chau Georgia Tech Partly based on materials

Visualization DOs &amp; DONTs Duen Horng (Polo) Chau Georgia Tech Partly based on materials

CSE 6242 / CX 4242 Sept 11, 2014 Visualization DOs &amp; DONTs Duen Horng (Polo) Chau Georgia Tech Partly based on materials by Professors Guy Lebanon, Jeffrey Heer, John Stasko, Christos Faloutsos Survey Why do you

778 views • 63 slides
Extensible Interactive Graphics data models (static) graphics but is less strong in

Extensible Interactive Graphics data models (static) graphics but is less strong in

Extensible Interactive Graphics Interactive

54 views • 4 slides
Shi Shiny ny app app wi with h THE HE go golem em and d ht html ml wi widget dgets

Shi Shiny ny app app wi with h THE HE go golem em and d ht html ml wi widget dgets

#CorrelCon2020 Build uilding ing a modul ularized Shi Shiny ny app app wi with h THE HE go golem em and d ht html ml wi widget dgets Dr. Cdric Scherer Freelancing Data Visualization Designer Leibniz Institute for Zoo &amp;

1.19k views • 87 slides

More recommend