applied machine learning applied machine learning
play

Applied Machine Learning Applied Machine Learning Naive Bayes - PowerPoint PPT Presentation

Mar 16, 2024 •604 likes •1.49k views

Applied Machine Learning Applied Machine Learning Naive Bayes Siamak Ravanbakhsh Siamak Ravanbakhsh COMP 551 COMP 551 (winter 2020) (winter 2020) 1 Learning objectives Learning objectives generative vs. discriminative classifier Naive

  1. Applied Machine Learning Applied Machine Learning Naive Bayes Siamak Ravanbakhsh Siamak Ravanbakhsh COMP 551 COMP 551 (winter 2020) (winter 2020) 1

  2. Learning objectives Learning objectives generative vs. discriminative classifier Naive Bayes classifier assumption different design choices 2

  3. Discreminative vs generative classification Discreminative vs generative classification so far we modeled the conditional distribution : p ( y ∣ x ) discriminative 3 . 1 image: https://rpsychologist.com

  4. Discreminative vs generative classification Discreminative vs generative classification so far we modeled the conditional distribution : p ( y ∣ x ) discriminative learn the joint distribution p ( y , x ) = p ( y ) p ( x ∣ y ) generative 3 . 1 image: https://rpsychologist.com

  5. Discreminative vs generative classification Discreminative vs generative classification so far we modeled the conditional distribution : p ( y ∣ x ) discriminative learn the joint distribution p ( y , x ) = p ( y ) p ( x ∣ y ) generative Bayes rule p ( c ) p ( x ∣ c ) p ( y = c ∣ x ) = p ( x ) 3 . 1 image: https://rpsychologist.com

  6. Discreminative vs generative classification Discreminative vs generative classification so far we modeled the conditional distribution : p ( y ∣ x ) discriminative learn the joint distribution p ( y , x ) = p ( y ) p ( x ∣ y ) generative prior class probability: frequency of observing this label Bayes rule p ( c ) p ( x ∣ c ) p ( y = c ∣ x ) = p ( x ∣ y = 1) p ( x ) p ( x ∣ y = 0) x 3 . 1 image: https://rpsychologist.com

  7. Discreminative vs generative classification Discreminative vs generative classification so far we modeled the conditional distribution : p ( y ∣ x ) discriminative learn the joint distribution p ( y , x ) = p ( y ) p ( x ∣ y ) generative prior class probability: frequency of observing this label likelihood of input features given the class label (input features for each label come from a different distribution) Bayes rule p ( c ) p ( x ∣ c ) p ( y = c ∣ x ) = p ( x ∣ y = 1) p ( x ) p ( x ∣ y = 0) x 3 . 1 image: https://rpsychologist.com

  8. Discreminative vs generative classification Discreminative vs generative classification so far we modeled the conditional distribution : p ( y ∣ x ) discriminative learn the joint distribution p ( y , x ) = p ( y ) p ( x ∣ y ) generative prior class probability: frequency of observing this label likelihood of input features given the class label (input features for each label come from a different distribution) Bayes rule p ( c ) p ( x ∣ c ) p ( y = c ∣ x ) = p ( x ∣ y = 1) p ( x ) p ( x ∣ y = 0) marginal probability of the input (evidence) x C ′ ∑ c =1 p ( x , c ) ′ 3 . 1 image: https://rpsychologist.com

  9. Discreminative vs generative classification Discreminative vs generative classification so far we modeled the conditional distribution : p ( y ∣ x ) discriminative learn the joint distribution p ( y , x ) = p ( y ) p ( x ∣ y ) generative prior class probability: frequency of observing this label likelihood of input features given the class label (input features for each label come from a different distribution) Bayes rule p ( c ) p ( x ∣ c ) p ( y = c ∣ x ) = p ( x ∣ y = 1) p ( x ) p ( x ∣ y = 0) posterior probability marginal probability of the input (evidence) x of a given class C ′ ∑ c =1 p ( x , c ) ′ 3 . 1 image: https://rpsychologist.com

  10. Discreminative vs generative classification Discreminative vs generative classification so far we modeled the conditional distribution : p ( y ∣ x ) discriminative learn the joint distribution p ( y , x ) = p ( y ) p ( x ∣ y ) generative prior class probability: frequency of observing this label likelihood of input features given the class label (input features for each label come from a different distribution) how to classify new input x? Bayes rule p ( c ) p ( x ∣ c ) p ( y = c ∣ x ) = p ( x ∣ y = 1) p ( x ) p ( x ∣ y = 0) posterior probability marginal probability of the input (evidence) x of a given class C ′ ∑ c =1 p ( x , c ) ′ 3 . 1 image: https://rpsychologist.com

  11. Example: Example: Bayes rule for classification Bayes rule for classification patient having cancer? y ∈ {yes, no} x ∈ {−, +} test results, a single binary feature p ( c ) p ( x ∣ c ) p ( c ∣ x ) = p ( x ) 3 . 2

  12. Example: Example: Bayes rule for classification Bayes rule for classification patient having cancer? y ∈ {yes, no} x ∈ {−, +} test results, a single binary feature prior: 1% of population has cancer p (yes) = .01 p ( c ) p ( x ∣ c ) p ( c ∣ x ) = p ( x ) 3 . 2

  13. Example: Bayes rule for classification Example: Bayes rule for classification patient having cancer? y ∈ {yes, no} x ∈ {−, +} test results, a single binary feature prior: 1% of population has cancer p (yes) = .01 p (+∣yes) = .9 likelihood: TP rate of the test (90%) p ( c ) p ( x ∣ c ) p ( c ∣ x ) = p ( x ) 3 . 2

  14. Example: Example: Bayes rule for classification Bayes rule for classification patient having cancer? y ∈ {yes, no} x ∈ {−, +} test results, a single binary feature prior: 1% of population has cancer p (yes) = .01 p (+∣yes) = .9 likelihood: TP rate of the test (90%) p ( c ) p ( x ∣ c ) p ( c ∣ x ) = p ( x ) FP rate of the test (5%) p (+) = p (yes) p (+∣yes) + p (no) p (+∣no) = .01 × .9 + .99 × .05 = .189 evidence: 3 . 2

  15. Example: Bayes rule for classification Example: Bayes rule for classification patient having cancer? y ∈ {yes, no} x ∈ {−, +} test results, a single binary feature prior: 1% of population has cancer p (yes) = .01 p (+∣yes) = .9 likelihood: TP rate of the test (90%) p ( c ) p ( x ∣ c ) p ( c ∣ x ) = p ( x ) FP rate of the test (5%) posterior: p (yes∣+) = .08 p (+) = p (yes) p (+∣yes) + p (no) p (+∣no) = .01 × .9 + .99 × .05 = .189 evidence: 3 . 2

  16. Example: Example: Bayes rule for classification Bayes rule for classification patient having cancer? y ∈ {yes, no} x ∈ {−, +} test results, a single binary feature prior: 1% of population has cancer p (yes) = .01 p (+∣yes) = .9 likelihood: TP rate of the test (90%) p ( c ) p ( x ∣ c ) p ( c ∣ x ) = p ( x ) FP rate of the test (5%) posterior: p (yes∣+) = .08 p (+) = p (yes) p (+∣yes) + p (no) p (+∣no) = .01 × .9 + .99 × .05 = .189 evidence: in a generative classifier likelihood & prior class probabilities are learned from data 3 . 2 Winter 2020 | Applied Machine Learning (COMP551)

  17. Generative classification Generative classification prior class probability: frequency of observing this label likelihood of input features given the class label (input features for each label come from a different distribution) p ( c ) p ( x ∣ c ) p ( y = c ∣ x ) = p ( x ) posterior probability marginal probability of the input (evidence) of a given class C ′ p ( x , c ) ∑ c =1 ′ Some generative classifiers: Gaussian Discriminant Analysis: the likelihood is multivariate Gaussian Naive Bayes: decomposed likelihood 4 . 1 image: https://rpsychologist.com

  18. Naive Bayes: Naive Bayes: model model number of input features D assumption about the likelihood p ( x ∣ y ) = p ( x ∣ y ) ∏ d =1 d 4 . 2

  19. Naive Bayes: Naive Bayes: model model number of input features D assumption about the likelihood p ( x ∣ y ) = p ( x ∣ y ) ∏ d =1 d when is this assumption correct? ⊥ ∣ when features are conditionally independent given the label x x y i j knowing the label, the value of one input feature gives us no information about the other input features 4 . 2

  20. Naive Bayes: Naive Bayes: model model number of input features D assumption about the likelihood p ( x ∣ y ) = p ( x ∣ y ) ∏ d =1 d when is this assumption correct? ⊥ ∣ when features are conditionally independent given the label x x y i j knowing the label, the value of one input feature gives us no information about the other input features chain rule of probability (true for any distribution) p ( x ∣ y ) = p ( x ∣ y ) p ( x ∣ y , x ) p ( x ∣ y , x , x ) … p ( x ∣ y , x , … , x ) 1 2 1 3 1 2 1 D −1 D 4 . 2

  21. Naive Bayes: Naive Bayes: model model number of input features D assumption about the likelihood p ( x ∣ y ) = p ( x ∣ y ) ∏ d =1 d when is this assumption correct? ⊥ ∣ when features are conditionally independent given the label x x y i j knowing the label, the value of one input feature gives us no information about the other input features chain rule of probability (true for any distribution) p ( x ∣ y ) = p ( x ∣ y ) p ( x ∣ y , x ) p ( x ∣ y , x , x ) … p ( x ∣ y , x , … , x ) 1 2 1 3 1 2 1 D −1 D conditional independence assumption x1, x2 give no extra information, so p ( x ∣ y , x , x ) = p ( x ∣ y ) 3 1 2 3 4 . 2

  22. Naive Bayes: objective Naive Bayes: objective given the training dataset (1) (1) ( N ) ( N ) D = {( x , y ), … , ( x , y )} maximize the joint likelihood (contrast with logistic regression) ( n ) ( n ) ℓ( w , u ) = log p ( x , y ) ∑ n u , w 4 . 3

  23. Naive Bayes: objective Naive Bayes: objective given the training dataset (1) (1) ( N ) ( N ) D = {( x , y ), … , ( x , y )} maximize the joint likelihood (contrast with logistic regression) ( n ) ( n ) ℓ( w , u ) = log p ( x , y ) ∑ n u , w ( n ) ( n ) ( n ) = log p ( y ) + log p ( x ∣ y ) ∑ n u w ( n ) ( n ) ( n ) = log p ( y ) + log p ( x ∣ y ) ∑ n ∑ n u w 4 . 3

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

MACHINE LEARNING Overview 1 1 APPLIED MACHINE LEARNING 2011-2012 APPLIED MACHINE LEARNING

MACHINE LEARNING Overview 1 1 APPLIED MACHINE LEARNING 2011-2012 APPLIED MACHINE LEARNING

APPLIED MACHINE LEARNING 2011-2012 APPLIED MACHINE LEARNING MACHINE LEARNING Overview 1 1 APPLIED MACHINE LEARNING 2011-2012 APPLIED MACHINE LEARNING Exam Format The exam lasts a total of 3 hours: - Upon entering the room, you must

373 views • 21 slides
Applied Machine Learning Introduction 1 APPLIED MACHINE LEARNING Practicalities Contact

Applied Machine Learning Introduction 1 APPLIED MACHINE LEARNING Practicalities Contact

APPLIED MACHINE LEARNING Applied Machine Learning Introduction 1 APPLIED MACHINE LEARNING Practicalities Contact Information of the Instructors APPLIED MACHINE LEARNING Practicalities Slides and exercises will be posted on the website of

1.46k views • 61 slides
APPLIED MACHINE LEARNING Methods for Clustering K-means, Soft K-means DBSCAN 1 MACHINE

APPLIED MACHINE LEARNING Methods for Clustering K-means, Soft K-means DBSCAN 1 MACHINE

MACHINE LEARNING - MSc Course APPLIED MACHINE LEARNING APPLIED MACHINE LEARNING Methods for Clustering K-means, Soft K-means DBSCAN 1 MACHINE LEARNING - MSc Course APPLIED MACHINE LEARNING Objectives Learn basic techniques for data

1.1k views • 74 slides
Introductory Applied Machine Learning The primary aim of the course is to provide the student with

Introductory Applied Machine Learning The primary aim of the course is to provide the student with

Introductory Applied Machine Learning The primary aim of the course is to provide the student with a set of practical tools that can be applied to solve real-world problems in machine learning. Nigel Goddard School of Informatics Machine

716 views • 8 slides
Applied Machine Learning Introduction 1 APPLIED MACHINE LEARNING Practicalities Slides and

Applied Machine Learning Introduction 1 APPLIED MACHINE LEARNING Practicalities Slides and

APPLIED MACHINE LEARNING Applied Machine Learning Introduction 1 APPLIED MACHINE LEARNING Practicalities Slides and exercises will be posted on the website of the class the day before class:

628 views • 37 slides
MACHINE LEARNING Linear and Weighted Regression Support Vector Regression 1 APPLIED MACHINE

MACHINE LEARNING Linear and Weighted Regression Support Vector Regression 1 APPLIED MACHINE

APPLIED MACHINE LEARNING MACHINE LEARNING Linear and Weighted Regression Support Vector Regression 1 APPLIED MACHINE LEARNING Classification (reminder) Maps N -dimensions input x N to discrete values y E.g.: x = [ Length , Color ]

931 views • 68 slides
How Developers Iterate on Machine Learning Workflows -- A Survey of the Applied Machine Learning

How Developers Iterate on Machine Learning Workflows -- A Survey of the Applied Machine Learning

How Developers Iterate on Machine Learning Workflows -- A Survey of the Applied Machine Learning Literature Doris Xin 1 , Litian Ma 1 , Shuchen Song 1 , Rong Ma 2 , Aditya Parameswaran 1 1 University of Illinois at Urbana-Champaign 2 Peking

337 views • 20 slides
APPLIED MACHINE LEARNING Methods for Reduction of Dimensionality through Linear Projection

APPLIED MACHINE LEARNING Methods for Reduction of Dimensionality through Linear Projection

APPLIED MACHINE LEARNING APPLIED MACHINE LEARNING Methods for Reduction of Dimensionality through Linear Projection Principal Component Analysis (PCA) 1 APPLIED MACHINE LEARNING Curse of Dimensionality Computational Costs O(N 2 ) O(N) N:

848 views • 69 slides
APPLIED MACHINE LEARNING Probability Density Functions Gaussian Mixture Models 1 APPLIED

APPLIED MACHINE LEARNING Probability Density Functions Gaussian Mixture Models 1 APPLIED

APPLIED MACHINE LEARNING APPLIED MACHINE LEARNING Probability Density Functions Gaussian Mixture Models 1 APPLIED MACHINE LEARNING Discrete Probabilities Consider two variables x and y taking discrete values over the intervals [1.....N ] and

688 views • 54 slides
Responsible Machine Learning INFO-4604, Applied Machine Learning University of Colorado Boulder

Responsible Machine Learning INFO-4604, Applied Machine Learning University of Colorado Boulder

Responsible Machine Learning INFO-4604, Applied Machine Learning University of Colorado Boulder November 13, 2018 Prof. Michael Paul Is Machine Learning Dangerous? Is Machine Learning Dangerous? Doomsday scenarios not likely any

554 views • 26 slides
Machine Learning Prof. Kuan-Ting Lai 2020/4/11 Applied Math for Machine Learning Linear

Machine Learning Prof. Kuan-Ting Lai 2020/4/11 Applied Math for Machine Learning Linear

Applied Math for Machine Learning Prof. Kuan-Ting Lai 2020/4/11 Applied Math for Machine Learning Linear Algebra Probability Calculus Optimization Linear Algebra Scalar real numbers Vector (1D) Has a magnitude

736 views • 54 slides
Applied Machine Learning Applied Machine Learning Regularization Siamak Ravanbakhsh Siamak

Applied Machine Learning Applied Machine Learning Regularization Siamak Ravanbakhsh Siamak

Applied Machine Learning Applied Machine Learning Regularization Siamak Ravanbakhsh Siamak Ravanbakhsh COMP 551 COMP 551 (winter 2020) (winter 2020) 1 Learning objectives Learning objectives Basic idea of overfitting and underfitting

404 views • 39 slides
Applied Machine Learning Applied Machine Learning Regularization Siamak Ravanbakhsh Siamak

Applied Machine Learning Applied Machine Learning Regularization Siamak Ravanbakhsh Siamak

Applied Machine Learning Applied Machine Learning Regularization Siamak Ravanbakhsh Siamak Ravanbakhsh COMP 551 COMP 551 (winter 2020) (winter 2020) 1 Learning objectives Learning objectives Basic idea of overfitting and underfitting

1.48k views • 135 slides
Applied Machine Learning Applied Machine Learning Convolutional Neural Networks Siamak

Applied Machine Learning Applied Machine Learning Convolutional Neural Networks Siamak

Applied Machine Learning Applied Machine Learning Convolutional Neural Networks Siamak Ravanbakhsh Siamak Ravanbakhsh COMP 551 COMP 551 (winter 2020) (winter 2020) 1 Learning objectives Learning objectives understand the convolution layer

1.34k views • 103 slides
Applied Machine Learning Applied Machine Learning Convolutional Neural Networks Siamak

Applied Machine Learning Applied Machine Learning Convolutional Neural Networks Siamak

Applied Machine Learning Applied Machine Learning Convolutional Neural Networks Siamak Ravanbakhsh Siamak Ravanbakhsh COMP 551 COMP 551 (winter 2020) (winter 2020) 1 Learning objectives Learning objectives understand the convolution layer

609 views • 25 slides
Applied Machine Learning Applied Machine Learning Linear Regression Siamak Ravanbakhsh Siamak

Applied Machine Learning Applied Machine Learning Linear Regression Siamak Ravanbakhsh Siamak

Applied Machine Learning Applied Machine Learning Linear Regression Siamak Ravanbakhsh Siamak Ravanbakhsh COMP 551 COMP 551 (winter 2020) (winter 2020) 1 Learning objectives Learning objectives linear model evaluation criteria how to find

1.22k views • 94 slides
T-61.3050 Machine Learning: Basic Principles Bayesian Networks Kai Puolam aki Laboratory of

T-61.3050 Machine Learning: Basic Principles Bayesian Networks Kai Puolam aki Laboratory of

Bayesian Networks Probabilistic Inference Estimating Parameters T-61.3050 Machine Learning: Basic Principles Bayesian Networks Kai Puolam aki Laboratory of Computer and Information Science (CIS) Department of Computer Science and

765 views • 60 slides
Bayesian networks Independence Bayesian networks Markov conditions Inference by

Bayesian networks Independence Bayesian networks Markov conditions Inference by

Bayesian networks Independence Bayesian networks Markov conditions Inference by enumeration rejection sampling Gibbs sampler Independence if P(A=a,B=a) = P(A=a)P(B=b) for all a and b, then we call A and B

733 views • 19 slides
CS 730/730W/830: Intro AI Bayesian Networks Approx. Inference Exact Inference 1 handout: slides

CS 730/730W/830: Intro AI Bayesian Networks Approx. Inference Exact Inference 1 handout: slides

CS 730/730W/830: Intro AI Bayesian Networks Approx. Inference Exact Inference 1 handout: slides final blog entries were due Wheeler Ruml (UNH) Lecture 27, CS 730 1 / 15 Bayesian Networks Example Reminder Approx. Inference Exact

333 views • 16 slides
Bayes Nets (Ch. 14) Announcements Homework 1 posted Bayesian Network A Bayesian network (Bayes

Bayes Nets (Ch. 14) Announcements Homework 1 posted Bayesian Network A Bayesian network (Bayes

Bayes Nets (Ch. 14) Announcements Homework 1 posted Bayesian Network A Bayesian network (Bayes net) is: (1) a directed graph (2) acyclic Additionally, Bayesian networks are assumed to be defined by conditional probability tables (3) P(x |

463 views • 32 slides
Adversarial Approaches to Bayesian Learning and Bayesian Approaches to Adversarial Robustness

Adversarial Approaches to Bayesian Learning and Bayesian Approaches to Adversarial Robustness

Adversarial Approaches to Bayesian Learning and Bayesian Approaches to Adversarial Robustness Ian Goodfellow, OpenAI Research Scientist NIPS 2016 Workshop on Bayesian Deep Learning Barcelona, 2016-12-10 Speculation on Three Topics Can we

769 views • 21 slides
Bayesian Deep Learning and Restricted Boltzmann Machines Narada Warakagoda Forsvarets

Bayesian Deep Learning and Restricted Boltzmann Machines Narada Warakagoda Forsvarets

Bayesian Deep Learning and Restricted Boltzmann Machines Narada Warakagoda Forsvarets Forskningsinstitutt ndw@ffi.no November 1, 2018 Narada Warakagoda (FFI) Short title November 1, 2018 1 / 56 Overview Probability Review 1 Bayesian

899 views • 56 slides
The LESO-PB building building control system 0.0015 Density estimate 0.0010 0.0005 0.0000 0

The LESO-PB building building control system 0.0015 Density estimate 0.0010 0.0005 0.0000 0

Integrating R in an advanced The LESO-PB building building control system 0.0015 Density estimate 0.0010 0.0005 0.0000 0 500 1000 1500 2000 2500 3000 3500 [lux] David Lindelf David Lindelf useR! 2006 useR! 2006 15/06/2006

449 views • 3 slides
Statistics Review of Probability Model Shiu-Sheng Chen Department of Economics National Taiwan

Statistics Review of Probability Model Shiu-Sheng Chen Department of Economics National Taiwan

Statistics Review of Probability Model Shiu-Sheng Chen Department of Economics National Taiwan University Fall 2019 Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 1 / 22 Probability Theory Section 1 Probability Theory Shiu-Sheng Chen

680 views • 22 slides

More recommend