active learning by the naive credal classifier
play

Active Learning by the Naive Credal Classifier Alessandro Antonucci - PowerPoint PPT Presentation

Dec 29, 2022 •325 likes •743 views

Active Learning by the Naive Credal Classifier Alessandro Antonucci , Giorgio Corani , Sandra Gabaglio Istituto Dalle Molle di Studi sullIntelligenza Artificiale - Lugano (Switzerland) ISIN/SUPSI - Lugano (Switzerland)

  1. Active Learning by the Naive Credal Classifier Alessandro Antonucci ∗ , Giorgio Corani ∗ , Sandra Gabaglio † ∗ Istituto “Dalle Molle” di Studi sull’Intelligenza Artificiale - Lugano (Switzerland) † ISIN/SUPSI - Lugano (Switzerland) PGM’12 - Granada, September 20, 2012

  2. Active Learning Class C (values in C ) attributes A := ( A 1 , . . . , A k ) Training Dataset Test Set/Instance (supervised) (unsupervised) c ( 1 ) , a ( 1 ) 1 , . . . , a ( 1 ) ∗ , ˜ ∗ , ˜ a 1 , . . . , ˜ a 1 , . . . , ˜ a k a k k c ( 2 ) , a ( 2 ) 1 , . . . , a ( 2 ) k . . . c ( n ) , a ( n ) 1 , . . . , a ( n ) k

  3. Active Learning Class C (values in C ) attributes A := ( A 1 , . . . , A k ) Training Dataset Test Set/Instance (supervised) (unsupervised) c ( 1 ) , a ( 1 ) 1 , . . . , a ( 1 ) ∗ , ˜ ∗ , ˜ a 1 , . . . , ˜ a 1 , . . . , ˜ a k a k k c ( 2 ) , a ( 2 ) 1 , . . . , a ( 2 ) k Classifier . . . c ( n ) , a ( n ) 1 , . . . , a ( n ) k

  4. Active Learning Class C (values in C ) attributes A := ( A 1 , . . . , A k ) Training Dataset Active Set Test Set/Instance (unsupervised) (supervised) (unsupervised) c ( 1 ) , a ( 1 ) 1 , . . . , a ( 1 ) ∗ , ˜ ∗ , ˜ a 1 , . . . , ˜ a 1 , . . . , ˜ a k a k k ∗ , a ( a ) 1 , . . . , a ( a ) c ( 2 ) , a ( 2 ) 1 , . . . , a ( 2 ) k k Classifier ∗ , a ( b ) 1 , . . . , a ( b ) . . . k ∗ , a ( c ) 1 , . . . , a ( c ) c ( n ) , a ( n ) 1 , . . . , a ( n ) k k

  5. Active Learning Class C (values in C ) attributes A := ( A 1 , . . . , A k ) Training Dataset Active Set Test Set/Instance (unsupervised) (supervised) (unsupervised) c ( 1 ) , a ( 1 ) 1 , . . . , a ( 1 ) ∗ , ˜ ∗ , ˜ a 1 , . . . , ˜ a 1 , . . . , ˜ a k a k k ∗ , a ( a ) 1 , . . . , a ( a ) c ( 2 ) , a ( 2 ) 1 , . . . , a ( 2 ) .3 k k Classifier ∗ , a ( b ) 1 , . . . , a ( b ) .8 . . . k ∗ , a ( c ) 1 , . . . , a ( c ) c ( n ) , a ( n ) 1 , . . . , a ( n ) .5 k k Active Learning Score

  6. Active Learning Class C (values in C ) attributes A := ( A 1 , . . . , A k ) Training Dataset Active Set Test Set/Instance (unsupervised) (supervised) (unsupervised) c ( 1 ) , a ( 1 ) 1 , . . . , a ( 1 ) ∗ , ˜ ∗ , ˜ a 1 , . . . , ˜ a 1 , . . . , ˜ a k a k k ∗ , a ( a ) 1 , . . . , a ( a ) c ( 2 ) , a ( 2 ) 1 , . . . , a ( 2 ) .3 k k Classifier . . . ∗ , a ( c ) 1 , . . . , a ( c ) c ( n ) , a ( n ) 1 , . . . , a ( n ) .5 k k ∗ , a ( b ) 1 , . . . , a ( b ) .8 Active k Learning Score

  7. Active Learning Class C (values in C ) attributes A := ( A 1 , . . . , A k ) Training Dataset Active Set Test Set/Instance (unsupervised) (supervised) (unsupervised) c ( 1 ) , a ( 1 ) 1 , . . . , a ( 1 ) ∗ , ˜ ∗ , ˜ a 1 , . . . , ˜ a 1 , . . . , ˜ a k a k k ∗ , a ( a ) 1 , . . . , a ( a ) c ( 2 ) , a ( 2 ) 1 , . . . , a ( 2 ) .3 k k Classifier . . . ∗ , a ( c ) 1 , . . . , a ( c ) c ( n ) , a ( n ) 1 , . . . , a ( n ) .5 k k c b , a ( b ) 1 , . . . , a ( b ) Active k Learning Annotation Score

  8. Active Learning Class C (values in C ) attributes A := ( A 1 , . . . , A k ) Training Dataset Active Set Test Set/Instance (unsupervised) (supervised) (unsupervised) c ( 1 ) , a ( 1 ) 1 , . . . , a ( 1 ) ∗ , ˜ ∗ , ˜ a 1 , . . . , ˜ a 1 , . . . , ˜ a k a k k ∗ , a ( a ) 1 , . . . , a ( a ) c ( 2 ) , a ( 2 ) 1 , . . . , a ( 2 ) .3 k k Classifier . . . Actively Learned Classifier ∗ , a ( c ) 1 , . . . , a ( c ) c ( n ) , a ( n ) 1 , . . . , a ( n ) .5 k k (more accurate) c b , a ( b ) 1 , . . . , a ( b ) Active k Learning Annotation Score

  9. Accuracy Trajectories constant AL score: random pick among active set instances (variance error decreases, accuracy increases)

  10. Accuracy Trajectories constant AL score: random pick among active set instances (variance error decreases, accuracy increases) Accuracy (on test set) Training Set N N+d N+2d N+kd . . . Size Active Set Active Set FULL EMPTY

  11. Accuracy Trajectories constant AL score: random pick among active set instances (variance error decreases, accuracy increases) Accuracy (on test set) Random Pick Training Set N N+d N+2d N+kd . . . Size Active Set Active Set FULL EMPTY

  12. Accuracy Trajectories constant AL score: random pick among active set instances (variance error decreases, accuracy increases) Accuracy AL algorithm (on test set) Random Pick AL algs should do better! Training Set N N+d N+2d N+kd . . . Size Active Set Active Set FULL EMPTY

  13. Naive Classifiers C A k A 1 A 2 . . . Given class C , attributes independent NAIVE BAYES (NBC) NAIVE CREDAL (NCC) A BN quantified from data by BNs quantified by set of priors a flat Dirichlet prior Dir ( s t ) Imprecise Dirichlet model � � P ( c , a ) = P ( c ) · � m Dir ( s t ) : t > 0 , � i t i = 1 T ≡ i = 1 P ( a i | c ) A set C ∗ ⊆ C of optimal Given test instance a , assigns class c ∗ := arg max c ∈C P ( c | f ) (undominated) classes ∀ c ′ , c ′′ ∈ C , dominance test Conservative dominance test P t ( c ′ , a ) P ( c ′′ | a ) = P ( c ′ , a ) P ( c ′ | a ) min t ∈T P t ( c ′′ , a ) > 1 P ( c ′′ , a ) > 1

  14. Naive Classifiers C A k A 1 A 2 . . . Given class C , attributes independent NAIVE BAYES (NBC) NAIVE CREDAL (NCC) A BN quantified from data by BNs quantified by set of priors a flat Dirichlet prior Dir ( s t ) Imprecise Dirichlet model � � P ( c , a ) = P ( c ) · � m Dir ( s t ) : t > 0 , � i t i = 1 T ≡ i = 1 P ( a i | c ) A set C ∗ ⊆ C of optimal Given test instance a , assigns class c ∗ := arg max c ∈C P ( c | f ) (undominated) classes ∀ c ′ , c ′′ ∈ C , dominance test Conservative dominance test P t ( c ′ , a ) P ( c ′′ | a ) = P ( c ′ , a ) P ( c ′ | a ) min t ∈T P t ( c ′′ , a ) > 1 P ( c ′′ , a ) > 1

  15. Naive Classifiers C A k A 1 A 2 . . . Given class C , attributes independent NAIVE BAYES (NBC) NAIVE CREDAL (NCC) A BN quantified from data by BNs quantified by set of priors a flat Dirichlet prior Dir ( s t ) Imprecise Dirichlet model � � P ( c , a ) = P ( c ) · � m Dir ( s t ) : t > 0 , � i t i = 1 T ≡ i = 1 P ( a i | c ) A set C ∗ ⊆ C of optimal Given test instance a , assigns class c ∗ := arg max c ∈C P ( c | f ) (undominated) classes ∀ c ′ , c ′′ ∈ C , dominance test Conservative dominance test P t ( c ′ , a ) P ( c ′′ | a ) = P ( c ′ , a ) P ( c ′ | a ) min t ∈T P t ( c ′′ , a ) > 1 P ( c ′′ , a ) > 1

  16. Naive Classifiers C A k A 1 A 2 . . . Given class C , attributes independent NAIVE BAYES (NBC) NAIVE CREDAL (NCC) A BN quantified from data by BNs quantified by set of priors a flat Dirichlet prior Dir ( s t ) Imprecise Dirichlet model � � P ( c , a ) = P ( c ) · � m Dir ( s t ) : t > 0 , � i t i = 1 T ≡ i = 1 P ( a i | c ) A set C ∗ ⊆ C of optimal Given test instance a , assigns class c ∗ := arg max c ∈C P ( c | f ) (undominated) classes ∀ c ′ , c ′′ ∈ C , dominance test Conservative dominance test P t ( c ′ , a ) P ( c ′′ | a ) = P ( c ′ , a ) P ( c ′ | a ) min t ∈T P t ( c ′′ , a ) > 1 P ( c ′′ , a ) > 1

  17. Uncertainty Samplings AL score ( a ) shows how hard-to-classify an instance is difficult/ambiguous instances give better contribution to learning Uncertainty Sampling Credal Uncertainty Sampling Based on NBC Set of NCC posteriors P ( C | a ) posterior P ( C | a ) The weaker the dominances, the The smaller the more hard-to-classify instances probability of the most If C = { c ′ , c ′′ } (binary class): probable class, the score ( a ) ≡ more hard-to-classify � P ( c ′ | a ) P ( c ′′ | a ) � − max min t P ( c ′′ | a ) , min t is the instance P ( c ′ | a ) More than two classes? score ( a ) ≡ − P ( c ∗ | a ) Max over all pairs ( c ′ , c ′′ ) ∈ C 2

  18. Uncertainty Samplings AL score ( a ) shows how hard-to-classify an instance is difficult/ambiguous instances give better contribution to learning Uncertainty Sampling Credal Uncertainty Sampling Based on NBC Set of NCC posteriors P ( C | a ) posterior P ( C | a ) The weaker the dominances, the The smaller the more hard-to-classify instances probability of the most If C = { c ′ , c ′′ } (binary class): probable class, the score ( a ) ≡ more hard-to-classify � P ( c ′ | a ) P ( c ′′ | a ) � − max min t P ( c ′′ | a ) , min t is the instance P ( c ′ | a ) More than two classes? score ( a ) ≡ − P ( c ∗ | a ) Max over all pairs ( c ′ , c ′′ ) ∈ C 2

  19. Uncertainty Samplings AL score ( a ) shows how hard-to-classify an instance is difficult/ambiguous instances give better contribution to learning Uncertainty Sampling Credal Uncertainty Sampling Based on NBC Set of NCC posteriors P ( C | a ) posterior P ( C | a ) The weaker the dominances, the The smaller the more hard-to-classify instances probability of the most If C = { c ′ , c ′′ } (binary class): probable class, the score ( a ) ≡ more hard-to-classify � P ( c ′ | a ) P ( c ′′ | a ) � − max min t P ( c ′′ | a ) , min t is the instance P ( c ′ | a ) More than two classes? score ( a ) ≡ − P ( c ∗ | a ) Max over all pairs ( c ′ , c ′′ ) ∈ C 2

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Outline Naive Credal Classifier 2: an extension of Naive Bayes Introducing NCC2 1 for

Outline Naive Credal Classifier 2: an extension of Naive Bayes Introducing NCC2 1 for

Introducing NCC2 Experimental Results Conclusions Introducing NCC2 Experimental Results Conclusions Outline Naive Credal Classifier 2: an extension of Naive Bayes Introducing NCC2 1 for delivering robust classifications Background Credal

377 views • 5 slides
The Multilabel Naive Credal Classifier Alessandro Antonucci and Giorgio Corani {

The Multilabel Naive Credal Classifier Alessandro Antonucci and Giorgio Corani {

The Multilabel Naive Credal Classifier Alessandro Antonucci and Giorgio Corani { alessandro,giorgio } @idsia.ch Istituto Dalle Molle di Studi sullIntelligenza Artificiale - Lugano (Switzerland) http://ipg.idsia.ch ISIPTA 15, Pescara,

785 views • 27 slides
3: Statistical Properties of Language Machine Learning and Real-world Data (MLRD) Paula Buttery

3: Statistical Properties of Language Machine Learning and Real-world Data (MLRD) Paula Buttery

3: Statistical Properties of Language Machine Learning and Real-world Data (MLRD) Paula Buttery (based on slides created by Simone Teufel) Lent 2019 Last session: We implemented a naive Bayes classifier We built a naive Bayes classifier. The

496 views • 26 slides
STAT 339 Naive Bayes Classification 8-10 March 2017 Colin Reimer Dawson Outline Naive Bayes

STAT 339 Naive Bayes Classification 8-10 March 2017 Colin Reimer Dawson Outline Naive Bayes

STAT 339 Naive Bayes Classification 8-10 March 2017 Colin Reimer Dawson Outline Naive Bayes Classification Naive Bayes Classifier Using MLE Parameter Estimates Bayesian Naive Bayes Continuous Features Mixed Feature Types Back to Supervised

530 views • 21 slides
Naive Bayes and Gaussian Bayes Classifier Ladislav Rampasek slides by Mengye Ren and others

Naive Bayes and Gaussian Bayes Classifier Ladislav Rampasek slides by Mengye Ren and others

Naive Bayes and Gaussian Bayes Classifier Ladislav Rampasek slides by Mengye Ren and others February 22, 2016 Naive Bayes and Gaussian Bayes Classifier February 22, 2016 1 / 21 Naive Bayes Bayes Rule: p ( t | x ) = p ( x | t ) p ( t ) p (

365 views • 21 slides
Applied Machine Learning Applied Machine Learning Naive Bayes Siamak Ravanbakhsh Siamak

Applied Machine Learning Applied Machine Learning Naive Bayes Siamak Ravanbakhsh Siamak

Applied Machine Learning Applied Machine Learning Naive Bayes Siamak Ravanbakhsh Siamak Ravanbakhsh COMP 551 COMP 551 (winter 2020) (winter 2020) 1 Learning objectives Learning objectives generative vs. discriminative classifier Naive

1.49k views • 87 slides
Applied Machine Learning Applied Machine Learning Naive Bayes Siamak Ravanbakhsh Siamak

Applied Machine Learning Applied Machine Learning Naive Bayes Siamak Ravanbakhsh Siamak

Applied Machine Learning Applied Machine Learning Naive Bayes Siamak Ravanbakhsh Siamak Ravanbakhsh COMP 551 COMP 551 (winter 2020) (winter 2020) 1 Learning objectives Learning objectives generative vs. discriminative classifier Naive

446 views • 27 slides
Spam Filtering with Naive Bayes Classifier Yuriy Arabskyy June 6, 2017 Table of contents What

Spam Filtering with Naive Bayes Classifier Yuriy Arabskyy June 6, 2017 Table of contents What

Spam Filtering with Naive Bayes Classifier Yuriy Arabskyy June 6, 2017 Table of contents What is spam? Different spam types Anti-Spam Techniques Probability theory basics Conditional probability Bayes Theorem Naive Bayes Theorem Spam

1.04k views • 27 slides
Introduction to Machine Learning Classification: Naive Bayes Learning goals 15 Understand the

Introduction to Machine Learning Classification: Naive Bayes Learning goals 15 Understand the

Introduction to Machine Learning Classification: Naive Bayes Learning goals 15 Understand the idea of Naive 10 Response Bayes x2 a b 5 Understand in which sense 0 Naive Bayes is a special QDA 0 5 10 15 x1 model NAIVE BAYES

317 views • 6 slides
SI425 : NLP Set 6 Logistic Regression Fall 2020 : Chambers Last time Naive Bayes Classifier

SI425 : NLP Set 6 Logistic Regression Fall 2020 : Chambers Last time Naive Bayes Classifier

SI425 : NLP Set 6 Logistic Regression Fall 2020 : Chambers Last time Naive Bayes Classifier Given X, what is the most probable Y? Y arg max P ( Y y ) P ( X | Y y ) = = new k i k y k i Problems with

571 views • 17 slides
Agenda Intro to Active Learning Activity Design Resources for Active Learning Lunch with Active

Agenda Intro to Active Learning Activity Design Resources for Active Learning Lunch with Active

Agenda Intro to Active Learning Activity Design Resources for Active Learning Lunch with Active Learning Veterans Wrap Up Introduction to Active Learning What is active learning? Brainstorm your ideas with your group and generate a

855 views • 42 slides
Multi-Task Active Learning Yi Zhang Outline Active Learning Multi-Task Active Learning

Multi-Task Active Learning Yi Zhang Outline Active Learning Multi-Task Active Learning

Multi-Task Active Learning Yi Zhang Outline Active Learning Multi-Task Active Learning Linguistic Annotations (ACL 08) Image Classification (CVPR 08) Current Work and Discussions Constraint-Driven Active Learning

945 views • 47 slides
CSE 473: Artificial Intelligence Autumn 2010 Machine Learning: Naive Bayes and Perceptron Luke

CSE 473: Artificial Intelligence Autumn 2010 Machine Learning: Naive Bayes and Perceptron Luke

CSE 473: Artificial Intelligence Autumn 2010 Machine Learning: Naive Bayes and Perceptron Luke Zettlemoyer Many slides over the course adapted from Dan Klein. 1 Outline Learning: Naive Bayes and Perceptron Naive Bayes models

837 views • 57 slides
Lecture 8: Maximum a Posteriori (MAP) Nave Bayes Classifier Applications Aykut Erdem

Lecture 8: Maximum a Posteriori (MAP) Nave Bayes Classifier Applications Aykut Erdem

Lecture 8: Maximum a Posteriori (MAP) Nave Bayes Classifier Applications Aykut Erdem November 2018 Hacettepe University Assignment 2 is out! It is due November 24 (i.e. in 2 weeks) Implement Naive Bayes classifier for fake

788 views • 65 slides
ACTIVE 1. WELCOME and JUMPING IN LEARNING 1. WHAT IS ACTIVE LEARNING? la Carte 1.

ACTIVE 1. WELCOME and JUMPING IN LEARNING 1. WHAT IS ACTIVE LEARNING? la Carte 1.

ACTIVE 1. WELCOME and JUMPING IN LEARNING 1. WHAT IS ACTIVE LEARNING? la Carte 1. CONSTRUCTING our TEACHING CHALLENGES in a low tech 1. ORCHESTRATION environment 1. SHOW-AND-TELL Selma Hamdani Psychology / DALC / Neuroscience

836 views • 50 slides
Naive Bayes case study Training set: 10,000 emails that are either SPAM or HAM Testing set:

Naive Bayes case study Training set: 10,000 emails that are either SPAM or HAM Testing set:

Naive Bayes case study Training set: 10,000 emails that are either SPAM or HAM Testing set: 1,000 additional emails Train a Naive Bayes classifier on (a subset of) the training set Predict SPAM/HAM on the test set and compute

775 views • 13 slides
Empirical Interpretation of Imprecise Probabilities Marco Cattaneo School of Mathematics and

Empirical Interpretation of Imprecise Probabilities Marco Cattaneo School of Mathematics and

Empirical Interpretation of Imprecise Probabilities Marco Cattaneo School of Mathematics and Physical Sciences University of Hull ISIPTA 17 ECSQARU 2017, Lugano, Switzerland 10 July 2017 introduction imprecise probabilities can have

311 views • 8 slides
Intro to Classification Sanity Check Project A Did everyone turn in their project? Any

Intro to Classification Sanity Check Project A Did everyone turn in their project? Any

Intro to Classification Sanity Check Project A Did everyone turn in their project? Any concern or questions? Project B released today Linear Regression KNN Classification Question: Last week we talked about

787 views • 30 slides
CEE 370 Environmental Engineering Principles Lecture #25 Water Quality Management III: Lakes

CEE 370 Environmental Engineering Principles Lecture #25 Water Quality Management III: Lakes

CEE 370 Lecture #25 11/6/2019 Print version Updated: 6 November 2019 CEE 370 Environmental Engineering Principles Lecture #25 Water Quality Management III: Lakes & toxic models Reading: Mihelcic & Zimmerman, Chapter 7 & 3.10

218 views • 20 slides
Computational Methods for Random Epidemiological Models M etodos Computacionales para el

Computational Methods for Random Epidemiological Models M etodos Computacionales para el

Computational Methods for Random Epidemiological Models M etodos Computacionales para el Estudio de Modelos Epidemiol ogicos con Incertidumbre Conferencias de Investigaci on para Posgrado 2016 Universidad Complutense de Madrid 24

1.1k views • 86 slides
Generalized Loopy 2U: A New Algorithm for Approximate Inference in Credal Networks Alessandro

Generalized Loopy 2U: A New Algorithm for Approximate Inference in Credal Networks Alessandro

Generalized Loopy 2U: A New Algorithm for Approximate Inference in Credal Networks Alessandro Antonucci, Marco Zaffalon, Sun Yi, Cassio de Campos IDSIA Lugano, Switzerland PGM 08 - Hirtshals September 19th, 2008 Imprecise probabilities

1.04k views • 75 slides
Coherence under uncertainty: Philosophical and psychological applications Niki Pfeifer 1 Giuseppe

Coherence under uncertainty: Philosophical and psychological applications Niki Pfeifer 1 Giuseppe

Coherence under uncertainty: Philosophical and psychological applications Niki Pfeifer 1 Giuseppe Sanfilippo 2 Angelo Gilio 3 1 Ludwig-Maximilians-University Munich, Germany 2 Department of Mathematics and Computer Science University of Palermo,

1.31k views • 103 slides
From Gauging Accuracy of Quantity What If a Model Is . . . Estimates to Gauging Accuracy and

From Gauging Accuracy of Quantity What If a Model Is . . . Estimates to Gauging Accuracy and

Outline Situations in Which . . . Situations in Which . . . From Gauging Accuracy of Quantity What If a Model Is . . . Estimates to Gauging Accuracy and Situations in Which . . . Example of Asymmetry Resolution of Field Measurements:

262 views • 14 slides
Evidence for lexically driven adaptation to novel accents in early development CIMSA April 7,

Evidence for lexically driven adaptation to novel accents in early development CIMSA April 7,

Evidence for lexically driven adaptation to novel accents in early development CIMSA April 7, 2013 The problem with accents The goal: understand meaning The task: BALL Adults and accents Role of top-down feedback in driving

646 views • 29 slides

More recommend