toward active learning in data selection
play

Toward Active Learning in Data Selection: Automatic Discovery of - PowerPoint PPT Presentation

Jan 20, 2023 •283 likes •550 views

Toward Active Learning in Data Selection: Automatic Discovery of Language Features During Elicitation Jonathan Clark Robert Frederking Lori Levin Language Technologies Institute Carnegie Mellon University Pittsburgh, PA Feature Detection

  1. Toward Active Learning in Data Selection: Automatic Discovery of Language Features During Elicitation Jonathan Clark Robert Frederking Lori Levin Language Technologies Institute Carnegie Mellon University Pittsburgh, PA

  2. Feature Detection • Grammatemes * - Language features that express grammatical meanings (such as number, person, tense) • Given a set of grammatemes and a structured corpus, can we determine if these grammatemes are expressed in a particular language? • e.g. Answers “Does this language distinguish singular nouns from plural nouns?” (“And if so, how?”) * Source: Alena Böhmová, Silvie Cinková, Eva Haji č ová. Annotation on the tectogrammatical layer in the Prague Dependency Treebank. 2005.

  3. Feature Detection The dog sleeps ((num sg)…) The dogs sleep ((num dl)…) The dogs sleep ((num pl)...)

  4. 犬が寝る 犬が寝る 犬が寝る Feature Detection Bilingual Person The dog sleeps ((num sg)…) The dogs sleep ((num dl)…) The dogs sleep ((num pl)...)

  5. 犬が寝る 犬が寝る 犬が寝る 犬が寝る 犬が寝る 犬が寝る 犬が寝る Feature Detection Bilingual Person The dog sleeps Marks Plural? NO ((num sg)…) Feature The dogs sleep Detection ((num dl)…) The dogs sleep Marks Dual? NO ((num pl)...)

  6. Data Selection • Given many potential training examples, select the ones that will help the target system most • Many Uses - Seen in Speech Recognition, Speech Synthesis, and Machine Translation • Corpus Navigation: Not all data is relevant for all languages • Helps when money or time is limited • e.g. Small Domains, MT Emergencies, and Minority Languages

  7. 犬が寝る 犬が寝る 犬が寝る 犬が寝る 犬が寝る 犬が寝る 犬が寝る Data Selection Bilingual Person The dog sleeps Marks Plural? NO ((num sg)…) Feature The dogs sleep Detection ((num dl)…) The dogs sleep Marks Dual? NO ((num pl)...)

  8. 犬が寝る 犬が寝る 犬が寝る 犬が寝る 犬が寝る 犬が寝る 犬が寝る Data Selection Bilingual Person The dog sleeps Marks Plural? NO ((num sg)…) Feature The dogs sleep Detection ((num dl)…) The dogs sleep Marks Dual? NO ((num pl)...) Data Selection (Corpus Navigation)

  9. 犬が寝る 犬が寝る 犬が寝る 犬が寝る 犬が寝る 犬が寝る 犬が寝る Data Selection Bilingual Person The dog sleeps Marks Plural? NO ((num sg)…) Feature The dogs sleep Detection ((num dl)…) The dogs sleep Marks Dual? NO ((num pl)...) Data Selection (Corpus Navigation) Implicational Universal: No Plural Marking --> No Dual Marking

  10. 犬が寝る 犬が寝る 犬が寝る 犬が寝る 犬が寝る 犬が寝る 犬が寝る 犬が寝る 犬が寝る 犬が寝る 犬が寝る 犬が寝る 犬が寝る 犬が寝る Data Selection Bilingual Person The dog sleeps Marks Plural? NO Marks Plural? NO The dog sleeps ((num sg)…) ((num sg)…) Feature The dogs sleep The dogs sleep Detection ((num dl)…) ((num dl)…) The dogs sleep Marks Dual? NO The dogs sleep Marks Dual? NO ((num pl)...) ((num pl)...) Data Selection (Corpus Navigation) Implicational Universal: No Plural Marking --> No Dual Marking

  11. Elicitation Corpus Entry context: Maria bakes cookies regularly or habitually. srcsent: Maria bakes cookies .

  12. Elicitation Corpus Entry context: Maria bakes cookies regularly or habitually. srcsent: Maria bakes cookies .

  13. Elicitation Corpus Entry context: Maria bakes cookies regularly or habitually. srcsent: Maria bakes cookies . tgtsent: Maria hornea galletas . aligned: ((1,1),(2,2),(3,3),(4,4))

  14. Elicitation Corpus Entry context: Maria bakes cookies regularly or habitually. srcsent: Maria bakes cookies . tgtsent: Maria hornea galletas . aligned: ((1,1),(2,2),(3,3),(4,4)) fstruct: [f1]( [f2](actor ((gender f)(anim human)(num sg))) [f3](undergoer ((person 3) (num dl))) (tense pres)) cstruct: [n1](S1 [n2](S [n3](NP [n4](NNP Maria)) [n5](VP [n6](VBZ bakes) [n7](NP [n8](NNS cookies))))) phimap: phi(n1)=f1; phi(n3)=f2; phi(n7)=f3; headmap: h(n1)=n2; h(n2)=n5; h(n3)=n4; h(n4)=n4; h(n5)=n6; h(n6)=n6; h(n7)=n8; h(n8)=n8;

  15. Example Deduction Rule # Perfective/Imperfective Aspect (rule (sentences (A (aspect perfective)) (B (aspect progressive)))

  16. Example Deduction Rule # Perfective/Imperfective Aspect (rule (sentences (A (aspect perfective)) (B (aspect progressive))) (overlap on)

  17. Example Deduction Rule # Perfective/Imperfective Aspect (rule (sentences (A (aspect perfective)) (B (aspect progressive))) (overlap on) (if 0.6 (different (target-lex (fnode (A))) (target-lex (fnode (B)))) (then (WALS ”Perfective/Imperfective Aspect” ”Grammatical marking”)))

  18. Example Deduction Rule # Perfective/Imperfective Aspect (rule (sentences (A (aspect perfective)) (B (aspect progressive))) (overlap on) (if 0.6 (different (target-lex (fnode (A))) (target-lex (fnode (B)))) (then (WALS ”Perfective/Imperfective Aspect” ”Grammatical marking”))) (if 0.4 (same (target-lex (fnode (A))) (target-lex (fnode (B)))) (then (WALS ”Perfective/Imperfective Aspect” ”No grammatical marking”))))

  19. Feature Detection Experiment • Corpus of 60 Spanish-English 100% sentences 80% • Tried to identify 21 features 60% from the World Atlas of 40% Language Structures 20% Precision Recall F1 Baseline 12 / 21 12 / 21 12 / 21 0% Experimental Experimental 19 / 21 19 / 21 19 / 21 Baseline

  20. Toward Corpus Navigation • Not all data is relevant for every language • Performed while a linguistically naive bilingual person translates sentences in GUI • After eliciting each sentence: * Apply feature detection * Choose the most valuable sentence to elicit next • Leverages knowledge from Greenbergian Implicational Universals (from Hal Daume’s database learned from WALS)

  21. Other Applications • Learning feature-annotated closed-class morphemes • Factored MT • Selection of data for automatic grammar induction for syntactic and hybrid MT systems • Aid for linguistics field work

  22. Language Resources • Result of Corpus Navigation is: 1. A resource dense with the “right” features 2. Highly structured; each language feature is linked with sentences that illustrate it 3. Word-aligned, feature-annotated sentences useful for studying divergences and MT

  23. Toward Active Learning in Data Selection: Automatic Discovery of Language Features During Elicitation Questions? Jonathan Clark Robert Frederking Lori Levin Language Technologies Institute Carnegie Mellon University Pittsburgh, PA

  24. WALS Features for Experiment Gender Distinctions in Independent Position of Interrogative Phrases in Content Personal Pronouns Questions Nominal and Locational Predication Position of Pronominal Possessive Affixes Occurrence of Nominal Plurality Position of Tense-Aspect Affixes Inclusive/Exclusive Distinction in Independent Order of Adjective and Noun Pronouns Inclusive/Exclusive Distinction in Verbal Order of Genitive and Noun Inflection Order of Numeral and Noun Semantic Distinctions of Evidentiality Order of Subject, Object and Verb The Future Tense Order of Subject and Verb Verbal Person Marking Order of Object and Verb ‘Want’ Complement Subjects Perfective/Imperfective Aspect Zero Copula for Predicate Nominals Politeness Distinctions in Pronouns

  25. Production Predicates fnode in-order source-lex target-lex *-uhead *-ihead same present not-present

  26. Elicitation Corpus Availability • Included in LDC’s Less Commonly Taught Languages (LCTL) Language Packs • 13 languages have already been translated by the LDC • Urdu language pack used in this year’s NIST MT Eval • Bengali queued for general release this year

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Active Learning with Active Learning with Model Selection Neil Rubens Sugiyama Lab / Tokyo

Active Learning with Active Learning with Model Selection Neil Rubens Sugiyama Lab / Tokyo

Active Learning with Active Learning with Model Selection Neil Rubens Sugiyama Lab / Tokyo Institute of Technology Active Learning (NLP Motivation) NLP (common scenario) Large amounts of unlabeled data Large amounts of unlabeled

706 views • 36 slides
Information-Based Objective Functions for Active Data Selection By David J. C. Mackay Presented

Information-Based Objective Functions for Active Data Selection By David J. C. Mackay Presented

Information-Based Objective Functions for Active Data Selection By David J. C. Mackay Presented by Aditya Sanghi and Grant Watson 1 Motivation Active learning A learning algorithm which is able to interactively query for more data

1.41k views • 107 slides
Release from Active Learning / Release from Active Learning / Model Selection Dilemma: Model

Release from Active Learning / Release from Active Learning / Model Selection Dilemma: Model

IJCNN2002 May 12-17, 2002 Release from Active Learning / Release from Active Learning / Model Selection Dilemma: Model Selection Dilemma: Optimizing Sample Points and Optimizing Sample Points and Models at the Same Time Models at the Same

532 views • 18 slides
10.1 Active Learning Review What questions should be asked in order to collect best data?

10.1 Active Learning Review What questions should be asked in order to collect best data?

CS/CNS/EE 253: Advanced Topics in Machine Learning Topic: Active Learning 2: the myopic version-space shrinking strategy Lecturer: Daniel Golovin Scribe: Pete Trautman Date: 8 February 2010 10.1 Active Learning Review What questions should be

264 views • 4 slides
Agenda Intro to Active Learning Activity Design Resources for Active Learning Lunch with Active

Agenda Intro to Active Learning Activity Design Resources for Active Learning Lunch with Active

Agenda Intro to Active Learning Activity Design Resources for Active Learning Lunch with Active Learning Veterans Wrap Up Introduction to Active Learning What is active learning? Brainstorm your ideas with your group and generate a

855 views • 42 slides
Multi-Task Active Learning Yi Zhang Outline Active Learning Multi-Task Active Learning

Multi-Task Active Learning Yi Zhang Outline Active Learning Multi-Task Active Learning

Multi-Task Active Learning Yi Zhang Outline Active Learning Multi-Task Active Learning Linguistic Annotations (ACL 08) Image Classification (CVPR 08) Current Work and Discussions Constraint-Driven Active Learning

945 views • 47 slides
ACTIVE 1. WELCOME and JUMPING IN LEARNING 1. WHAT IS ACTIVE LEARNING? la Carte 1.

ACTIVE 1. WELCOME and JUMPING IN LEARNING 1. WHAT IS ACTIVE LEARNING? la Carte 1.

ACTIVE 1. WELCOME and JUMPING IN LEARNING 1. WHAT IS ACTIVE LEARNING? la Carte 1. CONSTRUCTING our TEACHING CHALLENGES in a low tech 1. ORCHESTRATION environment 1. SHOW-AND-TELL Selma Hamdani Psychology / DALC / Neuroscience

836 views • 50 slides
Learning From Data Lecture 13 Validation and Model Selection The Validation Set Model Selection

Learning From Data Lecture 13 Validation and Model Selection The Validation Set Model Selection

Learning From Data Lecture 13 Validation and Model Selection The Validation Set Model Selection Cross Validation M. Magdon-Ismail CSCI 4100/6100 recap: Regularization Regularization combats the effects of noise by putting a leash on the

349 views • 31 slides
Active Learning Passive Learning Active Learning 1. Think 1. Acquisition of knowledge Ability

Active Learning Passive Learning Active Learning 1. Think 1. Acquisition of knowledge Ability

Active Learning Passive Learning Active Learning 1. Think 1. Acquisition of knowledge Ability 2. Decision making 2. Application of knowledge 3. Explain/communicate 1. Deep learning Approach to Uni-directional 2. Interactive Learning

282 views • 5 slides
Learning From Data Lecture 3 Is Learning Feasible? Outside the Data Probability to the Rescue

Learning From Data Lecture 3 Is Learning Feasible? Outside the Data Probability to the Rescue

Learning From Data Lecture 3 Is Learning Feasible? Outside the Data Probability to the Rescue Learning vs. Verification Selection Bias - A Cartoon M. Magdon-Ismail CSCI 4100/6100 recap: The Perceptron Learning Algorithm y = +1 y x

538 views • 27 slides
12.1 Active Learning: A Review When learning, it may be the case that getting the true labels of

12.1 Active Learning: A Review When learning, it may be the case that getting the true labels of

CS/CNS/EE 253: Advanced Topics in Machine Learning Topic: Active Learning Review and Kernel Methods Lecturer: Andreas Krause Scribe: Jonathan Krause Date: Feb. 17, 2010 12.1 Active Learning: A Review When learning, it may be the case that getting

213 views • 5 slides
Predator and Prey: Active Learning Is Social Learning Kenneth Ronkowitz Active Learning

Predator and Prey: Active Learning Is Social Learning Kenneth Ronkowitz Active Learning

Predator and Prey: Active Learning Is Social Learning Kenneth Ronkowitz Active Learning Symposium, Rutgers University, May 15, 2018 Typically part of the curriculum in biology, phrenology, biology, anatomy, craniometry, physiognomy, crime

484 views • 14 slides
Learning Loss for Active Learning Rymarczyk D., Zieliski B., Tabor J., Sadowski M., Titov M.

Learning Loss for Active Learning Rymarczyk D., Zieliski B., Tabor J., Sadowski M., Titov M.

Learning Loss for Active Learning Rymarczyk D., Zieliski B., Tabor J., Sadowski M., Titov M. Agenda 1. Active Learning introduction 2. Base methods in AL for Deep Learning 3. Learning Loss for Active Learning 4. Our ideas for Active

802 views • 43 slides
Introduction to Machine Learning Active Learning Barnabs Pczos 1 Credits Some of the

Introduction to Machine Learning Active Learning Barnabs Pczos 1 Credits Some of the

Introduction to Machine Learning Active Learning Barnabs Pczos 1 Credits Some of the slides are taken from Nina Balcan. 2 Classic Supervised Learning Paradigm is Insufficient Nowadays Modern applications: massive amounts of raw data.

1.17k views • 73 slides
Feature Selection Gavin Brown www.cs.man.ac.uk/~gbrown The Usual Supervised Learning Approach

Feature Selection Gavin Brown www.cs.man.ac.uk/~gbrown The Usual Supervised Learning Approach

COMP61011 : Machine Learning Feature Selection Gavin Brown www.cs.man.ac.uk/~gbrown The Usual Supervised Learning Approach data + labels Learning Algorithm Model Predicted label Testing data Predicting Recurrence of Lung Cancer ofi Only a

724 views • 17 slides
Active Learning: Rethinking Our Teaching to Promote Deeper Learning Facilitated by Ken Silvestri,

Active Learning: Rethinking Our Teaching to Promote Deeper Learning Facilitated by Ken Silvestri,

Active Learning: Rethinking Our Teaching to Promote Deeper Learning Facilitated by Ken Silvestri, CFE Instructional Designer Adapted for Institute of Technology By Menaka Abraham OVERVIEW Why Active Learning What is Active Learning

542 views • 18 slides
Fostering Innovation through Academic- Community Partnerships Yolande Chan, E. Marie Shantz

Fostering Innovation through Academic- Community Partnerships Yolande Chan, E. Marie Shantz

Fostering Innovation through Academic- Community Partnerships Yolande Chan, E. Marie Shantz Professor, Queens University Jeff Dixon, Associate Director, The Monieson Centre Agenda Overview of Partnerships and Projects Lessons Learned

792 views • 48 slides
Ef Effective Ma Marketing Creating an Engaging Environment Carol Gilbert, M.Ed, SNS 1 Fo

Ef Effective Ma Marketing Creating an Engaging Environment Carol Gilbert, M.Ed, SNS 1 Fo

5/13/20 Ef Effective Ma Marketing Creating an Engaging Environment Carol Gilbert, M.Ed, SNS 1 Fo Focus Points Identify components of marketing to increase participation Implement marketing strategies Create dining area

339 views • 9 slides
Outline 13.1 IR Effectiveness Measures 13.2 Probabilistic IR 13.3 Statistical Language Model

Outline 13.1 IR Effectiveness Measures 13.2 Probabilistic IR 13.3 Statistical Language Model

Outline 13.1 IR Effectiveness Measures 13.2 Probabilistic IR 13.3 Statistical Language Model 13.4 Latent-Topic Models 13.4.1 LSI based on SVD 13.4.2 pLSI and LDA 13.4.3 Skip-Gram Model 13.5 Learning to Rank Not only does God play dice,

795 views • 45 slides
Normal Accidents: Normal Accidents: A Book Report A Book Report Bill Tet zlaf f Bill Tet zlaf

Normal Accidents: Normal Accidents: A Book Report A Book Report Bill Tet zlaf f Bill Tet zlaf

Normal Accidents: Normal Accidents: A Book Report A Book Report Bill Tet zlaf f Bill Tet zlaf f Sept ember 6, 2001 Sept ember 6, 2001 Normal Accidents Normal Accidents Charles P errow P rincet on Universit y P ress, 1999 I SBM

736 views • 38 slides
The Ontology of States, Processes, and Events Antony Galton College of Engineering, Mathematics,

The Ontology of States, Processes, and Events Antony Galton College of Engineering, Mathematics,

The Ontology of States, Processes, and Events Antony Galton College of Engineering, Mathematics, and Physical Sciences, University of Exeter, UK InterOntology12, Keio University, Tokyo, February 2012 What is a Process? Processes are

436 views • 31 slides
A RuleBased Unsupervised Morphology Learning Framework Constan'ne Lignos, Erwin Chan*, Mitch

A RuleBased Unsupervised Morphology Learning Framework Constan'ne Lignos, Erwin Chan*, Mitch

A RuleBased Unsupervised Morphology Learning Framework Constan'ne Lignos, Erwin Chan*, Mitch Marcus, Charles Yang University of Pennsylvania, *University of Arizona Morpho Challenge 2009 CLEF 2009, 9/30/2009 De=ining the Task Applica'on

382 views • 13 slides
From NL to FOL From NL to Logic Semantics and the NLTK Scott Farrar CLMA, University of

From NL to FOL From NL to Logic Semantics and the NLTK Scott Farrar CLMA, University of

From NL to FOL Scott Farrar CLMA, University of Washington far- rar@u.washington.edu Review From NL to FOL From NL to Logic Semantics and the NLTK Scott Farrar CLMA, University of Washington farrar@u.washington.edu February 17, 2010

1.15k views • 77 slides
Polarized Electron Sources for the ILC and CLIC P. Adderley, J. Brittian, J.Clark, J. Grames, J.

Polarized Electron Sources for the ILC and CLIC P. Adderley, J. Brittian, J.Clark, J. Grames, J.

Polarized Electron Sources for the ILC and CLIC P. Adderley, J. Brittian, J.Clark, J. Grames, J. Hansknecht, M.Poelker, M. Stutzman, R. Suleiman Students: A. Jayaprakash , J. McCarter, K. Surles-Law Some perspective: Gun R&D Projects at

640 views • 51 slides

More recommend