a uni ed text annotation work ow for diverse goals
play

A Unied Text Annotation Workow for Diverse Goals Discovering - PowerPoint PPT Presentation

Nov 24, 2023 •303 likes •942 views

Janis Pagel Nils Reiter Ina Rsiger Sarah Schulz A Unied Text Annotation Workow for Diverse Goals Discovering phenomena not covered by a theory Strengthening denitions in a theory Often confused categories might be overlapping or

  1. Janis Pagel Nils Reiter Ina Rösiger Sarah Schulz A Uni�ed Text Annotation Work�ow for Diverse Goals

  2. Discovering phenomena not covered by a theory Strengthening de�nitions in a theory Often confused categories might be overlapping or at least unclear Uncovering implicit assumptions Manually annotated data can be analysed Which categories are how frequent in what context? Automatic tools can be evaluated How well do machines do this task? Supervised tools can be trained Why do we annotate? Empirical validation of theories Data creation Janis Pagel, Nils Reiter, Ina Rösiger, Sarah Schulz, Institute for Natural Language Processing (IMS), University of Stuttgart: Uni�ed Annotation Work�ow 2

  3. Manually annotated data can be analysed Which categories are how frequent in what context? Automatic tools can be evaluated How well do machines do this task? Supervised tools can be trained Why do we annotate? Empirical validation of theories Discovering phenomena not covered by a theory Strengthening de�nitions in a theory Often confused categories might be overlapping or at least unclear Uncovering implicit assumptions Data creation Janis Pagel, Nils Reiter, Ina Rösiger, Sarah Schulz, Institute for Natural Language Processing (IMS), University of Stuttgart: Uni�ed Annotation Work�ow 2

  4. Why do we annotate? Empirical validation of theories Discovering phenomena not covered by a theory Strengthening de�nitions in a theory Often confused categories might be overlapping or at least unclear Uncovering implicit assumptions Data creation Manually annotated data can be analysed Which categories are how frequent in what context? Automatic tools can be evaluated How well do machines do this task? Supervised tools can be trained Janis Pagel, Nils Reiter, Ina Rösiger, Sarah Schulz, Institute for Natural Language Processing (IMS), University of Stuttgart: Uni�ed Annotation Work�ow 2

  5. Analog annotation … Ideas attached to spans of text Sometimes fuzzy text spans Janis Pagel, Nils Reiter, Ina Rösiger, Sarah Schulz, Institute for Natural Language Processing (IMS), University of Stuttgart: Uni�ed Annotation Work�ow 3

  6. …and digital annotation Explicit assignment of categories to text spans Text spans are explicitly bounded (begin, end) Janis Pagel, Nils Reiter, Ina Rösiger, Sarah Schulz, Institute for Natural Language Processing (IMS), University of Stuttgart: Uni�ed Annotation Work�ow 4

  7. Pustejovsky and Stubbs (2012) Hovy and Lavid (2010) Circles Annotation (Circle) Well known in Computational Linguis- tics Automation-centered “Enrich with knowledge” Janis Pagel, Nils Reiter, Ina Rösiger, Sarah Schulz, Institute for Natural Language Processing (IMS), University of Stuttgart: Uni�ed Annotation Work�ow 5

  8. Hovy and Lavid (2010) Circles Annotation (Circle) Well known in Computational Linguis- tics Automation-centered “Enrich with knowledge” Pustejovsky and Stubbs (2012) Janis Pagel, Nils Reiter, Ina Rösiger, Sarah Schulz, Institute for Natural Language Processing (IMS), University of Stuttgart: Uni�ed Annotation Work�ow 5

  9. Circles Annotation (Circle) Well known in Computational Linguis- tics Automation-centered “Enrich with knowledge” Pustejovsky and Stubbs (2012) Hovy and Lavid (2010) Janis Pagel, Nils Reiter, Ina Rösiger, Sarah Schulz, Institute for Natural Language Processing (IMS), University of Stuttgart: Uni�ed Annotation Work�ow 5

  10. Gius and Jacke (2017) Circles Hermeneutic Circle Well known in Humanities Interpretation-centered “Retrieve knowledge” Janis Pagel, Nils Reiter, Ina Rösiger, Sarah Schulz, Institute for Natural Language Processing (IMS), University of Stuttgart: Uni�ed Annotation Work�ow 6

  11. Circles Hermeneutic Circle Well known in Humanities Interpretation-centered “Retrieve knowledge” Gius and Jacke (2017) Janis Pagel, Nils Reiter, Ina Rösiger, Sarah Schulz, Institute for Natural Language Processing (IMS), University of Stuttgart: Uni�ed Annotation Work�ow 6

  12. Work�ow Theoretical Data notion (Proto) annotation Annotation guidelines Anno- tated Analysis text/ corpus Interpretation Automation Janis Pagel, Nils Reiter, Ina Rösiger, Sarah Schulz, Institute for Natural Language Processing (IMS), University of Stuttgart: Uni�ed Annotation Work�ow 7

  13. Work�ow Theoretical Data notion (Proto) annotation Annotation guidelines Anno- tated Analysis text/ corpus Interpretation Automation Janis Pagel, Nils Reiter, Ina Rösiger, Sarah Schulz, Institute for Natural Language Processing (IMS), University of Stuttgart: Uni�ed Annotation Work�ow 7

  14. Work�ow Theoretical Data notion (Proto) annotation Annotation guidelines Anno- tated Analysis text/ corpus Interpretation Automation Janis Pagel, Nils Reiter, Ina Rösiger, Sarah Schulz, Institute for Natural Language Processing (IMS), University of Stuttgart: Uni�ed Annotation Work�ow 7

  15. Work�ow Theoretical Data notion (Proto) annotation Annotation guidelines Anno- tated Analysis text/ corpus Interpretation Automation Janis Pagel, Nils Reiter, Ina Rösiger, Sarah Schulz, Institute for Natural Language Processing (IMS), University of Stuttgart: Uni�ed Annotation Work�ow 7

  16. Work�ow Theoretical Data notion (Proto) annotation Annotation guidelines Anno- tated Analysis text/ corpus Interpretation Automation Janis Pagel, Nils Reiter, Ina Rösiger, Sarah Schulz, Institute for Natural Language Processing (IMS), University of Stuttgart: Uni�ed Annotation Work�ow 7

  17. Work�ow Theoretical Data notion (Proto) annotation Annotation guidelines Anno- tated Analysis text/ corpus Interpretation Automation Janis Pagel, Nils Reiter, Ina Rösiger, Sarah Schulz, Institute for Natural Language Processing (IMS), University of Stuttgart: Uni�ed Annotation Work�ow 7

  18. Work�ow Theoretical Data notion (Proto) annotation Annotation guidelines Anno- tated Analysis text/ corpus Interpretation Automation Janis Pagel, Nils Reiter, Ina Rösiger, Sarah Schulz, Institute for Natural Language Processing (IMS), University of Stuttgart: Uni�ed Annotation Work�ow 7

  19. Work�ow Theoretical Data notion (Proto) annotation Annotation guidelines Anno- tated Analysis text/ corpus Interpretation Automation Janis Pagel, Nils Reiter, Ina Rösiger, Sarah Schulz, Institute for Natural Language Processing (IMS), University of Stuttgart: Uni�ed Annotation Work�ow 7

  20. Work�ow Theoretical Data notion (Proto) annotation Annotation guidelines Anno- tated Analysis text/ corpus Interpretation Automation Janis Pagel, Nils Reiter, Ina Rösiger, Sarah Schulz, Institute for Natural Language Processing (IMS), University of Stuttgart: Uni�ed Annotation Work�ow 7

  21. Work�ow Theoretical Data notion (Proto) annotation Annotation guidelines Anno- tated Analysis text/ corpus Interpretation Automation Janis Pagel, Nils Reiter, Ina Rösiger, Sarah Schulz, Institute for Natural Language Processing (IMS), University of Stuttgart: Uni�ed Annotation Work�ow 7

  22. Theoretical Data notion (Proto) annotation Annotation guidelines Examples Pliny (Bradley, 2008) Anno- tated 3DH (Kleymann, Meister, Analysis text/ and Stange, 2018) corpus Interpretation Automation Goals of Annotation Exploratory Mostly note-taking Semi-organized Humanities centered Ideally completely free of presuppositions Janis Pagel, Nils Reiter, Ina Rösiger, Sarah Schulz, Institute for Natural Language Processing (IMS), University of Stuttgart: Uni�ed Annotation Work�ow 8

  23. Examples Pliny (Bradley, 2008) 3DH (Kleymann, Meister, and Stange, 2018) Goals of Annotation Theoretical Data notion Exploratory Mostly note-taking Semi-organized (Proto) Humanities centered annotation Annotation Ideally completely free of guidelines presuppositions Anno- tated Analysis text/ corpus Interpretation Automation Janis Pagel, Nils Reiter, Ina Rösiger, Sarah Schulz, Institute for Natural Language Processing (IMS), University of Stuttgart: Uni�ed Annotation Work�ow 8

  24. Examples Pliny (Bradley, 2008) 3DH (Kleymann, Meister, and Stange, 2018) Goals of Annotation Theoretical Data notion Exploratory Mostly note-taking Semi-organized (Proto) Humanities centered annotation Annotation Ideally completely free of guidelines presuppositions Anno- tated Analysis text/ corpus Interpretation Automation Janis Pagel, Nils Reiter, Ina Rösiger, Sarah Schulz, Institute for Natural Language Processing (IMS), University of Stuttgart: Uni�ed Annotation Work�ow 8

  25. Examples Pliny (Bradley, 2008) 3DH (Kleymann, Meister, and Stange, 2018) Goals of Annotation Theoretical Data notion Exploratory Mostly note-taking Semi-organized (Proto) Humanities centered annotation Annotation Ideally completely free of guidelines presuppositions Anno- tated Analysis text/ corpus Interpretation Automation Janis Pagel, Nils Reiter, Ina Rösiger, Sarah Schulz, Institute for Natural Language Processing (IMS), University of Stuttgart: Uni�ed Annotation Work�ow 8

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Lecture 2 Annotation tools & Segmentation Summary of Part 1 Annotation theory

Lecture 2 Annotation tools & Segmentation Summary of Part 1 Annotation theory

Center for Reflected Text Analytics Lecture 2 Annotation tools & Segmentation Summary of Part 1 Annotation theory Guidelines Inter-Annotator agreement Inter-subjective annotations Annotation exercise Discuss

671 views • 22 slides
Recognizing Emotions in Text Saima Aman Master s Thesis Presentation Supervisor: Dr. S.

Recognizing Emotions in Text Saima Aman Master s Thesis Presentation Supervisor: Dr. S.

Recognizing Emotions in Text Saima Aman Master s Thesis Presentation Supervisor: Dr. S. Szpakowicz University of Ottawa 2007 Agenda Introduction Problem Definition Related Work Data Emotion Annotation Annotation

748 views • 18 slides
Semantic annotation of unstructured and ungrammatical text Matthew Michelson & Craig A.

Semantic annotation of unstructured and ungrammatical text Matthew Michelson & Craig A.

Semantic annotation of unstructured and ungrammatical text Matthew Michelson & Craig A. Knoblock University of Southern California & Information Sciences Institute User Entered Text (on the web) User Entered Text (on the web)

621 views • 35 slides
Course Introduction SWEN-343 Welcome Goals for the Course Prepare you for real world Be ready

Course Introduction SWEN-343 Welcome Goals for the Course Prepare you for real world Be ready

Course Introduction SWEN-343 Welcome Goals for the Course Prepare you for real world Be ready to work with diverse systems Diverse teams Work with teams (intra team) Create & Maintain Enterprise apps How is this course different!? What

645 views • 48 slides
PoS Tagging June 2, 2009 Text Annotation Be ata B. Megyesi beata.megyesi@lingfil.uu.se 1

PoS Tagging June 2, 2009 Text Annotation Be ata B. Megyesi beata.megyesi@lingfil.uu.se 1

PoS Tagging June 2, 2009 Text Annotation Be ata B. Megyesi beata.megyesi@lingfil.uu.se 1 Goal ATA B. M EGYESI P O S T AGGING J UNE 2, 2009 What are the main components used for grammatical annotation? How do we get running

941 views • 75 slides
Semantic annotation of unstructured and ungrammatical text Matthew Michelson and Craig A.

Semantic annotation of unstructured and ungrammatical text Matthew Michelson and Craig A.

Semantic annotation of unstructured and ungrammatical text Matthew Michelson and Craig A. Knoblock Information Sciences Institute Department of Computer Science University of Southern California Ungrammatical & Unstructured Text

663 views • 29 slides
UIMA-based Annotation Type System for a Text Mining Architecture Udo Hahn, Ekaterina Buyko ,

UIMA-based Annotation Type System for a Text Mining Architecture Udo Hahn, Ekaterina Buyko ,

UIMA-based Annotation Type System for a Text Mining Architecture Udo Hahn, Ekaterina Buyko , Katrin Tomanek, Scott Piao, Yoshimasa Tsuruoka, John McNaught, Sophia Ananiadou Jena University Language and Information Engineering Lab & School

762 views • 38 slides
Annotation/Marking Text No one is smart enough to remember all the things he knows.

Annotation/Marking Text No one is smart enough to remember all the things he knows.

Annotation/Marking Text No one is smart enough to remember all the things he knows. ~Mark Twain Tuesday, November 27, 2012 Why bother? Tuesday, November 27, 2012 Why bother? To maintain engagement while reading in order to

542 views • 31 slides
Semantic Annotation of Clinical Text: The CLEF Corpus Angus Roberts, Robert Gaizauskas, Mark

Semantic Annotation of Clinical Text: The CLEF Corpus Angus Roberts, Robert Gaizauskas, Mark

Semantic Annotation of Clinical Text: The CLEF Corpus Angus Roberts, Robert Gaizauskas, Mark Hepple, George Demetriou, Yikun Guo, Andrea Setzer and Ian Roberts Natural Language Processing Group, University of Sheffield, UK Introduction

458 views • 31 slides
Annotation Analytics for Gene and Protein functions Nigam Shah, MBBS, PhD nigam@stanford.edu

Annotation Analytics for Gene and Protein functions Nigam Shah, MBBS, PhD nigam@stanford.edu

Annotation Analytics for Gene and Protein functions Nigam Shah, MBBS, PhD nigam@stanford.edu Annotation service Process textual metadata to automatically tag text with as many ontology terms as possible. 107 million calls, ~1000 GB data

273 views • 10 slides
> > Overview Goals of TM How does TM work Components History Problems

> > Overview Goals of TM How does TM work Components History Problems

> > Overview Goals of TM How does TM work Components History Problems particular to biomedical texts Examples 22/01/08 Tamara Polajnar > BRC > U Glasgow 2 > The Goal of Text Mining The small G-Protein Ras

1.04k views • 45 slides
Web Annotations Building the Experience Annotation An annotation is something added. It is not

Web Annotations Building the Experience Annotation An annotation is something added. It is not

Web Annotations Building the Experience Annotation An annotation is something added. It is not the primary content, or even the secondary content. It is the association. Annotation (cont.) Annotations exist already - Links - Block quotes

452 views • 14 slides
Childrens well-being in diverse migration contexts: Goals, design and preliminary findings

Childrens well-being in diverse migration contexts: Goals, design and preliminary findings

Childrens well-being in diverse migration contexts: Goals, design and preliminary findings from the FAMELO project. Jennifer E. Glick, Victor Agadjanian, Natalie D. Eggum-Wilkens Dirgha Ghimire, Sarah Hayford, Carlos Santos, Scott Yabiku

506 views • 26 slides
the annotation of the Encyclopedia of Life http://www.environments-eol.blogspot.com/ Evangelos

the annotation of the Encyclopedia of Life http://www.environments-eol.blogspot.com/ Evangelos

Identification of Environment Ontology terms in text and the annotation of the Encyclopedia of Life http://www.environments-eol.blogspot.com/ Evangelos Pafilis 1* , Sune Frankild 2 , Lucia Fanini 1 , Sarah Faulwetter 1 , Christina Pavloudi 1 ,

632 views • 49 slides
Here is some test text Here is some test text Here is some test text By Mou Mount nt_Everes

Here is some test text Here is some test text Here is some test text By Mou Mount nt_Everes

Here is some test text Here is some test text Here is some test text By Mou Mount nt_Everes erest_as_see een_ n_fro rom_ m_Dru rukai air2. r2.jp jpg: g: shri rimpo mpo196 1967de 7deri rivat ative e wor work: : Papa pa Li

370 views • 32 slides
Collective Annotation: From Crowdsourcing to Social Choice Ulle Endriss Institute for Logic,

Collective Annotation: From Crowdsourcing to Social Choice Ulle Endriss Institute for Logic,

Collective Annotation Tulane, 14 March 2014 Collective Annotation: From Crowdsourcing to Social Choice Ulle Endriss Institute for Logic, Language and Computation University of Amsterdam joint work with Raquel Fern andez, Justin

418 views • 16 slides
SOCI 210: Sociological Perspectives Oct. 6 1. Intersectionality 2. Interactional sociology 3.

SOCI 210: Sociological Perspectives Oct. 6 1. Intersectionality 2. Interactional sociology 3.

SOCI 210: Sociological Perspectives Oct. 6 1. Intersectionality 2. Interactional sociology 3. Status and boundaries 1 Intersectionality 2 Intersectionality Social categories Race, gender, disability, ethnicity, sexuality Central

230 views • 4 slides
Why relationships matter The magic ingredients Our journey so far and how relationship building

Why relationships matter The magic ingredients Our journey so far and how relationship building

Why relationships matter The magic ingredients Our journey so far and how relationship building has been pivotal Better Start Bradford is a partnership between the community and the services who support families of 0-3s and pregnant women

368 views • 19 slides
BISIMULATIONS FOR DIFFERENTIAL EQUATIONS http://sysma.imtlucca.it/tools/erode/ MOTIVATION 1 2

BISIMULATIONS FOR DIFFERENTIAL EQUATIONS http://sysma.imtlucca.it/tools/erode/ MOTIVATION 1 2

MIRCO TRIBASTONE IMT SCHOOL FOR ADVANCED STUDIES LUCCA JOINT WORK WITH LUCA CARDELLI, MAX TSCHAIKOWSKI, AND ANDREA VANDIN BISIMULATIONS FOR DIFFERENTIAL EQUATIONS http://sysma.imtlucca.it/tools/erode/ MOTIVATION 1 2 3 V in 1 2 3 Fig. 2:

236 views • 22 slides
Identifiability of dynamic networks with noisy and noise-free nodes Paul Van den Hof Coworkers:

Identifiability of dynamic networks with noisy and noise-free nodes Paul Van den Hof Coworkers:

Identifiability of dynamic networks with noisy and noise-free nodes Paul Van den Hof Coworkers: Harm Weerts, Arne Dankers 27 September 2016, ERNSI meeting, Cison di Valmarino, Italy Dynamic network r i external excitation v i process noise w i

364 views • 32 slides
Vagueness, graded truth and pairwise valuations Rossella Marrano Scuola Normale Superiore, Pisa

Vagueness, graded truth and pairwise valuations Rossella Marrano Scuola Normale Superiore, Pisa

Vagueness, graded truth and pairwise valuations Rossella Marrano Scuola Normale Superiore, Pisa The Future of Mathematical Fuzzy Logic Prague, 17 June 2016 MFL and vagueness Rossella Marrano (SNS) The future of MFL 17/6/2016 1 / 15 MFL

424 views • 31 slides
Digital Bubbles Plenary Commentary Hanne De Jaegher 1 Spectrums!

Digital Bubbles Plenary Commentary Hanne De Jaegher 1 Spectrums!

Digital Bubbles Plenary Commentary Hanne De Jaegher 1 Spectrums! Enactivism Cognitivism 2 Spectrums! Enactivism: Assumptions Self-organization and self-maintenance Experience Embodiment,

470 views • 7 slides
Large games and population protocols R. Ramanujam The Institute of Mathematical Sciences,

Large games and population protocols R. Ramanujam The Institute of Mathematical Sciences,

Large games and population protocols R. Ramanujam The Institute of Mathematical Sciences, Chennai, India email: jam@imsc.res.in Formal Methods Update Meeting, BITS-Pilani, Goa Campus July 20, 2018 First words . . . Thanks to Baskar for the

1.18k views • 102 slides
Competing Models? Loet Leydesdorff University of Amsterdam, Amsterdam School of Communication

Competing Models? Loet Leydesdorff University of Amsterdam, Amsterdam School of Communication

Peer Review vs Metrics: Competing Models? Loet Leydesdorff University of Amsterdam, Amsterdam School of Communication Research (ASCoR) loet@leydesdorff.net Models do not have to be compatible Peer review is another model (or

426 views • 19 slides

More recommend