csci 562 empirical methods in natural language processing
play

CSCI 562: EMPIRICAL METHODS IN NATURAL LANGUAGE PROCESSING 25 Aug - PowerPoint PPT Presentation

Dec 29, 2022 •555 likes •922 views

CSCI 562: EMPIRICAL METHODS IN NATURAL LANGUAGE PROCESSING 25 Aug 2009 WHAT WE WANT What weve got? A.L.I.C.E. WHAT WOULD YOU DO? Where is USC located? DATA 1980s: if you wanted a computer to know something, you had to program

  1. CSCI 562: EMPIRICAL METHODS IN NATURAL LANGUAGE PROCESSING 25 Aug 2009

  2. WHAT WE WANT What we’ve got? A.L.I.C.E.

  3. WHAT WOULD YOU DO? Where is USC located?

  4. DATA • 1980s: if you wanted a computer to “know” something, you had to program it in. • Now: just about everything has been posted on the Web by someone, somewhere. But, it is all in natural language.

  5. BAG OF WORDS USC’s two primary campuses, both located in the heart of Los Angeles, welcome thousands of guests and visitors each year. The 226-acre University Park campus, home to the College of Letters, Arts and Sciences, the Graduate Where is USC located? School, and most of USC’s professional schools, is adjacent to Exposition Park with its world-class museums and recreational facilities. A few miles to the northeast is the 61-acre Health Sciences campus, home to the Keck School of Medicine of USC and the School of Pharmacy as well as three major teaching hospitals. is 2 is 1 located 1 located 1 two 1 usc 1 usc 3 where 0 where 1 …

  6. AMBIGUITY Where can a snow leopard be seen?

  7. GRAMMAR What animal does a frog eat?

  8. GRAMMAR SBARQ WHNP SINV What animal VBZ NP VP does a frog VB NP eat t

  9. MULTILINGUALITY Tell me about Tai Yen-Hui.

  10. STRUCTURE • All language has various levels of structure , more than just a bag of words • Virtually all Natural Language Processing tasks involve inferring structure from text or transforming one kind of structure into another

  11. CS 562 - Lecture 1, part II • More about ambiguities • Key problems to address in this class • grammar formalisms • search algorithms • learning methods • Learning Examples CS 562 - Intro (part 2) 11

  12. More about Ambiguities • to middle school kids: what does this sentence mean? I saw her duck. Aravind Joshi lexical ambiguity (word-sense) CS 562 - Intro (part 2) 12

  13. More about Ambiguities • to middle school kids: what does this sentence mean? I eat sushi with tuna. Aravind Joshi structural ambiguity (PP-attachment) CS 562 - Intro (part 2) 13

  14. More about Ambiguities • to middle school kids: what does this sentence mean? I eat sushi with tuna. Aravind Joshi lexical ambiguity (word-sense) CS 562 - Intro (part 2) 14

  15. More about Ambiguities • to middle school kids: what does this sentence mean? Everybody loves somebody. Aravind Joshi ??? structural ambiguity (quantifier scope) CS 562 - Intro (part 2) 15

  16. More about Ambiguities • to middle school kids: what does this sentence mean? Buffalo buffalo Buffalo buffalo buffalo buffalo Buffalo buffalo Aravind Joshi Dogs dogs dog dog dogs. Police police police police police http://www.cse.buffalo.edu/~rapaport/BuffaloBuffalo/buffalobuffalo.html CS 562 - Intro (part 2) 16

  17. Ambiguities in Translation zi zhu zhong duan 自 助 终 端 self help terminal device CS 562 - Intro (part 2) 17

  18. Ambiguities in Translation CS 562 - Intro (part 2) 18

  19. or even... clear evidence that NLP is used in real life! CS 562 - Intro (part 2) 19

  20. Ambiguity Explosion I saw her duck. ... • how about... • I saw her duck with a telescope. • I saw her duck with a telescope in the garden... CS 562 - Intro (part 2) 20

  21. Ambiguity Explosion • exponential explosion of the search space • Q1: how to represent ambiguities (compactly)? • Q2: how to search over this space (efficiently)? • Q3: how to rank different hypotheses? S NP VP PRP VBD NP PP I saw PRP$ NN IN NP her duck with DT NN .. a telescope CS 562 - Intro (part 2) 21

  22. Answers... • Q1: how to represent ambiguities? • context-free grammar (unit 2) S • finite-state automata (unit I) NP VP • Q2: how to search in this space? PRP VBD NP PP I saw PRP$ NN IN NP • dynamic programming (units 1&2) her duck with DT NN • Q3: how to rank these hypotheses? a telescop • weighted grammar (units 1-3) • weights learned from data • (saw, with, telescope) seen more often in texts CS 562 - Intro (part 2) 22

  23. Why Learning? • learning is better than hand-written rules, because: • less work; easily adapts to new languages/domains • Powerset (now bing.com): 15 years for English grammar! • now they are writing their Chinese grammar... • and languages constantly change! • learning can work, and often works better! • machine translation: used to be dominated by rule-based • now statistical methods are better: google vs. systran • google learns from the web, and translates 40+ langs [also CS 567, Machine Learning, Fall 2009] CS 562 - Intro (part 2) 23

  24. Example - Rosetta Stone • the most famous (tri-)parallel text • machines can do the same job! (if given parallel text) • UN/EU/Ca proceedings, News, tech docs, ... CS 562 - Intro (part 2) 24

  25. A sci - fi example ( Knight, 1997 ) Y our assignment: translate this Centauri sentence into Arcturan farok crrrok hihok yorok clok kantok ok - yurp

  26. farok crrrok hihok yorok clok kantok ok - yurp 1c. ok - voon ororok sprok . 7c. lalok farok ororok lalok sprok izok enemok . 1a. at - voon bichat dat . 7a. wat jjat bichat wat dat vat eneat . 2c. ok - drubel ok - voon anok plok sprok . 8c. lalok brok anok plok nok . 2a. at - drubel at - voon pippat rrat dat . 8a. iat lat pippat rrat nnat . 3c. erok sprok izok hihok ghirok . 9c. wiwok nok izok kantok ok - yurp . 3a. totat dat arrat vat hilat . 9a. totat nnat quat oloat at - yurp . 4c. ok - voon anok drok brok jok . 10c. lalok mok nok yorok ghirok clok . 4a. at - voon krat pippat sat lat . 10a. wat nnat gat mat bat hilat . 5c. wiwok farok izok stok . 11c. lalok nok crrrok hihok yorok zanzanok . 5a. totat jjat quat cat . 11a. wat nnat arrat mat zanzanat . 6c. lalok sprok izok jok stok . 12c. lalok rarok nok izok hihok mok . 6a. wat dat krat quat cat . 12a. wat nnat forat arrat vat gat . ( Knight,1997 )

  27. farok crrrok hihok yorok clok kantok ok - yurp 1c. ok - voon ororok sprok . 7c. lalok farok ororok lalok sprok izok enemok . 1a. at - voon bichat dat . 7a. wat jjat bichat wat dat vat eneat . 2c. ok - drubel ok - voon anok plok sprok . 8c. lalok brok anok plok nok . 2a. at - drubel at - voon pippat rrat dat . 8a. iat lat pippat rrat nnat . 3c. erok sprok izok hihok ghirok . 9c. wiwok nok izok kantok ok - yurp . 3a. totat dat arrat vat hilat . 9a. totat nnat quat oloat at - yurp . 4c. ok - voon anok drok brok jok . 10c. lalok mok nok yorok ghirok clok . 4a. at - voon krat pippat sat lat . 10a. wat nnat gat mat bat hilat . 5c. wiwok farok izok stok . 11c. lalok nok crrrok hihok yorok zanzanok . 5a. totat jjat quat cat . 11a. wat nnat arrat mat zanzanat . 6c. lalok sprok izok jok stok . 12c. lalok rarok nok izok hihok mok . 6a. wat dat krat quat cat . 12a. wat nnat forat arrat vat gat . ( Knight,1997 )

  28. farok crrrok hihok yorok clok kantok ok - yurp 1c. ok - voon ororok sprok . 7c. lalok farok ororok lalok sprok izok enemok . 1a. at - voon bichat dat . 7a. wat jjat bichat wat dat vat eneat . 2c. ok - drubel ok - voon anok plok sprok . 8c. lalok brok anok plok nok . 2a. at - drubel at - voon pippat rrat dat . 8a. iat lat pippat rrat nnat . 3c. erok sprok izok hihok ghirok . 9c. wiwok nok izok kantok ok - yurp . 3a. totat dat arrat vat hilat . 9a. totat nnat quat oloat at - yurp . 4c. ok - voon anok drok brok jok . 10c. lalok mok nok yorok ghirok clok . 4a. at - voon krat pippat sat lat . 10a. wat nnat gat mat bat hilat . 5c. wiwok farok izok stok . 11c. lalok nok crrrok hihok yorok zanzanok . 5a. totat jjat quat cat . 11a. wat nnat arrat mat zanzanat . 6c. lalok sprok izok jok stok . 12c. lalok rarok nok izok hihok mok . 6a. wat dat krat quat cat . 12a. wat nnat forat arrat vat gat . ( Knight,1997 )

  29. farok crrrok hihok yorok clok kantok ok - yurp 1c. ok - voon ororok sprok . 7c. lalok farok ororok lalok sprok izok enemok . 1a. at - voon bichat dat . 7a. wat jjat bichat wat dat vat eneat . 2c. ok - drubel ok - voon anok plok sprok . 8c. lalok brok anok plok nok . 2a. at - drubel at - voon pippat rrat dat . 8a. iat lat pippat rrat nnat . 3c. erok sprok izok hihok ghirok . 9c. wiwok nok izok kantok ok - yurp . 3a. totat dat arrat vat hilat . 9a. totat nnat quat oloat at - yurp . 4c. ok - voon anok drok brok jok . 10c. lalok mok nok yorok ghirok clok . 4a. at - voon krat pippat sat lat . 10a. wat nnat gat mat bat hilat . 5c. wiwok farok izok stok . 11c. lalok nok crrrok hihok yorok zanzanok . 5a. totat jjat quat cat . 11a. wat nnat arrat mat zanzanat . 6c. lalok sprok izok jok stok . 12c. lalok rarok nok izok hihok mok . 6a. wat dat krat quat cat . 12a. wat nnat forat arrat vat gat . ( Knight,1997 )

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Assignment: Named Entity Recognition Empirical Methods in Natural Language Processing Philipp

Assignment: Named Entity Recognition Empirical Methods in Natural Language Processing Philipp

Assignment: Named Entity Recognition Empirical Methods in Natural Language Processing Philipp Koehn and Annette Leonhard 29 January 2007 based on the 2006 slides by Sebastian Riedel Outline Introduction 1. Information Extraction

449 views • 15 slides
Empirical Methods in Natural Language Processing Lecture 2 Introduction (II) Probability and

Empirical Methods in Natural Language Processing Lecture 2 Introduction (II) Probability and

Empirical Methods in Natural Language Processing Lecture 2 Introduction (II) Probability and Information Theory Philipp Koehn Lecture given by Tommy Herbert 10 January 2008 PK EMNLP 10 January 2008 1 Recap Given word counts we can

632 views • 10 slides
Empirical Methods in Natural Language Processing Lecture 4 Language Modeling (II): Smoothing and

Empirical Methods in Natural Language Processing Lecture 4 Language Modeling (II): Smoothing and

Empirical Methods in Natural Language Processing Lecture 4 Language Modeling (II): Smoothing and Back-Off Philipp Koehn 17 January 2008 PK EMNLP 17 January 2008 1 Language Modeling Example there is a big house i buy a house Training

628 views • 23 slides
Empirical Methods in Natural Language Processing Lecture 4 Language Modeling (II): Smoothing and

Empirical Methods in Natural Language Processing Lecture 4 Language Modeling (II): Smoothing and

Empirical Methods in Natural Language Processing Lecture 4 Language Modeling (II): Smoothing and Back-Off Philipp Koehn 17 January 2008 PK EMNLP 17 January 2008 1 Language Modeling Example there is a big house i buy a house Training

295 views • 12 slides
Natural Language Processing Lecture 11/13/2015 CSCI 5832 Susan W. Brown Natural Language

Natural Language Processing Lecture 11/13/2015 CSCI 5832 Susan W. Brown Natural Language

Natural Language Processing Lecture 11/13/2015 CSCI 5832 Susan W. Brown Natural Language Processing Were going to study what goes into getting computers to perform useful and interesting tasks involving human language. Speech and

464 views • 24 slides
IN5550: Neural Methods in Natural Language Processing IN5550 Neural Methods in Natural

IN5550: Neural Methods in Natural Language Processing IN5550 Neural Methods in Natural

IN5550: Neural Methods in Natural Language Processing IN5550 Neural Methods in Natural Language Processing Final Exam: Task overview Stephan Oepen, Lilja vrelid, Vinit Ravishankar & Erik Velldal University of Oslo April 25, 2019

383 views • 37 slides
CSCI 5832 Natural Language Processing Lecture 21 Jim Martin 4/24/07 CSCI 5832 Spring 2007 1

CSCI 5832 Natural Language Processing Lecture 21 Jim Martin 4/24/07 CSCI 5832 Spring 2007 1

CSCI 5832 Natural Language Processing Lecture 21 Jim Martin 4/24/07 CSCI 5832 Spring 2007 1 Today: 4/10 Compositional Semantics Syntax-driven methods of assigning semantics to sentences 4/24/07 CSCI 5832 Spring 2007 2 1 Meaning

615 views • 26 slides
IN5550: Neural Methods in Natural Language Processing IN5550 Neural Methods in Natural

IN5550: Neural Methods in Natural Language Processing IN5550 Neural Methods in Natural

IN5550: Neural Methods in Natural Language Processing IN5550 Neural Methods in Natural Language Processing Recurrent Neural Networks Lilja vrelid & Stephan Oepen University of Oslo March 14, 2019 Obligatory assignment 3

884 views • 27 slides
Natural Language Processing CSCI 4152/6509 Lecture 2 Introduction to Natural Language

Natural Language Processing CSCI 4152/6509 Lecture 2 Introduction to Natural Language

Natural Language Processing CSCI 4152/6509 Lecture 2 Introduction to Natural Language Processing Instructor: Vlado Keselj Time and date: 09:3510:25, 9-Jan-2020 Location: Dunn 135 CSCI 4152/6509, Vlado Keselj Lecture 2 1 / 8 Previous

320 views • 8 slides
Empirical Methods in Natural Language Processing Lecture 6 Tagging (II): Transformation-Based

Empirical Methods in Natural Language Processing Lecture 6 Tagging (II): Transformation-Based

Empirical Methods in Natural Language Processing Lecture 6 Tagging (II): Transformation-Based Learning and Maximum Entropy Models Philipp Koehn 24 January 2008 PK EMNLP 24 January 2008 1 Tagging as supervised learning Tagging is a

326 views • 9 slides
IN5550: Neural Methods in Natural Language Processing IN5550 Neural Methods in Natural

IN5550: Neural Methods in Natural Language Processing IN5550 Neural Methods in Natural

IN5550: Neural Methods in Natural Language Processing IN5550 Neural Methods in Natural Language Processing Transformers Jeremy Barnes University of Oslo March 31, 2020 Attention - tl;dr Pay attention to a weighted combination of input

552 views • 36 slides
L18 July 17, 2017 1 Lecture 18: Natural Language Processing II CSCI 1360E: Foundations for

L18 July 17, 2017 1 Lecture 18: Natural Language Processing II CSCI 1360E: Foundations for

L18 July 17, 2017 1 Lecture 18: Natural Language Processing II CSCI 1360E: Foundations for Informatics and Analytics 1.1 Overview and Objectives Last week, we introduced the concept of natural language processing, and in particular the

454 views • 19 slides
L16 July 18, 2018 1 Lecture 16: Natural Language Processing II CSCI 1360E: Foundations for

L16 July 18, 2018 1 Lecture 16: Natural Language Processing II CSCI 1360E: Foundations for

L16 July 18, 2018 1 Lecture 16: Natural Language Processing II CSCI 1360E: Foundations for Informatics and Analytics 1.1 Overview and Objectives Last week, we introduced the concept of natural language processing, and in particular the bag

572 views • 20 slides
Empirical Methods in Natural Language Processing Lecture 10 Parsing (II): Probabilistic parsing

Empirical Methods in Natural Language Processing Lecture 10 Parsing (II): Probabilistic parsing

Empirical Methods in Natural Language Processing Lecture 10 Parsing (II): Probabilistic parsing models Philipp Koehn 7 February 2008 Philipp Koehn EMNLP Lecture 10 7 February 2008 1 Parsing Task: build the syntactic tree for a sentence

471 views • 13 slides
Empirical Methods in Natural Language Processing Lecture 10 Parsing (II): Probabilistic parsing

Empirical Methods in Natural Language Processing Lecture 10 Parsing (II): Probabilistic parsing

Empirical Methods in Natural Language Processing Lecture 10 Parsing (II): Probabilistic parsing models Philipp Koehn 7 February 2008 Philipp Koehn EMNLP Lecture 10 7 February 2008 1 Parsing Task: build the syntactic tree for a sentence

710 views • 25 slides
Empirical Methods in Natural Language Processing Lecture 1 Introduction (I): Words and

Empirical Methods in Natural Language Processing Lecture 1 Introduction (I): Words and

Empirical Methods in Natural Language Processing Lecture 1 Introduction (I): Words and Probability Philipp Koehn Lecture given by Tommy Herbert 7 January 2008 PK EMNLP 7 January 2008 1 Welcome to EMNLP Lecturer: Philipp Koehn TA:

309 views • 9 slides
Loo ooking g into Ma o Malicious I Insider ers JPCERT/CC Koichiro Sparky Komiyama First

Loo ooking g into Ma o Malicious I Insider ers JPCERT/CC Koichiro Sparky Komiyama First

Loo ooking g into Ma o Malicious I Insider ers JPCERT/CC Koichiro Sparky Komiyama First Conference, Vienna Agenda Background Previous work Information leakage by malicious insider Our Research How to prevent 2 Insider

335 views • 30 slides
Evaluating the Impact of Officer Worn Body Cameras in the Phoenix Police Department Charles Katz,

Evaluating the Impact of Officer Worn Body Cameras in the Phoenix Police Department Charles Katz,

Evaluating the Impact of Officer Worn Body Cameras in the Phoenix Police Department Charles Katz, Mike Kurtenbach, David Choate, Justin Ready December 10, 2014 This project was supported by Grant No. 2013-DP-BX-K006 awarded by the Bureau of

693 views • 33 slides
Mine and Mineral Processing Virtual Workshop Se Sessi ssion 1 n 1 - Si Site e Ch

Mine and Mineral Processing Virtual Workshop Se Sessi ssion 1 n 1 - Si Site e Ch

Welcome to the CLU-IN Internet Seminar! EP EPA A Office e of R f Res esea earch h and nd Dev evel elopment pment's s Office e of S f Scienc ence e Policy y Mine and Mineral Processing Virtual Workshop Se Sessi ssion 1 n 1 - Si

509 views • 6 slides
NAT64 Operational Considerations draft-chen-v6ops-nat64-cpe-03.txt IETF 82- Taipei, Nov 2011 Gang

NAT64 Operational Considerations draft-chen-v6ops-nat64-cpe-03.txt IETF 82- Taipei, Nov 2011 Gang

NAT64 Operational Considerations draft-chen-v6ops-nat64-cpe-03.txt IETF 82- Taipei, Nov 2011 Gang Chen (chengang@chinamobile.com) Qibo Niu(niu.qibo@zte.com.cn) Changes since IETF#81 (-02 to -03) Added different deployment considerations

539 views • 5 slides
Explaining the Credibility of Emerging Claims on the Web and Social Media Kashyap Popat,

Explaining the Credibility of Emerging Claims on the Web and Social Media Kashyap Popat,

Where the Truth Lies : Explaining the Credibility of Emerging Claims on the Web and Social Media Kashyap Popat, Subhabrata Mukherjee, Jannik Strtgen, Gerhard Weikum WWW 2017 M OTIVATION Rapid spread of misinformation online"

393 views • 24 slides
and its Impact on Real Lives Christine Kenneally 14th International Digital Curation Conference

and its Impact on Real Lives Christine Kenneally 14th International Digital Curation Conference

Data, the Creation of History and its Impact on Real Lives Christine Kenneally 14th International Digital Curation Conference 5 February 2019 History and Real Lives How does history shape us? How do we know what we know? HOW WHAT WE

497 views • 24 slides
CS 4518 Mobile and Ubiquitous Computing Lecture 6: Databases, Camera, Face Detection Emmanuel

CS 4518 Mobile and Ubiquitous Computing Lecture 6: Databases, Camera, Face Detection Emmanuel

CS 4518 Mobile and Ubiquitous Computing Lecture 6: Databases, Camera, Face Detection Emmanuel Agu Administrivia Project 2 Emailed out last week Should be done in groups of 5 or 6 Due this Thursday, 11.59PM Can be done on

653 views • 62 slides
W HO IS A S TAKEHOLDER ? Technical Definition: Someone that sends or receives

W HO IS A S TAKEHOLDER ? Technical Definition: Someone that sends or receives

9/10/2015 K AUAI ITS A RCHITECTURE O PERATIONAL C ONCEPT W ORKSHOP 1 September 9, 2015 A GENDA Introductions and Welcome Stakeholders Review Operational Concepts Elements Projects / Future Efforts Agreements Customized

501 views • 18 slides

More recommend