learning dependency based compositional semantics
play

Learning Dependency-Based Compositional Semantics Semantic - PowerPoint PPT Presentation

Aug 26, 2022 •300 likes •1.74k views

Learning Dependency-Based Compositional Semantics Semantic Representations for Textual Inference Workshop Mar. 10, 2012 Percy Liang Google/Stanford joint work with Michael Jordan and Dan Klein Motivating Problem: Question Answering 2

  1. Basic DCS Trees DCS tree Constraints Database city city c ∈ city San Francisco 1 Chicago c 1 = ` 1 Boston 1 · · · ` ∈ loc loc 2 loc ` 2 = s 1 Mount Shasta California 1 San Francisco California CA s ∈ CA Boston Massachusetts · · · · · · CA California A DCS tree encodes a constraint satisfaction problem (CSP) 14

  2. Basic DCS Trees DCS tree Constraints Database city city c ∈ city San Francisco 1 Chicago c 1 = ` 1 Boston 1 · · · ` ∈ loc loc 2 loc ` 2 = s 1 Mount Shasta California 1 San Francisco California CA s ∈ CA Boston Massachusetts · · · · · · CA California A DCS tree encodes a constraint satisfaction problem (CSP) 14

  3. Basic DCS Trees DCS tree Constraints Database city c ∈ city city San Francisco 1 Chicago c 1 = ` 1 Boston 1 · · · ` ∈ loc loc 2 loc ` 2 = s 1 Mount Shasta California 1 San Francisco California s ∈ CA CA Boston Massachusetts · · · · · · CA California A DCS tree encodes a constraint satisfaction problem (CSP) Computation : dynamic programming ⇒ time = O ( # nodes ) 14

  4. Properties of DCS Trees city 1 1 1 2 loc traverse 2 1 1 1 state river 1 1 1 1 1 1 border major traverse 2 2 1 1 CA AZ 15

  5. Properties of DCS Trees city 1 1 1 2 loc traverse 2 1 1 1 state river 1 1 1 1 1 1 border major traverse 2 2 1 1 CA AZ Trees 15

  6. Properties of DCS Trees city 1 1 1 2 loc traverse 2 1 1 1 state river 1 1 1 1 1 1 border major traverse 2 2 1 1 CA AZ Linguistics Trees syntactic locality 15

  7. Properties of DCS Trees city 1 1 1 2 loc traverse 2 1 1 1 state river 1 1 1 1 1 1 border major traverse 2 2 1 1 CA AZ Linguistics Trees Computation syntactic locality e ffi cient interpretation 15

  8. Divergence between Syntactic and Semantic Scope most populous city in California 16

  9. Divergence between Syntactic and Semantic Scope most populous city in California Syntax city populous in most California 16

  10. Divergence between Syntactic and Semantic Scope most populous city in California Syntax Semantics city argmax ( λ x. city ( x ) ∧ loc ( x, CA ) , λ x. population ( x )) populous in most California 16

  11. Divergence between Syntactic and Semantic Scope most populous city in California Syntax Semantics city argmax ( λ x. city ( x ) ∧ loc ( x, CA ) , λ x. population ( x )) populous in most California 16

  12. Divergence between Syntactic and Semantic Scope most populous city in California Syntax Semantics city argmax ( λ x. city ( x ) ∧ loc ( x, CA ) , λ x. population ( x )) populous in most California Problem: syntactic scope is lower than semantic scope 16

  13. Divergence between Syntactic and Semantic Scope most populous city in California Syntax Semantics city argmax ( λ x. city ( x ) ∧ loc ( x, CA ) , λ x. population ( x )) populous in most California Problem: syntactic scope is lower than semantic scope If DCS trees look like syntax, how do we get correct semantics? 16

  14. Solution: Mark-Execute most populous city in California Superlatives ∗∗ x 1 x 1 city 1 1 1 1 population loc 2 c 1 argmax CA 17

  15. Solution: Mark-Execute most populous city in California Superlatives ∗∗ x 1 x 1 city 1 1 1 1 population loc Mark at syntactic scope 2 c 1 argmax CA 17

  16. Solution: Mark-Execute most populous city in California Superlatives ∗∗ Execute at semantic scope x 1 x 1 city 1 1 1 1 population loc Mark at syntactic scope 2 c 1 argmax CA 17

  17. Solution: Mark-Execute Alaska borders no states. Negation ∗∗ Execute at semantic scope x 1 x 1 border 1 2 1 1 AK state Mark at syntactic scope q no 17

  18. Solution: Mark-Execute Some river traverses every city. Quantification (narrow) ∗∗ Execute at semantic scope x 12 x 12 traverse 1 2 1 1 river city Mark at syntactic scope q q some every 17

  19. Solution: Mark-Execute Some river traverses every city. Quantification (wide) ∗∗ Execute at semantic scope x 21 x 21 traverse 1 2 1 1 river city Mark at syntactic scope q q some every 17

  20. Solution: Mark-Execute Some river traverses every city. Quantification (wide) ∗∗ Execute at semantic scope x 21 x 21 traverse 1 2 1 1 river city Mark at syntactic scope q q some every Analogy: Montague’s quantifying in, Carpenter’s scoping constructor 17

  21. Outline city 1 1 1 2 loc traverse 2 1 1 1 Representation state river 1 1 1 1 1 1 border major traverse 2 2 1 1 CA AZ x Learning θ z y w Experiments 18

  22. Graphical Model ∗∗ 1 2 z capital 1 1 CA database w 19

  23. Graphical Model ∗∗ 1 2 z capital 1 1 CA database Sacramento y w 19

  24. Graphical Model ∗∗ 1 2 z capital 1 1 Interpretation : p ( y | z, w ) CA (deterministic) database Sacramento y w 19

  25. Graphical Model capital of x California? ∗∗ 1 2 z capital 1 1 Interpretation : p ( y | z, w ) CA (deterministic) database Sacramento y w 19

  26. Graphical Model capital of x California? ∗∗ parameters 1 2 θ z capital 1 1 Interpretation : p ( y | z, w ) CA (deterministic) database Sacramento y w 19

  27. Graphical Model capital of x California? Semantic Parsing : p ( z | x, θ ) (probabilistic) ∗∗ parameters 1 2 θ z capital 1 1 Interpretation : p ( y | z, w ) CA (deterministic) database Sacramento y w 19

  28. Plan capital of x California? • What’s possible ? z ∈ Z ( x ) ∗∗ 1 parameters 2 • What’s probable ? p ( z | x, θ ) θ z capital 1 1 • Learning θ from ( x, y ) data CA database Sacramento y w 20

  29. Words to Predicates (Lexical Semantics) What is the most populous city in CA ? 21

  30. Words to Predicates (Lexical Semantics) CA What is the most populous city in CA ? Lexical Triggers: 1. String match CA ⇒ CA 21

  31. Words to Predicates (Lexical Semantics) argmax CA What is the most populous city in CA ? Lexical Triggers: 1. String match CA ⇒ CA 2. Function words (20 words) most ⇒ argmax 21

  32. Words to Predicates (Lexical Semantics) city city state state river river argmax population population CA What is the most populous city in CA ? Lexical Triggers: 1. String match CA ⇒ CA 2. Function words (20 words) most ⇒ argmax 3. Nouns/adjectives city ⇒ city state river population 21

  33. Predicates to DCS Trees (Compositional Semantics) C i,j = set of DCS trees for span [ i, j ] i j most populous city in California 22

  34. Predicates to DCS Trees (Compositional Semantics) C i,j = set of DCS trees for span [ i, j ] i k j most populous city in California 22

  35. Predicates to DCS Trees (Compositional Semantics) C i,j = set of DCS trees for span [ i, j ] C i,k C k,j i k j most populous city in California 22

  36. Predicates to DCS Trees (Compositional Semantics) C i,j = set of DCS trees for span [ i, j ] population city 1 c 1 argmax loc 2 1 CA C i,k C k,j i k j most populous city in California 22

  37. Predicates to DCS Trees (Compositional Semantics) C i,j = set of DCS trees for span [ i, j ] city 1 1 1 1 population loc 2 population city c 1 1 c argmax CA 1 argmax loc 2 1 CA C i,k C k,j i k j most populous city in California 22

  38. Predicates to DCS Trees (Compositional Semantics) C i,j = set of DCS trees for span [ i, j ] city 1 1 2 1 population loc 2 population city c 1 1 c argmax CA 1 argmax loc 2 1 CA C i,k C k,j i k j most populous city in California 22

  39. Predicates to DCS Trees (Compositional Semantics) C i,j = set of DCS trees for span [ i, j ] city 1 1 1 1 loc loc 2 2 population city 1 1 1 c population CA 1 argmax loc c 2 argmax 1 CA C i,k C k,j i k j most populous city in California 22

  40. Predicates to DCS Trees (Compositional Semantics) C i,j = set of DCS trees for span [ i, j ] city 1 1 2 1 loc loc 1 2 population city 1 1 1 c population CA 1 argmax loc c 2 argmax 1 CA C i,k C k,j i k j most populous city in California 22

  41. Predicates to DCS Trees (Compositional Semantics) C i,j = set of DCS trees for span [ i, j ] city 1 1 2 1 border loc 1 2 population city 1 1 1 c population CA 1 argmax loc c 2 argmax 1 CA C i,k C k,j i k j most populous city in California 22

  42. Predicates to DCS Trees (Compositional Semantics) C i,j = set of DCS trees for span [ i, j ] population 1 c 1 argmax city 1 population city 1 1 c loc 1 2 argmax loc 1 2 CA 1 CA C i,k C k,j i k j most populous city in California 22

  43. Plan capital of x California? • What’s possible ? z ∈ Z ( x ) ∗∗ 1 parameters 2 • What’s probable ? p ( z | x, θ ) θ z capital 1 1 • Learning θ from ( x, y ) data CA database Sacramento y w 23

  44. Log-linear Model 1 1 2 1 z : city city loc CA x : city in California 24

  45. Log-linear Model 1 1 2 1 z : city city loc CA x : city in California features ( x, z ) = ( ) ∈ R d 24

  46. Log-linear Model 1 1 2 1 z : city city loc CA x : city in California features ( x, z ) = ( ) in : 1 loc ∈ R d 24

  47. Log-linear Model 1 1 2 1 z : city city loc CA x : city in California features ( x, z ) = ( : 1 ) in : 1 loc ∈ R d 1 1 loc city 24

  48. Log-linear Model 1 1 2 1 z : city city loc CA x : city in California features ( x, z ) = ( ) in : 1 loc ∈ R d : 1 1 1 loc city · · · 24

  49. Log-linear Model 1 1 2 1 z : city city loc CA x : city in California features ( x, z ) = ( ) in : 1 loc ∈ R d : 1 1 1 loc city · · · score ( x, z ) = features ( x, z ) · θ 24

  50. Log-linear Model 1 1 2 1 z : city city loc CA x : city in California features ( x, z ) = ( ) in : 1 loc ∈ R d : 1 1 1 loc city · · · score ( x, z ) = features ( x, z ) · θ e score ( x,z ) p ( z | x, θ ) = z 02 Z ( x ) e score ( x,z 0 ) P 24

  51. Plan capital of x California? • What’s possible ? z ∈ Z ( x ) ∗∗ 1 parameters 2 • What’s probable ? p ( z | x, θ ) θ z capital 1 1 • Learning θ from ( x, y ) data CA database Sacramento y w 25

  52. Learning Objective Function: p ( y | z, w ) p ( z | x, θ ) Interpretation Semantic parsing 26

  53. Learning Objective Function: max θ p ( y | z, w ) p ( z | x, θ ) Interpretation Semantic parsing 26

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Learning Compositional Semantics for Introduction Open Domain Semantic Parsing Meaning

Learning Compositional Semantics for Introduction Open Domain Semantic Parsing Meaning

Learning Compositional Semantics Phong Le, Willem Zuidema Learning Compositional Semantics for Introduction Open Domain Semantic Parsing Meaning Representation Semantic Composition Experimental results Groningen Meaning Phong Le, Willem

683 views • 52 slides
Learning Compositional Semantics CS224U: Natural Language Understanding Feb. 9, 2012 Percy Liang

Learning Compositional Semantics CS224U: Natural Language Understanding Feb. 9, 2012 Percy Liang

Learning Compositional Semantics CS224U: Natural Language Understanding Feb. 9, 2012 Percy Liang Google/Stanford Review Last time: Mapping sentences to logical forms (FOL or lambda calculus) Alaska borders no states. x. state ( x )

1.88k views • 164 slides
SentiBERT: A Transferable Transformer-based Architecture for Compositional Sentiment Semantics Da

SentiBERT: A Transferable Transformer-based Architecture for Compositional Sentiment Semantics Da

SentiBERT: A Transferable Transformer-based Architecture for Compositional Sentiment Semantics Da Yin 1 , Tao Meng 2 , Kai-Wei Chang 2 1 Peking University 2 University of California, Los Angeles 1 Motivation Sentiment composition is

1.28k views • 13 slides
Lecture 16: More on Compositional Semantics, Verb Semantics Julia Hockenmaier

Lecture 16: More on Compositional Semantics, Verb Semantics Julia Hockenmaier

CS447: Natural Language Processing http://courses.engr.illinois.edu/cs447 Lecture 16: More on Compositional Semantics, Verb Semantics Julia Hockenmaier juliahmr@illinois.edu 3324 Siebel Center Admin Midterm: Regrade requests for midterm

713 views • 32 slides
Modeling Event Implications for Compositional Semantics Sai Qian Maxime Amblard Calligramme,

Modeling Event Implications for Compositional Semantics Sai Qian Maxime Amblard Calligramme,

Modeling Event Implications for Compositional Semantics Modeling Event Implications for Compositional Semantics Sai Qian Maxime Amblard Calligramme, LORIA & INRIA Nancy Grand-Est CAuLD Workshop: Logical Methods for Discourse December 13,

805 views • 36 slides
Synonymy in an approach to combined distributional and compositional semantics Ann Copestake and

Synonymy in an approach to combined distributional and compositional semantics Ann Copestake and

Synonymy in an approach to combined distributional and compositional semantics Synonymy in an approach to combined distributional and compositional semantics Ann Copestake and Aurlie Herbelot Computer Laboratory University of Cambridge

736 views • 21 slides
Mapping between Dependency Structures and Compositional Semantic Representations LREC 2010 Max

Mapping between Dependency Structures and Compositional Semantic Representations LREC 2010 Max

Intro Source Target Mapping Evaluation Conclusion Mapping between Dependency Structures and Compositional Semantic Representations LREC 2010 Max Jakob, Mark eta Lopatkov a, Valia Kordoni UFAL at Charles University in Prague,

174 views • 14 slides
Definitions and Proofs Structural Induction Three approaches to semantics compositional

Definitions and Proofs Structural Induction Three approaches to semantics compositional

Definitions and Proofs Structural Induction Three approaches to semantics compositional definitions Prove that the property holds for all the basis elements of the syntactic cat- natural semantics egory. structural operational

443 views • 5 slides
Lecture 19: Compositional Semantics Julia Hockenmaier juliahmr@illinois.edu 3324 Siebel Center

Lecture 19: Compositional Semantics Julia Hockenmaier juliahmr@illinois.edu 3324 Siebel Center

CS447: Natural Language Processing http://courses.engr.illinois.edu/cs447 Lecture 19: Compositional Semantics Julia Hockenmaier juliahmr@illinois.edu 3324 Siebel Center Lecture 19: Compositional Semantics : 1 t r a w P e i v r

833 views • 33 slides
Dependency Parsing & Feature-based Parsing Ling571 Deep Processing Techniques for NLP

Dependency Parsing & Feature-based Parsing Ling571 Deep Processing Techniques for NLP

Dependency Parsing & Feature-based Parsing Ling571 Deep Processing Techniques for NLP February 2, 2015 Roadmap Dependency parsing Graph-based dependency parsing Maximum spanning tree CLE Algorithm Learning

899 views • 87 slides
Dependency Grammars Topological Dependency Trees: A Constraint-based Account of Linear

Dependency Grammars Topological Dependency Trees: A Constraint-based Account of Linear

Dependency Grammars Topological Dependency Trees: A Constraint-based Account of Linear Precedence Extensible Dependency Grammar: A New Methodology Sibel Ciddi NPFL106 - Linguistics 2013 Summer Framework Immediate dependency

656 views • 18 slides
Cognitive Compositional Semantics using Continuation Dependencies William Schuler, Adam Wheeler

Cognitive Compositional Semantics using Continuation Dependencies William Schuler, Adam Wheeler

Cognitive Compositional Semantics using Continuation Dependencies William Schuler, Adam Wheeler Dept Linguistics, The Ohio State University August 25, 2014 William Schuler, Adam Wheeler Compositional Semantics using Continuation Dependencies

707 views • 25 slides
A Brief Introduction to Semantics CMSC 473/673 UMBC Outline Recap: dependency grammars and

A Brief Introduction to Semantics CMSC 473/673 UMBC Outline Recap: dependency grammars and

A Brief Introduction to Semantics CMSC 473/673 UMBC Outline Recap: dependency grammars and arc-standard dependency parsing Meaning from Syntax Structured Meaning: Semantic Frames and Roles What problem do they solve? Theory Computational

998 views • 83 slides
A Compositional Trace Semantics for Orc Dimitrios Vardoulakis and Mitchell Wand College of

A Compositional Trace Semantics for Orc Dimitrios Vardoulakis and Mitchell Wand College of

A Compositional Trace Semantics for Orc Dimitrios Vardoulakis and Mitchell Wand College of Computer and Information Science Northeastern University Overview of Orc Sites: autonomous computing units that do arithmetic, printing etc

374 views • 34 slides
Compositional Semantics for Composable Continuations From Abortive to Delimited Control Paul

Compositional Semantics for Composable Continuations From Abortive to Delimited Control Paul

Compositional Semantics for Composable Continuations From Abortive to Delimited Control Paul Downen Zena M. Ariola University of Oregon ICFP14 September 1, 2014 The big picture Effects that manipulate control flow, compositionally

431 views • 31 slides
Event Knowledge in Compositional Distributional Semantics Ludovica Pannitto Master Thesis in

Event Knowledge in Compositional Distributional Semantics Ludovica Pannitto Master Thesis in

Event Knowledge in Compositional Distributional Semantics Ludovica Pannitto Master Thesis in Digital Humanities - Language Technologies, University of Pisa Supervisor: Alessandro Lenci November 20, 2019 Contents Aim: investigate the use of

741 views • 34 slides
3 John Series Lesson #016 September 14, 2003 Dean Bible Ministries www.deanbibleministries.org

3 John Series Lesson #016 September 14, 2003 Dean Bible Ministries www.deanbibleministries.org

3 John Series Lesson #016 September 14, 2003 Dean Bible Ministries www.deanbibleministries.org Dr. Robert L. Dean, Jr. Ps 37:4 Delight yourself also in the Lord, And He shall give you the desires of your heart. Ps 37:5 Commit your way to

388 views • 16 slides
1 Thessalonians Series Lesson #020 May 28, 2015 Dean Bible Ministries

1 Thessalonians Series Lesson #020 May 28, 2015 Dean Bible Ministries

1 Thessalonians Series Lesson #020 May 28, 2015 Dean Bible Ministries www.deanbibleministries.org Dr. Robert L. Dean, Jr. Delight in the Lord 1 Thessalonians 1:8 STEP ONE: Claim a promise STEP TWO: Think through the doctrinal rationales

585 views • 19 slides
Community Led Housing Colerne Community Land and Asset Trust 1 What is Community Led Housing?

Community Led Housing Colerne Community Land and Asset Trust 1 What is Community Led Housing?

Community Led Housing Colerne Community Land and Asset Trust 1 What is Community Led Housing? The criteria of community led housing is: - That the community must be integrally involved throughout the process in key decisions; Community

430 views • 18 slides
Natural Language Processing with Deep Learning CS224N/Ling284 Christopher Manning Lecture 18:

Natural Language Processing with Deep Learning CS224N/Ling284 Christopher Manning Lecture 18:

Natural Language Processing with Deep Learning CS224N/Ling284 Christopher Manning Lecture 18: Tree Recursive Neural Networks, Constituency Parsing, and Sentiment Lecture Plan: Lecture 18: Tree Recursive Neural Networks, Constituency Parsing,

878 views • 73 slides
CAREERS & SKILLS ASSEMBLY Modern Languages Modern Languages at Boroughmuir and beyond

CAREERS & SKILLS ASSEMBLY Modern Languages Modern Languages at Boroughmuir and beyond

CAREERS & SKILLS ASSEMBLY Modern Languages Modern Languages at Boroughmuir and beyond You have the chance to learn French, German, Spanish, Mandarin and Italian Everyone studies at least one language until the end of S4 From S3,

554 views • 6 slides
MODELING WITH GRAPHS Alistair Jones Neo Technology What is modeling? Model Complexity

MODELING WITH GRAPHS Alistair Jones Neo Technology What is modeling? Model Complexity

MODELING WITH GRAPHS Alistair Jones Neo Technology What is modeling? Model Complexity Simple Complex Key-Value Document Column Graph Relational This talk compares Relational Whats a graph? Social ( network

742 views • 42 slides
Logic Programming Prolog as Language Temur Kutsia Research Institute for Symbolic Computation

Logic Programming Prolog as Language Temur Kutsia Research Institute for Symbolic Computation

Logic Programming Prolog as Language Temur Kutsia Research Institute for Symbolic Computation Johannes Kepler University Linz, Austria kutsia@risc.jku.at Prolog as Language Syntax Operators Equality Arithmetic Satisfying

612 views • 30 slides
Using Race, Ethnicity, While we wait to get started We are recording this webinar.

Using Race, Ethnicity, While we wait to get started We are recording this webinar.

Using Race, Ethnicity, While we wait to get started We are recording this webinar. Language & Disability To access captioning, click on captions show subtitles . (REALD) Data to For ASL interpreter access, you can

945 views • 26 slides

More recommend