Universal Decompositional Semantic Parsing Elias Stengel-Eskin - - PowerPoint PPT Presentation

▶
universal decompositional semantic parsing
SMART_READER_LITE
LIVE PREVIEW

Universal Decompositional Semantic Parsing Elias Stengel-Eskin - - PowerPoint PPT Presentation

Apr 25, 2023 463 likes •796 views

Universal Decompositional Semantic Parsing Elias Stengel-Eskin Aaron Steven White Sheng Zhang Benjamin Van Durme ACL 2020 Aaron White Sheng Zhang Ben Van Durme University of Rochester JHU Microsoft Research Johns Hopkins University Key

slide-1
SLIDE 1

Universal Decompositional Semantic Parsing

Elias Stengel-Eskin Aaron Steven White Sheng Zhang Benjamin Van Durme ACL 2020

slide-2
SLIDE 2

Ben Van Durme Johns Hopkins University Sheng Zhang JHU Microsoft Research Aaron White University of Rochester

slide-3
SLIDE 3

Key Takeaways

First parsing model for UDS Pipeline and end- to-end prediction Unique task: graph structure + scalar values, jointly

, , ,
  • f
Hiller Taiwan the Chechnya asked Bush to name leaders , and India Pakistan
  • f
Hiller Taiwan the Chechnya asked Bush to name leaders , and India Pakistan
  • f
Hiller Taiwan the Chechnya asked Bush to name leaders , and India Pakistan Embedder Encoder Hiller SOMETHING asked Bush(1) name leaders Bush(1) to … TOP asked TOP Hiller asked Bush(1) asked SMTH asked to SMTH name SMTH Bush(1) name Subspace Attribute Val factuality genericty genericity genericity time time time factual pred-dynamic pred-hypothetical pred-particular dur-days dur-minutes dur-seconds … 1.038 1.418
  • 0.892
1.418
  • 1.062
  • 0.912
1.260 Node attribute module Subspace Attribute Val protoroles protoroles protoroles protoroles protoroles awareness change-of-loc change-of-poss change-of-state existed-before …
  • 0.110
  • 0.039
0.000
  • 0.104
1.402 Edge attribute module Encoder Relation Embedder TOP Subspace Attribute Val Subspace Attribute Val
  • f

Hiller Taiwan the Chechnya asked Bush to name leaders , , and India Pakistan

Subspace Attribute Val
  • arg. edge
predicate node argument node syntax node instance edge nonhead edge syntax edge protoroles protoroles protoroles protoroles protoroles awareness change-of-loc change-of-poss change-of-state existed-before …
  • 0.110
  • 0.039
0.000
  • 0.104
1.402 factuality genericty genericity genericity time time time factual pred-dynamic pred-hypothetical pred-particular dur-days dur-minutes dur-seconds … 1.038 1.418
  • 0.892
1.418
  • 1.062
  • 0.912
1.260 genericity genericity genericity word-sense word-sense word-sense arg-abstract arg-kind arg-particular noun.act noun.cognition noun.food …
  • 1.112
1.195
  • 1.112
  • 3.000
  • 3.000
  • 3.000
semantic head edge
slide-4
SLIDE 4

Universal Decompositional Semantics (UDS)

  • Semantics representations (often)
  • Hard to annotate
  • Brittle to non-prototypical instances
  • UDS: decompose into simple questions
  • Scalar-valued, feature-based
  • Easy to annotate
  • Flexible
slide-5
SLIDE 5

LREC 2020

slide-6
SLIDE 6
  • f

Hiller Taiwan the Chechnya asked Bush to name leaders , , and India Pakistan

slide-7
SLIDE 7
  • f

Hiller Taiwan the Chechnya asked Bush to name leaders , , and India Pakistan

syntax node syntax edge

slide-8
SLIDE 8
  • f

Hiller Taiwan the Chechnya asked Bush to name leaders , , and India Pakistan

  • arg. edge

predicate node argument node syntax node syntax edge semantic head edge

event participant relation

slide-9
SLIDE 9
  • f

Hiller Taiwan the Chechnya asked Bush to name leaders , , and India Pakistan

  • arg. edge

predicate node argument node syntax node instance edge nonhead edge syntax edge semantic head edge

event participant relation

slide-10
SLIDE 10

Subspace Attribute Val Subspace Attribute Val

  • f

Hiller Taiwan the Chechnya asked Bush to name leaders , , and India Pakistan

Subspace Attribute Val

  • arg. edge

predicate node argument node syntax node instance edge nonhead edge syntax edge

protoroles protoroles protoroles protoroles protoroles awareness change-of-loc change-of-poss change-of-state existed-before …
  • 0.110
  • 0.039
0.000
  • 0.104
1.402 factuality genericty genericity genericity time time time factual pred-dynamic pred-hypothetical pred-particular dur-days dur-minutes dur-seconds … 1.038 1.418
  • 0.892
1.418
  • 1.062
  • 0.912
1.260 genericity genericity genericity word-sense word-sense word-sense arg-abstract arg-kind arg-particular noun.act noun.cognition noun.food …
  • 1.112
1.195
  • 1.112
  • 3.000
  • 3.000
  • 3.000

semantic head edge

slide-11
SLIDE 11

Universal Decompositional Semantics

  • Factuality
  • 3

3 Jo left Jo didn’t leave Cole thought that Jo had left

slide-12
SLIDE 12

Universal Decompositional Semantics

  • Factuality
  • Genericity
  • E.g. pred-particular
  • 3

3 I ate pizza today I ate pizza every day

slide-13
SLIDE 13

Universal Decompositional Semantics

  • Factuality
  • Genericity
  • Time
  • E.g. dur-minutes
  • 3

3 Tom was singing Tom left

slide-14
SLIDE 14

Universal Decompositional Semantics

  • Factuality
  • Genericity
  • Time
  • Wordsense
  • E.g. sup.person
  • 3

3 Sandy led Rufus by a leash Sandy led Rufus by a leash

slide-15
SLIDE 15

Universal Decompositional Semantics

  • Factuality
  • Genericity
  • Time
  • Wordsense
  • Semantic proto-roles
  • E.g. volition
  • 3

3 Derek broke the wishbone Derek broke his arm

slide-16
SLIDE 16

Why UDS?

  • Annotation flexibility
  • Crowdsourced
  • Simple questions
  • Rich meaning representation
  • Flexible inferences
  • Richer analysis

🤕

Hmmm, Patient? Or Theme?

😋

Was the participant changed during the event?

slide-17
SLIDE 17

What’s Transductive Parsing?

  • f

Hiller Taiwan the Chechnya asked Bush to name leaders , , and India Pakistan

, , ,
  • f
Hiller Taiwan the Chechnya asked Bush to name leaders , and India Pakistan
  • f
Hiller Taiwan the Chechnya asked Bush to name leaders , and India Pakistan
  • f
Hiller Taiwan the Chechnya asked Bush to name leaders , and India Pakistan

Embedder Encoder

Hiller SOMETHING asked Bush(1) name leaders Bush(1) to … TOP asked TOP Hiller asked Bush(1) asked SMTH asked to SMTH name SMTH Bush(1) name Subspace Attribute Val factuality genericty genericity genericity time time time factual pred-dynamic pred-hypothetical pred-particular dur-days dur-minutes dur-seconds … 1.038 1.418
  • 0.892
1.418
  • 1.062
  • 0.912
1.260

Node attribute module

Subspace Attribute Val protoroles protoroles protoroles protoroles protoroles awareness change-of-loc change-of-poss change-of-state existed-before …
  • 0.110
  • 0.039
0.000
  • 0.104
1.402

Edge attribute module Encoder Relation Embedder

TOP
slide-18
SLIDE 18

Arborescence

asked Hiller Bush(1) name leaders the of Che. Tai. Ind. and Pak. SOMETHING to Bush(1)

Copy re-entrant nodes

  • f

Hiller Taiwan the Chechnya asked Bush to name leaders , , and India Pakistan

  • arg. edge
predicate node argument node syntax node instance edge nonhead edge syntax edge semantic head edge

Flatten syntax graphs Explicitly represent embedded predicates Assign head labels

slide-19
SLIDE 19

Linearization

asked Hiller Bush(1) name leaders the of Che. Tai. Ind. and Pak. SOMETHING to Bush(1) asked Hiller SOMETHING Bush(1) name leaders Bush(1) to …

Pre-order linearization

slide-20
SLIDE 20

, , ,

  • f

Hiller Taiwan the Chechnya asked Bush to name leaders , and India Pakistan

  • f

Hiller Taiwan the Chechnya asked Bush to name leaders , and India Pakistan

  • f

Hiller Taiwan the Chechnya asked Bush to name leaders , and India Pakistan

Embedder Encoder

asked TOP

Encoder Relation Embedder

TOP

slide-21
SLIDE 21

, , ,

  • f

Hiller Taiwan the Chechnya asked Bush to name leaders , and India Pakistan

  • f

Hiller Taiwan the Chechnya asked Bush to name leaders , and India Pakistan

  • f

Hiller Taiwan the Chechnya asked Bush to name leaders , and India Pakistan

Embedder Encoder

Hiller asked TOP

Encoder Relation Embedder

TOP asked TOP

slide-22
SLIDE 22

, , ,

  • f

Hiller Taiwan the Chechnya asked Bush to name leaders , and India Pakistan

  • f

Hiller Taiwan the Chechnya asked Bush to name leaders , and India Pakistan

  • f

Hiller Taiwan the Chechnya asked Bush to name leaders , and India Pakistan

Embedder Encoder

Hiller asked TOP

Encoder Relation Embedder

TOP asked TOP

slide-23
SLIDE 23

, , ,

  • f

Hiller Taiwan the Chechnya asked Bush to name leaders , and India Pakistan

  • f

Hiller Taiwan the Chechnya asked Bush to name leaders , and India Pakistan

  • f

Hiller Taiwan the Chechnya asked Bush to name leaders , and India Pakistan

Embedder Encoder

Hiller SOMETHING asked Bush(1) name leaders Bush(1) to … TOP asked TOP Hiller asked Bush(1) asked SMTH asked to SMTH name SMTH Bush(1) name

Encoder Relation Embedder

TOP

slide-24
SLIDE 24

, , ,

  • f

Hiller Taiwan the Chechnya asked Bush to name leaders , and India Pakistan

  • f

Hiller Taiwan the Chechnya asked Bush to name leaders , and India Pakistan

  • f

Hiller Taiwan the Chechnya asked Bush to name leaders , and India Pakistan

Embedder Encoder

Hiller SOMETHING asked Bush(1) name leaders Bush(1) to … TOP asked TOP Hiller asked Bush(1) asked SMTH asked to SMTH name SMTH Bush(1) name

Subspace Attribute Val factuality genericty genericity genericity time time time factual pred-dynamic pred-hypothetical pred-particular dur-days dur-minutes dur-seconds … 1.038 1.418
  • 0.892
1.418
  • 1.062
  • 0.912
1.260

Node attribute module

Subspace Attribute Val protoroles protoroles protoroles protoroles protoroles awareness change-of-loc change-of-poss change-of-state existed-before …
  • 0.110
  • 0.039
0.000
  • 0.104
1.402

Edge attribute module Encoder Relation Embedder

TOP

slide-25
SLIDE 25

Evaluation Metrics

  • S score (Zhang et al. 2016)
  • Extension of SMATCH (Cai et al. 2013)
  • How well do two graphs match?
  • Structure and attributes
  • For attributes under oracle setting
  • Pearson’s R between predicted and gold attributes
  • F1 score on binarized values (> threshold, <= threshold)
slide-26
SLIDE 26

79.74 81.62 50 75 100

F1

Graph Structure Matching (Syntax included)

Pipeline Parser

slide-27
SLIDE 27

86.57 87.83 89.21 50 75 100

F1

Graph Structure Matching (Semantics only)

Pipeline Postprocessed End-to-end

slide-28
SLIDE 28

34.33

25 50 75 100

Pearson R

37.2 50.66

25 50 75 100

F1

Baseline F1 Model F1

slide-29
SLIDE 29

ψ(j, k) = tanh

  • 1 − |corr(νj − νj∗, νk − νk∗)|

|corr(νj∗, νk∗)|

  • <latexit sha1_base64="452uoXiYOheg4uQo2/QspsGljo=">ACi3icbVHbtNAEF2bWxoTeGxLysiUBKVyC6VWhCVKhASj0UibaU4ROvN2Nl4vbZ2x4jI8c/0k3jb1g7qYCGkVZ7dM6MZuZMmEth0PN+Oe69+w8ePmrtB8/2X2619l/dmyQnMY8Uxm+jpkBqRQMEKBEq5zDSwNJVyFycdav/oO2ohMfcVlDpOUxUpEgjO01LRzE4QC1UyKWI1qNpBbkRvcUiTPn1RgOEH1giU/MqCEXcoz59TYNIM16u1hrPtK56gSq+LWrJ/uViUB02KLlkHVpytabRf9SW6SVhWt+/TbAajZ7UzTtcbek3QbeBvQJds4mLa+RnMl6koJBLZszY93KclEyj4BLsjoWBnPGExTC2ULEUzKRsvKzoS8vMaJRp+xTShv27omSpMcs0tJkpw7m5q9Xk/7RxgdHpBQqLxAUXzeKCkxo/Vh6Exo4CiXFjCuhZ2V8jmzVqM9X9ua4N9deRtcHg394+HbL0fd8w8bO1rkgLwgPeKTE3JOPpMLMiLcaTlD58Q5dXfdN+479/061XU2Nc/JP+F+g15DMO/</latexit>

Pearson’s R between true attributes Pearson’s R between residuals

correlation well- captured systematic over/ under prediction no significant correlation present/ captured

  • 1

1

slide-30
SLIDE 30
slide-31
SLIDE 31
slide-32
SLIDE 32

Conclusions

  • Motivating
  • UDS as a dataset and task
  • Transductive paradigm for parsing
  • Showing
  • Challenges of UDS parsing (scalar + structure)
  • Benefits of end-to-end transductive system
  • Analyzing
  • interactions between UDS subspaces