Probabilistic Frame-Semantic Parsing Noah A. Smith Dipanjan Das - - PowerPoint PPT Presentation

▶
probabilistic frame semantic parsing
SMART_READER_LITE
LIVE PREVIEW

Probabilistic Frame-Semantic Parsing Noah A. Smith Dipanjan Das - - PowerPoint PPT Presentation

Nov 26, 2023 318 likes •1.09k views

Probabilistic Frame-Semantic Parsing Noah A. Smith Dipanjan Das Nathan Schneider Desai Chen School of Computer Science Carnegie Mellon University NAACL-HLT June 4, 2010 In a Nutshell Most models for semantics are very local (cascades of

slide-1
SLIDE 1

Probabilistic Frame-Semantic Parsing

Dipanjan Das Nathan Schneider Desai Chen

Noah A. Smith

NAACL-HLT June 4, 2010

School of Computer Science Carnegie Mellon University

slide-2
SLIDE 2

Das, Schneider, Chen and Smith, NAACL-HLT 2010

2

In a Nutshell

  • Most models for semantics are very local

(cascades of classifiers)

  • This work: towards more global modeling for rich

semantic processing (feature sharing among all semantic classes) (just two probabilistic models)

  • Our model outperforms the state of the art
  • Our framework lends itself to extensions and

improvements

slide-3
SLIDE 3

Das, Schneider, Chen and Smith, NAACL-HLT 2010

3

  • Introduction
  • Background and Datasets
  • Models and Results
  • Conclusion

Outline

slide-4
SLIDE 4

Das, Schneider, Chen and Smith, NAACL-HLT 2010

4

  • Introduction
  • Background and Datasets
  • Models and Results
  • Conclusion

Outline

slide-5
SLIDE 5

Das, Schneider, Chen and Smith, NAACL-HLT 2010

5

Overview

  • Annotate English sentences with semantic

representations

  • Combination of:
  • semantic frame (word sense) disambiguation
  • semantic role labeling
  • Frame and role repository: FrameNet (Fillmore et al., 2003)
slide-6
SLIDE 6

Das, Schneider, Chen and Smith, NAACL-HLT 2010

  • Theory developed by Fillmore (1982)
  • a word evokes a frame of semantic knowledge
  • a frame encodes a gestalt event or scenario
  • it has conceptual dependents filling roles

elaborating the frame instance

6

Frame Semantics

slide-7
SLIDE 7

Das, Schneider, Chen and Smith, NAACL-HLT 2010

  • Theory developed by Fillmore (1982)
  • a word evokes a frame of semantic knowledge
  • a frame encodes a gestalt event or scenario
  • it has conceptual dependents filling roles

elaborating the frame instance

7

the 1995 book by John Grisham

Frame Semantics

slide-8
SLIDE 8

Das, Schneider, Chen and Smith, NAACL-HLT 2010

  • Theory developed by Fillmore (1982)
  • a word evokes a frame of semantic knowledge
  • a frame encodes a gestalt event or scenario
  • it has conceptual dependents filling roles

elaborating the frame instance

8

the 1995 book by John Grisham

TEXT

Frame Semantics

slide-9
SLIDE 9

Das, Schneider, Chen and Smith, NAACL-HLT 2010

  • Theory developed by Fillmore (1982)
  • a word evokes a frame of semantic knowledge
  • a frame encodes a gestalt event or scenario
  • it has conceptual dependents filling roles

elaborating the frame instance

9

the 1995 book by John Grisham

  • a frame encodes a gestalt event or scenario

Frame Semantics

TEXT

slide-10
SLIDE 10

Das, Schneider, Chen and Smith, NAACL-HLT 2010

  • Theory developed by Fillmore (1982)
  • a word evokes a frame of semantic knowledge
  • a frame encodes a gestalt event or scenario
  • it has conceptual dependents filling roles

elaborating the frame instance

10

the 1995 book by John Grisham

  • a frame encodes a gestalt event or scenario

Author Time_of_creation

  • it has conceptual dependents filling roles

elaborating the frame instance

Frame Semantics

TEXT

slide-11
SLIDE 11

Das, Schneider, Chen and Smith, NAACL-HLT 2010

11

FrameNet

(Fillmore et al., 2003)

MAKE_NOISE Noisy_event Sound Sound_source Place Time

cough.v, gobble.v, hiss.v, ring.v, yodel.v, ...

slide-12
SLIDE 12

Das, Schneider, Chen and Smith, NAACL-HLT 2010

MAKE_NOISE Noisy_event Sound Sound_source Place Time

cough.v, gobble.v, hiss.v, ring.v, yodel.v, ...

12

lexical units frame roles

(Fillmore et al., 2003)

FrameNet

slide-13
SLIDE 13

Das, Schneider, Chen and Smith, NAACL-HLT 2010

13

relationships between frames and between roles

EVENT Place Time Event TRANSITIVE_ACTION Agent Patient Event Cause Place Time OBJECTIVE_INFLUENCE Dependent_entity Influencing_situation Place Time Influencing_entity CAUSE_TO_MAKE_NOISE Agent Sound_maker Cause Place Time MAKE_NOISE Noisy_event Sound Sound_source Place Time

cough.v, gobble.v, hiss.v, ring.v, yodel.v, ... blare.v, honk.v, play.v, ring.v, toot.v, ... — affect.v, effect.n, impact.n, impact.v, ... event.n, happen.v,

  • ccur.v, take place.v, ...

Inheritance relation Causative_of relation Excludes relation

Purpose

(Fillmore et al., 2003)

FrameNet

slide-14
SLIDE 14

Das, Schneider, Chen and Smith, NAACL-HLT 2010

14

  • Statistics:
  • 795 semantic frames
  • 7124 roles
  • 8379 lexical units (predicates)
  • 139,000 exemplar sentences containing one frame

annotation per sentence

FrameNet

slide-15
SLIDE 15

Das, Schneider, Chen and Smith, NAACL-HLT 2010

15

Marco Polo wrote an account of Asian society during the 13th century .

TEXT Author Topic

A Frame-Semantic Parse

slide-16
SLIDE 16

Das, Schneider, Chen and Smith, NAACL-HLT 2010

16

Marco Polo wrote an account of Asian society during the 13th century .

TEXT Author Topic

here, the ambiguous word evokes the TEXT frame

A Frame-Semantic Parse

slide-17
SLIDE 17

Das, Schneider, Chen and Smith, NAACL-HLT 2010

17

Marco Polo wrote an account of Asian society during the 13th century .

TEXT Author Topic

participants in the event or scenario

A Frame-Semantic Parse

slide-18
SLIDE 18

Das, Schneider, Chen and Smith, NAACL-HLT 2010

18

Marco Polo wrote an account of Asian society during the 13th century .

TEXT Author Topic

A Frame-Semantic Parse

frame-specific

participants in the event or scenario

slide-19
SLIDE 19

Das, Schneider, Chen and Smith, NAACL-HLT 2010

19

Why Frame-Semantic Parsing?

  • Combines lexical and predicate-argument semantics
  • Exploits meaningful primitives developed by experts
  • the FrameNet lexicon
  • Richer representation than PropBank style SRL
  • No inconsistent symbolic tags (ARG2-ARG5)

(Yi et al. 2007, Matsubayashi et al. 2009)

  • Patterns generalizing across frames and roles can be learned

(Matsubayashi et al. 2009)

slide-20
SLIDE 20

Das, Schneider, Chen and Smith, NAACL-HLT 2010

20

  • Introduction
  • Background and Datasets
  • Models and Results
  • Conclusion

Outline

slide-21
SLIDE 21

Das, Schneider, Chen and Smith, NAACL-HLT 2010

21

Early Work

  • Gildea and Jurafsky (2002)
  • Much smaller version of FrameNet
  • exemplar sentences
slide-22
SLIDE 22

Das, Schneider, Chen and Smith, NAACL-HLT 2010

22

SemEval 2007

  • Baker et al. (2007) organized the SemEval task on

frame structure extraction

  • first set of full text annotations available
  • released a corpus of ~2000 sentences with full

frame-semantic parses

  • Johansson and Nugues (2007) submitted the best

performing system

  • our baseline for comparison (J&N’07)
slide-23
SLIDE 23

Das, Schneider, Chen and Smith, NAACL-HLT 2010

23

SemEval 2007

  • SemEval 2007 dataset:
  • training set: 1941 sentences
  • test set: 120 sentences
  • Three domains
  • American National Corpus (travel)
  • Nuclear Threat Initiative (bureaucratic)
  • PropBank (news)
slide-24
SLIDE 24

Das, Schneider, Chen and Smith, NAACL-HLT 2010

24

SemEval 2007

  • Evaluation is done using the official SemEval script
  • Measures precision, recall and F1 score for

frames and arguments

  • Features a partial matching criterion for frame

identification

  • assigns score between 0 and 1 to closely

related frames in the FrameNet hierarchy

slide-25
SLIDE 25

Das, Schneider, Chen and Smith, NAACL-HLT 2010

25

  • Introduction
  • Background and Datasets
  • Models and Results
  • Conclusion

Outline

slide-26
SLIDE 26

Das, Schneider, Chen and Smith, NAACL-HLT 2010

26

Challenges

  • Several times more labels than traditional

shallow semantic parsing

  • Annotated data does not have gold

syntactic annotation

  • Very little labeled data
  • Identifying semantic frames for unknown

lexical units

  • Very sparse features
slide-27
SLIDE 27

Das, Schneider, Chen and Smith, NAACL-HLT 2010

27

Desired Structure

Everyone in Dublin seems intent on changing places with everyone else .

slide-28
SLIDE 28

Das, Schneider, Chen and Smith, NAACL-HLT 2010

28

Desired Structure

Everyone in Dublin seems intent on changing places with everyone else .

LOCATIVE_RELATION Figure Ground LOCALE Locale EXCHANGE Exchanger_1 Exchanger_2 Themes

PURPOSE

Agent Goal Phenomenon

APPEARANCE

Inference

slide-29
SLIDE 29

Das, Schneider, Chen and Smith, NAACL-HLT 2010

29

Three Subtasks:

  • Target identification
  • Identifying frame-evoking predicates

(nontrivial!)

  • Frame identification
  • Labeling each target with a frame type

(795 possibilities; ~WSD)

  • Argument identification
  • Finding each frame's arguments

(~SRL; roleset is frame-specific)

slide-30
SLIDE 30

Das, Schneider, Chen and Smith, NAACL-HLT 2010

29

Three Subtasks:

  • Target identification
  • Identifying frame-evoking predicates

(nontrivial!)

  • Frame identification
  • Labeling each target with a frame type

(795 possibilities; ~WSD)

  • Argument identification
  • Finding each frame's arguments

(~SRL; roleset is frame-specific)

sentence

predicates frames frames and arguments

slide-31
SLIDE 31

Das, Schneider, Chen and Smith, NAACL-HLT 2010

29

Three Subtasks:

  • Target identification
  • Identifying frame-evoking predicates

(nontrivial!)

  • Frame identification
  • Labeling each target with a frame type

(795 possibilities; ~WSD)

  • Argument identification
  • Finding each frame's arguments

(~SRL; roleset is frame-specific)

sentence

predicates frames frames and arguments

rule-based

probabilistic probabilistic

slide-32
SLIDE 32

Das, Schneider, Chen and Smith, NAACL-HLT 2010

30

  • Introduction
  • Background and Datasets
  • Models and Results
  • Target Identification
  • Frame Identification
  • Argument Identification
  • Final Results
  • Conclusion

Outline

slide-33
SLIDE 33

Das, Schneider, Chen and Smith, NAACL-HLT 2010

31

Target Identification

Everyone in Dublin seems intent on changing places with everyone else .

slide-34
SLIDE 34

Das, Schneider, Chen and Smith, NAACL-HLT 2010

32

  • Rule-based identification
  • list of all morphological variants of predicates in the lexicon
  • all prepositions filtered
  • support verbs were not identified
  • J&N’07 filtered these

Target Identification

Everyone in Dublin seems intent on changing places with everyone else .

slide-35
SLIDE 35

Das, Schneider, Chen and Smith, NAACL-HLT 2010

33

  • Introduction
  • Background and Datasets
  • Models and Results
  • Target Identification
  • Frame Identification
  • Argument Identification
  • Final Results
  • Conclusion

Outline

slide-36
SLIDE 36

Das, Schneider, Chen and Smith, NAACL-HLT 2010

34

Frame Identification

Everyone in Dublin seems intent on changing places with everyone else .

LOCATIVE_RELATION LOCALE EXCHANGE

PURPOSE APPEARANCE

slide-37
SLIDE 37

Das, Schneider, Chen and Smith, NAACL-HLT 2010

35

Frame Identification

Everyone in Dublin seems intent on changing places with everyone else .

LOCATIVE_RELATION LOCALE EXCHANGE

PURPOSE APPEARANCE

J&N’07 used several classifiers for this subtask

slide-38
SLIDE 38

Das, Schneider, Chen and Smith, NAACL-HLT 2010

36

Unseen LUs

from WordNet-extended set

sixth ∈ ORDINAL_NUMBERS? Y N

Seen LUs

sixth ∈ RELATIVE_TIME? sixth ∈ INGESTION?

… 1 classifier … 795 classifiers

Frame Identification

(Johansson and Nugues, 2007)

place

LOCALE PLACING

book in

slide-39
SLIDE 39

Das, Schneider, Chen and Smith, NAACL-HLT 2010

37

Unseen LUs

from WordNet-extended set

intent ∈ PURPOSE? Y N

Seen LUs

intent ∈ AIMING? intent ∈ INGESTION?

… 1 classifier … 795 classifiers

Frame Identification

(Johansson and Nugues, 2007)

place

LOCALE PLACING

book in

slide-40
SLIDE 40

Das, Schneider, Chen and Smith, NAACL-HLT 2010

38

Frame Identification

Our approach: One single model for frame identification

Everyone in Dublin seems intent on changing places with everyone else .

LOCATIVE_RELATION LOCALE EXCHANGE

PURPOSE APPEARANCE

slide-41
SLIDE 41

Das, Schneider, Chen and Smith, NAACL-HLT 2010

39

NN IN NNP VBZ NN IN VBG NNS IN NN RB .

Frame Identification

Assume POS tags and dependency trees to be given

Everyone in Dublin seems intent on changing places with everyone else .

slide-42
SLIDE 42

Das, Schneider, Chen and Smith, NAACL-HLT 2010

40

Frame Identification

Assume that target is connected to the frame through a prototype unit

t ℓ f t ℓ

f

NN IN NNP VBZ NN IN VBG NNS IN NN RB .

Everyone in Dublin seems intent on changing places with everyone else .

slide-43
SLIDE 43

Das, Schneider, Chen and Smith, NAACL-HLT 2010

NN

41

Frame Identification

PURPOSE

Assume that target is connected to the frame through a prototype unit

t ℓ f

?

NN IN NNP VBZ IN VBG NNS IN NN RB .

Everyone in Dublin seems intent on changing places with everyone else .

t ℓ

f

slide-44
SLIDE 44

Das, Schneider, Chen and Smith, NAACL-HLT 2010

42

Frame Identification

PURPOSE Goal Means Agent

aim.n, goal.n, object.n,

  • bjective.n, purpose.n,

target.n

Attribute Value

  • Consider the PURPOSE frame
slide-45
SLIDE 45

Das, Schneider, Chen and Smith, NAACL-HLT 2010

43

Frame Identification

PURPOSE Goal Means Agent

aim.n, goal.n, object.n,

  • bjective.n, purpose.n,

target.n

Attribute Value

{ }

ℓ∈

  • Consider the PURPOSE frame
slide-46
SLIDE 46

Das, Schneider, Chen and Smith, NAACL-HLT 2010

44

Frame Identification

PURPOSE Goal Means Agent

aim.n, goal.n, object.n,

  • bjective.n, purpose.n,

target.n

Attribute Value

{ }

ℓ∈

note that the target intent is unseen

  • Consider the PURPOSE frame
slide-47
SLIDE 47

Das, Schneider, Chen and Smith, NAACL-HLT 2010

  • Consider the PURPOSE frame

45

Frame Identification

PURPOSE Goal Means Agent

aim.n, goal.n, object.n,

  • bjective.n, purpose.n,

target.n

Attribute Value

{ }

ℓ∈

note that the target intent is unseen

but lexical semantic relationships between some and the target exist

ℓ

purpose ≈ intent

slide-48
SLIDE 48

Das, Schneider, Chen and Smith, NAACL-HLT 2010

46

Thus, we define a probabilistic model: Frame Identification

pθ(f, ℓ | t, x) ∝ exp θ⊤g(f, ℓ, t, x)

NN IN NNP VBZ NN IN VBG NNS IN NN RB .

Everyone in Dublin seems intent on changing places with everyone else .

slide-49
SLIDE 49

Das, Schneider, Chen and Smith, NAACL-HLT 2010

47

Thus, we define a probabilistic model: Frame Identification

pθ(f, ℓ | t, x) ∝ exp θ⊤g(f, ℓ, t, x)

WordNet relationships!

some features looking at the lexical and semantic relationships between and

ℓ f

NN IN NNP VBZ NN IN VBG NNS IN NN RB .

Everyone in Dublin seems intent on changing places with everyone else .

slide-50
SLIDE 50

Das, Schneider, Chen and Smith, NAACL-HLT 2010

  • ther features looking at the

whole sentence structure

48

Thus, we define a probabilistic model: Frame Identification

pθ(f, ℓ | t, x) ∝ exp θ⊤g(f, ℓ, t, x) x

NN IN NNP VBZ NN IN VBG NNS IN NN RB .

Everyone in Dublin seems intent on changing places with everyone else .

slide-51
SLIDE 51

Das, Schneider, Chen and Smith, NAACL-HLT 2010

49

Thus, we define a probabilistic model: Frame Identification

pθ(f, ℓ | t, x) ∝ exp θ⊤g(f, ℓ, t, x)

Note that is unknown

NN IN NNP VBZ NN IN VBG NNS IN NN RB .

Everyone in Dublin seems intent on changing places with everyone else .

ℓ

slide-52
SLIDE 52

Das, Schneider, Chen and Smith, NAACL-HLT 2010

50

Thus, we define a probabilistic model: Frame Identification

pθ(f, ℓ | t, x) ∝ exp θ⊤g(f, ℓ, t, x)

Marginalization of latent variable:

pθ(f | t, x) ∝

  • ℓ

exp θ⊤g(f, ℓ, t, x)

NN IN NNP VBZ NN IN VBG NNS IN NN RB .

Everyone in Dublin seems intent on changing places with everyone else .

slide-53
SLIDE 53

Das, Schneider, Chen and Smith, NAACL-HLT 2010

51

Frame Identification Inference:

NN IN NNP VBZ NN IN VBG NNS IN NN RB .

Everyone in Dublin seems intent on changing places with everyone else .

ˆ f ← argmaxf

  • ℓ

exp θ⊤g(f, ℓ, t, x)

slide-54
SLIDE 54

Das, Schneider, Chen and Smith, NAACL-HLT 2010

52

Frame Identification Inference: Training: maximum conditional likelihood

NN IN NNP VBZ NN IN VBG NNS IN NN RB .

Everyone in Dublin seems intent on changing places with everyone else .

ˆ f ← argmaxf

  • ℓ

exp θ⊤g(f, ℓ, t, x)

slide-55
SLIDE 55

Das, Schneider, Chen and Smith, NAACL-HLT 2010

53

Results

50 57 64 71 78 J&N’07 This Work This Work

74.2 68.3 64.0 74.2 61.0 56.4 74.2 77.5 73.9

Precision Recall F1

(Oracle Targets) (Model Targets)

Frame Identification

slide-56
SLIDE 56

Das, Schneider, Chen and Smith, NAACL-HLT 2010

54

Results

50 57 64 71 78 J&N’07 This Work This Work

74.2 68.3 64.0 74.2 61.0 56.4 74.2 77.5 73.9

Precision Recall F1

(Oracle Targets) (Model Targets)

Frame Identification

significant improvement

slide-57
SLIDE 57

Das, Schneider, Chen and Smith, NAACL-HLT 2010

55

Frame Identification

  • For gold standard targets, 210 out of 1058 lemmas

were unseen

  • 190 of these get some positive score for partial

frame matching

  • 4 of these exactly match
  • 44 get 0.5 or more, indicating close match
slide-58
SLIDE 58

Das, Schneider, Chen and Smith, NAACL-HLT 2010

56

  • Introduction
  • Background and Datasets
  • Models and Results
  • Target Identification
  • Frame Identification
  • Argument Identification
  • Final Results
  • Conclusion

Outline

slide-59
SLIDE 59

Das, Schneider, Chen and Smith, NAACL-HLT 2010

57

Argument Identification

Everyone in Dublin seems intent on changing places with everyone else .

EXCHANGE Exchanger_1 Exchanger_2 Themes

slide-60
SLIDE 60

Das, Schneider, Chen and Smith, NAACL-HLT 2010

58

Candidate spans

Two steps:

Argument Identification: The traditional approach

Everyone in Dublin

......

in Dublin

  • n changing places

changing places with everyone else places everyone

slide-61
SLIDE 61

Das, Schneider, Chen and Smith, NAACL-HLT 2010

59

Everyone in Dublin

......

Candidate spans

Two steps:

Argument Identification: The traditional approach

in Dublin

  • n changing places

changing places with everyone else places everyone

✔

binary filtering potential arguments

✗ ✗ ✗ ✗ ✔ ✔

slide-62
SLIDE 62

Das, Schneider, Chen and Smith, NAACL-HLT 2010

60

Everyone in Dublin

......

Candidate spans

Two steps:

Argument Identification: The traditional approach

in Dublin

  • n changing places

changing places with everyone else places everyone

✔ ✔ ✗ ✗ ✗ ✗ ✔

Exchanger_1 Exchanger_2 Themes

classification of arguments into different roles

slide-63
SLIDE 63

Das, Schneider, Chen and Smith, NAACL-HLT 2010

61

Everyone in Dublin

......

Candidate spans

Two steps:

Argument Identification: The traditional approach

in Dublin

  • n changing places

changing places with everyone else places everyone

✔ ✔ ✗ ✗ ✗ ✗ ✔

Exchanger_1 Exchanger_2 Themes

Two steps unnecessary

slide-64
SLIDE 64

Das, Schneider, Chen and Smith, NAACL-HLT 2010

62

......

Candidate spans Roleset for EXCHANGE

Argument Identification: Our approach

...

Everyone in Dublin in Dublin

  • n changing places

changing places with everyone else places everyone

Exchanger_1 Exchanger_2 Exchangers Theme_1 Theme_2 Themes Manner Means

slide-65
SLIDE 65

Das, Schneider, Chen and Smith, NAACL-HLT 2010

63

......

Candidate spans

Exchanger_1 Exchanger_2 Exchangers Theme_1 Theme_2 Themes Manner Means

Roleset for EXCHANGE

Argument Identification: Our approach

...

Everyone in Dublin in Dublin

  • n changing places

changing places with everyone else places everyone

  • ne step!

∅

slide-66
SLIDE 66

Das, Schneider, Chen and Smith, NAACL-HLT 2010

64

A probabilistic model: Argument Identification

pψ(r → s | f, t, x) ∝ exp ψ⊤h(r, s, f, t, x)

slide-67
SLIDE 67

Das, Schneider, Chen and Smith, NAACL-HLT 2010

65

A probabilistic model: Argument Identification

pψ(r → s | f, t, x) ∝ exp ψ⊤h(r, s, f, t, x)

features looking at the span, the frame, the role and the observed sentence structure

slide-68
SLIDE 68

Das, Schneider, Chen and Smith, NAACL-HLT 2010

Argument Identification A probabilistic model:

pψ(r → s | f, t, x) ∝ exp ψ⊤h(r, s, f, t, x)

Decoding: Best span for each role is selected For each frame, the best set of non-

  • verlapping arguments is decoded

together

66

slide-69
SLIDE 69

Das, Schneider, Chen and Smith, NAACL-HLT 2010

Argument Identification A probabilistic model:

pψ(r → s | f, t, x) ∝ exp ψ⊤h(r, s, f, t, x)

Training: Maximum conditional likelihood

67

slide-70
SLIDE 70

Das, Schneider, Chen and Smith, NAACL-HLT 2010

68

Results

50.000 59.572 69.145 78.717 88.290 Model Spans Oracle Spans

81.0 68.5 74.8 60.6 88.3 78.7

Precision Recall F1

Argument identification only, with gold targets and frames Argument Identification

slide-71
SLIDE 71

Das, Schneider, Chen and Smith, NAACL-HLT 2010

69

  • Introduction
  • Background and Datasets
  • Models and Results
  • Target Identification
  • Frame Identification
  • Argument Identification
  • Final Results
  • Conclusion

Outline

slide-72
SLIDE 72

Das, Schneider, Chen and Smith, NAACL-HLT 2010

70

Results

30.00 38.75 47.50 56.25 65.00 J&N’07 This work

50.2 45.6 41.9 38.4 62.8 56.0

Precision Recall F1

full frame-semantic parsing Full Frame-Semantic Parsing

slide-73
SLIDE 73

Das, Schneider, Chen and Smith, NAACL-HLT 2010

71

Results

30.00 38.75 47.50 56.25 65.00 J&N’07 This work

50.2 45.6 41.9 38.4 62.8 56.0

Precision Recall F1

full frame-semantic parsing

significant improvement

Full Frame-Semantic Parsing

slide-74
SLIDE 74

Das, Schneider, Chen and Smith, NAACL-HLT 2010

72

Conclusion

  • Best results to date on frame-semantic parsing
  • Only two probabilistic models instead of a cascade of

classifiers for the frame-semantic parsing task

  • Latent variable model for frame identification
  • Better modeling of the argument identification (SRL)

stage using only one model instead of two

  • Publicly available software: http://www.ark.cs.cmu.edu/SEMAFOR
slide-75
SLIDE 75

Thanks!

http://www.ark.cs.cmu.edu/SEMAFOR

slide-76
SLIDE 76

Thanks!

http://www.ark.cs.cmu.edu/SEMAFOR

JUDGMENT_DIRECT_ADDRESS