[PPT] - Annotating and Automatically Tagging Constructions of Causal PowerPoint Presentation

SLIDE 1

Annotating and Automatically Tagging Constructions of Causal Language

SLIDE 2

What Google displays for “why” questions

SLIDE 3

What Google displays for “why” questions

SLIDE 4

What Google displays for “why” questions could be a lot more helpful.

SLIDE 5

What Google displays for “why” questions could be a lot more helpful.

SLIDE 6

33% 12%

Such cause-and-effect questions & assertions are far from rare.

SLIDE 7

33% 12%

Such cause-and-effect questions & assertions are far from rare.

>5%

SLIDE 8

Why

CAUSATION

We’d like to be able to parse causal relationships in text.

SLIDE 9

CAUSATION

set the stage for Why

CAUSATION

We’d like to be able to parse causal relationships in text.

SLIDE 10

CAUSATION

set the stage for Why

CAUSATION

This style of analysis is known as “shallow semantic parsing.”

SLIDE 11

CAUSATION

set the stage for Why

CAUSATION

This style of analysis is known as “shallow semantic parsing.”

SLIDE 12

CAUSATION

set the stage for Why

CAUSATION

This style of analysis is known as “shallow semantic parsing.”

SLIDE 13

Task definition: connective discovery + argument identification

because

Connective discovery Argument identification

SLIDE 14

Causality is expressed in an enormous variety of ways.

SLIDE 15

Causality is expressed in an enormous variety of ways.

impede because of so

SLIDE 16

Causality is expressed in an enormous variety of ways.

impede because of so

pens the way for

SLIDE 17

Causality is expressed in an enormous variety of ways.

impede because of so

pens the way for

so that

SLIDE 18

Causality is expressed in an enormous variety of ways.

impede because of so

pens the way for

so that After The more the less

SLIDE 19

Each existing semantic parsing representation handles only a portion of this space.

SLIDE 20

Each existing semantic parsing representation handles only a portion of this space.

MAKE.02

made Verbs only

SLIDE 21

Each existing semantic parsing representation handles only a portion of this space.

CONTINGENCY:Cause

so Conjunctions & adverbs only

MAKE.02

made Verbs only

SLIDE 22

Each existing semantic parsing representation handles only a portion of this space.

CONTINGENCY:Cause

so Conjunctions & adverbs only

CAUSATION

made Words or constituents

nly

MAKE.02

made Verbs only

SLIDE 23

Each existing semantic parsing representation handles only a portion of this space.

CONTINGENCY:Cause

so Conjunctions & adverbs only

(one) meaning

ne

word

CAUSATION

made Words or constituents

nly

MAKE.02

made Verbs only

SLIDE 24

Construction grammar (CxG)

ffers a way forward.

SLIDE 25

Construction grammar (CxG)

ffers a way forward.

SLIDE 26

EXTREME(_______)  _____ ______ _______ so that

Construction grammar (CxG)

ffers a way forward.

SLIDE 27

EXTREME(_______)  _____ ______ _______ so that

Construction grammar (CxG)

ffers a way forward.

SLIDE 28

(It’s not just causality, either.)

SLIDE 29

(It’s not just causality, either.)

as as More than as as is

Comparatives

SLIDE 30

(It’s not just causality, either.)

in spite of no matter as as as More than as as is

Concessives Comparatives

SLIDE 31

Full CxG theory means “constructions all the way down”:

so offensive that I left

SLIDE 32

Full CxG theory means “constructions all the way down”:

〈so ___〉 EXTREME(__)

so offensive that I left

SLIDE 33

Full CxG theory means “constructions all the way down”:

〈so ___〉〈_ _ _ _ _ _〉 EXTREME(__)

so offensive that I left

SLIDE 34

Full CxG theory means “constructions all the way down”:

〈so ___〉〈_ _ _ _ _ _〉 EXTREME(__)

so offensive that I left

SLIDE 35

Full CxG theory means “constructions all the way down”:

〈that ____〉〈so ___〉 Comple- ment 〈_ _ _ _ _ _〉 EXTREME(__) ____

so offensive that I left

SLIDE 36

Full CxG theory means “constructions all the way down”:

〈that ____〉〈so ___〉 Comple- ment 〈_ _ _ _ _ _〉〈so ___that ____〉 EXTREME(__) ____ EXTREME(__)  ____

so offensive that I left

SLIDE 37

Full CxG theory means “constructions all the way down”:

〈that ____〉〈so ___〉 Comple- ment 〈_ _ _ _ _ _〉〈so ___that ____〉 EXTREME(__) ____ EXTREME(__)  ____

ffensive

I left so offensive that I left

SLIDE 38

… Tokenization POS tagging, syntactic parsing Construction recognition

The “constructions on top” approach reaps the low-hanging fruit from applying CxG to NLP .

SLIDE 39

… Tokenization POS tagging, syntactic parsing Construction recognition Tagging causal relations

The “constructions on top” approach reaps the low-hanging fruit from applying CxG to NLP .

SLIDE 40

“Constructions on top” borrows two key insights of CxG.

1. Words, multi-word expressions, and grammar

are all on equal footing as “learned pairings of form and function.”

2. Constructions pair patterns of surface forms

directly with meanings.

SLIDE 41

Using the “constructions on top” approach to applying CxG, we can:

SLIDE 42

richer, more flexible linguistic representations

Using the “constructions on top” approach to applying CxG, we can:

SLIDE 43

richer, more flexible linguistic representations
Design annotation guidelines & annotate a corpus

Using the “constructions on top” approach to applying CxG, we can:

SLIDE 44

richer, more flexible linguistic representations
Design annotation guidelines & annotate a corpus
automated machine learning taggers

Using the “constructions on top” approach to applying CxG, we can:

SLIDE 45

T

day’s talk:

The BECAUSE annotation scheme & corpus

f causal language

Causeway-L/Causeway-S: two pattern-based taggers for causal constructions DeepCx: a neural, transition-based tagger for causal constructions

SLIDE 46

T

day’s talk:

The BECAUSE annotation scheme & corpus

f causal language

Causeway-L/Causeway-S: two pattern-based taggers for causal constructions DeepCx: a neural, transition-based tagger for causal constructions

SLIDE 47

Previous projects have struggled to annotate real-world causality.

<e1>flu</e1> <e2>virus</e2>

Cause-Effect(e2, e1) = "true"

allocated equip

BEFORE-PRECONDITIONS

SLIDE 48

Existing shallow semantic parsing schemes include some elements of causal language.

… CAUSATION

made

SLIDE 49

Causal language: a clause or phrase in which

ne event, state, action, or entity

is explicitly presented as promoting or hindering another Causal language: a clause or phrase in which

ne event, state, action, or entity

is explicitly presented as promoting or hindering another

SLIDE 50

Connective: fixed constructional cue indicating a causal relationship

because prevented from causes

SLIDE 51

Connective: fixed constructional cue indicating a causal relationship

because prevented from causes

SLIDE 52

Cause: presented as producing effect Effect: presented as outcome

John trapped the fox it was threatening his chickens John the fox eating his chickens Ice cream consumption drowning

SLIDE 53

Connectives can be arbitrarily complex.

For to

pens the way for

SLIDE 54

We distinguish three types of causation.

because of

CONSEQUENCE

because

MOTIVATION

in order to

PURPOSE

SLIDE 55

F1 0.77 κ 0.70

Latest annotation scheme shows very good inter-annotator agreement.

SLIDE 56

We have annotated a small corpus with this scheme.

Total 121 4790 1803

BECAUSE = B E Cau S E

SLIDE 57

Actual corpus examples can get quite complex.

For to must allowed to Average causal sentence length: 30 words If prevents because

SLIDE 58

T

day’s talk:

The BECAUSE annotation scheme & corpus

f causal language

Causeway-L/Causeway-S: two pattern-based taggers for causal constructions DeepCx: a neural, transition-based tagger for causal constructions

SLIDE 59

T

day’s talk:

The BECAUSE annotation scheme & corpus

f causal language

Causeway-L/Causeway-S: two pattern-based taggers for causal constructions DeepCx: a neural, transition-based tagger for causal constructions

SLIDE 60

The computational task is challenging.

Long tail of causal connectives Requires sense disambiguation of connectives

for to for to

Complex output structure Combinatorial connective possibilities

SLIDE 61

from from

1. Pattern-based

connective discovery

SLIDE 62

I…died from worry …called me from your hotel from from

1. Pattern-based

connective discovery

2. Argument

identification

SLIDE 63

…called me from your hotel I…died from worry …called me from your hotel from from I…died from worry

1. Pattern-based

connective discovery

2. Argument

identification

3. Statistical classifier

to filter results

SLIDE 64

…called me from your hotel I…died from worry …called me from your hotel from from I…died from worry

1. Pattern-based

connective discovery

2. Argument

identification

3. Statistical classifier

to filter results

4. Remove duplicate

connectives

SLIDE 65

…called me from your hotel I…died from worry …called me from your hotel from from I…died from worry

1. Pattern-based

connective discovery

2. Argument

identification

3. Statistical classifier

to filter results

4. Remove duplicate

connectives

SLIDE 66

…called me from your hotel I…died from worry …called me from your hotel from from I…died from worry

1. Pattern-based

connective discovery

2. Argument

identification

3. Statistical classifier

to filter results

4. Remove duplicate

connectives

SLIDE 67

i.

Causeway-S: Syntax-based pipeline

ii.

Causeway-L: Lexical pattern-based pipeline

Causeway-S/Causeway-L: two pattern-based taggers for causal constructions

SLIDE 68

i.

Causeway-S: Syntax-based pipeline

ii.

Causeway-L: Lexical pattern-based pipeline

Causeway-S/Causeway-L: two pattern-based taggers for causal constructions

SLIDE 69

each construction is treated as a partially-fixed parse tree fragment.

SLIDE 70

each construction is treated as a partially-fixed parse tree fragment.

SLIDE 71

each construction is treated as a partially-fixed parse tree fragment.

advcl mark

because/IN

SLIDE 72

each construction is treated as a partially-fixed parse tree fragment.

advcl mark

because/IN

SLIDE 73

TRegex patterns are extracted in training, and matched at test time.

because

advcl mark

because/IN

(/^because_[0-9]+$/ <2 /^IN.*/ <1 mark > (/.*_[0-9]+/ <1 advcl > (/.*_[0-9]+/)))

SLIDE 74

TRegex patterns are extracted in training, and matched at test time.

because

advcl mark

because/IN

(/^because_[0-9]+$/ <2 /^IN.*/ <1 mark > (/.*_[0-9]+/ <1 advcl > (/.*_[0-9]+/))) (/^because_[0-9]+$/ <2 /^IN.*/ <1 mark > (/.*_[0-9]+/ <1 advcl > (/.*_[0-9]+/)))

TRegex 1

because

+

SLIDE 75

Argument heads are expanded to include most dependents.

care/VBP worry/VBP

SLIDE 76

Argument heads are expanded to include most dependents.

care/VBP worry/VBP

SLIDE 77

1. Causeway-S/Causeway-L: two simple systems

for tagging causal constructions

i.

Causeway-S: Syntax-based pipeline

ii.

Causeway-L: Lexical pattern-based pipeline

SLIDE 78

1. Causeway-S/Causeway-L: two simple systems

for tagging causal constructions

i.

Causeway-S: Syntax-based pipeline

ii.

Causeway-L: Lexical pattern-based pipeline

SLIDE 79

constructions are matched by regular expressions over word lemmas.

because

regex

because

+

(ˆ | )([ \ S]+ )+?(because/IN) ([ \ S]+ )+? (ˆ | )([ \ S]+ )+?(because/IN) ([ \ S]+ )+?

SLIDE 80

Arguments are labeled by a conditional random field.

… …

SLIDE 81

Both approaches use a soft vote of three classifiers as a filter.

SLIDE 82

Both approaches use a soft vote of three classifiers as a filter.

SLIDE 83

Both approaches use a soft vote of three classifiers as a filter.

SLIDE 84

Both approaches use a soft vote of three classifiers as a filter.

Example classifier features :

SLIDE 85

Our benchmark is a dependency path memorization heuristic.

27/ 4 0 / 8



14 / 1

…

SLIDE 86

11.5% 51.7% 13.4% 50.8% 31.6% 54.9% 54.6%

Connective discovery

SLIDE 87

11.5% 51.7% 13.4% 50.8% 31.6% 54.9% 54.6%

Connective discovery: Causeway outperforms the benchmark by ~20 points.

SLIDE 88

11.5% 51.7% 13.4% 50.8% 31.6% 54.9% 54.5%

Performance improves even more when Causeway is combined with the benchmark.

SLIDE 89

11.5% 51.7% 13.4% 50.8% 31.6% 54.9% 54.6%

The first stage gets high recall & low precision

SLIDE 90

11.5% 51.7% 13.4% 50.8% 31.6% 54.9% 54.6%

The first stage gets high recall & low precision, but the filters balance them out for a better F1.

SLIDE 91

Argument identification is passable given connective discovery, though effects are harder than causes.

SLIDE 92

Argument identification is passable given connective discovery, though effects are harder than causes.

SLIDE 93

T

day’s talk:

The BECAUSE annotation scheme & corpus

f causal language

Causeway-L/Causeway-S: two pattern-based taggers for causal constructions DeepCx: a neural, transition-based tagger for causal constructions

SLIDE 94

T

day’s talk:

The BECAUSE annotation scheme & corpus

f causal language

Causeway-L/Causeway-S: two pattern-based taggers for causal constructions DeepCx: a neural, transition-based tagger for causal constructions

SLIDE 95

Transition-based tagging builds a complex output structure using a sequence of simple operations.

SLIDE 96

The DeepCx transition scheme

SLIDE 97

The DeepCx transition scheme

SLIDE 98

The DeepCx transition scheme

SLIDE 99

The DeepCx transition scheme

Tagger state

SLIDE 100

The DeepCx transition scheme

Tagger state

NO- CONN

SLIDE 101

The DeepCx transition scheme

Tagger state

NO- CONN

SLIDE 102

The DeepCx transition scheme

Tagger state

NO- CONN

SLIDE 103

The DeepCx transition scheme

Tagger state

NO- CONN

SLIDE 104

The DeepCx transition scheme

Tagger state

NO- CONN

SLIDE 105

The DeepCx transition scheme

Tagger state

NO- CONN NEW-CONN

SLIDE 106