Predicting implicit and explicit questions Matthijs Westera COLT - - PowerPoint PPT Presentation

▶
predicting implicit and explicit questions
SMART_READER_LITE
LIVE PREVIEW

Predicting implicit and explicit questions Matthijs Westera COLT - - PowerPoint PPT Presentation

Jan 15, 2024 196 likes •802 views

Predicting implicit and explicit questions Matthijs Westera COLT kick-off workshop Predicting implicit and explicit questions Matthijs Westera COLT kick-off workshop = Computational Linguistics & Linguistic Theory The data of

slide-1
SLIDE 1

Predicting implicit and explicit questions

Matthijs Westera COLT kick-off workshop

slide-2
SLIDE 2

Predicting implicit and explicit questions

Matthijs Westera COLT kick-off workshop

Computational Linguistics & Linguistic Theory

=

slide-3
SLIDE 3

The data of linguistic theory:

slide-4
SLIDE 4

The data of linguistic theory:

( S e m a n t i c s & P r a g m a t i c s )

slide-5
SLIDE 5

The data of linguistic theory:

( S e m a n t i c s & P r a g m a t i c s )

slide-6
SLIDE 6

The data of computational ling:

slide-7
SLIDE 7

The data of computational ling:

slide-8
SLIDE 8

Main interest

messy crisp

slide-9
SLIDE 9

Main interest

enables

messy crisp

slide-10
SLIDE 10

Main interest

enables

How?

messy crisp

slide-11
SLIDE 11

Main interest

enables

How?

messy crisp

  • ...?
  • ...?
  • ...?
slide-12
SLIDE 12

Neural networks

slide-13
SLIDE 13

Neural networks

crisp

slide-14
SLIDE 14

Neural networks

messy crisp

slide-15
SLIDE 15

Neural networks

messy crisp ? ?

slide-16
SLIDE 16

Neural networks

messy crisp ? ?

  • cf. Aina et al. (2018) [SemEval]
slide-17
SLIDE 17

Distributional & formal semantics

slide-18
SLIDE 18

Distributional & formal semantics

cat red

animal

slide-19
SLIDE 19

Distributional & formal semantics

cat red

animal

messy

slide-20
SLIDE 20

Distributional & formal semantics

∃x (RED(x) ∧ CAT(x))

cat red

animal

messy

slide-21
SLIDE 21

Distributional & formal semantics

∃x (RED(x) ∧ CAT(x))

cat red

animal

messy crisp

slide-22
SLIDE 22

Distributional & formal semantics

∃x (RED(x) ∧ CAT(x))

cat red

animal

messy crisp

? ?

slide-23
SLIDE 23

Distributional & formal semantics

∃x (RED(x) ∧ CAT(x))

cat red

animal

messy crisp

? ?

Westera & Boleda (under review)

slide-24
SLIDE 24

Main interest

enables

messy crisp

How?

slide-25
SLIDE 25

Main interest

enables

  • how do NN models do it?

How?

messy crisp

slide-26
SLIDE 26

Main interest

enables

  • how do NN models do it?
  • how do DS and FS relate?

How?

messy crisp

slide-27
SLIDE 27

Main interest

enables

  • how do NN models do it?
  • how do DS and FS relate?
  • what does ling. theory say?

How?

messy crisp

slide-28
SLIDE 28

Main interest

enables

  • how do NN models do it?
  • how do DS and FS relate?
  • what does ling. theory say?

How?

messy crisp

slide-29
SLIDE 29

Pragmatics

slide-30
SLIDE 30

Pragmatics

€ PAINT

slide-31
SLIDE 31

Pragmatics

€ PAINT

slide-32
SLIDE 32

Pragmatics

slide-33
SLIDE 33

Pragmatics

slide-34
SLIDE 34

Pragmatics

slide-35
SLIDE 35

Main interest

How?

enables

  • how do NN models do it?
  • how do DS and FS relate?
  • what does ling. theory say?

messy crisp

slide-36
SLIDE 36

Main interest

How?

enables

  • how do NN models do it?
  • how do DS and FS relate?
  • ...

messy crisp

slide-37
SLIDE 37

Main interest

How?

enables

  • how do NN models do it?
  • how do DS and FS relate?
  • how are goals identifjed?

messy crisp

slide-38
SLIDE 38

Main interest

How?

enables

  • how do NN models do it?
  • how do DS and FS relate?
  • how are goals identifjed?

messy crisp

slide-39
SLIDE 39

Predicting implicit and explicit questions

Matthijs Westera COLT kick-off workshop

slide-40
SLIDE 40

Discourse goals

Pragmatic theory:

slide-41
SLIDE 41

Discourse goals

Pragmatic theory:

  • Discourse is organized around questions:
slide-42
SLIDE 42

Discourse goals

Pragmatic theory:

  • Discourse is organized around questions:

e.g., Westera 2017 (PhD thesis)

slide-43
SLIDE 43

Discourse goals

Pragmatic theory:

  • Discourse is organized around questions:

– Some explicit (“What color do you want?”) – Most are implicit:

e.g., Westera 2017 (PhD thesis)

slide-44
SLIDE 44

Discourse goals

Pragmatic theory:

  • Discourse is organized around questions:

– Some explicit (“What color do you want?”) – Most are implicit:

So, there's this party tonight... August, no! But it's perfect! It’s my last night in Aberdeen! I'll never be in this town again! No, we can't.

BookCorpus/590124__come-as-you-are.txt

e.g., Westera 2017 (PhD thesis)

slide-45
SLIDE 45

Discourse goals

Pragmatic theory:

  • Discourse is organized around questions:

– Some explicit (“What color do you want?”) – Most are implicit:

So, there's this party tonight... August, no! But it's perfect! It’s my last night in Aberdeen! I'll never be in this town again! No, we can't.

BookCorpus/590124__come-as-you-are.txt

S h a l l w e g

  • ?

?

e.g., Westera 2017 (PhD thesis)

slide-46
SLIDE 46

Discourse goals

Pragmatic theory:

  • Discourse is organized around questions:

– Some explicit (“What color do you want?”) – Most are implicit:

So, there's this party tonight... August, no! But it's perfect! It’s my last night in Aberdeen! I'll never be in this town again! No, we can't.

BookCorpus/590124__come-as-you-are.txt

S h a l l w e g

  • ?

?

e.g., Westera 2017 (PhD thesis)

slide-47
SLIDE 47

Question prediction

Get a model to predict implicit questions (goals), by training on explicit questions.

slide-48
SLIDE 48

Question prediction

Get a model to predict implicit questions (goals), by training on explicit questions.

= f

r e e s u p e r v i s i

  • n

!

slide-49
SLIDE 49

Question prediction

Get a model to predict implicit questions (goals), by training on explicit questions.

= f

r e e s u p e r v i s i

  • n

! Data:

  • Only dialogue contains enough questions.
slide-50
SLIDE 50

Question prediction

Get a model to predict implicit questions (goals), by training on explicit questions.

= f

r e e s u p e r v i s i

  • n

! Data:

  • Only dialogue contains enough questions.
  • Extract dialogues from movie scripts and books.

– E.g., 1M dialogues from BookCorpus.

slide-51
SLIDE 51

Question prediction

Get a model to predict implicit questions (goals), by training on explicit questions.

= f

r e e s u p e r v i s i

  • n

! Data:

  • Only dialogue contains enough questions.
  • Extract dialogues from movie scripts and books.

– E.g., 1M dialogues from BookCorpus.

Models (e.g.):

  • Simple word-level RNN. (Westera 2018 [SemDial])
slide-52
SLIDE 52

Question prediction

Get a model to predict implicit questions (goals), by training on explicit questions.

= f

r e e s u p e r v i s i

  • n

! Data:

  • Only dialogue contains enough questions.
  • Extract dialogues from movie scripts and books.

– E.g., 1M dialogues from BookCorpus.

Models (e.g.):

  • Simple word-level RNN. (Westera 2018 [SemDial])
  • Sentence-level RNN on pre-trained sent. embeddings.
slide-53
SLIDE 53

Example (aim)

BookCorpus/590124__come-as-you-are.txt

So, there's this party tonight… August, no! But it's perfect! It’s my last night in Aberdeen! I'll never be in this town again! No, we can't. So, there's this party tonight… August, no! But it's perfect! It’s my last night in Aberdeen! I'll never be in this town again! No, we can't.

slide-54
SLIDE 54

Example (aim)

BookCorpus/590124__come-as-you-are.txt

So, there's this party tonight…

Shall we go?

August, no! But it's perfect! It’s my last night in Aberdeen! I'll never be in this town again! No, we can't. So, there's this party tonight…

Shall we go?

August, no! But it's perfect! It’s my last night in Aberdeen! I'll never be in this town again! No, we can't.

slide-55
SLIDE 55

Example (aim)

BookCorpus/590124__come-as-you-are.txt

So, there's this party tonight…

Shall we go?

August, no!

Why do you even bring it up?

But it's perfect! It’s my last night in Aberdeen! I'll never be in this town again! No, we can't. So, there's this party tonight…

Shall we go?

August, no!

Why do you even bring it up?

But it's perfect! It’s my last night in Aberdeen! I'll never be in this town again! No, we can't.

slide-56
SLIDE 56

Example (aim)

BookCorpus/590124__come-as-you-are.txt

So, there's this party tonight…

Shall we go?

August, no!

Why do you even bring it up?

But it's perfect! It’s my last night in Aberdeen! I'll never be in this town again!

Can we go?

No, we can't. So, there's this party tonight…

Shall we go?

August, no!

Why do you even bring it up?

But it's perfect! It’s my last night in Aberdeen! I'll never be in this town again!

Can we go?

No, we can't.

slide-57
SLIDE 57

Example (aim)

BookCorpus/590124__come-as-you-are.txt

So, there's this party tonight…

Shall we go?

August, no!

Why do you even bring it up?

But it's perfect! It’s my last night in Aberdeen! I'll never be in this town again!

Can we go?

No, we can't.

Why can’t we go?

So, there's this party tonight…

Shall we go?

August, no!

Why do you even bring it up?

But it's perfect! It’s my last night in Aberdeen! I'll never be in this town again!

Can we go?

No, we can't.

Why can’t we go?

slide-58
SLIDE 58

Summary

enables

  • how do NN models do it?
  • how do DS and FS relate?
  • can we predict discourse goals?

How?

messy crisp

slide-59
SLIDE 59

Predicting implicit and explicit questions

Matthijs Westera COLT kick-off workshop