Practical Semantic Parsing for Spoken Language Understanding NAACL - - PowerPoint PPT Presentation

▶
practical semantic parsing for spoken language
SMART_READER_LITE
LIVE PREVIEW

Practical Semantic Parsing for Spoken Language Understanding NAACL - - PowerPoint PPT Presentation

Apr 17, 2023 265 likes •699 views

Practical Semantic Parsing for Spoken Language Understanding NAACL 2019 Marco Damonte 1 , Rahul Goel 2 , Tagyoung Chung 2 1 School of Informatics, University of Edinburgh 2 Amazon Alexa AI 1 / 42 What is the capital of California? Sacramento

slide-1
SLIDE 1

Practical Semantic Parsing for Spoken Language Understanding

NAACL 2019 Marco Damonte1, Rahul Goel2, Tagyoung Chung2

1 School of Informatics, University of Edinburgh 2 Amazon Alexa AI 1 / 42

slide-2
SLIDE 2

What is the capital of California? Sacramento Play the song Bohemian Rhapsody

Executable semantic parsing: the task of converting sentences into logical forms that can be directly used as queries.

2 / 42

slide-3
SLIDE 3

Contributions

1 Question Answering (Q&A) and Spoken Language Understanding (SLU) under the

same parsing framework:

  • Public Q&A corpora (English)
  • Proprietary Alexa SLU corpus (English)

2 Transfer learning to learn parsers on low-resource domains, for both Q&A and SLU:

  • Multi-task Learning
  • Pre-training

3 / 42

slide-4
SLIDE 4

SLU (Alexa) Data

Alexa data is annotated for intent/slot tagging:

Which cinemas screen Star|Title Wars|Title tonight|Time

Which we converted into trees: FindCinema Time Title Title tonight Wars Star

4 / 42

slide-5
SLIDE 5

SLU (Alexa) Data

Alexa data is annotated for intent/slot tagging:

Which cinemas screen Star|Title Wars|Title tonight|Time

Which we converted into trees: FindCinema Time Title Title tonight Wars Star

5 / 42

slide-6
SLIDE 6

SLU (Alexa) Data

Tree parsing allows to make more complex requests: and AddToListIntent PlayMusicIntent and MediaType apples

  • ranges

music Add apples and oranges to shopping list and play music

6 / 42

slide-7
SLIDE 7

SLU (Alexa) Data

DOMAIN SIZE TER NT WORDS closet 943 63 13 107 bookings 1280 10 19 42 cinema 13180 806 36 923 recipes 18721 530 40 643 search 23706 1621 51 1780

7 / 42

slide-8
SLIDE 8

SLU (Alexa) Data

DOMAIN SIZE TER NT WORDS closet 943 63 13 107 bookings 1280 10 19 42 cinema 13180 806 36 923 recipes 18721 530 40 643 search 23706 1621 51 1780

8 / 42

slide-9
SLIDE 9

Q&A Data

Overnight (Wang et al., 2015):

  • Questions annotated with Lambda DCS (Liang, 2013);
  • Divided in 8 domains;
  • Tree parsing.

getProperty Kobe Bryant reverse player num blocks

How many blocks were made by Kobe Bryant?

9 / 42

slide-10
SLIDE 10

Q&A Data

NLmaps (Lawrence and , 2016):

  • Questions about geographical facts;
  • No subdomains;
  • Tree parsing.

query area keyval name Edinburgh qtype count nwr keyval amenity prison

How many prisons does Edinburgh count?

10 / 42

slide-11
SLIDE 11

Q&A Data

DATASET DOMAIN SIZE TER NT Words Overnight publications 512 24 12 80 calendar 535 31 13 114 housing 601 34 13 109 recipes 691 30 13 121 restaurants 1060 40 13 144 basketball 1248 40 15 148 blocks 1276 30 13 99 social 2828 56 16 225 NLmaps 1200 160 24 280

11 / 42

slide-12
SLIDE 12

Q&A Data

DATASET DOMAIN SIZE TER NT Words Overnight publications 512 24 12 80 calendar 535 31 13 114 housing 601 34 13 109 recipes 691 30 13 121 restaurants 1060 40 13 144 basketball 1248 40 15 148 blocks 1276 30 13 99 social 2828 56 16 225 NLmaps 1200 160 24 280

12 / 42

slide-13
SLIDE 13

Parser

Which cinemas screen Star Wars tonight? STACK FindCinema FindCinema

13 / 42

slide-14
SLIDE 14

Parser

Which cinemas screen Star Wars tonight? STACK FindCinema FindCinema

14 / 42

slide-15
SLIDE 15

Parser

Which cinemas screen Star Wars tonight? STACK Title FindCinema FindCinema Title

15 / 42

slide-16
SLIDE 16

Parser

Which cinemas screen Star Wars tonight? STACK Star Title FindCinema FindCinema Title Star

16 / 42

slide-17
SLIDE 17

Parser

Which cinemas screen Star Wars tonight? STACK Title FindCinema FindCinema Title Star

17 / 42

slide-18
SLIDE 18

Parser

Which cinemas screen Star Wars tonight? STACK FindCinema FindCinema Title Star

18 / 42

slide-19
SLIDE 19

Parser

Which cinemas screen Star Wars tonight? STACK Title FindCinema FindCinema Title Title Star

19 / 42

slide-20
SLIDE 20

Parser

Which cinemas screen Star Wars tonight? STACK Wars Title FindCinema FindCinema Title Title Wars Star

20 / 42

slide-21
SLIDE 21

Parser

Which cinemas screen Star Wars tonight? STACK Title FindCinema FindCinema Title Title Wars Star

21 / 42

slide-22
SLIDE 22

Parser

Which cinemas screen Star Wars tonight? STACK FindCinema FindCinema Title Title Wars Star

22 / 42

slide-23
SLIDE 23

Parser

Which cinemas screen Star Wars tonight? STACK Time FindCinema FindCinema Time Title Title Wars Star

23 / 42

slide-24
SLIDE 24

Parser

Which cinemas screen Star Wars tonight? STACK tonight Time FindCinema FindCinema Time Title Title tonight Wars Star

24 / 42

slide-25
SLIDE 25

Parser

Transition-based parser of Cheng et al. (2017) + character-level embeddings and copy mechanism:

x0, x1, . . . , xn

HISTORY

. . .

BUFFER

. . .

STACK

. . . ATTENTION FEED-FORWARD LAYERS TER RED NT t0 . . . tn x0 . . . xn TER COPY nt0 . . . ntn NT

25 / 42

slide-26
SLIDE 26

Results

DATA TASK DOMAIN ACCURACY Overnight Q&A publications 26.1 calendar 32.1 housing 21.2 recipes 48.1 restaurants 33.7 basketball 66.5 blocks 22.8 social 50.9 NLMaps Q&A 60.7 Alexa SLU search 52.7 recipes 47.6 cinema 56.9 bookings 77.7 closet 44.1

26 / 42

slide-27
SLIDE 27

Results

DATA TASK DOMAIN BASELINE −Copy Overnight Q&A publications 26.1 +1.2 calendar 32.1 +6.0 housing 21.2

  • 2.2

recipes 48.1

  • 0.4

restaurants 33.7

  • 1.5

basketball 66.5

  • 1.3

blocks 22.8

  • 0.2

social 50.9

  • 6.0

NLMaps Q&A 60.7

  • 15.6

Alexa SLU search 52.7

  • 17.1

recipes 47.6

  • 6.7

cinema 56.9

  • 25.4

bookings 77.7

  • 5.4

closet 44.1 26.5

27 / 42

slide-28
SLIDE 28

Results

DATA TASK DOMAIN BASELINE −Attention Overnight Q&A publications 26.1 +6.8 calendar 32.1 +11.4 housing 21.2 +8.5 recipes 48.1 +10.2 restaurants 33.7 +3.6 basketball 66.5 +3.1 blocks 22.8 +2.3 social 50.9 +0.3 NLmaps Q&A 60.7

  • 17.2

Alexa SLU search 52.7

  • 17.8

recipes 47.6

  • 9.7

cinema 56.9

  • 21.4

bookings 77.7 77.7 closet 44.1

  • 8.2

28 / 42

slide-29
SLIDE 29

Reminder: Q&A Data

DATASET DOMAIN SIZE TER NT Words Overnight publications 512 24 12 80 calendar 535 31 13 114 housing 601 34 13 109 recipes 691 30 13 121 restaurants 1060 40 13 144 basketball 1248 40 15 148 blocks 1276 30 13 99 social 2828 56 16 225 NLmaps 1200 160 24 280

29 / 42

slide-30
SLIDE 30

Results

DATA TASK DOMAIN BASELINE −Attention Overnight Q&A publications 26.1 +6.8 calendar 32.1 +11.4 housing 21.2 +8.5 recipes 48.1 +10.2 restaurants 33.7 +3.6 basketball 66.5 +3.1 blocks 22.8 +2.3 social 50.9 +0.3 NLmaps Q&A 60.7

  • 17.2

Alexa SLU search 52.7

  • 17.8

recipes 47.6

  • 9.7

cinema 56.9

  • 21.4

bookings 77.7 77.7 closet 44.1

  • 8.2

30 / 42

slide-31
SLIDE 31

Transfer Learning: Pretraining

HIGH-RESOURCE DOMAIN

x0, x1, . . . , xn HISTORY . . . BUFFER . . . STACK . . . ATTENTION FEED-FORWARD LAYERS TER RED NT t0 . . . tn x0 . . . xn TER COPY nt0 . . . ntn NT

LOW-RESOURCE DOMAIN

x0, x1, . . . , xn HISTORY . . . BUFFER . . . STACK . . . ATTENTION FEED-FORWARD LAYERS TER RED NT t0 . . . tn x0 . . . xn TER COPY nt0 . . . ntn NT

31 / 42

slide-32
SLIDE 32

Transfer Learning: Pretraining

HIGH-RESOURCE DOMAIN

x0, x1, . . . , xn HISTORY . . . BUFFER . . . STACK . . . ATTENTION FEED-FORWARD LAYERS TER RED NT t0 . . . tn x0 . . . xn TER COPY nt0 . . . ntn NT

LOW-RESOURCE DOMAIN

x0, x1, . . . , xn HISTORY . . . BUFFER . . . STACK . . . ATTENTION FEED-FORWARD LAYERS TER RED NT t0 . . . tn x0 . . . xn TER COPY nt0 . . . ntn NT

32 / 42

slide-33
SLIDE 33

Transfer Learning: Pretraining

HIGH-RESOURCE DOMAIN

x0, x1, . . . , xn HISTORY . . . BUFFER . . . STACK . . . ATTENTION FEED-FORWARD LAYERS TER RED NT t0 . . . tn x0 . . . xn TER COPY nt0 . . . ntn NT

LOW-RESOURCE DOMAIN

x0, x1, . . . , xn HISTORY . . . BUFFER . . . STACK . . . ATTENTION FEED-FORWARD LAYERS TER RED NT t0 . . . tn x0 . . . xn TER COPY nt0 . . . ntn NT

33 / 42

slide-34
SLIDE 34

Transfer Learning: Pretraining

HIGH-RESOURCE DOMAIN

x0, x1, . . . , xn HISTORY . . . BUFFER . . . STACK . . . ATTENTION FEED-FORWARD LAYERS TER RED NT t0 . . . tn x0 . . . xn TER COPY nt0 . . . ntn NT

LOW-RESOURCE DOMAIN

x0, x1, . . . , xn HISTORY . . . BUFFER . . . STACK . . . ATTENTION FEED-FORWARD LAYERS TER RED NT t0 . . . tn x0 . . . xn TER COPY nt0 . . . ntn NT

34 / 42

slide-35
SLIDE 35

Transfer Learning: Multi-task Learning

x0, x1, . . . , xn

HISTORY

. . .

BUFFER

. . .

STACK

. . . ATTENTION FEED-FORWARD LAYERS TER RED NT t0 . . . tn x0 . . . xn TER COPY nt0 . . . ntn NT HR DOMAIN t0 . . . tn x0 . . . xn TER COPY LR DOMAIN

35 / 42

slide-36
SLIDE 36

Transfer Learning: Multi-task Learning

x0, x1, . . . , xn

HISTORY

. . .

BUFFER

. . .

STACK

. . . ATTENTION FEED-FORWARD LAYERS TER RED NT t0 . . . tn x0 . . . xn TER COPY nt0 . . . ntn NT HR DOMAIN t0 . . . tn x0 . . . xn TER COPY LR DOMAIN

36 / 42

slide-37
SLIDE 37

Trasfer Learning: Multi-task Learning

x0, x1, . . . , xn

HISTORY

. . .

BUFFER

. . .

STACK

. . . ATTENTION FEED-FORWARD LAYERS TER RED NT t0 . . . tn x0 . . . xn TER COPY nt0 . . . ntn NT HR DOMAIN t0 . . . tn x0 . . . xn TER COPY LR DOMAIN

37 / 42

slide-38
SLIDE 38

Transfer Learning: Results on Overnight (Q&A)

Q&A transfer learning helps for low-resource domains housing publications 29.6 32.9 38.1 37.3 38.1 40.4 BASELINE MULTI-TASK LEARNING PRETRAING

38 / 42

slide-39
SLIDE 39

Multi-task Learning for Alexa (SLU)

x0, x1, . . . , xn

HISTORY

. . .

BUFFER

. . .

STACK

. . . ATTENTION FEED-FORWARD LAYERS TER RED NT t0 . . . tn x0 . . . xn TER COPY nt0 . . . ntn NT HR DOMAIN t0 . . . tn x0 . . . xn TER COPY nt0 . . . ntn NT LR DOMAIN

39 / 42

slide-40
SLIDE 40

Transfer Learning: Results on Alexa (SLU)

SLU transfer learning helps for low-resource domains: booking closet 77.7 44.1 81.2 52.5 78.9 50.8 BASELINE MULTI-TASK LEARNING PRETRAING

40 / 42

slide-41
SLIDE 41

Transfer learning: from SLU to Q&A

  • Recipe domain exist in both Q&A and SLU;
  • Pretrain with SLU’s recipe for Q&A’s recipes;
  • Results: 58.3 → 61.1.

41 / 42

slide-42
SLIDE 42

Takeaways

  • Executable semantic parsing unifies Q&A and SLU;
  • One model for all is fine but some choices must be revisited (e.g. attention, copy);
  • Transfer learning for low-resource domains on Q&A and SLU.

42 / 42