A Simple Method for Commonsense Reasoning Trieu and Quoc - - PowerPoint PPT Presentation

▶
a simple method for commonsense reasoning
SMART_READER_LITE
LIVE PREVIEW

A Simple Method for Commonsense Reasoning Trieu and Quoc - - PowerPoint PPT Presentation

Sep 22, 2022 257 likes •819 views

A Simple Method for Commonsense Reasoning Trieu and Quoc Google AI Residency Program | g.co/airesidency The trophy cannot fit in the suitcase because it is too big. Google AI Residency Program | g.co/airesidency ?? The trophy

slide-1
SLIDE 1

Trieu and Quoc

A Simple Method for Commonsense Reasoning

slide-2
SLIDE 2

Google AI Residency Program | g.co/airesidency

slide-3
SLIDE 3

Google AI Residency Program | g.co/airesidency

  • The trophy cannot fit in the suitcase because it is too big.
slide-4
SLIDE 4

Google AI Residency Program | g.co/airesidency

  • The trophy cannot fit in the suitcase because it is too big.

??

slide-5
SLIDE 5

Google AI Residency Program | g.co/airesidency

  • The trophy cannot fit in the suitcase because it is too big.

??

slide-6
SLIDE 6

Google AI Residency Program | g.co/airesidency

  • The trophy cannot fit in the suitcase because it is too big.

Winograd Schema Challenge

??

slide-7
SLIDE 7

Coreference Resolution

Google AI Residency Program | g.co/airesidency

  • The trophy cannot fit in the suitcase because it is too big.

Winograd Schema Challenge

??

Winograd Schema Challenge

slide-8
SLIDE 8

Google AI Residency Program | g.co/airesidency

Winograd Schema Challenge

The racecar zoomed by the school bus because [it] was going so fast.

slide-9
SLIDE 9

Google AI Residency Program | g.co/airesidency

Winograd Schema Challenge

The racecar zoomed by the school bus because [it] was going so fast.

slide-10
SLIDE 10

Google AI Residency Program | g.co/airesidency

Winograd Schema Challenge

The racecar zoomed by the school bus because [it] was going so fast. Frank was pleased when Bill said [he] was the winner of the competition.

slide-11
SLIDE 11

Coreference Resolution

Google AI Residency Program | g.co/airesidency

  • The trophy cannot fit in the suitcase because it is too big.

Winograd Schema Challenge

??

Winograd Schema Challenge

slide-12
SLIDE 12

Coreference Resolution

Google AI Residency Program | g.co/airesidency

  • The trophy cannot fit in the suitcase because it is too big.

Winograd Schema Challenge

??

Expensive to construct commonsensereasoning.org < 300 samples. Winograd Schema Challenge

slide-13
SLIDE 13

Google AI Residency Program | g.co/airesidency

  • The trophy cannot fit in the suitcase because it is too big.

Winograd Schema Challenge

??

Random Guess: ~50% Human: ~90% Human: ~90% Random Guess: ~50%

slide-14
SLIDE 14

Google AI Residency Program | g.co/airesidency

  • The trophy cannot fit in the suitcase because it is too big.

Winograd Schema Challenge

??

Random Guess: ~50% SOTA: ~53% Human: ~90% SOTA: ~53% Human: ~90% Random Guess: ~50%

slide-15
SLIDE 15

Google AI Residency Program | g.co/airesidency

  • The trophy cannot fit in the suitcase because it is too big.

Winograd Schema Challenge

??

Random Guess: ~50% SOTA: ~53% Human: ~90% SOTA: ~53% Human: ~90% Random Guess: ~50%

Wordnet(1995) ConceptNet(2004)-17m Cyc(1984) Google Search API

Rule-based reasoning, Hand-crafted features.

slide-16
SLIDE 16

Google AI Residency Program | g.co/airesidency

  • The trophy cannot fit in the suitcase because it is too big.

Winograd Schema Challenge

??

Random Guess: ~50% SOTA: ~53% Human: ~90% SOTA: ~53% Human: ~90% Random Guess: ~50%

Word2Vec Wordnet(1995) ConceptNet(2004)-17m Cyc(1984) Google Search API

slide-17
SLIDE 17

Google AI Residency Program | g.co/airesidency

  • The trophy cannot fit in the suitcase because it is too big.

Winograd Schema Challenge

??

Random Guess: ~50% SOTA: ~53% Human: ~90% SOTA: ~53% Human: ~90% Random Guess: ~50%

Word2Vec + Supervised DeepNN Wordnet(1995) ConceptNet(2004)-17m Cyc(1984) Google Search API

slide-18
SLIDE 18

Google AI Residency Program | g.co/airesidency

  • The trophy cannot fit in the suitcase because it is too big.

Winograd Schema Challenge

??

Random Guess: ~50% SOTA: ~53% Human: ~90% SOTA: ~53% Human: ~90% Random Guess: ~50%

Word2Vec + Supervised DeepNN Wordnet(1995) ConceptNet(2004)-17m Cyc(1984) Google Search API

slide-19
SLIDE 19

Google AI Residency Program | g.co/airesidency

  • The trophy cannot fit in the suitcase because it is too big.

Winograd Schema Challenge

??

Random Guess: ~50% SOTA: ~53% Human: ~90% SOTA: ~53% Human: ~90% Random Guess: ~50% Ours LM: ~64%

slide-20
SLIDE 20

Google AI Residency Program | g.co/airesidency

  • The trophy cannot fit in the suitcase because it is too big.

Method

slide-21
SLIDE 21

Google AI Residency Program | g.co/airesidency

  • The trophy cannot fit in the suitcase because it is too big.

The trophy cannot fit in the suitcase because the trophy is too big .

Method

slide-22
SLIDE 22

Google AI Residency Program | g.co/airesidency

  • The trophy cannot fit in the suitcase because it is too big.

Method

The trophy cannot fit in the suitcase because the trophy is too big . The trophy cannot fit in the suitcase because the suitcase is too big .

slide-23
SLIDE 23

Google AI Residency Program | g.co/airesidency

  • The trophy cannot fit in the suitcase because it is too big.

Method

The trophy cannot fit in the suitcase because the trophy is too big . The trophy cannot fit in the suitcase because the suitcase is too big .

More probable?

slide-24
SLIDE 24

Google AI Residency Program | g.co/airesidency

  • The trophy cannot fit in the suitcase because it is too big.

Method

The trophy cannot fit in the suitcase because the trophy is too big . The trophy cannot fit in the suitcase because the suitcase is too big .

More probable?

slide-25
SLIDE 25

Google AI Residency Program | g.co/airesidency

  • The trophy cannot fit in the suitcase because it is too big.

Method

The trophy cannot fit in the suitcase because the trophy is too big . The trophy cannot fit in the suitcase because the suitcase is too big .

More probable?

slide-26
SLIDE 26

Google AI Residency Program | g.co/airesidency

  • The trophy cannot fit in the suitcase because it is too big.

Method

The trophy cannot fit in the suitcase because the trophy is too big . The trophy cannot fit in the suitcase because the suitcase is too big .

More probable?

slide-27
SLIDE 27

Google AI Residency Program | g.co/airesidency

  • The trophy cannot fit in the suitcase because it is too big.

Method

The trophy cannot fit in the suitcase because the trophy is too big . The trophy cannot fit in the suitcase because the suitcase is too big .

slide-28
SLIDE 28

Google AI Residency Program | g.co/airesidency

  • The trophy cannot fit in the suitcase because it is too big.

Method

The trophy cannot fit in the suitcase because the trophy is too big . The trophy cannot fit in the suitcase because the suitcase is too big .

Language Model

slide-29
SLIDE 29

Google AI Residency Program | g.co/airesidency

  • The trophy cannot fit in the suitcase because it is too big.

Method

The trophy cannot fit in the suitcase because the trophy is too big . The trophy cannot fit in the suitcase because the suitcase is too big .

Language Model

slide-30
SLIDE 30

Google AI Residency Program | g.co/airesidency

  • The trophy cannot fit in the suitcase because it is too big.

Method

The trophy cannot fit in the suitcase because the trophy is too big . The trophy cannot fit in the suitcase because the suitcase is too big .

Language Model

slide-31
SLIDE 31

Google AI Residency Program | g.co/airesidency

  • The trophy cannot fit in the suitcase because it is too big.

Method

The trophy cannot fit in the suitcase because the trophy is too big . The trophy cannot fit in the suitcase because the suitcase is too big .

Language Model

slide-32
SLIDE 32

Google AI Residency Program | g.co/airesidency

  • The trophy cannot fit in the suitcase because it is too big.

Method

The trophy cannot fit in the suitcase because the trophy is too big . The trophy cannot fit in the suitcase because the suitcase is too big .

Language Model

slide-33
SLIDE 33

Google AI Residency Program | g.co/airesidency

  • The trophy cannot fit in the suitcase because it is too big.

Method

The trophy cannot fit in the suitcase because the trophy is too big . The trophy cannot fit in the suitcase because the suitcase is too big .

Language Model

slide-34
SLIDE 34

Google AI Residency Program | g.co/airesidency

  • The trophy cannot fit in the suitcase because it is too big.

Method

The trophy cannot fit in the suitcase because the trophy is too big . The trophy cannot fit in the suitcase because the suitcase is too big .

Language Model

slide-35
SLIDE 35

Google AI Residency Program | g.co/airesidency

  • The trophy cannot fit in the suitcase because it is too big.

Method

The trophy cannot fit in the suitcase because the trophy is too big . The trophy cannot fit in the suitcase because the suitcase is too big .

Language Model

slide-36
SLIDE 36

Google AI Residency Program | g.co/airesidency

  • The trophy cannot fit in the suitcase because it is too big.

Method

The trophy cannot fit in the suitcase because the trophy is too big . The trophy cannot fit in the suitcase because the suitcase is too big .

Language Model

slide-37
SLIDE 37

The trophy cannot fit in the suitcase because the trophy is too big . The trophy cannot fit in the suitcase because the suitcase is too big .

  • The trophy cannot fit in the suitcase because it is too big.

Method

Language Model

Google AI Residency Program | g.co/airesidency

slide-38
SLIDE 38

The trophy cannot fit in the suitcase because the trophy is too big . The trophy cannot fit in the suitcase because the suitcase is too big .

  • The trophy cannot fit in the suitcase because it is too big.

Method

Language Model

Google AI Residency Program | g.co/airesidency

slide-39
SLIDE 39

The trophy cannot fit in the suitcase because the trophy is too big . The trophy cannot fit in the suitcase because the suitcase is too big .

  • The trophy cannot fit in the suitcase because it is too big.

Method

Language Model

=

Google AI Residency Program | g.co/airesidency

0.0 1.0 ∞

slide-40
SLIDE 40

The trophy cannot fit in the suitcase because the trophy is too big . The trophy cannot fit in the suitcase because the suitcase is too big .

  • The trophy cannot fit in the suitcase because it is too big.

Method

Language Model

=

Google AI Residency Program | g.co/airesidency

0.0 1.0 ∞

slide-41
SLIDE 41

The trophy cannot fit in the suitcase because the trophy is too big . The trophy cannot fit in the suitcase because the suitcase is too big .

  • The trophy cannot fit in the suitcase because it is too big.

Method

Language Model

=

Google AI Residency Program | g.co/airesidency

0.0 1.0 ∞

slide-42
SLIDE 42

The trophy cannot fit in the suitcase because the trophy is too big . The trophy cannot fit in the suitcase because the suitcase is too big .

  • The trophy cannot fit in the suitcase because it is too big.

Method

Language Model

=

Google AI Residency Program | g.co/airesidency

0.0 1.0 ∞

  • The trophy cannot fit in the suitcase because it is too small.
slide-43
SLIDE 43

The trophy cannot fit in the suitcase because the trophy is too big . The trophy cannot fit in the suitcase because the suitcase is too big .

  • The trophy cannot fit in the suitcase because it is too big.

Method

Language Model

=

Google AI Residency Program | g.co/airesidency

0.0 1.0 ∞

  • The trophy cannot fit in the suitcase because it is too small.
slide-44
SLIDE 44

The trophy cannot fit in the suitcase because the trophy is too big . The trophy cannot fit in the suitcase because the suitcase is too big .

  • The trophy cannot fit in the suitcase because it is too big.

Method

Language Model

=

Google AI Residency Program | g.co/airesidency

0.0 1.0 ∞

  • The trophy cannot fit in the suitcase because it is too small.
slide-45
SLIDE 45

Capturing special words

Google AI Residency Program | g.co/airesidency

slide-46
SLIDE 46

Capturing special words

Google AI Residency Program | g.co/airesidency

  • Vocabulary of 800K words, including common names
slide-47
SLIDE 47

Capturing special words

Google AI Residency Program | g.co/airesidency

slide-48
SLIDE 48

Results

Google AI Residency Program | g.co/airesidency

slide-49
SLIDE 49

Results

Google AI Residency Program | g.co/airesidency

Gutenberg Books LM-1-Billion CommonCrawl SQuAD

slide-50
SLIDE 50

Results

Google AI Residency Program | g.co/airesidency

Gutenberg Books LM-1-Billion CommonCrawl SQuAD + STORIES

STORIES = CommonCrawl documents w/ largest overlapping n-grams

slide-51
SLIDE 51

Results

Google AI Residency Program | g.co/airesidency

Gutenberg Books LM-1-Billion CommonCrawl SQuAD + STORIES

STORIES = CommonCrawl documents w/ largest overlapping n-grams

slide-52
SLIDE 52

Diversity of Training Data

Google AI Residency Program | g.co/airesidency

slide-53
SLIDE 53

Preprint and Code for Reproducing

Google AI Residency Program | g.co/airesidency

slide-54
SLIDE 54

Takeaway

Google AI Residency Program | g.co/airesidency

  • Deep NN can capture Commonsense.
  • Commonsense representation does not

have to be Graphs or Tuples, but - might as well be vectors.

slide-55
SLIDE 55

Thank you!

Q & A

Google AI Residency Program | g.co/airesidency