Commonsense in Natural Language Yonatan Bisk , Rowan Zellers, Ronan - - PowerPoint PPT Presentation

β–Ά
commonsense in natural language
SMART_READER_LITE
LIVE PREVIEW

Commonsense in Natural Language Yonatan Bisk , Rowan Zellers, Ronan - - PowerPoint PPT Presentation

Sep 20, 2022 101 likes β€’303 views

PIQA: Reasoning about Physical Commonsense in Natural Language Yonatan Bisk , Rowan Zellers, Ronan Le Bras, Jianfeng Gao and Yejin Choi Whats the meaning of a word? 01 ) This is silly right?

slide-1
SLIDE 1

PIQA: Reasoning about Physical Commonsense in Natural Language

Yonatan Bisk, Rowan Zellers, Ronan Le Bras, Jianfeng Gao and Yejin Choi

slide-2
SLIDE 2

What’s the meaning of a word? π‘ž π‘₯𝑒 π‘₯0β€¦π‘’βˆ’1) π‘ž π‘₯𝑗 π‘₯0β€¦π‘—βˆ’1, π‘₯𝑗+1…𝑂) This is silly right?

slide-3
SLIDE 3
slide-4
SLIDE 4
slide-5
SLIDE 5

https://www.instructables.com/id/Stylish-Duct-Tape-Backpack/

slide-6
SLIDE 6

Context

6

Place a cutting board or another flat object over the top half of the backpack. Place about 10

  • lbs. (4.5 kg) of heavy objects on top. Leave the weights in place for at least 2 hours. (You can

accelerate the process by using heavier objects, or by sitting on the cutting board as you read through the rest of this Instructable!)

Core tool: Heavy flat object To press a ____ you should place ____ on it for ____ hours. Required component: Time Desire: Something smushed

slide-7
SLIDE 7

Annotation

  • 1. Visit an Instructables.com page to collect inspiration
  • 2. Goal: What is a related sub-task?
  • 3. Solution: How would you go about achieving this?
  • 4. Trick: What similar answer would fail to achieve the goal?

7

slide-8
SLIDE 8

Physical Commonsense

8

To separate egg whites from the yolk using a water bottle, you should … Squeeze the water bottle and press it against the yolk. Release, which creates suction and lifts the yolk. Place the water bottle and press it against the yolk. Keep pushing, which creates suction and lifts the yolk.

A B

βœ” ❌

slide-9
SLIDE 9

Physical Commonsense

9

To separate egg whites from the yolk using a water bottle, you should … Squeeze the water bottle and press it against the yolk. Release, which creates suction and lifts the yolk. Place the water bottle and press it against the yolk. Keep pushing, which creates suction and lifts the yolk.

A B βœ” ❌

slide-10
SLIDE 10

Tools and Dataset Bias

  • What can I use whisk eggs?
  • What can I use to pop a balloon?
  • What can I use to close an electrical circuit?
  • What can I use to dig a small hole in the garden?
  • What can I use to spread butter?
  • What can I use to open a paint can?
  • What can I use to let air out a tire?

10

Bad idea: Asking humans questions

Fork Fork Fork Fork Fork Fork Fork

slide-11
SLIDE 11

Annotation Artifacts

11

Gururangan, Suchin, et al. "Annotation artifacts in natural language inference data." (2018) Poliak, Adam, et al. "Hypothesis only baselines in natural language inference." (2018) What about… Adversarial Filtering (Zellers 2018) β†’ specifically AFLite (Sakaguchi 2020) Solution:

Don’t go anywhere!

Gratuitous Popups!

slide-12
SLIDE 12

AFLite

12

All data Classifiers Classify everything Keep boundary

5,000 examples / classifier

slide-13
SLIDE 13

Dataset & Baselines

  • Trivial patterns removed
  • Dataset balanced
  • 16K Training, 2K Validation, 3K Testing

13 50 55 60 65 70 75 80 85 90 95 100 Valid Test GPT BERT-Large RoBERTa-Large

Human

slide-14
SLIDE 14

A single word

14

slide-15
SLIDE 15

Common replacements

15

slide-16
SLIDE 16

Qualitative

16

slide-17
SLIDE 17

What do we want to learn?

Hollow Long & thin Flexible Straws Y Y Y Toothpicks N Y N Chopsticks N Y N PVC Pipe Y Y Y Steel Pipe Y Y N

slide-18
SLIDE 18

How do we want to learn it? A little bit of both?

slide-19
SLIDE 19

Knowledge

Physical Commonsense

Written Knowledge

*Shapes are not to scale

What’s out here?

Thanks!

http://yonatanbisk.com/piqa