It Takes Two to Tango: Towards Theory of AIs Mind It Takes Two to - PowerPoint PPT Presentation

Vicki’s Quirks • Imperfect vision • Limited capability to understand language • Can’t reason about common-sense • Limited vocabulary • Doesn’t understand question-image relevance • Heavily influenced by dataset biases Vicki Slide: 12

Vicki’s Quirks • Imperfect vision • Limited capability to understand language • Can’t reason about common-sense • Limited vocabulary • Doesn’t understand question-image relevance • Heavily influenced by dataset biases Q1, I1 A1 Vicki Slide: 12

Vicki’s Quirks • Imperfect vision • Limited capability to understand language • Can’t reason about common-sense • Limited vocabulary • Doesn’t understand question-image relevance • Heavily influenced by dataset biases Q1, I1 A1 Q2, I2 A2 Vicki Slide: 12

Vicki’s Quirks • Imperfect vision • Limited capability to understand language • Can’t reason about common-sense • Limited vocabulary • Doesn’t understand question-image relevance • Heavily influenced by dataset biases Q1, I1 A1 Q2, I2 A2 Vicki Slide: 12

Vicki’s Quirks • Imperfect vision • Limited capability to understand language • Can’t reason about common-sense • Limited vocabulary • Doesn’t understand question-image relevance • Heavily influenced by dataset biases Q1, I1 A1 Q2, I2 A2 Vicki Qn, In An Slide: 12

Vicki’s Quirks • Imperfect vision • Limited capability to understand language • Can’t reason about common-sense • Limited vocabulary • Doesn’t understand question-image relevance • Heavily influenced by dataset biases Q1, I1 A1 Q2, I2 A2 Helps us pick on Vicki Vicki’s quirks Qn, In An Slide: 12

Vicki’s Quirks Slide: 13

Vicki’s Quirks What color is the grass? Blue Slide: 13

Vicki’s Quirks What color is the grass? Blue What are the people doing? Eating Slide: 13

Vicki’s Quirks What color is the grass? Blue What are the people doing? Eating How many people are there? 4 What is the man holding? Fire Hydrant Slide: 13

ToAIM Slide: 14

ToAIM • To study/evaluate ToAIM Large-scale experiments on MTurk Slide: 14

ToAIM • To study/evaluate ToAIM Large-scale experiments on MTurk Subjects on AMT Vicki Slide: 14

ToAIM • To study/evaluate ToAIM Large-scale experiments on MTurk Task Interface Subjects on AMT Vicki Slide: 14

ToAIM • To study/evaluate ToAIM Large-scale experiments on MTurk Task Interface Subjects on AMT Vicki Failure Prediction Knowledge Prediction Slide: 14

ToAIM Slide: 15

ToAIM • Failure Prediction Slide: 15

ToAIM • Failure Prediction How many people are there? Slide: 15

ToAIM • Failure Prediction How many people are there? Subject thinks Vicki will answer correctly Slide: 15

ToAIM • Failure Prediction Correctly How many people are there? Subject thinks Vicki will answer correctly Slide: 15

ToAIM • Failure Prediction Slide: 16

ToAIM Slide: 17

ToAIM • Knowledge Prediction Slide: 17

ToAIM • Knowledge Prediction How many people are there? Slide: 17

ToAIM • Knowledge Prediction 4 How many people are there? Subject thinks Vicki will answer 4 Slide: 17

ToAIM • Knowledge Prediction 4 4 How many people are there? Subject thinks Vicki will answer 4 Slide: 17

ToAIM • Knowledge Prediction Slide: 18

ToAIM Slide: 19