Counterfactual Visual Explanations Yash Goyal Ziyan Wu Jan Ernst - - PowerPoint PPT Presentation

Counterfactual Visual Explanations

Dhruv Batra (Georgia Tech) Devi Parikh (Georgia Tech) Yash Goyal (Georgia Tech) Jan Ernst (Siemens) Ziyan Wu (Siemens) Stefan Lee (Georgia Tech)

An XAI Question: Why did the model predict Eared Grebe instead of Horned Grebe?

Counterfactual Visual Explanations

2

Eared Grebe Horned Grebe Western Grebe Pied-Bill Grebe Herring Gull

Bird Classification Deep Network

A

An XAI Question: Why did the model predict Eared Grebe instead of Horned Grebe?

Counterfactual Visual Explanations

3

Eared Grebe Horned Grebe Western Grebe Pied-Bill Grebe Herring Gull

Bird Classification Deep Network

A For input X, why did the model predict Y instead of Z?

An XAI Question: Why did the model predict Eared Grebe instead of Horned Grebe?

Counterfactual Visual Explanations

4

Eared Grebe Horned Grebe Western Grebe Pied-Bill Grebe Herring Gull

Bird Classification Deep Network

A Explanation through Counterfactual: If X was X*, then the outcome would have been Z rather than Y. For input X, why did the model predict Y instead of Z?

An XAI Question: Why did the model predict Eared Grebe instead of Horned Grebe?

5

Eared Grebe Horned Grebe Western Grebe Pied-Bill Grebe Herring Gull

Bird Classification Deep Network

Explanation through Counterfactual: What would have to change in image A to make the model predict Horned Grebe? A

Counterfactual Visual Explanations

An XAI Question: Why did the model predict Eared Grebe instead of Horned Grebe?

6

Eared Grebe Horned Grebe Western Grebe Pied-Bill Grebe Herring Gull

Bird Classification Deep Network

A B

Counterfactual Visual Explanations

An image where the network predicts Horned Grebe.

Explanation through Counterfactual: What would have to change in image A to make the model predict Horned Grebe?

7

What would have to change in image A to make the model predict Horned Grebe? A B

An image where the network predicts Horned Grebe.

Counterfactual Visual Explanations

8

What would have to change in image A to make the model predict Horned Grebe? A B

An image where the network predicts Horned Grebe.

If looked more like

Counterfactual Visual Explanations

9

What would have to change in image A to make the model predict Horned Grebe? A B

An image where the network predicts Horned Grebe.

If looked more like If looked more like

Counterfactual Visual Explanations

10

What would have to change in image A to make the model predict Horned Grebe? A B

An image where the network predicts Horned Grebe.

If looked more like If looked more like How can we identify these region pairs important to the model?

Counterfactual Visual Explanations

11

Bird Classification Deep Network

How can we identify these region pairs important to the model?

Counterfactual Visual Explanations

12

… ℎ𝑥 𝑒 ℎ 𝑒 𝑥 𝑔(𝐽)

Spatial Features

Spatial Feature Extractor

Bird Classification Deep Network

How can we identify these region pairs important to the model?

Counterfactual Visual Explanations

13

𝑕(𝑔 𝐽 )

log 𝑄 𝑧. 𝐽) log 𝑄 𝑧/ 𝐽) ⋮ ⋮ log 𝑄(𝑧 1 |𝐽)

… ℎ𝑥 𝑒 ℎ 𝑒 𝑥 𝑔(𝐽)

Spatial Features

Spatial Feature Extractor Decision Network

How can we identify these region pairs important to the model?

Counterfactual Visual Explanations

Bird Classification Deep Network

14

…

𝑔(𝐽3)

Query Image Features

…

𝑔(𝐽4)

Distractor Image Features

A B

Bird Classification Deep Network

Counterfactual Visual Explanations

15

…

𝑔(𝐽3)

Query Image Features

…

𝑔(𝐽4)

Distractor Image Features

… … … … … … … … … … … …

𝑄

Permutation Matrix

A B

* dramatization – permutation is in feature space.

Bird Classification Deep Network

Counterfactual Visual Explanations

16

…

𝑔(𝐽3)

Query Image Features

…

𝑔(𝐽4)

Distractor Image Features

… … … … … … … … … … … …

𝑄

Permutation Matrix

A

Bird Classification Deep Network

…

1 − 𝑏

Inverted Gating Vector

…

𝑏

Gating Vector

B

* dramatization – permutation is in feature space.

Counterfactual Visual Explanations

17

…

𝑔(𝐽3)

Query Image Features

…

𝑔(𝐽4)

Distractor Image Features

… … … … … … … … … … … …

𝑄

Permutation Matrix

A

Bird Classification Deep Network

…

1 − 𝑏

Inverted Gating Vector

…

𝑏

Gating Vector

…

𝑔(𝐽∗)

Counterfactual Edit Image Features

B

* dramatization – permutation is in feature space.

Counterfactual Visual Explanations

18

…

𝑔(𝐽3)

Query Image Features

…

𝑔(𝐽4)

Distractor Image Features

… … … … … … … … … … … …

𝑄

Permutation Matrix

A

Bird Classification Deep Network

…

1 − 𝑏

Inverted Gating Vector

…

𝑏

Gating Vector

…

𝑔(𝐽∗)

Counterfactual Edit Image Features

* dramatization – replacement is in feature space.

B

* dramatization – permutation is in feature space.

Counterfactual Visual Explanations

…

𝑔(𝐽3)

Query Image Features

…

𝑔(𝐽4)

Distractor Image Features

… … … … … … … … … … … …

𝑄

Permutation Matrix

A

Bird Classification Deep Network

…

1 − 𝑏

Inverted Gating Vector

…

𝑏

Gating Vector

…

𝑔(𝐽∗)

Counterfactual Edit Image Features

* dramatization – replacement is in feature space.

B

* dramatization – permutation is in feature space.

19

Counterfactual Visual Explanation Generation: Find 1) binary gating vector 𝒃, and 2) a permutation matrix 𝑄

Counterfactual Visual Explanations

…

𝑔(𝐽3)

Query Image Features

…

𝑔(𝐽4)

Distractor Image Features

… … … … … … … … … … … …

𝑄

Permutation Matrix

A

Bird Classification Deep Network

…

1 − 𝑏

Inverted Gating Vector

…

𝑏

Gating Vector

…

𝑔(𝐽∗)

Counterfactual Edit Image Features

* dramatization – replacement is in feature space.

B

* dramatization – permutation is in feature space.

20

Counterfactual Visual Explanation Generation: Find 1) binary gating vector 𝒃, and 2) a permutation matrix 𝑄 such that the model changes its decision to the distractor class

Counterfactual Visual Explanations

…

𝑔(𝐽3)

Query Image Features

…

𝑔(𝐽4)

Distractor Image Features

… … … … … … … … … … … …

𝑄

Permutation Matrix

A

Bird Classification Deep Network

…

1 − 𝑏

Inverted Gating Vector

…

𝑏

Gating Vector

…

𝑔(𝐽∗)

Counterfactual Edit Image Features

* dramatization – replacement is in feature space.

B

* dramatization – permutation is in feature space.

21

Counterfactual Visual Explanation Generation: Find 1) binary gating vector 𝒃, and 2) a permutation matrix 𝑄 such that the model changes its decision to the distractor class with the fewest edits (i.e. 𝒏𝒋𝒐 𝒃 𝟐)

Counterfactual Visual Explanations

Results – Single Edit

22

Eared Grebe Horned Grebe

Query Image A Distractor Image B Composite Image (for visualization only)

If the highlighted region in image A looked like the highlighted region in image B, then image A is more likely to be classified as class B.

Results – Single Edit

23

Eared Grebe Horned Grebe

Query Image A Distractor Image B Composite Image (for visualization only)

If the highlighted region in image A looked like the highlighted region in image B, then image A is more likely to be classified as class B.

Results – Single Edit

24

Query Image A Distractor Image B Composite Image (for visualization only)

Results – Single Edit

25

Query Image A Distractor Image B Composite Image (for visualization only)

Eared Grebe Horned Grebe Olive sided Flycatcher Myrtle Warbler Blue Grosbeak Indigo Bunting

Results – Single Edit

26

Query Image A Distractor Image B Composite Image (for visualization only)

Machine Teaching – Bird Classification

27

Do our counterfactual explanations help untrained participants learn to identify fine-grained classes?

Machine Teaching – Bird Classification

28

Training

Feedback: Sorry, its not a Bravo. It is actually an Alpha.

Do our counterfactual explanations help untrained participants learn to identify fine-grained classes?

Machine Teaching – Bird Classification

29

Testing Training

Feedback: Sorry, its not a Bravo. It is actually an Alpha.

Do our counterfactual explanations help untrained participants learn to identify fine-grained classes?

Machine Teaching – Bird Classification

30

Do our counterfactual explanations help untrained participants learn to identify fine-grained classes?

71.09 74.29 78.77

50 55 60 65 70 75 80 85 Instant Feedback (IF) IF + Feature Attribution Explanation (GradCAM) IF + Counterfactual Explanation

Questions?

Stop by our poster at #149 in Pacific Ballroom! Eared Grebe Horned Grebe

Query Image A Distractor Image B Composite Image (for visualization only)

Counterfactual Visual Explanations