Published @ ICCV 2017 Context Many try to explain CNN predictions - - PowerPoint PPT Presentation

Published @ ICCV 2017

Context

• Many try to explain CNN predictions
• Good overview: CVPR 2018 tutorial on Interpretable ML for CV
• https://interpretablevision.github.io/
• Studies show existing methods that use gradients are problematic
• Today: a 'good' explenation method

What is an explenation?

• A rule that predicts the response of f to certain inputs
• Examples:
• f(x) = +1

if x contains a cat

• f(x) = f(x')

if x and x' are related by a rotation. x' is perturbed version

• Rules tested using data
• Quality of a rule: generalization to unseen data
• Rules can be discovered and learned

Explenations for CNN's: Saliency

• What region of the image is important to get decision f(x)?
• Idea: delete parts of x until posterior drops
• Deletion = blurring
• Task: find smallest mask m that minimizes f(x) significantly

Artefacts

• Naively learning the mask introduces artefacts
• Remember: explenation should generalize! So if the image x changes,

explenation should still hold.

• Solution 1: apply mask with random offsets during optimization
• Solution 2: regularize mask: smoother / more natural perturbations

Better interpretability

• Mask highlights only essential evidence.
• Other methods often find 'irrelevant' evidence.

Spurious correlation

• Method finds CNN errors

Better understanding

• Use extra annotations of Imagenet + masks to improve understanding
• Animal faces are more important than feet for CNN's

Adverserial images have strange masks

Detecting adverserial images

• After blurring the 'adverserial' mask,

CNN can recover original prediction in 40% of the cases

Take home

• Saliency != gradient
• Proposed method can be used to diagnose and understand CNN's
• Paper with extensive, proper evaluation
• Proposed method can be slow (requires 300 iterations of Adam)