Interpretable and Accurate Fine-grained Recognition via Region - - PowerPoint PPT Presentation

Interpretable and Accurate Fine-grained Recognition via Region Grouping

Zixuan Huang 1, Yin Li 2,1

• 1. Department of Computer Sciences
• 2. Department of Biostatistics & Medical Informatics

CVPR 2020 Interpretable and Accurate Fine-grained Recognition via Region Grouping 2

Part segmentation Part attribution Yellow-headed blackbird Only image-level label required! Input

Interpretation = part segmentation + part attribution

CVPR 2020 Interpretable and Accurate Fine-grained Recognition via Region Grouping 3

Related work

Zhang et al., CVPR’18 Zhou et al., CVPR’16 Chen et al., NeurIPS’19 Brendel et al., ICLR’19 Selvaraju et al., ICCV’17 Feng & Vedaldi, ICCV’17

CVPR 2020 Interpretable and Accurate Fine-grained Recognition via Region Grouping 4

Part segmentation Yellow-headed blackbird Input

Interpretation = part segmentation part attribution +

CVPR 2020 Interpretable and Accurate Fine-grained Recognition via Region Grouping 5

Input image Feature map Feature vectors Part assignment Part dictionary Part segmentation Region features

• Encode each part into
• ne vector
• Assign feature vectors

to different centers

Part segmentation via region grouping

CVPR 2020 Interpretable and Accurate Fine-grained Recognition via Region Grouping 6

Part segmentation Part attribution Yellow-headed blackbird

Interpretation = part segmentation part attribution +

CVPR 2020 Interpretable and Accurate Fine-grained Recognition via Region Grouping 7

Part segmentation Region features Attention map

• Generate region-based

attention

• Attention-guided

classification

Part attribution via region attention

Cliff swallow

Attention selects important regions for classification

CVPR 2020 Interpretable and Accurate Fine-grained Recognition via Region Grouping 8

Learning with image-level labels

How does an object part occur in natural images?

U-shaped distribution

CVPR 2020 Interpretable and Accurate Fine-grained Recognition via Region Grouping 9

Regularization by part occurrence

Match the empirical distribution to prior using Earth-Mover distance Max-pooling of part assignment as a part detector

Part assignment for bird head Max 0.98 0.04 0.92 0.13 … Empirical distribution Minibatch of N samples Calculate Earth-Mover distance

CVPR 2020 Interpretable and Accurate Fine-grained Recognition via Region Grouping 10

Results - CUB200

70 75 80 85 90

S T N K e r n e l M A

• C

N N K E R L D F L

• C

N N T A S N R e s N e t 1 1 O u r s

Accuracy (%)

5 10 15 20 25

DFF SCOPS Ours

Localization Error (%)

Bird species classification (accuracy) Bird landmark localization (interpretability)

CVPR 2020 Interpretable and Accurate Fine-grained Recognition via Region Grouping 11

Qualitative results

Input Assignment Attention

CVPR 2020 Interpretable and Accurate Fine-grained Recognition via Region Grouping 12

Results - iNaturalist2017

50 55 60 65 70

SSN TASN ResNet101 Ours

Accuracy (%)

5 10 15

CAM / Grad-CAM Guided Grad-CAM Ours

Pointing Error (%)

Species classification (accuracy) Pointing game using attention (interpretability)

See our paper for more results on iNaturalist and CelebA datasets

CVPR 2020 Interpretable and Accurate Fine-grained Recognition via Region Grouping 13

Conclusion

• An interpretable and accurate model for fine-grained classification
• Region grouping + attention = interpretability
• A novel prior as regularization
• Strong performance over challenging datasets

Thank you!

Project website: https://www.biostat.wisc.edu/~yli/cvpr2020-interp/