Interpreting Adversarial Trained Convolutional Neural Networks Tianyuan Zhang , Zhanxing Zhu Peking University 1600012888@pku.edu.cn zhanxing.zhu@pku.edu.cn Poster: Pacific Ballroom #148 � 1
Contents • Normally trained CNNs typically lack of interpretability • Biased towards textures • Hypothesis: Adversarially trained CNNs could improve interpretability • Capture more semantic features: shapes. • Systematic experiments to validate the hypothesis • Discussions � 2
Normally Trained CNN • Interpreting normally trained CNN: texture bias Published as a conference paper at ICLR 2019 I MAGE N ET - TRAINED CNN S ARE BIASED TOWARDS TEXTURE ; INCREASING SHAPE BIAS IMPROVES ACCURACY AND ROBUSTNESS Robert Geirhos Patricia Rubisch University of T¨ ubingen & IMPRS-IS University of T¨ ubingen & U. of Edinburgh robert.geirhos@bethgelab.org p.rubisch@sms.ed.ac.uk Claudio Michaelis Matthias Bethge ∗ University of T¨ ubingen & IMPRS-IS University of T¨ ubingen claudio.michaelis@bethgelab.org matthias.bethge@bethgelab.org Felix A. Wichmann ∗ Wieland Brendel ∗ University of T¨ ubingen University of T¨ ubingen felix.wichmann@uni-tuebingen.de wieland.brendel@bethgelab.org (a) Texture image (b) Content image (c) Texture-shape cue conflict 81.4% Indian elephant 71.1% tabby cat 63.9% Indian elephant 10.3% indri 17.3% grey fox 26.4% indri � 3 8.2% 3.3% 9.6% black swan Siamese cat black swan
Fraction of 'shape' decisions Fraction of 'shape' decisions 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● Shape categories Shape categories ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Fraction of 'texture' decisions Fraction of 'texture' decisions Augmented Stylized- ImageNet could improve shape bias. � 4
Are there any other models that could improve shape bias? Adversarially trained CNNs! � 5
Adversarial Examples • Deep neural networks are easily fooled by adversarial examples. Not robust! f(x;w*) P(“panda”) = 57.7% f(x;w*) P(“gibbon”) = 99.3% ?! � 6
Adversarial Training <latexit sha1_base64="u2ls1kIdYnIemBsgoRHdXNC6+s=">ADFnicbVJdi9NAFJ3Er7V+bFcfblYCi2WJVkEBVlY1AWfZEW7u9Dphsl0kg47mcTMZGmJ8yt8a/4oMivopv/htv2i7u1w2Bw7nLm5mbhQ0tg+Ov5167fuHlr7Xbrzt179fbGw/2TV6VXAx5rvLyMGZGKnF0EqrxGFRCpbFShzEx6+a/sGJKI3M9Qc7L8Q4Y6mWieTMIhVteAP6CehEKMugQUp8BHrCSlEYqXLd6tJMaoigBmqnAkUOaMbsNI6R2gUX1TX0YAYDmEMfqJEZNpGwplC+Bod+DhMTuwIgAql0JH0ZvBildlHM3pLmU7tuNWFNJLpNOY+q3rBYOGSzN2tNVHzS5sA01KxmtasNJKPOt9xHuzvtPzNw5YehqjdObKotquR26I31lAjxZDHAm6JTru6jdCTaDRcFlEK5Ah6xqL2r/oZOcV5nQlitmzCgMCjum1yuhGvRyoiC8WOWihFCzTJhxvXitzroIjOBJC/x1RYW7FlHzTJj5lmMymZR5mKvIa/qjSqbPB/XUheVFZovD0oqBTaH5o7ARJaCWzVHwHgpcVbgU4arsniTWriE8OInXwb7W5sh4ndPOzsvV+tYI4/IY9IjIXlGdsgbskeGhHufva/ed+H/8X/5v/0fy2lvrfyPCTnyv/9DwPD8fs=</latexit> <latexit sha1_base64="u2ls1kIdYnIemBsgoRHdXNC6+s=">ADFnicbVJdi9NAFJ3Er7V+bFcfblYCi2WJVkEBVlY1AWfZEW7u9Dphsl0kg47mcTMZGmJ8yt8a/4oMivopv/htv2i7u1w2Bw7nLm5mbhQ0tg+Ov5167fuHlr7Xbrzt179fbGw/2TV6VXAx5rvLyMGZGKnF0EqrxGFRCpbFShzEx6+a/sGJKI3M9Qc7L8Q4Y6mWieTMIhVteAP6CehEKMugQUp8BHrCSlEYqXLd6tJMaoigBmqnAkUOaMbsNI6R2gUX1TX0YAYDmEMfqJEZNpGwplC+Bod+DhMTuwIgAql0JH0ZvBildlHM3pLmU7tuNWFNJLpNOY+q3rBYOGSzN2tNVHzS5sA01KxmtasNJKPOt9xHuzvtPzNw5YehqjdObKotquR26I31lAjxZDHAm6JTru6jdCTaDRcFlEK5Ah6xqL2r/oZOcV5nQlitmzCgMCjum1yuhGvRyoiC8WOWihFCzTJhxvXitzroIjOBJC/x1RYW7FlHzTJj5lmMymZR5mKvIa/qjSqbPB/XUheVFZovD0oqBTaH5o7ARJaCWzVHwHgpcVbgU4arsniTWriE8OInXwb7W5sh4ndPOzsvV+tYI4/IY9IjIXlGdsgbskeGhHufva/ed+H/8X/5v/0fy2lvrfyPCTnyv/9DwPD8fs=</latexit> <latexit sha1_base64="u2ls1kIdYnIemBsgoRHdXNC6+s=">ADFnicbVJdi9NAFJ3Er7V+bFcfblYCi2WJVkEBVlY1AWfZEW7u9Dphsl0kg47mcTMZGmJ8yt8a/4oMivopv/htv2i7u1w2Bw7nLm5mbhQ0tg+Ov5167fuHlr7Xbrzt179fbGw/2TV6VXAx5rvLyMGZGKnF0EqrxGFRCpbFShzEx6+a/sGJKI3M9Qc7L8Q4Y6mWieTMIhVteAP6CehEKMugQUp8BHrCSlEYqXLd6tJMaoigBmqnAkUOaMbsNI6R2gUX1TX0YAYDmEMfqJEZNpGwplC+Bod+DhMTuwIgAql0JH0ZvBildlHM3pLmU7tuNWFNJLpNOY+q3rBYOGSzN2tNVHzS5sA01KxmtasNJKPOt9xHuzvtPzNw5YehqjdObKotquR26I31lAjxZDHAm6JTru6jdCTaDRcFlEK5Ah6xqL2r/oZOcV5nQlitmzCgMCjum1yuhGvRyoiC8WOWihFCzTJhxvXitzroIjOBJC/x1RYW7FlHzTJj5lmMymZR5mKvIa/qjSqbPB/XUheVFZovD0oqBTaH5o7ARJaCWzVHwHgpcVbgU4arsniTWriE8OInXwb7W5sh4ndPOzsvV+tYI4/IY9IjIXlGdsgbskeGhHufva/ed+H/8X/5v/0fy2lvrfyPCTnyv/9DwPD8fs=</latexit> <latexit sha1_base64="u2ls1kIdYnIemBsgoRHdXNC6+s=">ADFnicbVJdi9NAFJ3Er7V+bFcfblYCi2WJVkEBVlY1AWfZEW7u9Dphsl0kg47mcTMZGmJ8yt8a/4oMivopv/htv2i7u1w2Bw7nLm5mbhQ0tg+Ov5167fuHlr7Xbrzt179fbGw/2TV6VXAx5rvLyMGZGKnF0EqrxGFRCpbFShzEx6+a/sGJKI3M9Qc7L8Q4Y6mWieTMIhVteAP6CehEKMugQUp8BHrCSlEYqXLd6tJMaoigBmqnAkUOaMbsNI6R2gUX1TX0YAYDmEMfqJEZNpGwplC+Bod+DhMTuwIgAql0JH0ZvBildlHM3pLmU7tuNWFNJLpNOY+q3rBYOGSzN2tNVHzS5sA01KxmtasNJKPOt9xHuzvtPzNw5YehqjdObKotquR26I31lAjxZDHAm6JTru6jdCTaDRcFlEK5Ah6xqL2r/oZOcV5nQlitmzCgMCjum1yuhGvRyoiC8WOWihFCzTJhxvXitzroIjOBJC/x1RYW7FlHzTJj5lmMymZR5mKvIa/qjSqbPB/XUheVFZovD0oqBTaH5o7ARJaCWzVHwHgpcVbgU4arsniTWriE8OInXwb7W5sh4ndPOzsvV+tYI4/IY9IjIXlGdsgbskeGhHufva/ed+H/8X/5v/0fy2lvrfyPCTnyv/9DwPD8fs=</latexit> <latexit sha1_base64="wZnUQB5AvoGEfSOmTH7qxkynWHw=">AC93icbVJda9RAFJ3Er7pau9VHXy4uhV0sJSmCghSKWvBJKrptYbMNk9lJdujMJGQmswf8QXHxTx1b/im/Gm90t9uGwOHc87c3ExWSWFsFP0Nwlu379y9t3a/8+Dh+qON7ubjI1M2NeNDVsqyPsmo4VJoPrTCSn5S1ZyqTPLj7Oxt2z/+wmsjSv3Zzis+VrTQIheMWqTSzWA9UJDCg4SO+WgodEUTvNMqQOwKfOQR9msA1zGEBihIJW20oYlQjfoQMfn0ie2xFAwqVER96fwetV5gDN6K1FMbXjzhYUqVgmnce4D74fbdcoejp7gA1B7AHSV5T5pK1lbgWZ9S1p8N/H9i5i8JY+80Tm8alTqxF/tTfWMCPF8McCHonBv4tNuLdqJFwXUQr0CPrOow7f5JiVrFNeWSWrMKI4qO3ZtLpPcd5LG8IqyM1rwEUJNFTdjt/hvHraQmUBe1vhqCwv2osNRZcxcZahsF2Wu9lrypt6osfmrsRO6aizXbHlQ3kiwJbSXACai5szKOQLKaoGzAptSXJXFq9LBJcRXP/k6ONrdiRF/fNHbf7Naxp5Sp6RPonJS7JP3pNDMiQsMHX4HvwI5yH38Kf4a+lNAxWnifkUoW/wF5ruXD</latexit> <latexit sha1_base64="wZnUQB5AvoGEfSOmTH7qxkynWHw=">AC93icbVJda9RAFJ3Er7pau9VHXy4uhV0sJSmCghSKWvBJKrptYbMNk9lJdujMJGQmswf8QXHxTx1b/im/Gm90t9uGwOHc87c3ExWSWFsFP0Nwlu379y9t3a/8+Dh+qON7ubjI1M2NeNDVsqyPsmo4VJoPrTCSn5S1ZyqTPLj7Oxt2z/+wmsjSv3Zzis+VrTQIheMWqTSzWA9UJDCg4SO+WgodEUTvNMqQOwKfOQR9msA1zGEBihIJW20oYlQjfoQMfn0ie2xFAwqVER96fwetV5gDN6K1FMbXjzhYUqVgmnce4D74fbdcoejp7gA1B7AHSV5T5pK1lbgWZ9S1p8N/H9i5i8JY+80Tm8alTqxF/tTfWMCPF8McCHonBv4tNuLdqJFwXUQr0CPrOow7f5JiVrFNeWSWrMKI4qO3ZtLpPcd5LG8IqyM1rwEUJNFTdjt/hvHraQmUBe1vhqCwv2osNRZcxcZahsF2Wu9lrypt6osfmrsRO6aizXbHlQ3kiwJbSXACai5szKOQLKaoGzAptSXJXFq9LBJcRXP/k6ONrdiRF/fNHbf7Naxp5Sp6RPonJS7JP3pNDMiQsMHX4HvwI5yH38Kf4a+lNAxWnifkUoW/wF5ruXD</latexit> <latexit sha1_base64="wZnUQB5AvoGEfSOmTH7qxkynWHw=">AC93icbVJda9RAFJ3Er7pau9VHXy4uhV0sJSmCghSKWvBJKrptYbMNk9lJdujMJGQmswf8QXHxTx1b/im/Gm90t9uGwOHc87c3ExWSWFsFP0Nwlu379y9t3a/8+Dh+qON7ubjI1M2NeNDVsqyPsmo4VJoPrTCSn5S1ZyqTPLj7Oxt2z/+wmsjSv3Zzis+VrTQIheMWqTSzWA9UJDCg4SO+WgodEUTvNMqQOwKfOQR9msA1zGEBihIJW20oYlQjfoQMfn0ie2xFAwqVER96fwetV5gDN6K1FMbXjzhYUqVgmnce4D74fbdcoejp7gA1B7AHSV5T5pK1lbgWZ9S1p8N/H9i5i8JY+80Tm8alTqxF/tTfWMCPF8McCHonBv4tNuLdqJFwXUQr0CPrOow7f5JiVrFNeWSWrMKI4qO3ZtLpPcd5LG8IqyM1rwEUJNFTdjt/hvHraQmUBe1vhqCwv2osNRZcxcZahsF2Wu9lrypt6osfmrsRO6aizXbHlQ3kiwJbSXACai5szKOQLKaoGzAptSXJXFq9LBJcRXP/k6ONrdiRF/fNHbf7Naxp5Sp6RPonJS7JP3pNDMiQsMHX4HvwI5yH38Kf4a+lNAxWnifkUoW/wF5ruXD</latexit> <latexit sha1_base64="wZnUQB5AvoGEfSOmTH7qxkynWHw=">AC93icbVJda9RAFJ3Er7pau9VHXy4uhV0sJSmCghSKWvBJKrptYbMNk9lJdujMJGQmswf8QXHxTx1b/im/Gm90t9uGwOHc87c3ExWSWFsFP0Nwlu379y9t3a/8+Dh+qON7ubjI1M2NeNDVsqyPsmo4VJoPrTCSn5S1ZyqTPLj7Oxt2z/+wmsjSv3Zzis+VrTQIheMWqTSzWA9UJDCg4SO+WgodEUTvNMqQOwKfOQR9msA1zGEBihIJW20oYlQjfoQMfn0ie2xFAwqVER96fwetV5gDN6K1FMbXjzhYUqVgmnce4D74fbdcoejp7gA1B7AHSV5T5pK1lbgWZ9S1p8N/H9i5i8JY+80Tm8alTqxF/tTfWMCPF8McCHonBv4tNuLdqJFwXUQr0CPrOow7f5JiVrFNeWSWrMKI4qO3ZtLpPcd5LG8IqyM1rwEUJNFTdjt/hvHraQmUBe1vhqCwv2osNRZcxcZahsF2Wu9lrypt6osfmrsRO6aizXbHlQ3kiwJbSXACai5szKOQLKaoGzAptSXJXFq9LBJcRXP/k6ONrdiRF/fNHbf7Naxp5Sp6RPonJS7JP3pNDMiQsMHX4HvwI5yH38Kf4a+lNAxWnifkUoW/wF5ruXD</latexit> • Adversarial training for defensing adversarial examples: • A robust optimization problem Projected Gradient Descent � min max δ ∈ S ` ( f ( x + � ; ✓ ) , y ) E ( x,y ) ∼ D θ k δ k ε min E ( x,y ) ∼ D [ ` ( f ( x ; ✓ ) , y )] Standard training θ • Interpreting adversarially trained CNNs ( AT-CNNs ) • What have AT-CNNs learned to make them robust? • Compared with standard CNNs, AT-CNNs tend to be more shape-biased. � 7
Two ways for interpreting AT-CNNs • Qualitative method (Lots of people did this) • Visualizing sensitivity maps � 8
Recommend
More recommend
