Arxiv, 8 dec 2018
Main question How does a GAN represent our visual world internally? How do activations in the generator correlate with concepts in the output image?
Dissection
Results In the dining room dataset, a unit emerges to match dining table regions. More interestingly, the matched tables have different colors, materials, geometry, viewpoints, and levels of clutter: the only obvious commonality among these regions is the concept of a table. This unit’s featuremap correlates to the fully supervised segmentation model (Xiao et al., 2018) with a high IoU of 0.34.
Results • The set of all object classes matched by the units of a GAN provides a map of what a GAN has learned about the data. • Conference room GAN learns chairs, table, etc., • Bedroom GAN learns bed, window, etc. • Many units represent parts of objects: for example, the conference room GAN contains separate units for the body and head of a person. • Depth influences unit semantics
Debugging GANs • By examining unit semantics, we confirm that [contribution of ProGAN] increases not only the realism of results, but also the diversity of concepts represented by units: the number of types of objects, parts, and materials matching units increases by more than 40%.
Causal intervention
Fixing GANS Strikingly, this simple manual change to a network beats state- of-the-art GANs models.
GAN image editing: removal
GAN image editing: insertion
• https://www.youtube.com/watch?v=yVCgUYe4JTM
Recommend
More recommend
