Arxiv, 8 dec 2018 Main question How does a GAN represent our visual - - PowerPoint PPT Presentation

▶

Sep 23, 2023 247 likes •388 views

Arxiv, 8 dec 2018 Main question How does a GAN represent our visual world internally? How do activations in the generator correlate with concepts in the output image? Dissection Results In the dining room dataset, a unit emerges to match

SLIDE 1

Arxiv, 8 dec 2018

SLIDE 2

Main question

How does a GAN represent our visual world internally? How do activations in the generator correlate with concepts in the

utput image?

SLIDE 3

Dissection

SLIDE 4

Results

In the dining room dataset, a unit emerges to match dining table

regions. More interestingly, the matched tables have different colors,

materials, geometry, viewpoints, and levels of clutter: the only obvious commonality among these regions is the concept of a table. This unit’s featuremap correlates to the fully supervised segmentation model (Xiao et al., 2018) with a high IoU of 0.34.

SLIDE 5

Results

The set of all object classes matched by the units of a GAN provides a

map of what a GAN has learned about the data.

Conference room GAN learns chairs, table, etc.,
Bedroom GAN learns bed, window, etc.
Many units represent parts of objects: for example, the conference

room GAN contains separate units for the body and head of a person.

Depth influences unit semantics

SLIDE 6

SLIDE 7

Debugging GANs

By examining unit semantics, we confirm that [contribution of

ProGAN] increases not only the realism of results, but also the diversity of concepts represented by units: the number of types of

bjects, parts, and materials matching units increases by more than

40%.

SLIDE 8

Causal intervention

SLIDE 9

Fixing GANS

Strikingly, this simple manual change to a network beats state-

f-the-art GANs models.

SLIDE 10

GAN image editing: removal

SLIDE 11

GAN image editing: insertion

SLIDE 12

https://www.youtube.com/watch?v=yVCgUYe4JTM