High-Fidelity Image Generation With Fewer Labels Michael - - PowerPoint PPT Presentation

▶

Nov 19, 2023 649 likes •812 views

High-Fidelity Image Generation With Fewer Labels Michael Tschannen* Mario Lucic* Marvin Rituer* Xiaohua Zhai Olivier Bachem Sylvain Gelly *equal contribution Generative Adversarial Networks (GANs): Recent Progress BigGAN (Brock,

SLIDE 1

High-Fidelity Image Generation With Fewer Labels

Michael Tschannen* Mario Lucic* Marvin Rituer* Xiaohua Zhai Olivier Bachem Sylvain Gelly

*equal contribution

SLIDE 2

Generative Adversarial Networks (GANs): Recent Progress

P 3

BigGAN (Brock, Donahue, Simonyan 2019)

SLIDE 3

Generative Adversarial Networks (GANs): Recent Progress

P 4

BigGAN (Brock, Donahue, Simonyan 2019) class-conditional

SLIDE 4

Generative Adversarial Networks (GANs): Recent Progress

Conditioning reduces the diverse generation problem to a per-class problem

P 5

BigGAN (Brock, Donahue, Simonyan 2019) class-conditional

SLIDE 5

Generative Adversarial Networks (GANs): Recent Progress

Conditioning reduces the diverse generation problem to a per-class problem

P 6

BigGAN (Brock, Donahue, Simonyan 2019) SS-GAN (Chen et al. 2019) class-conditional unsupervised

SLIDE 6

Generative Adversarial Networks (GANs): Recent Progress

Conditioning reduces the diverse generation problem to a per-class problem

P 7

BigGAN (Brock, Donahue, Simonyan 2019) SS-GAN (Chen et al. 2019) class-conditional unsupervised

Unsupervised models are considerably less powergul

SLIDE 7

This work: How to close the gap between conditional and unsupervised GANs?

Generative Adversarial Networks (GANs): Recent Progress

P 8

BigGAN (Brock, Donahue, Simonyan 2019) SS-GAN (Chen et al. 2019) class-conditional unsupervised

SLIDE 8

Proposed methods: Overview

P 9

Replace ground-truth labels with synthetic/inferred labels

➜ No changes in the GAN architecture required

Infer labels for the real data using self-supervised and

semi-supervised learning techniques

SLIDE 9

Proposed methods: Pre-training

P 10

1. Learn a semantic representation F of the data using self-supervision by rotation prediction (Gidaris et al. 2018) 2. Clustering or semi-supervised learning on the representation F 3. Train GAN with inferred labels

SLIDE 10

Proposed methods: Co-training

P 11

Semi-supervised classifjcation head on discriminator

SLIDE 11

Improve pre- and co-training methods

P 12

Rotation-self supervision during GAN training (Chen et al. 2019)

SLIDE 12

Clustering (SS) is unsupervised SOTA (FID 22.0)
S2GAN (20%) and S3GAN (10%) match BigGAN (100%)
S3GAN (20%) outpergorms BigGAN (100%) (SOTA)

Results

P 13

BigGAN (100%)

SLIDE 13

Samples: BigGAN (our implementation) vs proposed

P 14

S3GAN (10%) BigGAN (100%)

256 x 256 px

SLIDE 14

Results

P 16

S3GAN (10%)

256 x 256 px

SLIDE 15

Code, pretrained models and Colabs: github.com/google/compare_gan Check out our poster #13 tonight 6:30-9:00 pm!

P 17