Introduction to GANs LSGAN SAGAN MIX+GAN Ian Goodfellow, Sta ff - - PowerPoint PPT Presentation

▶
introduction to gans
SMART_READER_LITE
LIVE PREVIEW

Introduction to GANs LSGAN SAGAN MIX+GAN Ian Goodfellow, Sta ff - - PowerPoint PPT Presentation

Jan 25, 2024 839 likes •1.18k views

CoGAN ID-CGAN LR-GAN MedGAN CGAN IcGAN DiscoGAN LS-GAN b-GAN LAPGAN MPM-GAN AdaGAN AMGAN iGAN InfoGAN IAN CatGAN Introduction to GANs LSGAN SAGAN MIX+GAN Ian Goodfellow, Sta ff Research Scientist, Google Brain McGAN CVPR

slide-1
SLIDE 1

Ian Goodfellow, Staff Research Scientist, Google Brain CVPR Tutorial on GANs Salt Lake City, 2018-06-22

Introduction to GANs

3D-GAN AC-GAN AdaGAN SAGAN AL-CGAN ALI AMGAN AnoGAN ArtGAN b-GAN Bayesian GAN BEGAN BiGAN BS-GAN CGAN CCGAN CatGAN CoGAN Context-RNN-GAN C-RNN-GAN C-VAE-GAN CycleGAN DTN DCGAN DiscoGAN DR-GAN DualGAN EBGAN f-GAN FF-GAN GAWWN GoGAN GP-GAN IAN iGAN IcGAN ID-CGAN InfoGAN LAPGAN LR-GAN LS-GAN LSGAN MGAN MAGAN MAD-GAN MalGAN MARTA-GAN McGAN MedGAN MIX+GAN MPM-GAN

slide-2
SLIDE 2

(Goodfellow 2018)

Generative Modeling: Density Estimation

Training Data Density Function

slide-3
SLIDE 3

(Goodfellow 2018)

Generative Modeling: Sample Generation

Training Data Sample Generator (CelebA) (Karras et al, 2017)

slide-4
SLIDE 4

(Goodfellow 2018)

Adversarial Nets Framework

x sampled from data Differentiable function D D(x) tries to be near 1 Input noise z Differentiable function G x sampled from model D D tries to make D(G(z)) near 0, G tries to make D(G(z)) near 1

(Goodfellow et al., 2014)

slide-5
SLIDE 5

(Goodfellow 2018)

Self-Play

1959: Arthur Samuel’s checkers agent (Silver et al, 2017) (Bansal et al, 2017) (OpenAI, 2017)

slide-6
SLIDE 6

(Goodfellow 2018)

3.5 Years of Progress on Faces

2014 2015 2016 2017

(Brundage et al, 2018)

slide-7
SLIDE 7

(Goodfellow 2018)

<2 Years of Progress on ImageNet

Odena et al 2016 Miyato et al 2017 Zhang et al 2018

slide-8
SLIDE 8

(Goodfellow 2018)

Self-Attention GAN

State of the art FID on ImageNet: 1000 categories, 128x128 pixels (Zhang et al., 2018) Indigo Bunting Goldfish Redshank Saint Bernard Tiger Cat Stone Wall Broccoli Geyser

slide-9
SLIDE 9

(Goodfellow 2018)

From GAN to SAGAN

  • Depth and Convolution
  • Class-conditional generation
  • Spectral Normalization
  • Hinge loss
  • Two-timescale update rule
  • Self-attention
slide-10
SLIDE 10

(Goodfellow 2018)

No Convolution Needed to Solve Simple Tasks

Original GAN, 2014

slide-11
SLIDE 11

(Goodfellow 2018)

Depth and Convolution for Harder Tasks

Original GAN (CIFAR-10) DCGAN (ImageNet) No convolution One convolutional layer Many convolutional layers (Radford et al, 2015)

slide-12
SLIDE 12

(Goodfellow 2018)

From GAN to SAGAN

  • Depth and Convolution
  • Class-conditional generation
  • Spectral Normalization
  • Hinge loss
  • Two-timescale update rule
  • Self-attention
slide-13
SLIDE 13

(Goodfellow 2018)

Class-Conditional GANs

(Mirza and Osindero, 2014)

slide-14
SLIDE 14

(Goodfellow 2018)

AC-GAN: Specialist Generators

(Odena et al, 2016)

slide-15
SLIDE 15

(Goodfellow 2018)

SN-GAN: Shared Generator

(Miyato et al, 2017)

slide-16
SLIDE 16

(Goodfellow 2018)

From GAN to SAGAN

  • Depth and Convolution
  • Class-conditional generation
  • Spectral Normalization
  • Hinge loss
  • Two-timescale update rule
  • Self-attention
slide-17
SLIDE 17

(Goodfellow 2018)

Spectral Normalization

(Miyato et al, 2017)

slide-18
SLIDE 18

(Goodfellow 2018)

From GAN to SAGAN

  • Depth and Convolution
  • Class-conditional generation
  • Spectral Normalization
  • Hinge loss
  • Two-timescale update rule
  • Self-attention
slide-19
SLIDE 19

(Goodfellow 2018)

Hinge Loss

(Miyato et al 2017, Lim and Ye 2017, Tran et al 2017)

slide-20
SLIDE 20

(Goodfellow 2018)

From GAN to SAGAN

  • Depth and Convolution
  • Class-conditional generation
  • Spectral Normalization
  • Hinge loss
  • Two-timescale update rule
  • Self-attention
slide-21
SLIDE 21

(Goodfellow 2018)

Two-Timescale Update Rule

slide-22
SLIDE 22

(Goodfellow 2018)

From GAN to SAGAN

  • Depth and Convolution
  • Class-conditional generation
  • Spectral Normalization
  • Hinge loss
  • Two-timescale update rule
  • Self-attention
slide-23
SLIDE 23

(Goodfellow 2018)

Self-Attention

Use layers from Wang et al 2018

slide-24
SLIDE 24

(Goodfellow 2018)

Applying GANs

  • Semi-supervised Learning
  • Model-based optimization
  • Extreme personalization
  • Program synthesis
slide-25
SLIDE 25

(Goodfellow 2018)

Supervised Discriminator for Semi-Supervised Learning

Input Real Hidden units Fake Input Real dog Hidden units Fake Real cat

(Odena 2016, Salimans et al 2016) Learn to read with 100 labels rather than 60,000

slide-26
SLIDE 26

(Goodfellow 2018)

Semi-Supervised Classification

MNIST: 100 training labels -> 80 test mistakes SVHN: 1,000 training labels -> 4.3% test error CIFAR-10: 4,000 labels -> 14.4% test error (Dai et al 2017)

slide-27
SLIDE 27

(Goodfellow 2018)

Designing DNA to optimize protein binding

(Killoran et al, 2017)

slide-28
SLIDE 28

(Goodfellow 2018)

Personalized GANufacturing

(Hwang et al 2018)

slide-29
SLIDE 29

(Goodfellow 2018)

SPIRAL

(Ganin et al, 2018) Synthesizing Programs for Images Using Reinforced Adversarial Learning

slide-30
SLIDE 30

(Goodfellow 2018)

Other applications

  • Planning
  • World Models for RL agents
  • Fairness and Privacy
  • Missing data
  • Topics covered at workshop:
  • Training data for other agents (Philip Isola, Taesung Park, Jun-Yan Zhu)
  • Inference in other probabilistic models (Mihaela Rosca)
  • Domain adaptation (Judy Hoffman)
  • Imitation Learning (Stefano Ermon)
slide-31
SLIDE 31

(Goodfellow 2018)

Track updates at the GAN Zoo

https://github.com/hindupuravinash/the-gan-zoo

slide-32
SLIDE 32

(Goodfellow 2018)

Questions