Generative Adversarial Networks MIX+GAN Ian Goodfellow, Sta ff - - PowerPoint PPT Presentation

Ian Goodfellow, Staff Research Scientist, Google Brain ICCV Tutorial on GANs Venice, 2017-10-22

Generative Adversarial Networks

3D-GAN AC-GAN AdaGAN AffGAN AL-CGAN ALI AMGAN AnoGAN ArtGAN b-GAN Bayesian GAN BEGAN BiGAN BS-GAN CGAN CCGAN CatGAN CoGAN Context-RNN-GAN C-RNN-GAN C-VAE-GAN CycleGAN DTN DCGAN DiscoGAN DR-GAN DualGAN EBGAN f-GAN FF-GAN GAWWN GoGAN GP-GAN IAN iGAN IcGAN ID-CGAN InfoGAN LAPGAN LR-GAN LS-GAN LSGAN MGAN MAGAN MAD-GAN MalGAN MARTA-GAN McGAN MedGAN MIX+GAN MPM-GAN GMAN alpha-GAN WGAN-GP

(Goodfellow 2017)

Generative Modeling

• Density estimation
• Sample generation

Training examples Model samples

(Goodfellow 2017)

Maximum Likelihood

θ∗ = arg max

θ

Ex∼pdata log pmodel(x | θ)

(Goodfellow 2017)

Adversarial Nets Framework

x sampled from data Differentiable function D D(x) tries to be near 1 Input noise z Differentiable function G x sampled from model D D tries to make D(G(z)) near 0, G tries to make D(G(z)) near 1

(Goodfellow et al., 2014)

(Goodfellow 2017)

What can you do with GANs?

• Simulated environments and training data
• Missing data
• Semi-supervised learning
• Multiple correct answers
• Realistic generation tasks
• Simulation by prediction
• Solve inference problems
• Learn useful embeddings

(Goodfellow 2017)

(Goodfellow 2017)

GANs for simulated training data

(Shrivastava et al., 2016)

(Raffel, 2017)

GANs for domain adaptation

(Bousmalis et al., 2016)

(Goodfellow 2017)

What can you do with GANs?

• Simulated environments and training data
• Missing data
• Semi-supervised learning
• Multiple correct answers
• Realistic generation tasks
• Simulation by prediction
• Solve inference problems
• Learn useful embeddings

(Goodfellow 2017)

Generative modeling reveals a face

(Yeh et al., 2016)

(Goodfellow 2017)

What can you do with GANs?

• Simulated environments and training data
• Missing data
• Semi-supervised learning
• Multiple correct answers
• Realistic generation tasks
• Simulation by prediction
• Solve inference problems
• Learn useful embeddings

(Goodfellow 2017)

Supervised Discriminator

Input Real Hidden units Fake Input Real dog Hidden units Fake Real cat

(Odena 2016, Salimans et al 2016)

(Goodfellow 2017)

What can you do with GANs?

• Simulated environments and training data
• Missing data
• Semi-supervised learning
• Multiple correct answers
• Realistic generation tasks
• Simulation by prediction
• Solve inference problems
• Learn useful embeddings

(Goodfellow 2017)

Next Video Frame Prediction

Ground Truth MSE Adversarial

(Lotter et al 2016) What happens next?

(Goodfellow 2017)

Ground Truth MSE Adversarial

Next Video Frame Prediction

(Lotter et al 2016)

(Raffel, 2017)

Next Video Frame(s) Prediction

(Mathieu et al. 2015) Mean Squared Error Mean Absolute Error Adversarial

(Goodfellow 2017)

What can you do with GANs?

• Simulated environments and training data
• Missing data
• Semi-supervised learning
• Multiple correct answers
• Realistic generation tasks
• Simulation by prediction
• Solve inference problems
• Learn useful embeddings

(Goodfellow 2017)

Which of these are real photos ?

(work by vue.ai covered by Quartz)

(Goodfellow 2017)

What can you do with GANs?

• Simulated environments and training data
• Missing data
• Semi-supervised learning
• Multiple correct answers
• Realistic generation tasks
• Simulation by prediction
• Solve inference problems
• Learn useful embeddings

(Goodfellow 2017)

Vector Space Arithmetic

• +

=

Man with glasses Man Woman Woman with Glasses (Radford et al, 2015)

(Goodfellow 2017)

How long until GANs can do this?

Training examples Model samples

(Goodfellow 2017)

AC-GANs

(Odena et al., 2016)

(Goodfellow 2017)

Track updates at the GAN Zoo

https://github.com/hindupuravinash/the-gan-zoo

(Goodfellow 2017)

Questions?