GANs for Creativity and Design MIX+GAN Ian Goodfellow, Sta ff - - PowerPoint PPT Presentation

▶
gans for creativity and design
SMART_READER_LITE
LIVE PREVIEW

GANs for Creativity and Design MIX+GAN Ian Goodfellow, Sta ff - - PowerPoint PPT Presentation

Jan 08, 2024 838 likes •1.07k views

CoGAN ID-CGAN LR-GAN MedGAN CGAN IcGAN A ff GAN DiscoGAN LS-GAN b-GAN LAPGAN MPM-GAN AdaGAN AMGAN iGAN LSGAN InfoGAN DRAGAN IAN CatGAN GAN-GP GANs for Creativity and Design MIX+GAN Ian Goodfellow, Sta ff Research Scientist,

slide-1
SLIDE 1

Ian Goodfellow, Staff Research Scientist, Google Brain NIPS Workshop on ML for Creativity and Design Long Beach, CA 2017-12-08

GANs for Creativity and Design

3D-GAN AC-GAN AdaGAN AffGAN AL-CGAN ALI AMGAN AnoGAN ArtGAN b-GAN Bayesian GAN BEGAN BiGAN BS-GAN CGAN CCGAN CatGAN CoGAN Context-RNN-GAN C-RNN-GAN C-VAE-GAN CycleGAN DTN DCGAN DiscoGAN DR-GAN DualGAN EBGAN f-GAN FF-GAN GAWWN GoGAN GP-GAN IAN iGAN IcGAN ID-CGAN InfoGAN LAPGAN LR-GAN LS-GAN LSGAN MGAN MAGAN MAD-GAN MalGAN MARTA-GAN McGAN MedGAN MIX+GAN MPM-GAN SN-GAN Progressive GAN DRAGAN GAN-GP

slide-2
SLIDE 2

(Goodfellow 2017)

no mention of realism

slide-3
SLIDE 3

(Goodfellow 2017)

Generative Modeling

  • Density estimation
  • Sample generation

Training examples Model samples

slide-4
SLIDE 4

(Goodfellow 2017)

Adversarial Nets Framework

x sampled from data Differentiable function D D(x) tries to be near 1 Input noise z Differentiable function G x sampled from model D D tries to make D(G(z)) near 0, G tries to make D(G(z)) near 1

(Goodfellow et al., 2014)

slide-5
SLIDE 5

(Goodfellow 2017)

Imagination

(Merriam Webster)

slide-6
SLIDE 6

(Goodfellow 2017)

What is in this image?

(Yeh et al., 2016)

“not present to the senses”

slide-7
SLIDE 7

(Goodfellow 2017)

Generative modeling reveals a face

(Yeh et al., 2016)

“not present to the senses”

slide-8
SLIDE 8

(Goodfellow 2017)

Celebrities who have never existed

(Karras et al., 2017) “never before wholly perceived in reality”

slide-9
SLIDE 9

(Goodfellow 2017)

Is imperfect mimicry

  • riginality?

(Karras et al., 2017)

slide-10
SLIDE 10

(Goodfellow 2017)

Creative Adversarial Networks

(Elgammal et al., 2017) See this afternoon’s keynote

slide-11
SLIDE 11

(Goodfellow 2017)

GANs for design

  • A lower bar than “true creativity”
  • A tool that assists a human designer
slide-12
SLIDE 12

(Goodfellow 2017)

GANs for simulated training data

(Shrivastava et al., 2016)

slide-13
SLIDE 13

(Goodfellow 2017)

iGAN

youtube (Zhu et al., 2016)

slide-14
SLIDE 14

(Goodfellow 2017)

Introspective Adversarial Networks

youtube (Brock et al., 2016)

slide-15
SLIDE 15

(Goodfellow 2017)

Image to Image Translation

Input Ground truth Output

(Isola et al., 2016)

Aerial to Map Labels to Street Scene

input

  • utput

input

  • utput
slide-16
SLIDE 16

(Goodfellow 2017)

Unsupervised Image-to-Image Translation

(Liu et al., 2017) Day to night

slide-17
SLIDE 17

(Goodfellow 2017)

CycleGAN

(Zhu et al., 2017)

slide-18
SLIDE 18

(Goodfellow 2017)

vue.ai

slide-19
SLIDE 19

(Goodfellow 2017)

vue.ai

✓ ✓

slide-20
SLIDE 20

(Goodfellow 2017)

Future directions

  • Beyond realism: train the discriminator to estimate how

appealing an artifact is, in addition to or instead of modeling whether the design is statistically similar to past designs

  • Extreme personalization: highly automate design to generate

artifacts to fit each customer or appeal to each customer’s tastes

  • GAN-based simulators to help test artifacts being designed

(vue.ai is a first step in this direction)

slide-21
SLIDE 21

(Goodfellow 2017)

Conclusion

  • GANs are useful tools for design
  • GANs have a form of imagination
  • It is debatable whether GANs are “original” enough

to count as truly creative. Though designed to perfectly mimic a pattern, they can be used to do more than that