Generative Adversarial Networks (GANs) By: Ismail Elezi - - PowerPoint PPT Presentation

▶
generative adversarial networks gans
SMART_READER_LITE
LIVE PREVIEW

Generative Adversarial Networks (GANs) By: Ismail Elezi - - PowerPoint PPT Presentation

Aug 24, 2023 166 likes •983 views

Generative Adversarial Networks (GANs) By: Ismail Elezi ismail.elezi@gmail.com Supervised Learning vs Unsupervised Learning Supervised Learning vs Unsupervised Learning Supervised Learning vs Unsupervised Learning Supervised Learning vs

slide-1
SLIDE 1

Generative Adversarial Networks (GANs)

By: Ismail Elezi ismail.elezi@gmail.com

slide-2
SLIDE 2

Supervised Learning vs Unsupervised Learning

slide-3
SLIDE 3

Supervised Learning vs Unsupervised Learning

slide-4
SLIDE 4

Supervised Learning vs Unsupervised Learning

slide-5
SLIDE 5

Supervised Learning vs Unsupervised Learning

slide-6
SLIDE 6

Supervised Learning vs Unsupervised Learning

slide-7
SLIDE 7

Supervised Learning vs Unsupervised Learning

slide-8
SLIDE 8

Supervised Learning vs Unsupervised Learning

slide-9
SLIDE 9

Supervised Learning vs Unsupervised Learning

slide-10
SLIDE 10
slide-11
SLIDE 11
slide-12
SLIDE 12

Generative Adversarial Networks

slide-13
SLIDE 13

Generative Adversarial Networks

Credit: Thilo Stadelmann

slide-14
SLIDE 14

Minimax Game on GANs

slide-15
SLIDE 15

Minimax Game on GANs

slide-16
SLIDE 16

Minimax Game on GANs

slide-17
SLIDE 17

Minimax Game on GANs

slide-18
SLIDE 18

Minimax Game on GANs

slide-19
SLIDE 19

Alternative Cost Function

slide-20
SLIDE 20

GAN Training Algorithm

Ian Goodfellow et al, Generative Adversarial Networks, NIPS 2014

slide-21
SLIDE 21
slide-22
SLIDE 22
slide-23
SLIDE 23
slide-24
SLIDE 24
slide-25
SLIDE 25

Generating Digits

https://github.com/TheRevanchist/Generative_Adversarial_Networks/tree/master/gan

slide-26
SLIDE 26

Conditional GANs

What if we want to generate only images of one particular class. Idea: Give the labels of the classes (in one-hot format) to both the generator and discriminator. For the generator concatenate the noise coming from latent space with the

  • ne hot vector. Similarly, the discriminator receives in input both the image

and its label.

slide-27
SLIDE 27

Conditional GANs

Mirza and Osindero, Conditional Generative Adversarial Networks, NIPS 2014

slide-28
SLIDE 28
slide-29
SLIDE 29
slide-30
SLIDE 30

Generating Digits

https://github.com/TheRevanchist/Generative_Adversarial_Networks/tree/master/cgan

slide-31
SLIDE 31

Any idea how to improve GANs?

slide-32
SLIDE 32

Deep Convolutional GANs (DCGAN)

Radford, Metz and Chintala, Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks, ICLR 2016

slide-33
SLIDE 33

Deep Convolutional GANs (DCGAN)

Radford, Metz and Chintala, Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks, ICLR 2016

slide-34
SLIDE 34
slide-35
SLIDE 35
slide-36
SLIDE 36
slide-37
SLIDE 37

Deep Convolutional GANs (DCGAN)

Radford, Metz and Chintala, Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks, ICLR 2016

slide-38
SLIDE 38

Radford, Metz and Chintala, Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks, ICLR 2016

slide-39
SLIDE 39

However, During Training

slide-40
SLIDE 40

Mode Collapse

https://github.com/TheRevanchist/Generative_Adversarial_Networks/tree/master/dcgan

slide-41
SLIDE 41

Possible Fixes to Mode Collapse

  • (Not scientific) Soft labeling, instead of giving to the discriminator labels

1/0, give to it 0.8/0.2

  • (Definitely not scientific) Checkpoint the net, and every time mode

collapse occurs, load the net from the previous checkpoint.

  • (A bit more scientific) LSGAN, other types of cost functions.
  • (Scientific) Wasserstein GAN
  • (Even more scientific) Improved Wasserstein GAN, Dirac Gan etc
slide-42
SLIDE 42

The GAN Zoo

https://github.com/hindupuravinash/the-gan-zoo

slide-43
SLIDE 43

Does it Really Matter?!

Lucic et al, Are GANs Created Equal? A Large-Scale Study, NIPS 2018

slide-44
SLIDE 44

Goodfellow, CVPR tutorial, 2018

slide-45
SLIDE 45

Goodfellow, CPVP tutorial, 2018

slide-46
SLIDE 46

Goodfellow, CPVP tutorial, 2018

slide-47
SLIDE 47

Goodfellow, CPVP tutorial, 2018

slide-48
SLIDE 48

GANs for Time Series

Hyland et al, Real-valued (medical) time series generation with recurrent conditional GANs, arXiv 2017

slide-49
SLIDE 49

Efros, ICCV tutorial, 2017

slide-50
SLIDE 50

Efros, ICCV tutorial, 2017

slide-51
SLIDE 51
slide-52
SLIDE 52
slide-53
SLIDE 53
slide-54
SLIDE 54
slide-55
SLIDE 55
slide-56
SLIDE 56
slide-57
SLIDE 57
slide-58
SLIDE 58
slide-59
SLIDE 59
slide-60
SLIDE 60
slide-61
SLIDE 61
slide-62
SLIDE 62
slide-63
SLIDE 63
slide-64
SLIDE 64
slide-65
SLIDE 65

For much more look at: https://github.com/junyanz/pytorch-CycleGAN-and-pix2pix

slide-66
SLIDE 66
slide-67
SLIDE 67

My GAN-story

slide-68
SLIDE 68

Problems

1) Our images are 2000 x 2000. At 700 (+ delta) by 700 (+delta) images, even a VOLTA V100 runs out of memory

slide-69
SLIDE 69

Problems

1) Our images are 2000 x 2000. At 700 by 700 images, even a VOLTA V100 runs out of memory

  • Solution 1: train in patches, generate large images.
slide-70
SLIDE 70

Problems

1) Our images are 2000 x 2000. At 700 by 700 images, even a VOLTA V100 runs out of memory

  • Solution 1: train in patches, generate large images. It doesn’t work.
slide-71
SLIDE 71

Problems

1) Our images are 2000 x 2000. At 700 by 700 images, even a VOLTA V100 runs out of memory

  • Solution 1: train in patches, generate large images. It doesn’t work.
  • Solution 2: make the nets more efficient. Train on float16 (NVIDIA

Apex) and use gradient checkpointing.

slide-72
SLIDE 72

Problems

1) Our images are 2000 x 2000. At 700 by 700 images, even a VOLTA V100 runs out of memory

  • Solution 1: train in patches, generate large images. It doesn’t work.
  • Solution 2: make the nets more efficient. Train on float16 (NVIDIA

Apex) and use gradient checkpointing. It works.

slide-73
SLIDE 73

Digression: Half precision training

slide-74
SLIDE 74

Digression: Gradient Checkpointing

slide-75
SLIDE 75

Digression: Gradient Checkpointing

slide-76
SLIDE 76

Digression: Gradient Checkpointing

slide-77
SLIDE 77

Digression: Gradient Checkpointing

https://github.com/TheRevanchist/pytorch-CycleGAN-and-pix2pix

slide-78
SLIDE 78

Problems

1) Our images are 2000 x 2000. At 700 by 700 images, even a VOLTA V100 runs out of memory

  • Solution 1: train in patches, generate large images. It doesn’t work.
  • Solution 2: make the nets more efficient. Train on float16 (NVIDIA

Apex) and use gradient checkpointing. It works. 2) Bigger images, less likely that we will be able to generate meaningful images (mode collapse)

slide-79
SLIDE 79

Problems

1) Our images are 2000 x 2000. At 700 by 700 images, even a VOLTA V100 runs out of memory

  • Solution 1: train in patches, generate large images. It doesn’t work.
  • Solution 2: make the nets more efficient. Train on float16 (NVIDIA

Apex) and use gradient checkpointing. It works. 2) Bigger images, less likely that we will be able to generate meaningful images (mode collapse)

  • Solution 1: more careful training and hyperparameter optimization.
  • Solution 2: different loss functions, maybe Wasserstein GANs (or

the improved version of it), researchy stuff.

  • Solution 3: progressive training and/or BigGan-inspired approach.
slide-80
SLIDE 80

Thank You!