PixelGAN Autoencoders Alireza Makhzani, Brendan Frey Machine - - PowerPoint PPT Presentation

▶
pixelgan autoencoders
SMART_READER_LITE
LIVE PREVIEW

PixelGAN Autoencoders Alireza Makhzani, Brendan Frey Machine - - PowerPoint PPT Presentation

Mar 06, 2023 228 likes •520 views

PixelGAN Autoencoders Alireza Makhzani, Brendan Frey Machine learning Group University of Toronto CIFAR Deep Learning Summer School Montreal, Canada June 29 th , 2017 Alireza Makhzani PixelGAN Autoencoders 1 / 27 Outline 1. Background

slide-1
SLIDE 1

Alireza Makhzani PixelGAN Autoencoders

/ 27

CIFAR Deep Learning Summer School Montreal, Canada Machine learning Group University of Toronto

PixelGAN Autoencoders

Alireza Makhzani, Brendan Frey June 29th, 2017

1

slide-2
SLIDE 2

Alireza Makhzani PixelGAN Autoencoders

/ 27

Outline

2

  • 1. Background
  • PixelCNNs
  • Variational Autoencoders
  • Adversarial Autoencoders
  • 2. PixelGAN Autoencoders
  • Gaussian Priors
  • Categorical Priors

✦ Clustering ✦ Semi-supervised Learning

slide-3
SLIDE 3

Alireza Makhzani PixelGAN Autoencoders

/ 27

PixelCNNs

3

✦ Learn the image statistics directly at the pixel level. ✦ Good at modelling low-level pixel statistics. ✦ Samples lack global structure. ✦ Lacking latent representation. ✦ Conditional PixelCNNs can learn conditional densities. van den Oord et al., 2016

slide-4
SLIDE 4

Alireza Makhzani PixelGAN Autoencoders

/ 27

Variational Autoencoders

4

log p(x) > Eq(z|x)[ log(p(x|z)] KL(q(z|x)kp(z))

4

Kingma et al., 2013 ✦ Good at capturing the global structure, but samples are blurry. ✦ Learn hierarchical representations useful for down-stream tasks. ✦ Attempts at combining PixelCNN with VAEs:

  • PixelVAE (Gulrajani et al., 2016)
  • VLAE (Chen et al., 2017)
slide-5
SLIDE 5

Alireza Makhzani PixelGAN Autoencoders

/ 27

Adversarial Autoencoders

5

B

D

Code Space

  • f MNIST:

Gaussian Prior Mixture of Gaussians

Makhzani et al., 2015

slide-6
SLIDE 6

Alireza Makhzani PixelGAN Autoencoders

/ 27

Outline

6

  • 1. Background
  • PixelCNNs
  • Variational Autoencoders
  • Adversarial Autoencoders
  • 2. PixelGAN Autoencoders
  • Gaussian Priors
  • Categorical Priors

✦ Clustering ✦ Semi-supervised Learning

slide-7
SLIDE 7

Alireza Makhzani PixelGAN Autoencoders

/ 27

Limitations of Variational/Adversarial Autoencoders

7

✦ All the image statistics are captured by the single latent vector.

label, style global and local

VAE

None

z x p(z) p(x|z)

Latent Variable Deterministic (factorized Gaussians)

slide-8
SLIDE 8

Alireza Makhzani PixelGAN Autoencoders

/ 27

PixelGAN Autoencoders

8

✦ The image statistics are captured jointly by the latent vector and the autoregressive decoder. PixelCNN Latent Variable

z x p(z) p(x|z)

slide-9
SLIDE 9

Alireza Makhzani PixelGAN Autoencoders

/ 27

PixelGAN Autoencoders

9

PixelCNN Latent Variable

z x p(z) p(x|z)

GAN ✦ The image statistics are captured jointly by the latent vector and the autoregressive decoder.

slide-10
SLIDE 10

Alireza Makhzani PixelGAN Autoencoders

/ 27

PixelGAN Autoencoders

10

PixelGAN (Gaussian)

Global (low-frequency) Local (high-frequency)

PixelGAN (Categorical)

Discrete (label) Continuous (Style)

PixelCNN Latent Variable

z x p(z) p(x|z)

GAN ✦ The image statistics are captured jointly by the latent vector and the autoregressive decoder.

slide-11
SLIDE 11

Alireza Makhzani PixelGAN Autoencoders

/ 27

PixelGAN Autoencoders

11

Semi-supervised Learning PixelCNN Latent Variable

z x p(z) p(x|z)

GAN PixelGAN (Gaussian)

Global (low-frequency) Local (high-frequency)

PixelGAN (Categorical)

Discrete (label) Continuous (Style)

✦ The image statistics are captured jointly by the latent vector and the autoregressive decoder.

slide-12
SLIDE 12

Alireza Makhzani PixelGAN Autoencoders

/ 27

PixelGAN Autoencoders

12

Cost function of PixelGAN = Reconstruction + Adversarial Cost

slide-13
SLIDE 13

Alireza Makhzani PixelGAN Autoencoders

/ 27

Outline

13

  • 1. Background
  • PixelCNNs
  • Variational Autoencoders
  • Adversarial Autoencoders
  • 2. PixelGAN Autoencoders
  • Gaussian Priors
  • Categorical Priors

Clustering Semi-supervised Learning

slide-14
SLIDE 14

Alireza Makhzani PixelGAN Autoencoders

/ 27

Global vs. Local Decomposition

14

slide-15
SLIDE 15

Alireza Makhzani PixelGAN Autoencoders

/ 27

Code Space

15

Code Space

  • f MNIST:
slide-16
SLIDE 16

Alireza Makhzani PixelGAN Autoencoders

/ 27

Outline

16

  • 1. Background
  • PixelCNNs
  • Variational Autoencoders
  • Adversarial Autoencoders
  • 2. PixelGAN Autoencoders
  • Gaussian Priors
  • Categorical Priors

✦ Clustering ✦ Semi-supervised Learning

slide-17
SLIDE 17

Alireza Makhzani PixelGAN Autoencoders

/ 27

PixelGAN Autoencoders with Categorical Priors

17

slide-18
SLIDE 18

Alireza Makhzani PixelGAN Autoencoders

/ 27

Discrete vs. Continuous Decomposition (Clustering)

18

slide-19
SLIDE 19

Alireza Makhzani PixelGAN Autoencoders

/ 27

Discrete vs. Continuous Decomposition (Clustering)

19

0.3% Error rate

slide-20
SLIDE 20

Alireza Makhzani PixelGAN Autoencoders

/ 27

Unsupervised Clustering

20

slide-21
SLIDE 21

Alireza Makhzani PixelGAN Autoencoders

/ 27

Unsupervised Clustering

21

slide-22
SLIDE 22

Alireza Makhzani PixelGAN Autoencoders

/ 27

Unsupervised Clustering

22

5% Error rate

slide-23
SLIDE 23

Alireza Makhzani PixelGAN Autoencoders

/ 27

Outline

23

  • 1. Background
  • PixelCNNs
  • Variational Autoencoders
  • Adversarial Autoencoders
  • 2. PixelGAN Autoencoders
  • Gaussian Priors
  • Categorical Priors

✦ Clustering ✦ Semi-supervised Learning

slide-24
SLIDE 24

Alireza Makhzani PixelGAN Autoencoders

/ 27

Semi-supervised Learning

24

Semi-supervised Learning

slide-25
SLIDE 25

Alireza Makhzani PixelGAN Autoencoders

/ 27

Semi-supervised Learning

25

Semi-supervised Learning

slide-26
SLIDE 26

Alireza Makhzani PixelGAN Autoencoders

/ 27

Semi-supervised Classification

26

slide-27
SLIDE 27

Alireza Makhzani PixelGAN Autoencoders

/ 27 27

Thank you!