[PPT] - HexaGAN: Generative Adversarial Nets for Real World Classification PowerPoint Presentation

SLIDE 1

Uiwon Hwang , Dahuin Jung, and Sungroh Yoon

HexaGAN: Generative Adversarial Nets for Real World Classification

Seoul National University Electrical and Computer Engineering

∗

speaker

∗

SLIDE 2

Missing data problem
Missing data imputation
Imputing missing elements in a data level
Class imbalance problem
Class-conditional generation
Imputing the entire elements of a sample

conditioned on a label

Missing label problem
Semi-supervised learning
Imputing missing class labels using a classifier
Keyword: Imputation

Problem Definition

𝑦$ 𝑦% 𝑦& 𝑧 𝑦$ 𝑦% 𝑦& 𝑧 𝑦$ 𝑦% 𝑦& 𝑧 𝑦$ 𝑦% 𝑦& 𝑧 𝑦$ 𝑦% 𝑦& 𝑧 𝑦$ 𝑦% 𝑦& 𝑧

SLIDE 3

Missing data problem
Missing data imputation
filling missing elements in a data level
Class imbalance problem
Class-conditional generation
Imputing the entire elements of a sample

conditioned on a label

Missing label problem
Semi-supervised learning
Imputing missing class labels using a classifier
Keyword: Imputation

Problem Definition

SLIDE 4

Missing data problem
Missing data imputation
filling missing elements in a data level
Class imbalance problem
Class-conditional generation
Imputing the entire elements of a sample

conditioned on a label

Missing label problem
Semi-supervised learning
Imputing missing class labels using a classifier
Keyword: Imputation

Problem Definition

𝑦$ 𝑦% 𝑦& 𝑧 𝑦$ 𝑦% 𝑦& 𝑧 𝑦$ 𝑦% 𝑦& 𝑧 𝑦$ 𝑦% 𝑦& 𝑧

SLIDE 5

Missing data problem
Missing data imputation
filling missing elements in a data level
Class imbalance problem
Class-conditional generation
Imputing the entire elements of a sample

conditioned on a label

Missing label problem
Semi-supervised learning
Imputing missing class labels using a classifier
Keyword: Imputation

Problem Definition

SLIDE 6

Missing data problem
Missing data imputation
filling missing elements in a data level
Class imbalance problem
Class-conditional generation
Imputing the entire elements of a sample

conditioned on a label

Missing label problem
Semi-supervised learning
Imputing missing class labels using a classifier

→ Keyword: Imputation

Problem Definition

SLIDE 7

We propose a generative adversarial network to solve the problems in real world classification

simultaneously

Overview of HexaGAN

SLIDE 8

Missing data (element-wise) imputation (to solve the missing data problem)
Components
𝐹: transfers both labeled and unlabeled instances into the hidden space
𝐻*+: imputes missing data
𝐸*+ - $:/: distinguishes b/w missing and non-missing elements

Addressing Three Problems

SLIDE 9

Missing data (element-wise) imputation (to solve the missing data problem)
Components
𝐹: transfers both labeled and unlabeled instances into the hidden space
𝐻*+: imputes missing data
𝐸*+ - $:/: distinguishes b/w missing and non-missing elements

Addressing Three Problems

SLIDE 10

Missing data (element-wise) imputation (to solve the missing data problem)
Components
𝐹: transfers both labeled and unlabeled instances into the hidden space
𝐻*+: imputes missing data
𝐸*+ - $:/: distinguishes b/w missing and non-missing elements

Addressing Three Problems

SLIDE 11

Missing data (element-wise) imputation (to solve the missing data problem)
Components
𝐹: transfers both labeled and unlabeled instances into the hidden space
𝐻*+: imputes missing data
𝐸*+ - $:/: distinguishes b/w missing and non-missing elements
A novel element-wise adversarial loss function and gradient penalty
max

345

min

845

Addressing Three Problems

SLIDE 12

Class conditional generation (to solve the class imbalance problem)
Components
𝐻93: creates conditional hidden vectors 𝐢;
𝐸93: determines whether a hidden vector is from the dataset or has been created by 𝐻93
𝐻*+ generates the entire elements conditioned on the minority class

Addressing Three Problems

SLIDE 13

Class conditional generation (to solve the class imbalance problem)
Components
𝐻93: creates conditional hidden vectors 𝐢;
𝐸93: determines whether a hidden vector is from the dataset or has been created by 𝐻93
𝐻*+ generates the entire elements conditioned on the minority class

Addressing Three Problems

SLIDE 14

Class conditional generation (to solve the class imbalance problem)
Components
𝐻93: creates conditional hidden vectors 𝐢;
𝐸93: determines whether a hidden vector is from the dataset or has been created by 𝐻93
𝐻*+ generates the entire elements conditioned on the minority class
Losses
WGAN loss + zero-centered gradient penalty
Add Loss of 𝐻*+ calculated from 𝐲

=; and the cross entropy of 𝐲 =, 𝐳; to 𝐻93

Addressing Three Problems

SLIDE 15

Semi-supervised learning (to solve the missing label problem)
Components
𝐷: estimates class labels. This also works as the label generator
𝐸*+ - /A$: distinguishes b/w real and pseudo (fake) labels

Addressing Three Problems

SLIDE 16

Semi-supervised learning (to solve the missing label problem)
Components
𝐷: estimates class labels. This also works as the label generator
𝐸*+ - /A$: distinguishes b/w real and pseudo (fake) labels
We adopt the pseudo-labeling technique of TripleGAN (Li et al., NIPS 2017)
The two components are related adversarially
Addressing Three Problems

SLIDE 17

Semi-supervised learning (to solve the missing label problem)
Components
𝐷: estimates class labels. This also works as the label generator
𝐸*+ - /A$: distinguishes b/w real and pseudo (fake) labels
We adopt the pseudo-labeling technique of TripleGAN (Li et al., NIPS 2017)
The two components are related adversarially
Addressing Three Problems

SLIDE 18

Not three separate models, this is ONEmodel dubbed HexaGAN
The six components of HexaGAN interplay to solve the problems effectively

Overview of HexaGAN(revisited)

SLIDE 19

Theorem 1: Global optimality of 𝑞 𝑦 𝑛D = 1 = 𝑞(𝑦|𝑛D = 0) for HexaGAN
A generator distribution 𝑞(𝑦|𝑛D = 0) is a global optimum for the min-max game of 𝑯𝑵𝑱 and 𝑬𝑵𝑱,

if and only if 𝒒 𝒚 𝒏𝒋 = 𝟐 = 𝒒(𝒚|𝒏𝒋 = 𝟏) for all 𝒚 ∈ ℝ/, except possibly on a set of zero Lebesgue measure.

Theorems

SLIDE 20

Theorem 1: Global optimality of 𝑞 𝑦 𝑛D = 1 = 𝑞(𝑦|𝑛D = 0) for HexaGAN
A generator distribution 𝑞(𝑦|𝑛D = 0) is a global optimum for the min-max game of 𝑯𝑵𝑱 and 𝑬𝑵𝑱,

if and only if 𝒒 𝒚 𝒏𝒋 = 𝟐 = 𝒒(𝒚|𝒏𝒋 = 𝟏) for all 𝒚 ∈ ℝ/, except possibly on a set of zero Lebesgue measure.

Theorem 2: The adversarial loss for semi-supervised learning is the ODM cost
Output distribution matching (ODM) cost function (Sutskeveret al., ICLR workshop 2016)
the global minimum of the supervised cost function is also a global minimum of the ODM cost function
Optimizing the adversarial losses for C and 𝐸*+ - /A$ imposes an unsupervised constraint on C.

Then, the adversarial losses for semi-supervised learning in HexaGAN satisfy the definition of the ODM cost.

Theorems

SLIDE 21

Missing data imputation
HexaGAN shows good performances on various real world datasets
Medical, financial, vision, …

Experimental Results

SLIDE 22

e define the three problems (missing data, class imbalance, and missing label) in real-world classification from the perspective of imputation Our framework is simple to use and works automatically when the absence of data and label (𝐧and 𝑛X) is indicated. We loss function and gradient penalty for element-wise imputation. Our imputation performance produces stable, state-of-the-art results. Our method significantly outperforms cascading combinations of the existing state-of-the-art methods.

For more details, please visit our poster session!
June 11th (Today), 6:30 – 9:00 pm, Pacific Ballroom #20

Conclusions

HexaGAN

𝑭 𝑫 𝑯𝑵𝑱 𝑬𝑵𝑱 𝑬𝑫𝑯 𝑯𝑫𝑯 Missing Data

Real World Classification

Missing Label Class Imbalance

SLIDE 23

e define the three problems (missing data, class imbalance, and missing label) in real-world classification from the perspective of imputation Our framework is simple to use and works automatically when the absence of data and label (𝐧and 𝑛X) is indicated. We loss function and gradient penalty for element-wise imputation. Our imputation performance produces stable, state-of-the-art results. Our method significantly outperforms cascading combinations of the existing state-of-the-art methods.