In Deep Learning Anima Anandkumar & Zachary Lipton DATA - - PowerPoint PPT Presentation

▶
in deep learning
SMART_READER_LITE
LIVE PREVIEW

In Deep Learning Anima Anandkumar & Zachary Lipton DATA - - PowerPoint PPT Presentation

Dec 29, 2022 49 likes •319 views

Addressing Data Scarcity In Deep Learning Anima Anandkumar & Zachary Lipton DATA AUGMENTATION To improve generalization, augment training data. In computer vision. Simple techniques: rotation, cropping, noise In speech

slide-1
SLIDE 1

Addressing Data Scarcity In Deep Learning

Anima Anandkumar & Zachary Lipton

slide-2
SLIDE 2

DATA AUGMENTATION

  • To improve generalization, augment training data.
  • In computer vision. Simple techniques: rotation, cropping, noise
  • In speech recognition: Additive background noise and spectral

transform

  • More sophisticated approaches ?
slide-3
SLIDE 3

PREDICTIVE VS GENERATIVE MODELS

y x y x

P(y | x) P(x | y)

  • Impressive gains with deep learning
  • Information loss (domain of y << x)
  • Far more challenging
  • Need to model latent variations
slide-4
SLIDE 4

DATA AUGMENTATION 1: MIXED REALITY GAN

Merits

  • Captures statistics of

natural images

  • Learnable

Peril

  • Quality of generated

images not high

  • Introduces artifacts

Our GAN-based framework – Mr.GANs – narrows gap between synthetic and real data

GAN Merits

  • High-quality rendering
  • Full annotation for free
  • Generate infinite data

Peril Synthetic Data

  • Domain mis-match
  • Rendering for visual

appeal and not classification

slide-5
SLIDE 5

MIXED-REALITY 
 GENERATIVE ADVERSARIAL NETWORKS (MR-GAN)
 


Tan Nguyen, Hao Chen, Zachary Lipton, Leo Dirac, Stefano Soatto, A.

slide-6
SLIDE 6


 MIXED-REALITY GENERATIVE ADVERSARIAL NETWORKS (MR-GAN)
 


  • Two domains X and Y.
  • CycleGAN: Transforms from

domain X to Y and viceversa.

  • Enforcing Cycle consistency:

F(G(X)) ~ X.

  • MR-GAN: Progressive

CycleGAN

slide-7
SLIDE 7


 MIXED-REALITY GENERATIVE ADVERSARIAL NETWORKS (MR-GAN)
 


slide-8
SLIDE 8

CLASSIFICATION RESULTS ON CIFAR-DA4

1million synthetic from 3D

  • models. 0.25% real from
  • CIFAR100. 4 classes.

Improvement over training

  • n real data:
  • Real + Synthetic:

5.43% (Stage 0)

  • Real + CycleGAN:

5.09% (Stage 1)

  • Real + Mr.GAN:
  • 8.85% (Stage 2)
slide-9
SLIDE 9

THE REAL, THE SYNTHETIC AND THE REFINED

Real Synthetic Refined Real Refined Synthetic

  • Mr. GAN pushes both real and synthetic images

closer to one another

slide-10
SLIDE 10

(R-S): SYNTHETIC -> REFINED SYNTHETIC

Refined Synthetic Synthetic

slide-11
SLIDE 11

(R-S): REAL -> REFINED REAL

Refined Real Real

slide-12
SLIDE 12

PREDICTIVE VS GENERATIVE MODELS

y x y x

P(y | x) P(x | y) One model to do both?

  • SOTA prediction from CNN models.
  • What class of p(x|y) yield CNN models for p(y|x)?
slide-13
SLIDE 13

LATENT-DEPENDENT DEEP RENDERING MODEL (LD-DRM)
 


Nhat Ho, Tan Nguyen, Ankit Patel, A. , Michael Jordan, Richard Baraniuk

slide-14
SLIDE 14

LATENT-DEPENDENT DEEP RENDERING MODEL (LD-DRM)

  • bject

category intermediate rendering image latent variables

Design joint priors for latent variables based on reverse-engineering CNN predictive architectures

slide-15
SLIDE 15

LATENT-DEPENDENT DEEP RENDERING MODEL (LD-DRM)

slide-16
SLIDE 16

LATENT-DEPENDENT DEEP RENDERING MODEL (LD-DRM)

slide-17
SLIDE 17

LATENT-DEPENDENT DEEP RENDERING MODEL (LD-DRM)

slide-18
SLIDE 18

STATISTICAL GUARANTEES FOR THE LD-DRM

  • Generalization for prediction depends on the generative model
  • Better generalization when no. of active rendering paths minimized
  • Rendering path normalization: new form of regularization
  • Improves performance significantly.

Training loss in the CNNs equivalent to likelihood in LD-DRM

slide-19
SLIDE 19

SEMI-SUPERVISED LEARNING RESULTS

Error rate percentage on CIFAR-10 Error rate percentage on CIFAR-100

LD-DRM achieves comparable results to state-of-the-art SSL methods

slide-20
SLIDE 20

DATA AUGMENTATION 3: SYMBOLIC EXPRESSIONS

Goal: Learn a domain of functions (sin, cos, log, add…)

  • Training on numerical input-output does not generalize.

Data Augmentation with Symbolic Expressions

  • Efficiently encode relationships between functions.

Solution:

  • Design networks to use both: symbolic + numerical
slide-21
SLIDE 21

COMMON STRUCTURE: TREES

  • Symbolic expression trees. Function evaluation tree.
  • Decimal trees: encode numbers with decimal representation (numerical).
  • Can encode any expression, function evaluation and number.
slide-22
SLIDE 22

STRUCTURE : TREE LSTM

slide-23
SLIDE 23

RESULTS: EQUATION COMPLETION & FUNCTION EVAL

slide-24
SLIDE 24

RESULTS: EQUATION VERIFICATION

Generalization to unseen depth

slide-25
SLIDE 25

RESULTS SUMMARIZED

  • Vastly Improved numerical evaluation: 90% over function-fitting baseline.
  • Generalization to verifying symbolic equations of higher depth
  • Combining symbolic + numerical data helps in better generalization

for both tasks: symbolic and numerical evaluation.

LSTM: Symbolic TreeLSTM: Symbolic TreeLSTM: symbolic + numeric 76.40 % 93.27 % 96.17 %

slide-26
SLIDE 26
  • Collection: Active learning and partial feedback
  • Aggregation: Crowdsourcing models
  • Augmentation:
  • Graphics rendering + GANs
  • Semi-supervised learning
  • Symbolic expressions




CONCLUSION

Data scarcity needs to be addressed in a number of ways

slide-27
SLIDE 27

Thank you