Bias and Generalization in Deep Generative Models Shengjia Zhao*, - PowerPoint PPT Presentation

Feb 22, 2023 •143 likes •350 views

Bias and Generalization in Deep Generative Models Shengjia Zhao, Hongyu Ren, Arianna Yuan, Jiaming Song, Noah Goodman and Stefano Ermon *equal contribution Success in Generative Modeling of Images Brock A, et al. "Large scale gan

Bias and Generalization in Deep Generative Models Shengjia Zhao*, Hongyu Ren*, Arianna Yuan, Jiaming Song, Noah Goodman and Stefano Ermon *equal contribution
Success in Generative Modeling of Images Brock A, et al. "Large scale gan training for high fidelity natural image synthesis."
Goal: Understanding Generalization How do generative models generalize?
Generalization Example: Object Count
Empirical Study of Generalization: Method • Design datasets • Train generative models (VAE, GAN, PixelCNN) • Observe generalization behavior • Find common patterns
Generalization Example: Object Count
Generalization in Feature Space: Numerosity Generates a log-normal Frequency Frequency shaped distribution 2 1 2 3 4 # Objects # Objects Training Distribution Generated Distribution (Observed)
Multiple Numerosities Frequency ? 2 7 # Objects Training Distribution
Multiple Numerosities: Only 2 Frequency Frequency 2 7 # Objects 1 2 3 4 # Objects Training Distribution Generated Distribution
Multiple Numerosities: Only 7 Frequency Frequency 2 7 # Objects 6 7 8 9 # Objects Training Distribution Generated Distribution
Multiple Numerosities: Additive Hypothesis Frequency Frequency 2 7 # Objects 1 2 3 4 6 7 8 9 # Objects Training Distribution Generated Distribution (Observed)
Additive Hypothesis with 2 and 4 Objects Frequency Frequency 2 4 1 2 3 4 5 6 # Objects # Objects Training Distribution Generated Distribution ( Hypothesized )
Actual Result: Prototype Enhancement 3 objects most likely, even though no training Frequency Frequency image contains 3 objects! 2 4 1 2 3 4 5 6 # Objects # Objects Training Distribution Generated Distribution ( Observed )
Prototype Enhancement Similar pattern for other features: Frequency color , size , location Frequency 2 4 1 2 3 4 5 6 # Objects # Objects Training Distribution Generated Distribution ( Observed )
Multiple Features
Memorization vs. Generalization
Memorization vs. Generalization
Different Setups, Similar Results - Different features (shape, color, size, numerosity, etc.) - Different models: (VAE, GAN, PixelCNN, etc.) - Different architectures (fully connected, convolutional, etc.) - Different hyper-parameters (network size, learning rate, etc.)
Conclusion • New methodology: design datasets to probe generative models • Observed common patterns across different setups Welcome to our poster session for further discussions! Tuesday 5-7pm @ Room 210 & 230 AB #6 Code available at github.com/ermongroup/BiasAndGeneralization

Download Presentation

Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Invertible Generative Models for Inverse Problems Mitigating Representation Error and Dataset Bias

Invertible Generative Models for Inverse Problems Mitigating Representation Error and Dataset Bias M. Asim, M. Daniels, O. Leong, P . Hand, A. Ahmed Inverse Problems with Generative Models as Image Priors Inverse Problems with Generative Models

369 views • 22 slides

Probability Functional Descent: A Unifying Perspective on GANs, VI, and RL Casey Chu <

Probability Functional Descent: A Unifying Perspective on GANs, VI, and RL Casey Chu < caseychu@stanford.edu > Jose Blanchet Peter Glynn Deep generative models Deep generative models Variational inference Deep generative models

275 views • 16 slides

Deep Generative models for Inverse Problems Alex Dimakis joint work with Ashish Bora, Dave Van

Deep Generative models for Inverse Problems Alex Dimakis joint work with Ashish Bora, Dave Van Veen and Ajil Jalal, Sriram Vishwanath and Eric Price, UT Austin Outline Generative Models Using generative models for Inverse

1.17k views • 101 slides

Learning Deep Generative Models Inference & Representation Lecture 12 Rahul G. Krishnan

Introduction Variational Inference Deep Generative Models Summary Learning Deep Generative Models Inference & Representation Lecture 12 Rahul G. Krishnan Fall 2015 Rahul G. Krishnan Learning Deep Generative Models Introduction

1.22k views • 83 slides

Deep-Learning: Unsupervised Generative models Deep Belief Networks Deep Stacked AutoEncoders

Deep-Learning: Unsupervised Generative models Deep Belief Networks Deep Stacked AutoEncoders Generative Adversarial Networks Pr. Fabien MOUTARDE Center for Robotics MINES ParisTech PSL Universit Paris Fabien.Moutarde@mines-paristech.fr

777 views • 23 slides

Structure: Exposition General problem Purpose / stated goal(s) Experimental setup

Structure: Exposition General problem Purpose / stated goal(s) Experimental setup summary Result summary 1 Experimental Evaluation: Bias and Generalization in Deep Generative Models Nicholay Topin MLD, Carnegie Mellon

352 views • 22 slides

Deep Hybrid Models: Bridging Discriminative and Generative Approaches Volodymyr Kuleshov and

A New Framework For Hybrid Models An Application: Deep Hybrid Models Supervised and Semi-Supervised Experiments Deep Hybrid Models: Bridging Discriminative and Generative Approaches Volodymyr Kuleshov and Stefano Ermon Department of Computer

523 views • 29 slides

A Tutorial on Deep Probabilistic Generative Models Ryan P. Adams Princeton University Machine

A Tutorial on Deep Probabilistic Generative Models Ryan P. Adams Princeton University Machine Learning Summer School Buenos Aires, Argentina June 2018 lips.cs.princeton.edu @ryan_p_adams Tutorial Outline What is generative modeling?

2.33k views • 156 slides

Variational Inference and Generative Models CS 294-112: Deep Reinforcement Learning Sergey

Variational Inference and Generative Models CS 294-112: Deep Reinforcement Learning Sergey Levine Class Notes 1. Homework 3 due next Wednesday 2. Accept CMT peer review invitations These are required (part of your final project grade)

652 views • 34 slides

Generating & Designing DNA with Deep Generative Models Nathan Killoran, Leo J. Lee, Andrew

Generating & Designing DNA with Deep Generative Models Nathan Killoran, Leo J. Lee, Andrew Delong, David Duvenaud, Brendan J. Fey Presented by Yomna Omar on February 16th, 2018 What is DNA sequencing? The process of determining the order

533 views • 22 slides

What deep generative models can do for you: Opportunities, challenges, and open questions Giulia

What deep generative models can do for you: Opportunities, challenges, and open questions Giulia Fanti Carnegie Mellon University 1 Kiran Zinan Lin Hao Liang Alankar Jain Thekumparampil Chen Wang Sewoong Oh Vyas Sekar 2 Classifying

540 views • 51 slides

4 Deep Generative Models BVM 2018 Tutorial: Advanced Deep Learning Methods Jens Petersen Dept.

4 Deep Generative Models BVM 2018 Tutorial: Advanced Deep Learning Methods Jens Petersen Dept. of Neuroradiology, Heidelberg University Hospital Div. of Medical Image Computing, DKFZ Heidelberg Faculty of Physics & Astronomy, Heidelberg

1.48k views • 65 slides

Deep Generative Models for Clustering: A Semi-supervised and Unsupervised Approach Jhosimar

Deep Generative Models for Clustering: A Semi-supervised and Unsupervised Approach Jhosimar George Arias Figueroa Advisor: Gerberth Ad n Ram rez Rivera Masters Thesis Defense February 19, 2018 Motivation Huge amount of unlabeled

1.26k views • 125 slides

Synthesizing 3D Shapes via Modeling Multi-View Depth Maps and Silhouettes with Deep Generative

Synthesizing 3D Shapes via Modeling Multi-View Depth Maps and Silhouettes with Deep Generative Networks Amir A. Soltani Haibin Huang Jiajun Wu Tejas Kulkarni Josh Tenenbaum Samples Out-of-Sample Generalization 07/21/2017 Motivation -

783 views • 55 slides

Semi-supervised Learning with Deep Generative Models Diedrik P. Kingma, Danilo J. Rezende, Shakir

Semi-supervised Learning with Deep Generative Models Diedrik P. Kingma, Danilo J. Rezende, Shakir Mohamed, Max Welling What is Deep Learning very good at? Classifying highly structured data -ImageNet -Part of Speech Tagging -MNIST

195 views • 18 slides

Computational Systems Biology Deep Learning in the Life Sciences 6.802 6.874 20.390 20.490

Computational Systems Biology Deep Learning in the Life Sciences 6.802 6.874 20.390 20.490 HST.506 David Gifford Lecture 9 March 5, 2020 Generative Models http://mit6874.github.io 1 Why generative models? We can sample new examples from a

747 views • 52 slides

Data Preprocessing Week 2 Topics Topics Data Types Data Repositories Data

Data Preprocessing Week 2 Topics Topics Data Types Data Repositories Data Preprocessing Present homework assignment #1 Team Homework Assignment #2 Team Homework Assignment #2 Read pp 227 Read pp. 227 240, pp. 250

926 views • 67 slides

Motivation Garbage-in, garbage-out Cannot get good mining results from bad data Data

Motivation Garbage-in, garbage-out Cannot get good mining results from bad data Data Preprocessing Need to understand data properties to select the right technique and parameter values Data cleaning Mirek Riedewald Data

511 views • 7 slides

Elementary numerosities and measures notions The main result Open Emanuele Bottazzi,

Elementary numerosity and measures Emanuele Bottazzi Preliminary Elementary numerosities and measures notions The main result Open Emanuele Bottazzi, University of Trento challenges Naples - Konstanz Model Theory Days 2013 Finitely

452 views • 24 slides

How to measure the size of sets Vieri Benci Dipartimento di Matematica Applicata U. Dini

How to measure the size of sets Vieri Benci Dipartimento di Matematica Applicata U. Dini Universit di Pisa 28th May 2006 Vieri Benci (DMA) Size of Sets 28th May 2006 1 / 41 Work in collaboration with M. Di Nasso and M. Forti Vieri

1.1k views • 80 slides

Learning quantities from vision and language Raffaella Bernardi University of Trento March 23,

Learning quantities from vision and language Raffaella Bernardi University of Trento March 23, 2017 Raffaella Bernardi (University of Trento) LaVi: quantifiers March 23, 2017 1 / 44 Cardinals and Quantifiers Three of the animals are dogs.

1.09k views • 62 slides

Prototype Selection Using Polyhedron Curvature Benyamin Ghojogh, Fakhri Karray, Mark Crowley

Anomaly Detection and Prototype Selection Using Polyhedron Curvature Benyamin Ghojogh, Fakhri Karray, Mark Crowley Canadian AI conference, 2020 1 Anomaly Detection finding outliers or anomalies which differ significantly from the normal

391 views • 23 slides

SCATTERPLOTS: TASKS, DATA AND DESIGN A. Sarikaya and M. Gleicher Presented By: IEEE Transaction

SCATTERPLOTS: TASKS, DATA AND DESIGN A. Sarikaya and M. Gleicher Presented By: IEEE Transaction on Visualization and Computer Graphics Shareen Mahmud 1 WHAT IS A TRADITIONAL SCATTERPLOT? Encodes two quantitative variables using the

670 views • 18 slides

Money illusion Jean-Robert Tyran U Vienna Outline Introduction Evidence Surveys

The Econom ics of Money illusion Jean-Robert Tyran U Vienna Outline Introduction Evidence Surveys (Stephens and Tyran WP 2012) Neuroscience Field (labor, housing, stock markets ) Tyran (Science 2007), Fehr and

802 views • 34 slides