size noise tradeoffs in generative networks
play

Size-Noise Tradeoffs in Generative Networks Bolton Bailey Matus - PowerPoint PPT Presentation

Dec 15, 2022 •200 likes •263 views

Size-Noise Tradeoffs in Generative Networks Bolton Bailey Matus Telgarsky Univ. of Illinois Urbana-Champaign November 29, 2018 Generative networks Easy distribution X R n . What can Y be? Hard distribution Y R d . Generator Network g :

  1. Size-Noise Tradeoffs in Generative Networks Bolton Bailey Matus Telgarsky Univ. of Illinois Urbana-Champaign November 29, 2018

  2. Generative networks Easy distribution X ∈ R n . What can Y be? Hard distribution Y ∈ R d . Generator Network g : X → Y . Previous Work ◮ Universal approximation theorem: Shallow networks approximate continuous functions. ◮ “On the ability of neural nets to express distributions”: Upper bounds for representability & shallow depth separation. Our Contribution: Wasserstein Error Bounds ◮ ( n < d ) Tight error bounds ≈ (Width) Depth → This is a deep lower bound. ◮ ( n = d ) Switching distributions ≈ polylog (1 / Error ). → ◮ ( n > d ) Trivial networks approximate normal by addition.

  3. Increasing Uniform Noise ( n < d = kn ) Networks going from Uniform [0 , 1] n to [0 , 1] kn : � � Depth Optimal Error ≈ (Width) − . k − 1 Upper Bound Proof: Space filling curve. Lower Bound Proof: Affine piece counting. ≈ ≈ ≈ ≈

  4. Normal ↔ Uniform ( n = d = 1) Normal → Uniform: Upper Bound Approximate the normal CDF with Taylor series. + + ≈ → Size = polylog(1 / Error). Uniform → Normal: Upper Bound Approximate the inverse CDF using binary search. Normal CDF Normal CDF Local variables for executing binary search Size = polylog(1 / Error). Lower bounds Size > log(1 / Error) with more affine piece counting.

  5. High Dimensional Uniform to Normal ( n > d ) Summing independent uniform distributions approximates a normal. → → With a version of Berry-Esseen, we have: � Error ≈ 1 / Number of inputs . Poster 10:45 AM - 12:45 PM Room 210 & 230 AB #141

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Generative networks part 2: GANs 23 / 54 Recap on generative networks Generative networks provide

Generative networks part 2: GANs 23 / 54 Recap on generative networks Generative networks provide

Generative networks part 2: GANs 23 / 54 Recap on generative networks Generative networks provide a way to sample from any distribution. 1. Sample z , where denotes an efficiently sampleable distribution (e.g., uniform or Gaussian). 2.

486 views • 47 slides
Generative Adversarial Networks Aaron Mishkin UBC MLRG 2018W2 1 Generative Adversial Networks

Generative Adversarial Networks Aaron Mishkin UBC MLRG 2018W2 1 Generative Adversial Networks

Generative Adversarial Networks Aaron Mishkin UBC MLRG 2018W2 1 Generative Adversial Networks Two imaginary celebrities that were dreamed up by a random number generator. https://research.nvidia.com/publication/2017-10

665 views • 39 slides
Visualizing size-security tradeoffs for lattice-based encryption Daniel J. Bernstein Horizontal

Visualizing size-security tradeoffs for lattice-based encryption Daniel J. Bernstein Horizontal

Visualizing size-security tradeoffs for lattice-based encryption Daniel J. Bernstein Horizontal axis: ciphertext size Why focus on size instead of CPU time? Fitting into existing frameworks and protocols. Data from Google. Long term:

835 views • 23 slides
Conditional Generative Adversarial Networks (and a brief look at image-to-image translation)

Conditional Generative Adversarial Networks (and a brief look at image-to-image translation)

Conditional Generative Adversarial Networks (and a brief look at image-to-image translation) Final Presentation Peter Bromley Generative Models What is generative modeling? Data: Samples from high dimensional probability distribution p real

621 views • 25 slides
Generative Adversarial Networks presented by Ian Goodfellow presentation co-developed with Aaron

Generative Adversarial Networks presented by Ian Goodfellow presentation co-developed with Aaron

Generative Adversarial Networks presented by Ian Goodfellow presentation co-developed with Aaron Courville 1 In todays talk Generative Adversarial Networks Goodfellow et al., NIPS 2014 Conditional Generative

2.84k views • 40 slides
LAB MEETING: A Connection Between Generative Adversarial Networks, Inverse Reinforcement Learning

LAB MEETING: A Connection Between Generative Adversarial Networks, Inverse Reinforcement Learning

LAB MEETING: A Connection Between Generative Adversarial Networks, Inverse Reinforcement Learning and Energy-Based models Suwon Suh POSTECH MLG Feb, 13, 2017 Goal Understanding Basic Models 1) Generative Adversarial Networks (GAN) 2)

722 views • 12 slides
Generative Deep Learning Prof. Kuan-Ting Lai 2020/5/12 DeepFake (Intro) Generative Recurrent

Generative Deep Learning Prof. Kuan-Ting Lai 2020/5/12 DeepFake (Intro) Generative Recurrent

Generative Deep Learning Prof. Kuan-Ting Lai 2020/5/12 DeepFake (Intro) Generative Recurrent Networks Douglas Eck (2002), Music Generation using LSTM Alex Graves, Generating Sequences With Recurrent Neural Networks, arXiv (2013),

854 views • 67 slides
Generative Adversarial Networks Ian Goodfellow Research Scientist GPU Technology Conference

Generative Adversarial Networks Ian Goodfellow Research Scientist GPU Technology Conference

Generative Adversarial Networks Ian Goodfellow Research Scientist GPU Technology Conference San Jose, California 2016-04-05 Generative Modeling Have training examples: x p train ( x ) Want a model that can draw samples: x p

317 views • 16 slides
Generative Adversarial Networks (GANs) Ian Goodfellow, OpenAI Research Scientist Presentation at

Generative Adversarial Networks (GANs) Ian Goodfellow, OpenAI Research Scientist Presentation at

Generative Adversarial Networks (GANs) Ian Goodfellow, OpenAI Research Scientist Presentation at Berkeley Artificial Intelligence Lab, 2016-08-31 Generative Modeling Density estimation Sample generation Training examples Model samples

897 views • 37 slides
Generative Adversarial Networks (GANs) Ian Goodfellow, OpenAI Research Scientist Re-Work Deep

Generative Adversarial Networks (GANs) Ian Goodfellow, OpenAI Research Scientist Re-Work Deep

Generative Adversarial Networks (GANs) Ian Goodfellow, OpenAI Research Scientist Re-Work Deep Learning Summit San Francisco, 2017-01-26 Generative Modeling Density estimation Sample generation Training examples Model samples

384 views • 22 slides
CSC321 Lecture 19: Generative Adversarial Networks Roger Grosse Roger Grosse CSC321 Lecture 19:

CSC321 Lecture 19: Generative Adversarial Networks Roger Grosse Roger Grosse CSC321 Lecture 19:

CSC321 Lecture 19: Generative Adversarial Networks Roger Grosse Roger Grosse CSC321 Lecture 19: Generative Adversarial Networks 1 / 25 Overview In generative modeling, wed like to train a network that models a distribution, such as a

920 views • 25 slides
Adversarial Training Attacks on Deep Networks and Generative Adversarial Networks Erkut Erdem

Adversarial Training Attacks on Deep Networks and Generative Adversarial Networks Erkut Erdem

images from Geris Game (Pixar, 1997) Adversarial Training Attacks on Deep Networks and Generative Adversarial Networks Erkut Erdem Aykut Erdem Levent Karacan Computer Vision Lab, Hacettepe University Outline Part 1: Attacks on

1.11k views • 72 slides
Generative Adversarial Networks (GANs) Ian Goodfellow, Research Scientist MLSLP Keynote, San

Generative Adversarial Networks (GANs) Ian Goodfellow, Research Scientist MLSLP Keynote, San

Generative Adversarial Networks (GANs) Ian Goodfellow, Research Scientist MLSLP Keynote, San Francisco 2016-09-13 Generative Modeling Density estimation Sample generation Training examples Model samples (Goodfellow 2016) Conditional

550 views • 28 slides
Introduction to Generative Models (and GANs) Haoqiang Fan fhq@megvii.com Nov. 2017 Figures

Introduction to Generative Models (and GANs) Haoqiang Fan fhq@megvii.com Nov. 2017 Figures

Introduction to Generative Models (and GANs) Haoqiang Fan fhq@megvii.com Nov. 2017 Figures adapted from NIPS 2016 Tutorial Generative Adversarial Networks Generative Models: Learning the Distributions Discriminative: learns the likelihood

1.37k views • 77 slides
Generative Adversarial Networks (GANs) Ian Goodfellow, OpenAI Research Scientist Presentation at

Generative Adversarial Networks (GANs) Ian Goodfellow, OpenAI Research Scientist Presentation at

Generative Adversarial Networks (GANs) Ian Goodfellow, OpenAI Research Scientist Presentation at AI With the Best, 2016-09-24 Generative Modeling Density estimation Sample generation Training examples Model samples (Goodfellow 2016)

344 views • 16 slides
Generative Adversarial Networks Wanyue Li

Generative Adversarial Networks Wanyue Li

Generating Fundus Fluorescence Angiography Images from Structure Fundus Images Using Generative Adversarial Networks Wanyue Li (wanyueli93@126.com) University of Science and Technology of China (USTC)

144 views • 10 slides
Building T EX Live Karl Berry ( karl@tug.org ) TL overview http://tug.org/texlive 2/11

Building T EX Live Karl Berry ( karl@tug.org ) TL overview http://tug.org/texlive 2/11

Building T EX Live Karl Berry ( karl@tug.org ) TL overview http://tug.org/texlive 2/11 cross-platform T EX distribution, yearly release, DVD sponsored by TUG, DANTE e.V., all user groups basis for distros (Debian, Fedora, BSD, . .

267 views • 11 slides
SHARK-FV 2014 The MOOD Ideas MOOD for multi-mat. flows The good The not-yet-good-enough

SHARK-FV 2014 The MOOD Ideas MOOD for multi-mat. flows The good The not-yet-good-enough

LA-UR 14-22765 Extension of the MOOD Method to the Saurel-Petitpas-Berry Model for Multi-Material Compressible Flows Fourth-order 2D results: The good &amp; the not-yet-good-enough a Steven Diot , a Marianne Franois, b Edward Dendy. a Fluid

897 views • 67 slides
On the Rate of Convergence in the Quantum Central Limit Theorem M. Cramer Ulm University on

On the Rate of Convergence in the Quantum Central Limit Theorem M. Cramer Ulm University on

On the Rate of Convergence in the Quantum Central Limit Theorem M. Cramer Ulm University on work with F.G.S.L. Brando Microsoft Research and University College London M. Guta University of Nottingham The Rate of Convergence in the Central

779 views • 60 slides
Whats new with OpenDNSSEC Berry van Halderen Nlnet Labs / OpenNetLabs Place of OpenDNSSEC

Whats new with OpenDNSSEC Berry van Halderen Nlnet Labs / OpenNetLabs Place of OpenDNSSEC

Whats new with OpenDNSSEC Berry van Halderen Nlnet Labs / OpenNetLabs Place of OpenDNSSEC DNSSEC adds a new dimension to DNS; Zone files do no longer sit statically in your nameserver; DNSSEC requires constant resigning, key management

361 views • 8 slides
Scaling up phylogene/c networks to genome-size data

Scaling up phylogene/c networks to genome-size data

Scaling up phylogene/c networks to genome-size data Co-supervisor Supervisor 70 % 30 % Vincent Berry Celine Scornavacca Prof, LIRMM lab CR2,

517 views • 14 slides
Explicit Optimal Binary Pebbling for One-Way Hash Chain Reversal Berry Schoenmakers Coding &amp;

Explicit Optimal Binary Pebbling for One-Way Hash Chain Reversal Berry Schoenmakers Coding &amp;

Explicit Optimal Binary Pebbling for One-Way Hash Chain Reversal Berry Schoenmakers Coding &amp; Crypto group Dept of Mathematics &amp; Computer Science TU Eindhoven, The Netherlands berry@win.tue.nl 20 th Financial Cryptography Tuesday,

1.12k views • 97 slides
Mo Mode del Parame meter Estima mation n with h Da Data Assi Assimilation on usi sing g

Mo Mode del Parame meter Estima mation n with h Da Data Assi Assimilation on usi sing g

Mo Mode del Parame meter Estima mation n with h Da Data Assi Assimilation on usi sing g NICAM AM-LE LETKF Sh Shunji Ko Kotsuki 1 , Y. Sato 2 , K. Terasaki 1 , H. Yashiro 1 H. Tomita 1 , M. Satoh 3 , and T. Miyoshi 1 1. RIKEN Center

388 views • 20 slides
Trena Wilkerson, NCTM President Robert Berry, NCTM Past-President #nctmchange #nctmchange

Trena Wilkerson, NCTM President Robert Berry, NCTM Past-President #nctmchange #nctmchange

#nctmchange Trena Wilkerson, NCTM President Robert Berry, NCTM Past-President #nctmchange #nctmchange #nctmchange NCTM #nctmchange #nctmchange Christa Jackson Eric Milou Iowa State University Rowan University Ames Iowa Glassboro, New

456 views • 34 slides

More recommend