generative modeling by estimating gradients of the data
play

Generative Modeling by Estimating Gradients of the Data Distribution - PowerPoint PPT Presentation

Jul 14, 2023 •614 likes •1.12k views

Generative Modeling by Estimating Gradients of the Data Distribution Yang Song, Stefano Ermon David Zimmerer Medical Image Analysis ( #MIA-san-mia ) DKFZ Author Division Generative modeling 02.11.16 | Page2 2 | Author Division

  1. Generative Modeling by Estimating Gradients of the Data Distribution Yang Song, Stefano Ermon David Zimmerer Medical Image Analysis ( #MIA-san-mia ) DKFZ

  2. Author Division Generative modeling 02.11.16 | Page2 2 |

  3. Author Division Generative modeling 02.11.16 | Page3 3 |

  4. Author Division Generative modeling 02.11.16 | Page4 4 |

  5. Author Division Generative modeling 02.11.16 | Page5 5 |

  6. Author Division Generative modeling 02.11.16 | Page6 6 |

  7. Author Division Generative modeling 02.11.16 | Page7 7 |

  8. Author Division Generative modeling 02.11.16 | Page8 8 |

  9. Author Division “New” Idea: 02.11.16 | Page9 Generative Modeling by Estimating Gradients of the Data Distribution 9 |

  10. Author Division “New” Idea: 02.11.16 | Page10 Generative Modeling by Estimating Gradients of the Data Distribution 10 |

  11. Author Division “New” Idea: 02.11.16 | Page11 Generative Modeling by Estimating Gradients of the Data Distribution 11 |

  12. Author Division “New” Idea: 02.11.16 | Page12 Generative Modeling by Estimating Gradients of the Data Distribution 12 |

  13. Author Division Score matching 02.11.16 | Page13 [1] A. Hyvärinen. Estimation of non-normalized statistical models by score matching. Journal of Machine Learning Research, 6(Apr):695–709, 2005. 13 |

  14. Author Division Score matching 02.11.16 | Page14 → [1] A. Hyvärinen. Estimation of non-normalized statistical models by score matching. Journal of Machine Learning Research, 6(Apr):695–709, 2005. 14 |

  15. Author Division Score matching 02.11.16 | Page15 → [1] A. Hyvärinen. Estimation of non-normalized statistical models by score matching. Journal of Machine Learning Research, 6(Apr):695–709, 2005. 15 |

  16. Author Division Score matching 02.11.16 | Page16 → [1] A. Hyvärinen. Estimation of non-normalized statistical models by score matching. Journal of Machine Learning Research, 6(Apr):695–709, 2005. 16 |

  17. Author Division Score matching 02.11.16 | Page17 → → [1] A. Hyvärinen. Estimation of non-normalized statistical models by score matching. Journal of Machine Learning Research, 6(Apr):695–709, 2005. 17 |

  18. Author Division Trivial Implementation (on MNIST) 02.11.16 | Page18 18 |

  19. Author Division Trivial Implementation (on MNIST) 02.11.16 | Page19 19 |

  20. Author Division Trivial Implementation (on MNIST) 02.11.16 | Page20 20 |

  21. Author Division Spoiler: Improved Results 02.11.16 | Page21 21 |

  22. Author Division Spoiler: Improved Results 02.11.16 | Page22 22 |

  23. Author Division Spoiler: Improved Results 02.11.16 | Page23 23 |

  24. Author Division Spoiler: Improved Results 02.11.16 | Page24 24 |

  25. Author Division What are the problems ? 02.11.16 | Page25 25 |

  26. Author Division What are the problems ? 02.11.16 | Page26 26 |

  27. Author Division What are the problems ? 02.11.16 | Page27 27 |

  28. Author Division What are the problems ? 02.11.16 | Page28 → 28 |

  29. Author Division What are the problems ? 02.11.16 | Page29 → 29 |

  30. Author Division What are the problems ? 02.11.16 | Page30 → 30 |

  31. Author Division What are the problems ? 02.11.16 | Page31 → → 31 |

  32. Author Division How to sample: 02.11.16 | Page32 32 |

  33. Author Division How to sample: 02.11.16 | Page33 33 |

  34. Author Division How to sample: 02.11.16 | Page34 34 |

  35. Author Division How to sample: 02.11.16 | Page35 35 |

  36. Author Division How to sample: 02.11.16 | Page36 36 |

  37. Author Division How to sample: 02.11.16 | Page37 37 |

  38. Author Division Approach 02.11.16 | Page38 38 |

  39. Author Division Approach 02.11.16 | Page39 39 |

  40. Author Division Results 02.11.16 | Page40 40 |

  41. Author Division Results: Qualitative 02.11.16 | Page41 41 |

  42. Author Division Results: Qualitative 02.11.16 | Page42 42 |

  43. Author Division Results: Quantitative 02.11.16 | Page43 43 |

  44. Author Division Results: Reproducible 02.11.16 | Page44 → [2] A. Matosevic. Reproducibility Challenge – Generative Modeling by Estimating Gradients of the Data Distribution, NeurIPS 2019 Reproducibility Challenge Blind Report, https://openreview.net/forum?id=SkxCSTqG6H 44 |

  45. Author Division Results: Reproducible 02.11.16 | Page45 → [2] A. Matosevic. Reproducibility Challenge – Generative Modeling by Estimating Gradients of the Data Distribution, NeurIPS 2019 Reproducibility Challenge Blind Report, https://openreview.net/forum?id=SkxCSTqG6H 45 |

  46. Author Division Results: Reproducible 02.11.16 | Page46 → [2] A. Matosevic. Reproducibility Challenge – Generative Modeling by Estimating Gradients of the Data Distribution, NeurIPS 2019 Reproducibility Challenge Blind Report, https://openreview.net/forum?id=SkxCSTqG6H 46 |

  47. Author Division Results: Reproducible 02.11.16 | Page47 → [2] A. Matosevic. Reproducibility Challenge – Generative Modeling by Estimating Gradients of the Data Distribution, NeurIPS 2019 Reproducibility Challenge Blind Report, https://openreview.net/forum?id=SkxCSTqG6H 47 |

  48. Author Division Results: Reproducible 02.11.16 | Page48 → [2] A. Matosevic. Reproducibility Challenge – Generative Modeling by Estimating Gradients of the Data Distribution, NeurIPS 2019 Reproducibility Challenge Blind Report, https://openreview.net/forum?id=SkxCSTqG6H 48 |

  49. Author Division The End 02.11.16 | Page49 49 |

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

CoT: Cooperative Training for Generative Modeling of Discrete Data

CoT: Cooperative Training for Generative Modeling of Discrete Data

CoT: Cooperative Training for Generative Modeling of Discrete Data https://github.com/desire2020/CoT Sidi Lu, Lantao Yu, Siyuan Feng, Yaoming Zhu, Weinan Zhang, and Yong Yu Shanghai Jiao Tong University Autoregressive Models Autoregressive

399 views • 16 slides
Modeling Velocity Gradients in an OBC, First-Break Positioning Algorithm Noel Zinn Western

Modeling Velocity Gradients in an OBC, First-Break Positioning Algorithm Noel Zinn Western

Modeling Velocity Gradients in an OBC, First-Break Positioning Algorithm Noel Zinn Western Geophysical EAGE Geneva 1997 In this talk I provide an overview of the modeling of vertical and lateral velocity gradients that can be sources of

403 views • 25 slides
Conditional Generative Adversarial Networks (and a brief look at image-to-image translation)

Conditional Generative Adversarial Networks (and a brief look at image-to-image translation)

Conditional Generative Adversarial Networks (and a brief look at image-to-image translation) Final Presentation Peter Bromley Generative Models What is generative modeling? Data: Samples from high dimensional probability distribution p real

621 views • 25 slides
Mining for Cost Estimating Relations from Limited Complex Data Modeling Approaches for NASA

Mining for Cost Estimating Relations from Limited Complex Data Modeling Approaches for NASA

Mining for Cost Estimating Relations from Limited Complex Data Modeling Approaches for NASA Robotic Earth/Space Science Projects 2015 ICEAA Workshop 6/11/15 Shawn Hayes and Mark Jacobs Victory Solutions MIPSS Team 1 OUTLINE 1. Introduction

537 views • 36 slides
Dealing with Data Gradients: Backing Out & Calibration Nathaniel Osgood Agent-Based

Dealing with Data Gradients: Backing Out & Calibration Nathaniel Osgood Agent-Based

Dealing with Data Gradients: Backing Out & Calibration Nathaniel Osgood Agent-Based Modeling Bootcamp for Health Researchers August 24, 2011 A Key Deliverable! Reference mode Specification & Learning Parameter sensitivity

665 views • 51 slides
for Speech Synthesis and Sensor Data Augmentation Deep Generative Neural Network Speech Text

for Speech Synthesis and Sensor Data Augmentation Deep Generative Neural Network Speech Text

Deep Generative Modeling for Speech Synthesis and Sensor Data Augmentation Deep Generative Neural Network Speech Text Praveen Narayanan Ford Motor Company PROJECT DESCRIPTION Use of DNNs increasingly prevalent as a solution for many data

893 views • 53 slides
Vanishing and exploding gradients RECURREN T N EURAL N ETW ORK S F OR LAN GUAGE MODELIN G IN P

Vanishing and exploding gradients RECURREN T N EURAL N ETW ORK S F OR LAN GUAGE MODELIN G IN P

Vanishing and exploding gradients RECURREN T N EURAL N ETW ORK S F OR LAN GUAGE MODELIN G IN P YTH ON David Cecchini Data Scientist Training RNN models RECURRENT NEURAL NETWORKS FOR LANGUAGE MODELING IN PYTHON Example: a = f ( W , a , x )

623 views • 28 slides
Statistical modeling responses of bioassessment indices to eutrophication stressor gradients

Statistical modeling responses of bioassessment indices to eutrophication stressor gradients

Statistical modeling responses of bioassessment indices to eutrophication stressor gradients Presentation to Stakeholder WG Friday, March 17, 2017 Objectives Support decisions on numeric guidance (e.g., a numeric objective) for

441 views • 25 slides
Flat Metric Minimization with Applications in Generative Modeling Thomas M ollenhoff Daniel

Flat Metric Minimization with Applications in Generative Modeling Thomas M ollenhoff Daniel

Flat Metric Minimization with Applications in Generative Modeling Thomas M ollenhoff Daniel Cremers Motivation Latent concepts often induce an orientation of the data. 1/12 Simard et al. 1992, 1998; Rifai et al. 2011 Motivation Latent

855 views • 49 slides
Scalable Deep Generative Modeling for Sparse Graphs Hanjun Dai 1 , Azade Nazi 1 , Yujia Li 2 , Bo

Scalable Deep Generative Modeling for Sparse Graphs Hanjun Dai 1 , Azade Nazi 1 , Yujia Li 2 , Bo

Scalable Deep Generative Modeling for Sparse Graphs Hanjun Dai 1 , Azade Nazi 1 , Yujia Li 2 , Bo Dai 1 , Dale Schuurmans 1 1 Google Brain, 2 DeepMind Graph generative models Given a set of graphs {G 1 , G 2 , , G N }, fit a probabilistic model

516 views • 39 slides
Blended Conditional Gradients: The unconditioning of conditional gradients Joint work with Gabor

Blended Conditional Gradients: The unconditioning of conditional gradients Joint work with Gabor

Blended Conditional Gradients: The unconditioning of conditional gradients Joint work with Gabor Braun, Sebastian Pokutta, Steve Wright Dan Tu 1/9 CONDITIONAL GRADIENTS: PROJECTION-FREE Given a polytope ! , solve the optimization problem: min

109 views • 9 slides
An overview of Boosting Yoav Freund UCSD Plan of talk Generative vs. non-generative

An overview of Boosting Yoav Freund UCSD Plan of talk Generative vs. non-generative

An overview of Boosting Yoav Freund UCSD Plan of talk Generative vs. non-generative modeling Boosting Alternating decision trees Boosting and over-fitting Applications 2 Toy Example Computer receives telephone call

735 views • 61 slides
Modeling Heterogeneous Statistical Patterns in High- dimensional Data by Adversarial

Modeling Heterogeneous Statistical Patterns in High- dimensional Data by Adversarial

Modeling Heterogeneous Statistical Patterns in High- dimensional Data by Adversarial Distributions: An Unsupervised Generative Framework (FIRD) Han Zhang 1 Wenhao Zheng 3 Charley Chen 1 Kevin Gao 1 Yao Hu 3 Ling Huang 2 Wei Xu 1 1 Tsinghua

409 views • 30 slides
HACK TB Estimating subnational TB burden in Pakistan Kate LeGrand, MPH, CPH Jennifer Ross, MD,

HACK TB Estimating subnational TB burden in Pakistan Kate LeGrand, MPH, CPH Jennifer Ross, MD,

HACK TB Estimating subnational TB burden in Pakistan Kate LeGrand, MPH, CPH Jennifer Ross, MD, MPH January 7, 2020 Overview 1. Tuberculosis epidemiology 2. Sources of data 3. Data challenges in the TB landscape 4. IHME geospatial modeling

345 views • 16 slides
Estimating Daily Streamlow at Ungaged Sites in NH: A Modeling Update Neil M. Fennessey

Estimating Daily Streamlow at Ungaged Sites in NH: A Modeling Update Neil M. Fennessey

Estimating Daily Streamlow at Ungaged Sites in NH: A Modeling Update Neil M. Fennessey HYSR/Hydrologic Services HYSRConsult@gmail.com NH Department of Environmental Services Concord, NH March 29, 2018 Estimating Daily Flows at Ungaged Sites

533 views • 32 slides
LEARNING GENERATIVE MODELS ACROSS INCOMPARABLE SPACES Cha harlot otte Bunne unne , David

LEARNING GENERATIVE MODELS ACROSS INCOMPARABLE SPACES Cha harlot otte Bunne unne , David

LEARNING GENERATIVE MODELS ACROSS INCOMPARABLE SPACES Cha harlot otte Bunne unne , David Alvarez-Melis, Andreas Krause, Stefanie Jegelka Pos oster #173 173 Poster #173 173 & Charlotte Bunne Generative Modeling generative = network

250 views • 13 slides
Logical foundations of databases Diego Figueira Gabriele Puppis CNRS LaBRI

Logical foundations of databases Diego Figueira Gabriele Puppis CNRS LaBRI

day 5 ESSLLI 2016 Bolzano, Italy Logical foundations of databases Diego Figueira Gabriele Puppis CNRS LaBRI Recap Acyclic Conjunctive Q ueries Join Trees Evaluation of ACQ (LOGCFL-complete) Ears, GYO algorithm

592 views • 38 slides
Bisimulation, Logics and Metrics for Labelled Markov Processes Prakash Panangaden School of

Bisimulation, Logics and Metrics for Labelled Markov Processes Prakash Panangaden School of

Bisimulation, Logics and Metrics for Labelled Markov Processes Prakash Panangaden School of Computer Science McGill University and Research Visitor: The Alan Turing Institute Logic for Data Science Seminar The Alan Turing Institute 8th June

1.55k views • 124 slides
Crowdsourcing a Corpus for Clickbait Spoiling July 4th, 2019 Jana Puschmann 1. Referee:

Crowdsourcing a Corpus for Clickbait Spoiling July 4th, 2019 Jana Puschmann 1. Referee:

Bachelors thesis defence Crowdsourcing a Corpus for Clickbait Spoiling July 4th, 2019 Jana Puschmann 1. Referee: Prof. Dr. Benno Stein 2. Referee: PD Dr. Andreas Jakoby Clickbait The term clickbait refers to social media messages

624 views • 45 slides
Model-comparison Games with Algebraic Rules Bjarki Holm University of Cambridge Computer

Model-comparison Games with Algebraic Rules Bjarki Holm University of Cambridge Computer

Model-comparison Games with Algebraic Rules Bjarki Holm University of Cambridge Computer Laboratory Newton Institute March Games in finite model theory Logic Corresponding game First-order logic Ehrenfeucht-Frass game

873 views • 75 slides
The CLIC Beam Delivery System R. Tom as Thanks to the input of many: D. Angal-Kalinin, G.

The CLIC Beam Delivery System R. Tom as Thanks to the input of many: D. Angal-Kalinin, G.

The CLIC Beam Delivery System R. Tom as Thanks to the input of many: D. Angal-Kalinin, G. Burt, B. Dalena, J.L. Fernandez, L. Gatignon, E. Mar n, M. Modena, J. Osborne, J. Resta, G. Rumolo, H. Schmickler, D. Schulte, A. Seryi, J.

402 views • 14 slides
Game comonads & generalised quantifiers Adam O Conghaile joint with Anuj Dawar BCTCS 2020

Game comonads & generalised quantifiers Adam O Conghaile joint with Anuj Dawar BCTCS 2020

Game comonads & generalised quantifiers Adam O Conghaile joint with Anuj Dawar BCTCS 2020 O Conghaile, Dawar Game comonads & generalised quantifiers BCTCS 2020 1 / 31 Content Logic & Computation: Descriptive Complexity

863 views • 31 slides
Neural Networks: Optimization Part 1 Intro to Deep Learning, Fall 2020 1 Story so far

Neural Networks: Optimization Part 1 Intro to Deep Learning, Fall 2020 1 Story so far

Neural Networks: Optimization Part 1 Intro to Deep Learning, Fall 2020 1 Story so far Neural networks are universal approximators Can model any odd thing Provided they have the right architecture We must train them to approximate

1.89k views • 143 slides
Signal v. Appropriation: Why Do Investors Fund Patenting Firms? Michael Risch, Villanova Law

Signal v. Appropriation: Why Do Investors Fund Patenting Firms? Michael Risch, Villanova Law

Signal v. Appropriation: Why Do Investors Fund Patenting Firms? Michael Risch, Villanova Law David Ratigan, Villanova Economics Venture funding Its hard out there for a startup Hard to get funding High failure rate It gets

306 views • 18 slides

More recommend