a randomized likelihood method for data reduction in
play

A Randomized Likelihood Method for Data Reduction in Large-scale - PowerPoint PPT Presentation

Sep 10, 2022 •209 likes •991 views

A Randomized Likelihood Method for Data Reduction in Large-scale Inverse Problems Ellen B. Le Tan Bui-Thanh Aaron Myers Institute for Computational Engineering and Sciences (ICES) The University of Texas at Austin SIAM CSE 15 Salt Lake City

  1. A Randomized Likelihood Method for Data Reduction in Large-scale Inverse Problems Ellen B. Le Tan Bui-Thanh Aaron Myers Institute for Computational Engineering and Sciences (ICES) The University of Texas at Austin SIAM CSE 15 Salt Lake City π -day, 2015 Bui-Thanh, Le, Myers (ICES, UT Austin) Big data meets big inversions 1 / 31

  2. Big data, big (inverse) problems An inverse problem: find parameters of a model given real observations. Bui-Thanh, Le, Myers (ICES, UT Austin) Big data meets big inversions 2 / 31

  3. Big data, big (inverse) problems An inverse problem: find parameters of a model given real observations. y obs is our (noisy) data vector, Bui-Thanh, Le, Myers (ICES, UT Austin) Big data meets big inversions 2 / 31

  4. Big data, big (inverse) problems An inverse problem: find parameters of a model given real observations. y obs is our (noisy) data vector, ex. temperature measurements on a thermal fin ǫ ∼ N ( 0 , Γ) y obs := w ( x j ) + ǫ j , j = 1 , . . . N j Bui-Thanh, Le, Myers (ICES, UT Austin) Big data meets big inversions 2 / 31

  5. Big data, big (inverse) problems An inverse problem: find parameters of a model given real observations. y obs is our (noisy) data vector, ex. temperature measurements on a thermal fin ǫ ∼ N ( 0 , Γ) y obs := w ( x j ) + ǫ j , j = 1 , . . . N j u is the parameter we want Bui-Thanh, Le, Myers (ICES, UT Austin) Big data meets big inversions 2 / 31

  6. Big data, big (inverse) problems An inverse problem: find parameters of a model given real observations. y obs is our (noisy) data vector, ex. temperature measurements on a thermal fin ǫ ∼ N ( 0 , Γ) y obs := w ( x j ) + ǫ j , j = 1 , . . . N j u is the parameter we want Physics model: −∇ · ( e u ∇ w ) = 0 in Ω − e u ∇ w · n = Bi w on ∂ Ω \ Γ R , − e u ∇ w · n = − 1 on Γ R , Bui-Thanh, Le, Myers (ICES, UT Austin) Big data meets big inversions 2 / 31

  7. Big data, big (inverse) problems An inverse problem: find parameters of a model given real observations. y obs is our (noisy) data vector, ex. temperature measurements on a thermal fin ǫ ∼ N ( 0 , Γ) y obs := w ( x j ) + ǫ j , j = 1 , . . . N j u is the parameter we want Physics model:  −∇ · ( e u ∇ w ) = 0 in Ω   − e u ∇ w · n = Bi w on ∂ Ω \ Γ R , ⇒ G ( u ) = w , Our forward map  − e u ∇ w · n = − 1  on Γ R , Bui-Thanh, Le, Myers (ICES, UT Austin) Big data meets big inversions 2 / 31

  8. Big data, big (inverse/optimization) problems Minimize the cost. Bui-Thanh, Le, Myers (ICES, UT Austin) Big data meets big inversions 3 / 31

  9. Big data, big (inverse/optimization) problems Minimize the cost. Cost is � � u ; y obs , u 0 J s.t. −∇ · ( e u ∇ w ) = 0 in Ω − e u ∇ w · n = Bi w on ∂ Ω \ Γ R , − e u ∇ w · n = − 1 on Γ R , Bui-Thanh, Le, Myers (ICES, UT Austin) Big data meets big inversions 3 / 31

  10. Big data, big (inverse/optimization) problems Minimize the cost. Cost is � � � � := 1 2 � y obs − G ( u ) u ; y obs , u 0 � � J � 2 Γ � �� � data misfit s.t. −∇ · ( e u ∇ w ) = 0 in Ω − e u ∇ w · n = Bi w on ∂ Ω \ Γ R , − e u ∇ w · n = − 1 on Γ R , Bui-Thanh, Le, Myers (ICES, UT Austin) Big data meets big inversions 3 / 31

  11. Big data, big (inverse/optimization) problems Minimize the cost. Cost is � � � � := 1 1 2 � y obs − G ( u ) u ; y obs , u 0 2 � u − u 0 � 2 � � J + � C 2 Γ � �� � � �� � data misfit some regularization s.t. −∇ · ( e u ∇ w ) = 0 in Ω − e u ∇ w · n = Bi w on ∂ Ω \ Γ R , − e u ∇ w · n = − 1 on Γ R , Bui-Thanh, Le, Myers (ICES, UT Austin) Big data meets big inversions 3 / 31

  12. More data, higher numerical rank −∇ · ( e u ∇ w ) = 0 in Ω − e u ∇ w · n = Bi w on ∂ Ω \ Γ R , − e u ∇ w · n = − 1 on Γ R , Bui-Thanh, Le, Myers (ICES, UT Austin) Big data meets big inversions 4 / 31

  13. More data, higher numerical rank −∇ · ( e u ∇ w ) = 0 in Ω − e u ∇ w · n = Bi w on ∂ Ω \ Γ R , − e u ∇ w · n = − 1 on Γ R , 6 5 4 3 2 1 0 0.2 0.4 0.6 0.8 1 observation locations Bui-Thanh, Le, Myers (ICES, UT Austin) Big data meets big inversions 4 / 31

  14. More data, higher numerical rank −∇ · ( e u ∇ w ) = 0 in Ω − e u ∇ w · n = Bi w on ∂ Ω \ Γ R , − e u ∇ w · n = − 1 on Γ R , 2 6 10 2001 data 993 data Misfit Hessian singular values 497 data 5 249 data 0 125 data 10 63 data 4 3 −2 10 2 −4 1 10 0 0.2 0.4 0.6 0.8 1 0 10 20 30 40 50 60 observation locations index Bui-Thanh, Le, Myers (ICES, UT Austin) Big data meets big inversions 4 / 31

  15. More data, higher numerical rank −∇ · ( e u ∇ w ) = 0 in Ω − e u ∇ w · n = Bi w on ∂ Ω \ Γ R , − e u ∇ w · n = − 1 on Γ R , 6 5 4 3 2 1 0 0.2 0.4 0.6 0.8 1 observation locations Bui-Thanh, Le, Myers (ICES, UT Austin) Big data meets big inversions 5 / 31

  16. More data, higher numerical rank −∇ · ( e u ∇ w ) = 0 in Ω − e u ∇ w · n = Bi w on ∂ Ω \ Γ R , − e u ∇ w · n = − 1 on Γ R , 2 6 10 Misfit Hessian singular values 5 2001 data 1 10 1001 data 501 data 4 251 data 0 10 126 data 63 data 3 −1 10 2 −2 1 10 0 0.2 0.4 0.6 0.8 1 0 10 20 30 40 50 60 observation locations index Bui-Thanh, Le, Myers (ICES, UT Austin) Big data meets big inversions 5 / 31

  17. Big data issues in large-scale inverse problems Big data issues More data 1 Bui-Thanh, Le, Myers (ICES, UT Austin) Big data meets big inversions 6 / 31

  18. Big data issues in large-scale inverse problems Big data issues More data → higher rank 1 Bui-Thanh, Le, Myers (ICES, UT Austin) Big data meets big inversions 6 / 31

  19. Big data issues in large-scale inverse problems Big data issues More data → higher rank → more PDE solves 1 Bui-Thanh, Le, Myers (ICES, UT Austin) Big data meets big inversions 6 / 31

  20. Big data issues in large-scale inverse problems Big data issues More data → higher rank → more PDE solves → more $$$ 1 Bui-Thanh, Le, Myers (ICES, UT Austin) Big data meets big inversions 6 / 31

  21. Big data issues in large-scale inverse problems Big data issues More data → higher rank → more PDE solves → more $$$ 1 There’s a lot of redundancy in big data 2 Bui-Thanh, Le, Myers (ICES, UT Austin) Big data meets big inversions 6 / 31

  22. Big data issues in large-scale inverse problems Big data issues More data → higher rank → more PDE solves → more $$$ 1 There’s a lot of redundancy in big data 2 Furthermore, carrying big data along (I/O, data moving, etc) 3 large-scale inversion is costly Bui-Thanh, Le, Myers (ICES, UT Austin) Big data meets big inversions 6 / 31

  23. Big data issues in large-scale inverse problems Big data issues More data → higher rank → more PDE solves → more $$$ 1 There’s a lot of redundancy in big data 2 Furthermore, carrying big data along (I/O, data moving, etc) 3 large-scale inversion is costly Challenge How to reduce the cost for big-data-meets-big-inverse-problems? Bui-Thanh, Le, Myers (ICES, UT Austin) Big data meets big inversions 6 / 31

  24. Big data issues in large-scale inverse problems Big data issues More data → higher rank → more PDE solves → more $$$ 1 There’s a lot of redundancy in big data 2 Furthermore, carrying big data along (I/O, data moving, etc) 3 large-scale inversion is costly Challenge How to reduce the cost for big-data-meets-big-inverse-problems? An answer: reduce the amount of data. Bui-Thanh, Le, Myers (ICES, UT Austin) Big data meets big inversions 6 / 31

  25. Big data issues in large-scale inverse problems Big data issues More data → higher rank → more PDE solves → more $$$ 1 There’s a lot of redundancy in big data 2 Furthermore, carrying big data along (I/O, data moving, etc) 3 large-scale inversion is costly Challenge How to reduce the cost for big-data-meets-big-inverse-problems? An answer: reduce the amount of data. BUT HOW? Bui-Thanh, Le, Myers (ICES, UT Austin) Big data meets big inversions 6 / 31

  26. A Randomized Likelihood Method for Big Data � � �� misfit = 1 2 y obs − G ( u ) � Γ − 1 � � 2 � 2 Bui-Thanh, Le, Myers (ICES, UT Austin) Big data meets big inversions 7 / 31

  27. A Randomized Likelihood Method for Big Data � � �� misfit = 1 2 y obs − G ( u ) � Γ − 1 � � 2 � 2 � �� 1 � εε T � � 2 y obs − G ( u ) Γ − 1 � � , e.g. ε ∼ N ( 0 , I ) � E ε 2 � 2 Bui-Thanh, Le, Myers (ICES, UT Austin) Big data meets big inversions 7 / 31

  28. A Randomized Likelihood Method for Big Data � � �� misfit = 1 2 y obs − G ( u ) � Γ − 1 � � 2 � 2 � �� = 1 � εε T � � 2 y obs − G ( u ) Γ − 1 � � , e.g. ε ∼ N ( 0 , I ) � E ε 2 � 2 Bui-Thanh, Le, Myers (ICES, UT Austin) Big data meets big inversions 7 / 31

  29. A Randomized Likelihood Method for Big Data � � �� misfit = 1 2 y obs − G ( u ) � Γ − 1 � � 2 � 2 � �� = 1 � εε T � � 2 y obs − G ( u ) Γ − 1 � � , e.g. ε ∼ N ( 0 , I ) � E ε 2 � 2 �� �� 2 � � = 1 y obs − G ( u ) ε T Γ − 1 2 E ε 2 Bui-Thanh, Le, Myers (ICES, UT Austin) Big data meets big inversions 7 / 31

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

1 Being Normal, Simultaneously Maximizing Likelihood with Uniform Now have two equations, two

1 Being Normal, Simultaneously Maximizing Likelihood with Uniform Now have two equations, two

Likelihood of Data Maximum Likelihood Estimator The Maximum Likelihood Estimator (MLE) of , Consider n I.I.D. random variables X 1 , X 2 , ..., X n is the value of that maximizes L ( ) X i a sample from density function f (X

419 views • 3 slides
Randomized methods for machine learning David Lopez-Paz, FAIR May 17, 2016

Randomized methods for machine learning David Lopez-Paz, FAIR May 17, 2016

Randomized methods for machine learning David Lopez-Paz, FAIR May 17, 2016 http://tinyurl.com/randomized-practical Some random examples Building atomic bombs Truncated SVD Dimensionality reduction Kernel methods for big data Nonlinear

970 views • 81 slides
On a Data Assimilation Method coupling Kalman Filtering, MCRE Concept and PGD Model Reduction for

On a Data Assimilation Method coupling Kalman Filtering, MCRE Concept and PGD Model Reduction for

On a Data Assimilation Method coupling Kalman Filtering, MCRE Concept and PGD Model Reduction for Real-Time Updating of Structural Mechanics Model 2016 SIAM Conference on Uncertainty Quantification Basile Marchand 1 , Ludovic Chamoin 1 ,

1.15k views • 75 slides
Relation between Five Data Assimilation Methods for a Simplistic Weakly Non-linear Problem Trond

Relation between Five Data Assimilation Methods for a Simplistic Weakly Non-linear Problem Trond

Relation between Five Data Assimilation Methods for a Simplistic Weakly Non-linear Problem Trond Mannseth Uni Research CIPR Methods Methods Randomized Maximum Likelihood (RML) (Simultaneous estimation, Iterative updates) Methods Randomized

779 views • 55 slides
Logistic Regression, Gradient Descent, and Newton Method Matthieu R. Bloch 1 Maximum Likelihood

Logistic Regression, Gradient Descent, and Newton Method Matthieu R. Bloch 1 Maximum Likelihood

1 (1) no clear methodology. Hence, we must resort to a numerical algorithm to find the solution of (7) Here, this means that (6) (5) (4) Tie log likelihood can therefore be written as (3) In case you are not familiar with this way of

152 views • 3 slides
Lecture 5 Substitution method, and randomized algorithms! Announcements HW2 is posted! Due

Lecture 5 Substitution method, and randomized algorithms! Announcements HW2 is posted! Due

Lecture 5 Substitution method, and randomized algorithms! Announcements HW2 is posted! Due Friday. Please send any OAE letters to Luna Frank-Fischer (luna16@stanford.edu) by April 28. Lines at office hours: we know they are long.

1.08k views • 30 slides
Stat 5102 Lecture Slides: Deck 3 Likelihood Inference Charles J. Geyer School of Statistics

Stat 5102 Lecture Slides: Deck 3 Likelihood Inference Charles J. Geyer School of Statistics

Stat 5102 Lecture Slides: Deck 3 Likelihood Inference Charles J. Geyer School of Statistics University of Minnesota 1 Likelihood Inference We have learned one very general method of estimation: the method of moments. Now we learn another:

1.5k views • 113 slides
1 Quick recap of EM negative total-data log-likelihood free energy likelihood p

1 Quick recap of EM negative total-data log-likelihood free energy likelihood p

Statistical Modeling and Analysis of Neural Data (NEU 560) Princeton University, Spring 2018 Jonathan Pillow Lecture 17 notes: Latent variable models and EM Thurs, 4.12 1 Quick recap of EM negative total-data log-likelihood free energy

459 views • 5 slides
(Randomized) Localized Model Order Reduction Kathrin Smetana (University of Twente) March 24,

(Randomized) Localized Model Order Reduction Kathrin Smetana (University of Twente) March 24,

(Randomized) Localized Model Order Reduction Kathrin Smetana (University of Twente) March 24, 2020 ICERM Workshop Algorithms for Dimension and Complexity Reduction K Smetana (k.smetana@utwente.nl) Localized Model Order Reduction March

1.23k views • 88 slides
Likelihood Functions The likelihood function answers the question: What does the sensor tell about

Likelihood Functions The likelihood function answers the question: What does the sensor tell about

ambiguous sensor data ( P D < 1 , F > 0 ) k } n k n k + 1 possible interpretations of the sensor data Z k = { z j j =1 ! E 0 : the object was not detected; n k false data in the Field of View ( FoV ) E j , j = 1 , . . . , n k :

866 views • 68 slides
The Case for Lower is Better No randomized trial has established the optimal target for LDL-C

The Case for Lower is Better No randomized trial has established the optimal target for LDL-C

The Case for Lower is Better No randomized trial has established the optimal target for LDL-C for CVD risk reduction Vs Biological plausibility Apo B lipoproteins linked to atherogenesis Epidemiology data Mendelian

248 views • 8 slides
A Mendelian Randomized Controlled Trial of Long Term Reduction in Low-Density Lipoprotein

A Mendelian Randomized Controlled Trial of Long Term Reduction in Low-Density Lipoprotein

A Mendelian Randomized Controlled Trial of Long Term Reduction in Low-Density Lipoprotein Cholesterol Beginning Early in Life Brian A. Ference, M.D., M.Phil., M.Sc. ACC.12 | Chicago | 26 March 2012 Disclosures None ACC.12 |

579 views • 29 slides
SAS data reduction Haydyn Mertens (EMBL-Hamburg) Data reduction steps Acquisition Reduction

SAS data reduction Haydyn Mertens (EMBL-Hamburg) Data reduction steps Acquisition Reduction

SAS data reduction Haydyn Mertens (EMBL-Hamburg) Data reduction steps Acquisition Reduction Parameters SAXS instrumentation 2D images to 1D profile SAXS invariants - Sample - Integration - Rg - Buffer - Normalisation - I0 (MM) -

705 views • 67 slides
A Course in Applied Econometrics Outline Lecture 16 1. Introduction 2. Generalized Method

A Course in Applied Econometrics Outline Lecture 16 1. Introduction 2. Generalized Method

A Course in Applied Econometrics Outline Lecture 16 1. Introduction 2. Generalized Method of Moments Estimation Generalized Method of Moments and Empirical Likelihood 3. Empirical Likelihood 4. Computational Issues Guido Imbens 5. A

379 views • 9 slides
Maximum Likelihood Estimation CS 446 Maximum likelihood: abstract formulation Weve had one

Maximum Likelihood Estimation CS 446 Maximum likelihood: abstract formulation Weve had one

Maximum Likelihood Estimation CS 446 Maximum likelihood: abstract formulation Weve had one main meta-algorithm) this semester: (Regularized) ERM principle: pick the model that minimizes an average loss over training data. 1 / 76

643 views • 45 slides
Maximum Likelihood Estimation CS 446 Maximum likelihood: abstract formulation Weve had one

Maximum Likelihood Estimation CS 446 Maximum likelihood: abstract formulation Weve had one

Maximum Likelihood Estimation CS 446 Maximum likelihood: abstract formulation Weve had one main meta-algorithm this semester: (Regularized) ERM principle: pick the model that minimizes an average loss over training data. 1 / 70

599 views • 42 slides
Computational Methods for Inverse Problems in Geophysics Russell J. Hewett Mathematics &

Computational Methods for Inverse Problems in Geophysics Russell J. Hewett Mathematics &

Computational Methods for Inverse Problems in Geophysics Russell J. Hewett Mathematics & CMDA, Virginia Tech Theory and Experience in Solving Inverse Problems in Geophysics Workshop Uppsala University April 9, 2019 RJH (Virginia Tech)

1.51k views • 148 slides
Week 2: Greedy Algorithms Karan Singh 373F19 - Karan Singh 1 Recap Divide & Conquer

Week 2: Greedy Algorithms Karan Singh 373F19 - Karan Singh 1 Recap Divide & Conquer

CSC373 Week 2: Greedy Algorithms Karan Singh 373F19 - Karan Singh 1 Recap Divide & Conquer Master theorem Counting inversions in ( log ) Finding closest pair of points in 2 in log Fast

648 views • 47 slides
Phaseless Inverse Scattering Problems and Global Convergence Michael V. Klibanov University of

Phaseless Inverse Scattering Problems and Global Convergence Michael V. Klibanov University of

Phaseless Inverse Scattering Problems and Global Convergence Michael V. Klibanov University of North Carolina at Charlotte, USA This talk reflects my research activity in 2015-2016 Acknowledgment This work was supported by US Army Research

717 views • 49 slides
Divide-and-Conquer Lecturer: Shi Li Department of Computer Science and Engineering University at

Divide-and-Conquer Lecturer: Shi Li Department of Computer Science and Engineering University at

CSE 431/531: Analysis of Algorithms Divide-and-Conquer Lecturer: Shi Li Department of Computer Science and Engineering University at Buffalo Outline Divide-and-Conquer 1 Counting Inversions 2 Quicksort and Selection 3 Quicksort Lower

1.1k views • 95 slides
Lo Low-de depth pth qu quantu tum m cir ircuit its for r computing ting dis iscr cret

Lo Low-de depth pth qu quantu tum m cir ircuit its for r computing ting dis iscr cret

Lo Low-de depth pth qu quantu tum m cir ircuit its for r computing ting dis iscr cret ete e logarit ithms hms on bin inary ellip iptic tic curves Rainer Steinwandt (based on joint work with Martin Rtteler) Dlog log com

382 views • 13 slides
Lecture 3: Inversion and Chaining Disclaimer: In this lecture, we drop the names of the judgments

Lecture 3: Inversion and Chaining Disclaimer: In this lecture, we drop the names of the judgments

Lecture 3: Inversion and Chaining Disclaimer: In this lecture, we drop the names of the judgments eph and pers , as it is clear form the context, which formula is index how. Full linear logic contains a rich set of connectives for building

333 views • 14 slides
Seismic Pre-Stack Igneous Rock Characterization: Guanaco Field Study case, Neuquina Basin,

Seismic Pre-Stack Igneous Rock Characterization: Guanaco Field Study case, Neuquina Basin,

YPF - Pblica Seismic Pre-Stack Igneous Rock Characterization: Guanaco Field Study case, Neuquina Basin, Argentina Hernn De Simone Mario Sigismondi Marcelo Barrionuevo YPF S.A YPF - Pblica YPF - Pblica Schedule Location and

273 views • 5 slides
Inversion and extension of the finite Hilbert transform Guillermo P . Curbera Universidad de

Inversion and extension of the finite Hilbert transform Guillermo P . Curbera Universidad de

Inversion and extension of the finite Hilbert transform Guillermo P . Curbera Universidad de Sevilla September 11, 2019 Workshop on Banach spaces and Banach lattices ICMAT Guillermo P . Curbera (Univ. Sevilla) The finite Hilbert transform

1.03k views • 63 slides

More recommend