maximum likelihood estimator lecture 10 biostatistics 602
play

Maximum Likelihood Estimator Lecture 10 Biostatistics 602 - - PowerPoint PPT Presentation

Jan 13, 2024 •233 likes •1.08k views

. . February 12th, 2013 Biostatistics 602 - Lecture 10 Hyun Min Kang February 12th, 2013 Hyun Min Kang Maximum Likelihood Estimator Lecture 10 Biostatistics 602 - Statistical Inference . Summary . . MLE Recap . . . . . . . . . 1

  1. . . February 12th, 2013 Biostatistics 602 - Lecture 10 Hyun Min Kang February 12th, 2013 Hyun Min Kang Maximum Likelihood Estimator Lecture 10 Biostatistics 602 - Statistical Inference . Summary . . MLE Recap . . . . . . . . . 1 / 20 . . . . . . . . . . . . .

  2. . . February 12th, 2013 Biostatistics 602 - Lecture 10 Hyun Min Kang 5 How can you find an MLE? . . 4 What is a maximum likelihood estimator (MLE)? . . estimator? 3 What are advantages and disadvantages of method of moment . . 2 What is a method of moment estimator? . . 1 What is a point estimator, and a point estimate? . . Last Lecture Summary . MLE Recap . . . . . . . . . 2 / 20 . . . . . . . . . . . . .

  3. . . February 12th, 2013 Biostatistics 602 - Lecture 10 Hyun Min Kang 5 How can you find an MLE? . . 4 What is a maximum likelihood estimator (MLE)? . . estimator? 3 What are advantages and disadvantages of method of moment . . 2 What is a method of moment estimator? . . 1 What is a point estimator, and a point estimate? . . Last Lecture Summary . MLE Recap . . . . . . . . . 2 / 20 . . . . . . . . . . . . .

  4. . . February 12th, 2013 Biostatistics 602 - Lecture 10 Hyun Min Kang 5 How can you find an MLE? . . 4 What is a maximum likelihood estimator (MLE)? . . estimator? 3 What are advantages and disadvantages of method of moment . . 2 What is a method of moment estimator? . . 1 What is a point estimator, and a point estimate? . . Last Lecture Summary . MLE Recap . . . . . . . . . 2 / 20 . . . . . . . . . . . . .

  5. . . February 12th, 2013 Biostatistics 602 - Lecture 10 Hyun Min Kang 5 How can you find an MLE? . . 4 What is a maximum likelihood estimator (MLE)? . . estimator? 3 What are advantages and disadvantages of method of moment . . 2 What is a method of moment estimator? . . 1 What is a point estimator, and a point estimate? . . Last Lecture Summary . MLE Recap . . . . . . . . . 2 / 20 . . . . . . . . . . . . .

  6. . . February 12th, 2013 Biostatistics 602 - Lecture 10 Hyun Min Kang 5 How can you find an MLE? . . 4 What is a maximum likelihood estimator (MLE)? . . estimator? 3 What are advantages and disadvantages of method of moment . . 2 What is a method of moment estimator? . . 1 What is a point estimator, and a point estimate? . . Last Lecture Summary . MLE Recap . . . . . . . . . 2 / 20 . . . . . . . . . . . . .

  7. . n February 12th, 2013 Biostatistics 602 - Lecture 10 Hyun Min Kang X k n . . . n . 3 / 20 MLE . Recap - Method of Moment Estimator . Summary . . . . . . . . Recap . . . . . . . . . . . . . • Point Estimation - Estimate θ or τ ( θ ) . • Method of Moment m 1 = 1 ∑ X i = E X = µ 1 m 2 = 1 i = E X 2 = µ 2 ∑ X 2 m k = 1 i = E X k = µ k ∑

  8. • Easy to implement • Easy to understand • Estimators can be improved; use as initial value to get other • No guarantee that the estimator will fall into the range of valid . . February 12th, 2013 Biostatistics 602 - Lecture 10 Hyun Min Kang parameter space. estimators i n n 4 / 20 . . . i.i.d. Recap - Example of Method of Moment Estimator . . . . . . . Summary Recap MLE . . . . . . . . . . . . . ∼ N ( µ, σ 2 ) X 1 , · · · , X n µ = X ˆ σ 2 = E X 2 = 1 µ 2 + ˆ ∑ X 2 ˆ i =1 σ 2 = ∑ ( X i − X ) 2 / n ˆ

  9. • Easy to understand • Estimators can be improved; use as initial value to get other • No guarantee that the estimator will fall into the range of valid . . February 12th, 2013 Biostatistics 602 - Lecture 10 Hyun Min Kang parameter space. estimators i n n 4 / 20 i.i.d. MLE . . Recap - Example of Method of Moment Estimator Summary . . . . . . . . Recap . . . . . . . . . . . . . ∼ N ( µ, σ 2 ) X 1 , · · · , X n µ = X ˆ σ 2 = E X 2 = 1 µ 2 + ˆ ∑ X 2 ˆ i =1 σ 2 = ∑ ( X i − X ) 2 / n ˆ • Easy to implement

  10. • Estimators can be improved; use as initial value to get other • No guarantee that the estimator will fall into the range of valid . i.i.d. February 12th, 2013 Biostatistics 602 - Lecture 10 Hyun Min Kang parameter space. estimators i n n . 4 / 20 . . Recap - Example of Method of Moment Estimator Summary . . . . . . . . MLE Recap . . . . . . . . . . . . . ∼ N ( µ, σ 2 ) X 1 , · · · , X n µ = X ˆ σ 2 = E X 2 = 1 µ 2 + ˆ ∑ X 2 ˆ i =1 σ 2 = ∑ ( X i − X ) 2 / n ˆ • Easy to implement • Easy to understand

  11. • No guarantee that the estimator will fall into the range of valid . i.i.d. February 12th, 2013 Biostatistics 602 - Lecture 10 Hyun Min Kang parameter space. estimators i n n . 4 / 20 Recap . . Summary . MLE . . . . . . . Recap - Example of Method of Moment Estimator . . . . . . . . . . . . . ∼ N ( µ, σ 2 ) X 1 , · · · , X n µ = X ˆ σ 2 = E X 2 = 1 µ 2 + ˆ ∑ X 2 ˆ i =1 σ 2 = ∑ ( X i − X ) 2 / n ˆ • Easy to implement • Easy to understand • Estimators can be improved; use as initial value to get other

  12. . i.i.d. February 12th, 2013 Biostatistics 602 - Lecture 10 Hyun Min Kang parameter space. estimators i n n . 4 / 20 Recap - Example of Method of Moment Estimator MLE . . . . . . . . . Recap . Summary . . . . . . . . . . . . . ∼ N ( µ, σ 2 ) X 1 , · · · , X n µ = X ˆ σ 2 = E X 2 = 1 µ 2 + ˆ ∑ X 2 ˆ i =1 σ 2 = ∑ ( X i − X ) 2 / n ˆ • Easy to implement • Easy to understand • Estimators can be improved; use as initial value to get other • No guarantee that the estimator will fall into the range of valid

  13. . . February 12th, 2013 Biostatistics 602 - Lecture 10 Hyun Min Kang is called the likelihood function. f x x defined by L x is observed, the function of Given that X n i.i.d. . 5 / 20 . Definition . . . . . . . . . Recap MLE . Summary Recap - Likelihood Function . . . . . . . . . . . . . . X 1 , · · · , X n ∼ f X ( x | θ ) . The join distribution of X = ( X 1 , · · · , X n ) is ∏ f X ( x | θ ) = f X ( x i | θ ) i =1

  14. . . February 12th, 2013 Biostatistics 602 - Lecture 10 Hyun Min Kang is called the likelihood function. n i.i.d. . . Definition . Recap - Likelihood Function Summary . . . . . . . . . Recap MLE . 5 / 20 . . . . . . . . . . . . . X 1 , · · · , X n ∼ f X ( x | θ ) . The join distribution of X = ( X 1 , · · · , X n ) is ∏ f X ( x | θ ) = f X ( x i | θ ) i =1 Given that X = x is observed, the function of θ defined by L ( θ | x ) = f ( x | θ )

  15. . . February 12th, 2013 Biostatistics 602 - Lecture 10 Hyun Min Kang i.i.d. . Summary Recap - Example Likelihood Function MLE Recap . . . . . . . . . 6 / 20 . . . . . . . . . . . . . • X 1 , X 2 , X 3 , X 4 ∼ Bernoulli ( p ) , 0 < p < 1 . • x = (1 , 1 , 1 , 1) T • Intuitively, it is more likely that p is larger than smaller. • L ( p | x ) = f ( x | p ) = ∏ 4 i =1 p x i (1 − p ) 1 − x i = p 4 .

  16. • For one-dimensional parameter, negative second order derivative • For two-dimensional parameter, suppose L • Check boundary points to see whether boundary gives global maximum. • Use numerical methods • Or perform direct maximization, using inequalities, or properties of L is the likelihood function. Then we need to show (a) L or February 12th, 2013 . Biostatistics 602 - Lecture 10 implies local maximum. If the function is NOT differentiable with respect to . the function. Hyun Min Kang (b) Determinant of second-order derivative is positive . . 2 Check second-order derivative to check local maximum. . . . . . . . . . Recap MLE . Summary How do we find MLE? . . 1 Find candidates that makes first order derivative to be zero . . 7 / 20 . . . . . . . . . . . . . If the function is differentiable with respect to θ ,

  17. • For one-dimensional parameter, negative second order derivative • For two-dimensional parameter, suppose L • Check boundary points to see whether boundary gives global maximum. • Use numerical methods • Or perform direct maximization, using inequalities, or properties of L is the likelihood function. Then we need to show (a) L or February 12th, 2013 . Biostatistics 602 - Lecture 10 implies local maximum. If the function is NOT differentiable with respect to . the function. Hyun Min Kang (b) Determinant of second-order derivative is positive . . 2 Check second-order derivative to check local maximum. . . . . . . . . . Recap MLE . Summary How do we find MLE? . . 1 Find candidates that makes first order derivative to be zero . . 7 / 20 . . . . . . . . . . . . . If the function is differentiable with respect to θ ,

  18. • For one-dimensional parameter, negative second order derivative • For two-dimensional parameter, suppose L • Check boundary points to see whether boundary gives global maximum. • Use numerical methods • Or perform direct maximization, using inequalities, or properties of L is the likelihood function. Then we need to show (a) L or February 12th, 2013 . Biostatistics 602 - Lecture 10 implies local maximum. If the function is NOT differentiable with respect to . the function. Hyun Min Kang (b) Determinant of second-order derivative is positive . . 2 Check second-order derivative to check local maximum. . . . . . . . . . Recap MLE . Summary How do we find MLE? . . 1 Find candidates that makes first order derivative to be zero . . 7 / 20 . . . . . . . . . . . . . If the function is differentiable with respect to θ ,

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

1 Being Normal, Simultaneously Maximizing Likelihood with Uniform Now have two equations, two

1 Being Normal, Simultaneously Maximizing Likelihood with Uniform Now have two equations, two

Likelihood of Data Maximum Likelihood Estimator The Maximum Likelihood Estimator (MLE) of , Consider n I.I.D. random variables X 1 , X 2 , ..., X n is the value of that maximizes L ( ) X i a sample from density function f (X

419 views • 3 slides
Lecture 3. Inadmissibility of Maximum Likelihood Estimate and James-Stein Estimator Yuan Yao

Lecture 3. Inadmissibility of Maximum Likelihood Estimate and James-Stein Estimator Yuan Yao

Lecture 3. Inadmissibility of Maximum Likelihood Estimate and James-Stein Estimator Yuan Yao Hong Kong University of Science and Technology March 4, 2020 Outline Recall: PCA in Noise Maximum Likelihood Estimate Example: Multivariate Normal

496 views • 46 slides
Biostatistics Preparatory Course: Methods and Computing Lecture 9 Maximum Likelihood &amp; the

Biostatistics Preparatory Course: Methods and Computing Lecture 9 Maximum Likelihood &amp; the

Biostatistics Preparatory Course: Methods and Computing Lecture 9 Maximum Likelihood &amp; the Bootstrap Methods and Computing Harvard University Department of Biostatistics 1 / 16 Overview: Maximum Likelihood Estimation Consider estimating a

455 views • 18 slides
Maximum Likelihood properties Maximum parsimony Maximum likelihood Experimental design

Maximum Likelihood properties Maximum parsimony Maximum likelihood Experimental design

N. Salamin c Sept 2007 Lecture outline Maximum likelihood in phylogenetics Definition Phylogenetics and bioinformatics for evolution Maximum likelihood and models Likelihood of a tree Computational complexity Statistical Maximum

697 views • 30 slides
k-Maximum Likelihood Estimator for mixtures of generalized Gaussians ICPR 2012, Tokyo, Japan

k-Maximum Likelihood Estimator for mixtures of generalized Gaussians ICPR 2012, Tokyo, Japan

Motivation and background k -Maximum Likelihood estimator Mixtures of generalized Gaussian distribution k-Maximum Likelihood Estimator for mixtures of generalized Gaussians ICPR 2012, Tokyo, Japan Olivier Schwander Aurlien Schutz Yannick

722 views • 22 slides
Profile Maximum Likelihood: An Optimal, Universal, Plug-and-Play Functional Estimator Yi Hao and

Profile Maximum Likelihood: An Optimal, Universal, Plug-and-Play Functional Estimator Yi Hao and

Profile Maximum Likelihood: An Optimal, Universal, Plug-and-Play Functional Estimator Yi Hao and Alon Orlitsky, UCSD 0 / 19 Outline Property estimation Plug-in estimators Prior results Profile maximum likelihood Results Simple, unified,

389 views • 20 slides
A Maximum Likelihood estimator of a Markov model for disease activity in chronic diseases that

A Maximum Likelihood estimator of a Markov model for disease activity in chronic diseases that

A Maximum Likelihood estimator of a Markov model for disease activity in chronic diseases that alternate between relapse and remission, for annually aggregated partial observations. Sixten Borg. The Swedish Institute for Health Economics (IHE),

363 views • 13 slides
Lecture 7: Maximum Likelihood Estimation (MLE) Maximum a Posteriori (MAP) Aykut Erdem

Lecture 7: Maximum Likelihood Estimation (MLE) Maximum a Posteriori (MAP) Aykut Erdem

Lecture 7: Maximum Likelihood Estimation (MLE) Maximum a Posteriori (MAP) Aykut Erdem October 2016 Hacettepe University Administrative Assignment 2 will be out on Thursday It is due November 10 (i.e. in 2 weeks) You will

831 views • 70 slides
Lecture 8: Maximum Likelihood Estimation (MLE) (contd.) Maximum a posteriori (MAP)

Lecture 8: Maximum Likelihood Estimation (MLE) (contd.) Maximum a posteriori (MAP)

Lecture 8: Maximum Likelihood Estimation (MLE) (contd.) Maximum a posteriori (MAP) estimation Nave Bayes Classifier Aykut Erdem March 2016 Hacettepe University Last time Flipping a Coin I have a coin, if I flip it, whats the

932 views • 82 slides
Maximum likelihood models Tues. Feb. 27, 2018 1 Overview of today Informal notion of

Maximum likelihood models Tues. Feb. 27, 2018 1 Overview of today Informal notion of

COMP 546 Lecture 14 Maximum likelihood models Tues. Feb. 27, 2018 1 Overview of today Informal notion of likelihood Formal definition of likelihood as conditional probability Maximum likelihood problems (sketch) 2 Scene

960 views • 40 slides
Outline n Maximum likelihood (ML) n Priors, and maximum a posteriori (MAP) n

Outline n Maximum likelihood (ML) n Priors, and maximum a posteriori (MAP) n

Maximum Likelihood (ML), Expectation Maximization (EM) Pieter Abbeel UC Berkeley EECS Many slides adapted from Thrun, Burgard and Fox, Probabilistic Robotics Outline n Maximum likelihood (ML) n Priors, and maximum a posteriori (MAP) n

649 views • 23 slides
Maximum Likelihood Estimation CS 446 Maximum likelihood: abstract formulation Weve had one

Maximum Likelihood Estimation CS 446 Maximum likelihood: abstract formulation Weve had one

Maximum Likelihood Estimation CS 446 Maximum likelihood: abstract formulation Weve had one main meta-algorithm this semester: (Regularized) ERM principle: pick the model that minimizes an average loss over training data. 1 / 70

599 views • 42 slides
Maximum Likelihood Estimation CS 446 Maximum likelihood: abstract formulation Weve had one

Maximum Likelihood Estimation CS 446 Maximum likelihood: abstract formulation Weve had one

Maximum Likelihood Estimation CS 446 Maximum likelihood: abstract formulation Weve had one main meta-algorithm) this semester: (Regularized) ERM principle: pick the model that minimizes an average loss over training data. 1 / 76

643 views • 45 slides
ECON 626: Applied Microeconomics Lecture 11: Maximum Likelihood Estimation Professors: Pamela

ECON 626: Applied Microeconomics Lecture 11: Maximum Likelihood Estimation Professors: Pamela

ECON 626: Applied Microeconomics Lecture 11: Maximum Likelihood Estimation Professors: Pamela Jakiela and Owen Ozier Maximum Likelihood: Motivation So far, weve been thinking about average treatment effects, but the ATE may or may not be

564 views • 23 slides
Bayes Estimator Lecture 15 Biostatistics 602 - Statistical Inference . . Summary Conjugate

Bayes Estimator Lecture 15 Biostatistics 602 - Statistical Inference . . Summary Conjugate

. . March 12th, 2013 Biostatistics 602 - Lecture 15 Hyun Min Kang March 12th, 2013 Hyun Min Kang Bayes Estimator Lecture 15 Biostatistics 602 - Statistical Inference . . Summary Conjugate Family . Bayes Estimator Bayesian Statistics

1.35k views • 88 slides
CS 287 Advanced Robotics (Fall 2019) Lecture 13: Kalman Smoother, Maximum A Posteriori, Maximum

CS 287 Advanced Robotics (Fall 2019) Lecture 13: Kalman Smoother, Maximum A Posteriori, Maximum

CS 287 Advanced Robotics (Fall 2019) Lecture 13: Kalman Smoother, Maximum A Posteriori, Maximum Likelihood, Expectation Maximization Pieter Abbeel UC Berkeley EECS Outline n Kalman smoothing n Maximum a posteriori sequence n Maximum likelihood

865 views • 68 slides
Value Based Care Major Trends &amp; Innovation in Data Analytics David Bolton Vice President,

Value Based Care Major Trends &amp; Innovation in Data Analytics David Bolton Vice President,

Value Based Care Major Trends &amp; Innovation in Data Analytics David Bolton Vice President, Industry Solutions Visualization is the last mile 3 Early Success: Sahlgrenska, Sweden Delivering Value Based Care (Orthopaedics) 4

648 views • 30 slides
Network Security Games Saurabh Amin Massachusetts Institute of Technology ACCESS-FORCES CPS

Network Security Games Saurabh Amin Massachusetts Institute of Technology ACCESS-FORCES CPS

Network Security Games Saurabh Amin Massachusetts Institute of Technology ACCESS-FORCES CPS workshop KTH, October 26-27, 2015 Amin (MIT) FORCES October 26, 2015 1 / 46 FORCES National Science Foundation (NSF) sponsored CPS Frontiers

754 views • 46 slides
Entanglement Entropy, Quantum Field Theory, and Holography Matthew Headrick Brandeis University

Entanglement Entropy, Quantum Field Theory, and Holography Matthew Headrick Brandeis University

Entanglement Entropy, Quantum Field Theory, and Holography Matthew Headrick Brandeis University August 6, 2014 YITP Workshop on Quantum Information Physics Contents 0 Intro &amp; disclaimer 2 1 1 Basic definitions 2 2 Entanglement

312 views • 10 slides
credal networks under epistemic irrelevance Jasper De Bock 7

credal networks under epistemic irrelevance Jasper De Bock 7

WPMSIIP 2013 credal networks under epistemic irrelevance Jasper De Bock 7 September 2013, Lugano Research group SYSTeMS Jasper De Bock Gert de

730 views • 41 slides
RAxML vs. FastTree: A Comparison of Two Maximum Likelihood Phylogeny Estimation Methods Mia

RAxML vs. FastTree: A Comparison of Two Maximum Likelihood Phylogeny Estimation Methods Mia

RAxML vs. FastTree: A Comparison of Two Maximum Likelihood Phylogeny Estimation Methods Mia Schoening RAxML vs. FastTree RAxML FastTree Implements standard SPR-based Uses combination of Neighbor-Joining, hill-climbing

1.08k views • 12 slides
Exiting from QE by Fumio Hayshi and Junko Koeda Federal Reserve Bank of San Francisco March 28

Exiting from QE by Fumio Hayshi and Junko Koeda Federal Reserve Bank of San Francisco March 28

Exiting from QE by Fumio Hayshi and Junko Koeda Federal Reserve Bank of San Francisco March 28 th 2014 Roger E. A. Farmer, Distinguished Professor, UCLA What this Paper Does Uses a structural VAR with endogenous regime switching to study

522 views • 27 slides
Es Estima timation tion of of Wei eibull bull Para Parame meter ers s us usin ing g Ma

Es Estima timation tion of of Wei eibull bull Para Parame meter ers s us usin ing g Ma

An Oral Presentation for Wind Energy International Conference Es Estima timation tion of of Wei eibull bull Para Parame meter ers s us usin ing g Ma Maxi ximum mum Li Likeli elihoo hood d Meth Method od for or Win ind P d

445 views • 16 slides
Meade Street Bridge Design Alternative Study Planning &amp; Design Open Forum Stakeholder

Meade Street Bridge Design Alternative Study Planning &amp; Design Open Forum Stakeholder

Meade Street Bridge Design Alternative Study Planning &amp; Design Open Forum Stakeholder Meeting #1 January 10, 2011 Presented by: Dan Biggs, RLA Tonights Meeting Purpose: Schedule: Open House (part I) Introduce Project 1. 6:30

445 views • 17 slides

More recommend