Examples Method Theory Simulations Conclusion A general procedure to combine estimators Fr´ ed´ eric Lavancier Laboratoire de Math´ ematiques Jean Leray University of Nantes Joint work with Paul Rochet (University of Nantes)
Examples Method Theory Simulations Conclusion Introduction Let θ be an unknown quantity in a statistical model. Consider a collection of k estimators T 1 , ..., T k of θ . Aim: combining these estimators to obtain a better estimate.
Examples Method Theory Simulations Conclusion Some examples 1 The method 2 Theoretical results 3 Simulations : back to the examples 4 Conclusion 5
Examples Method Theory Simulations Conclusion Some examples 1 The method 2 Theoretical results 3 Simulations : back to the examples 4 Conclusion 5
Examples Method Theory Simulations Conclusion Example 1 : mean and median Let x 1 , . . . , x n be n i.i.d realisations of an unknown distribution on the real line. Assume this distribution is symmetric around some parameter θ ( θ ∈ R ). Two natural choices to estimate θ : the mean T 1 = ¯ x n the median T 2 = x ( n / 2) The idea to combine these two estimators comes from Pierre Simon de Laplace. In the Second Supplement of the Th´ eorie Analytique des Probabilit´ es (1812), he wrote : ” En combinant les r´ esultats de ces deux m´ ethodes, on peut obtenir un r´ esultat dont la loi de probabilit´ e des erreurs soit plus rapidement d´ ecroissante.” [ In combining the results of these two methods, one can obtain a result whose probability law of error will be more rapidly decreasing. ]
Examples Method Theory Simulations Conclusion Example 1 : mean and median Let x 1 , . . . , x n be n i.i.d realisations of an unknown distribution on the real line. Assume this distribution is symmetric around some parameter θ ( θ ∈ R ). Two natural choices to estimate θ : the mean T 1 = ¯ x n the median T 2 = x ( n / 2) The idea to combine these two estimators comes from Pierre Simon de Laplace. In the Second Supplement of the Th´ eorie Analytique des Probabilit´ es (1812), he wrote : ” En combinant les r´ esultats de ces deux m´ ethodes, on peut obtenir un r´ esultat dont la loi de probabilit´ e des erreurs soit plus rapidement d´ ecroissante.” [ In combining the results of these two methods, one can obtain a result whose probability law of error will be more rapidly decreasing. ]
Examples Method Theory Simulations Conclusion Example 1 : mean and median Laplace considered the combination λ 1 ¯ x n + λ 2 x ( n / 2) with λ 1 + λ 2 = 1. 1. He proved that the asymptotic law of this combination is Gaussian (in 1812)! 2. Minimizing the asymptotic variance in λ 1 , λ 2 , he concluded that if the underlying distribution is Gaussian, then the best combination is to take λ 1 = 1 and λ 2 = 0. for other distributions, the best combination depends on the distribution: ” L’ignorance o` u l’on est de la loi de probabilit´ e des erreurs des observations rend cette correction impraticable” [ When one does not know the distribution of the errors of observation this correction is not feasible. ]
Examples Method Theory Simulations Conclusion Example 1 : mean and median Laplace considered the combination λ 1 ¯ x n + λ 2 x ( n / 2) with λ 1 + λ 2 = 1. 1. He proved that the asymptotic law of this combination is Gaussian (in 1812)! 2. Minimizing the asymptotic variance in λ 1 , λ 2 , he concluded that if the underlying distribution is Gaussian, then the best combination is to take λ 1 = 1 and λ 2 = 0. for other distributions, the best combination depends on the distribution: ” L’ignorance o` u l’on est de la loi de probabilit´ e des erreurs des observations rend cette correction impraticable” [ When one does not know the distribution of the errors of observation this correction is not feasible. ]
Examples Method Theory Simulations Conclusion Example 1 : mean and median Laplace considered the combination λ 1 ¯ x n + λ 2 x ( n / 2) with λ 1 + λ 2 = 1. 1. He proved that the asymptotic law of this combination is Gaussian (in 1812)! 2. Minimizing the asymptotic variance in λ 1 , λ 2 , he concluded that if the underlying distribution is Gaussian, then the best combination is to take λ 1 = 1 and λ 2 = 0. for other distributions, the best combination depends on the distribution: ” L’ignorance o` u l’on est de la loi de probabilit´ e des erreurs des observations rend cette correction impraticable” [ When one does not know the distribution of the errors of observation this correction is not feasible. ]
Examples Method Theory Simulations Conclusion Example 1 : mean and median Laplace considered the combination λ 1 ¯ x n + λ 2 x ( n / 2) with λ 1 + λ 2 = 1. 1. He proved that the asymptotic law of this combination is Gaussian (in 1812)! 2. Minimizing the asymptotic variance in λ 1 , λ 2 , he concluded that if the underlying distribution is Gaussian, then the best combination is to take λ 1 = 1 and λ 2 = 0. for other distributions, the best combination depends on the distribution: ” L’ignorance o` u l’on est de la loi de probabilit´ e des erreurs des observations rend cette correction impraticable” [ When one does not know the distribution of the errors of observation this correction is not feasible. ] Is it possible to estimate λ 1 and λ 2 ?
Examples Method Theory Simulations Conclusion Example 2 : Weibull model Let x 1 , . . . , x n i.i.d with respect to the Weibull distribution � β − 1 � x f ( x ) = β e − ( x /η ) β , x > 0 . η η We consider 3 standard methods to estimate β and η the maximum likelihood estimator (ML) the method of moments (MM) the ordinary least squares method or Weibull plot (OLS)
Recommend
More recommend
