Another look at estimating parameters in systems of ordinary - PowerPoint PPT Presentation

Another look at estimating parameters in systems of ordinary differential equations via regularization c ∗ Ivan Vujaˇ ci´ Seyed Mahdi Mahmoudi ∗∗ , Ernst Wit ∗∗ ∗ Department of Mathematics, Vrije Universiteit Amsterdam, The Netherlands ∗∗ Department of Statistics and Probability, University of Groningen, The Netherlands Van Dantzig seminar, March 6, 2014 Ivan Vujaˇ ci´ c (VU) Generalized Tikhonov regularization for ODEs Van Dantzig seminar, March 6, 2014 1 / 47

Introduction Ivan Vujaˇ ci´ c (VU) Generalized Tikhonov regularization for ODEs Van Dantzig seminar, March 6, 2014 2 / 47

Motivation System of ordinary differential equations (ODEs) in the standard form � x ′ ( t ) = f ( x ( t ) , t ; θ ) , t ∈ [ 0 , T ] , (1) x ( 0 ) = ξ , where x ( t ) , ξ ∈ R d and θ ∈ R p . x ( t ; θ , ξ ) denotes the solution of (1) for given ξ , θ . Many processes in science and engineering are modelled by (1). Ivan Vujaˇ ci´ c (VU) Generalized Tikhonov regularization for ODEs Van Dantzig seminar, March 6, 2014 3 / 47

Example: The FitzHugh-Nagumo neural spike potential equations � x ′ 1 ( t ) = c { x 1 ( t ) − x 1 ( t ) 3 / 3 + x 2 ( t ) } , x ′ 2 ( t ) = − 1 c { x 1 ( t ) − a + bx 2 ( t ) } . x 1 represents the voltage across an axon membrane. x 2 summarizes outward currents. 2 1 x 1 0 Example: −1 −2 0 5 10 15 20 ξ 1 = − 1, ξ 2 = 1. time a = 0 . 2, b = 0 . 2, c = 3. 1.0 0.5 x 2 0.0 −1.0 0 5 10 15 20 time Ivan Vujaˇ ci´ c (VU) Generalized Tikhonov regularization for ODEs Van Dantzig seminar, March 6, 2014 4 / 47

The problem Noisy observations of x ( t ; θ 0 , ξ 0 ) of some states of the system are available: y i ( t j ) = x i ( t j ; θ 0 , ξ 0 )+ ε i ( t j ) , i = 1 ,..., d 1 ; j = 1 ,..., n . where 0 ≤ t 1 ≤ ··· ≤ t n ≤ T . For simplicity, we consider Gaussian errors. Goal Estimate θ 0 from the data Y, where Y = ( y i ( t i )) ij . This is inverse problem for the coefficients in a system of ODEs. If ξ 0 is not known it is considered as parameter and estimated as well. Ivan Vujaˇ ci´ c (VU) Generalized Tikhonov regularization for ODEs Van Dantzig seminar, March 6, 2014 5 / 47

FhNdata from R package ’CollocInfer’ ● ● ● ● ● 2 ● ● ● ● ● ● ● ● ● ● ● ● ● ● 1 ● ● ● ● x 1 0 ● ● ● −1 ● ● ● ● ● ● ● ● ● ● ● ● −2 ● ● ● 0 5 10 15 20 time 2 ● ● ● ● ● ● ● 1 ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● x 2 0 ● ● ● ● ● ● ● ● ● ● ● −1 ● ● ● ● −2 0 5 10 15 20 time Ivan Vujaˇ ci´ c (VU) Generalized Tikhonov regularization for ODEs Van Dantzig seminar, March 6, 2014 6 / 47

Some existing approaches Non-linear least squares (MLE) 1 Smooth and match estimators 2 Generalized profiling procedure 3 Ivan Vujaˇ ci´ c (VU) Generalized Tikhonov regularization for ODEs Van Dantzig seminar, March 6, 2014 7 / 47

Non-linear least squares Numerical solution � x ( t ; θ , ξ ) of the ODE system. 1 Criterion M n ( θ , ξ ) . 2 d 1 n ∑ ∑ M n ( θ , ξ ) = − x i ( t j ; θ , ξ )) , log p ( y i ( t j ) | � i = 1 j = 1 where p ( y i ( t j ) | � x i ( t j ; θ , ξ )) is the probability density function of the data. NLS estimator is √ n -consistent and asymptotically efficient. Assumption: the maximum step size of the numerical solver goes to zero. Otherwise NLS is not consistent. [Xue et al., 2010] Ivan Vujaˇ ci´ c (VU) Generalized Tikhonov regularization for ODEs Van Dantzig seminar, March 6, 2014 8 / 47

Reference Xue, H.,Miao, H. and Wu, Hulin (2010). Sieve estimation of constant and time-varying coefficients in nonlinear ordinary differential equation models by considering both numerical error and measurement error. Annals of statistics , 38:2351–2387. Ivan Vujaˇ ci´ c (VU) Generalized Tikhonov regularization for ODEs Van Dantzig seminar, March 6, 2014 9 / 47

Smooth and match estimator Smoother � x ( t ) 1 Criterion M n ( θ ) 2 � T x ′ ( t ) − f ( � x ( t ) , θ ) � q w ( t ) d t . M n ( θ ) = 0 � � The √ n -consistency was shown for: regression splines for 0 < q ≤ ∞ . [Brunel et al., 2008] kernel estimator for q = 2. [Gugushvili and Klaassen, 2012] Asymptotic normality was shown for regression splines for q = 2. [Brunel et al., 2008] Van Dantzig seminar, March 6, 2014 10 / Ivan Vujaˇ ci´ c (VU) Generalized Tikhonov regularization for ODEs 47

References Brunel, N. J. et al. (2008). Parameter estimation of ode’s via nonparametric estimators. Electronic Journal of Statistics , 2:1242–1267. Gugushvili, S. and Klaassen, C. A. J. (2012). √ n -consistent parameter estimation for systems of ordinary differential equations: bypassing numerical integration via smoothing. Bernoulli , 18:1061–1098. Van Dantzig seminar, March 6, 2014 11 / Ivan Vujaˇ ci´ c (VU) Generalized Tikhonov regularization for ODEs 47

Smooth and match estimator: integral criterion Smoother � x ( t ) 1 Criterion M n ( θ , ξ ) 2 � T � t 0 f ( x ( t ) , θ ) d s � 2 d t . M n ( θ , ξ ) = 0 � � x ( t ) − ξ − For f ( x ( t ) , θ ) = g ( x ( t )) θ , g : R d → R d × p √ n -consistency was shown for: local polynomials [Dattner and Klaassen(2013)]. certain step function estimator in [Vujacic et al.(2014)]. Van Dantzig seminar, March 6, 2014 12 / Ivan Vujaˇ ci´ c (VU) Generalized Tikhonov regularization for ODEs 47

References Dattner, I., Klaassen, C.A.: Estimation in systems of ordinary differential equations linear in the parameters. arXiv preprint arXiv:1305.4126, (2013) Vujaˇ ci´ c, I., Dattner, I., Gonz´ alez, J., Wit, E. : Time-course window estimator for ordinary differential equations linear in the parameters. Statistics and Computing, (2014) (To appear in Statistics and Computing. Published online. ) Van Dantzig seminar, March 6, 2014 13 / Ivan Vujaˇ ci´ c (VU) Generalized Tikhonov regularization for ODEs 47

Generalized profiling procedure Model based smoother � x ( t ; θ , ξ ) , where � x = argmin x ∈ X m J ( x ) . 1 Criterion M n ( θ , ξ ) 2 Inner criterion � T d 1 n d 0 { x ′ i ( t ) − f i ( x ( t ) , t , θ ) } 2 d t , ∑ ∑ ∑ J ( x ) = − log p ( y i ( t j ) | x i ( t j ; θ , ξ ))+ λ w i i = 1 j = 1 i = 1 Outer criterion d 1 n ∑ ∑ M n ( θ , ξ ) = − log p ( y i ( t j ) | � x i ( t j ; θ , ξ )) . i = 1 j = 1 The estimator is consistent and asymptotically efficient. [Ramsay et al.(2007)] The only frequentist approach that can handle partially observed systems. Van Dantzig seminar, March 6, 2014 14 / Ivan Vujaˇ ci´ c (VU) Generalized Tikhonov regularization for ODEs 47

Reference Ramsay, J.O., Hooker, G., Campbell, D., Cao, J.: Parameter estimation for differential equations: a generalized smoothing approach. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 69 (5): 741–796, (2007) Van Dantzig seminar, March 6, 2014 15 / Ivan Vujaˇ ci´ c (VU) Generalized Tikhonov regularization for ODEs 47

Summary The framework: Stochastic or deterministic approximation � x of the solution. Criterion function M n . Van Dantzig seminar, March 6, 2014 16 / Ivan Vujaˇ ci´ c (VU) Generalized Tikhonov regularization for ODEs 47

This talk For simplicity let ξ 0 be known. Otherwise, define augmented vector θ ∗ = ( θ , ξ ) . The framework: 1 . � x ( θ ) = argmin x ∈ X m T α , γ ( x | θ ) , � θ n = argmin θ ∈ Θ M n ( θ | � x ( θ ) , Y ) . 2 . We consider log-likelihood criterion M n . Aim Define T α , γ such that: It yields asymptotically efficient estimator. It can handle partially observed systems. Van Dantzig seminar, March 6, 2014 17 / Ivan Vujaˇ ci´ c (VU) Generalized Tikhonov regularization for ODEs 47

Structure of the rest of the presentation Background on regularization theory. 1 Applying the regularization theory to ODE problem. 2 Asymptotic results. 3 Conceptual comparison with the generalized profiling procedure. 4 Only theory in this talk; no simulation studies. Van Dantzig seminar, March 6, 2014 18 / Ivan Vujaˇ ci´ c (VU) Generalized Tikhonov regularization for ODEs 47

1. Background on regularization theory. Van Dantzig seminar, March 6, 2014 19 / Ivan Vujaˇ ci´ c (VU) Generalized Tikhonov regularization for ODEs 47

References Vasin, V. V. and Ageev, A. L. (1995). Ill-posed problems with a priori information , volume 3. Walter de Gruyter. Engl, H. W., Hanke, M., and Neubauer, A. (1996). Regularization of inverse problems , volume 375. Springer. P¨ oschl, C. (2008). Tikhonov regularization with general residual term . University Innsbruck. Van Dantzig seminar, March 6, 2014 20 / Ivan Vujaˇ ci´ c (VU) Generalized Tikhonov regularization for ODEs 47

Well-posedness in the sense of Hadamard Let F : X → Y where X , Y are linear normed spaces and consider the equation F ( x ) = y , (2) x ∈ X , y ∈ Y . The problem (2) is well-posed in the sense of Hadamard on ( X , Y ) if: The solution of (2) exists. 1 It is unique. 2 It is continuous with respect to y . 3 The problem (2) is ill-posed on ( X , Y ) if it is not well-posed. Van Dantzig seminar, March 6, 2014 21 / Ivan Vujaˇ ci´ c (VU) Generalized Tikhonov regularization for ODEs 47