Efficient Parameter Estimation for ODE Models from Relative Data - PowerPoint PPT Presentation


 MASAMB 2016 Efficient Parameter Estimation for ODE Models from 
 Relative Data Using Hierarchical Optimization Sabrina Krause, Carolin Loos, Jan Hasenauer 
 Helmholtz Zentrum München Institute of Computational Biology 
 Data-driven Computational Modelling Cambridge, 02/10/16

Parameter Estimation

Parameter Estimation ODE model: dx dt = f ( θ , x ( t , θ )) , x ( 0 , θ ) = x 0 ( θ ) dynamics y ( t ) = h ( θ , x ( t , θ )) observables

Parameter Estimation ODE model: dx dt = f ( θ , x ( t , θ )) , x ( 0 , θ ) = x 0 ( θ ) dynamics y ( t ) = h ( θ , x ( t , θ )) observables ε k ∼ N ( 0 , σ 2 ) , Measurements: y k = h ( θ , x ( t k , θ )) + ε k , k = 1 , . . . , n t ¯

Parameter Estimation ODE model: dx dt = f ( θ , x ( t , θ )) , x ( 0 , θ ) = x 0 ( θ ) dynamics y ( t ) = h ( θ , x ( t , θ )) observables ε k ∼ N ( 0 , σ 2 ) , Measurements: y k = h ( θ , x ( t k , θ )) + ε k , k = 1 , . . . , n t ¯ Maximize the likelihood function: � ¯ � 2 �� � � 1 − 1 y k − h ( θ , x ( t k , θ )) max p ( D| θ ) = 2 πσ 2 exp � √ 2 σ θ k

Parameter Estimation ODE model: dx dt = f ( θ , x ( t , θ )) , x ( 0 , θ ) = x 0 ( θ ) dynamics y ( t ) = h ( θ , x ( t , θ )) observables ε k ∼ N ( 0 , σ 2 ) , Measurements: y k = h ( θ , x ( t k , θ )) + ε k , k = 1 , . . . , n t ¯ Minimize the negative log likelihood function: � ¯ � 2 � � J ( θ ) = 1 y k − h ( θ , x ( t k , θ )) log ( 2 πσ 2 ) + min � 2 σ θ k

Parameter Estimation ODE model: dx dt = f ( θ , x ( t , θ )) , x ( 0 , θ ) = x 0 ( θ ) dynamics y ( t ) = h ( θ , x ( t , θ )) observables ε k ∼ N ( 0 , σ 2 ) , Measurements: y k = h ( θ , x ( t k , θ )) + ε k , k = 1 , . . . , n t ¯ Minimize the negative log likelihood function: � ¯ � 2 � � J ( θ ) = 1 y k − h ( θ , x ( t k , θ )) log ( 2 πσ 2 ) + min � 2 σ θ k Optimization problem with n θ parameters

Multi-Start Optimization output parameter 2 time parameter 1

Multi-Start Optimization output parameter 2 time parameter 1

Multi-Start Optimization output parameter 2 time parameter 1

Multi-Start Optimization output parameter 2 time parameter 1

Multi-Start Optimization output parameter 2 time parameter 1

Multi-Start Optimization output parameter 2 time parameter 1

Multi-Start Optimization output parameter 2 time parameter 1

Multi-Start Optimization output parameter 2 time parameter 1

Multi-Start Optimization output parameter 2 time parameter 1

Multi-Start Optimization output parameter 2 time parameter 1

Multi-Start Optimization output parameter 2 time negative log-likelihood 1. local optimum global optimum optimizer runs parameter 1

https://github.com/ICB-DCM/PESTO Multi-Start Optimization https://github.com/ICB-DCM/AMICI output parameter 2 time negative log-likelihood 1. local optimum global optimum optimizer runs parameter 1

Parameter Estimation ODE model: dx dt = f ( θ , x ( t , θ )) , x ( 0 , θ ) = x 0 ( θ ) dynamics y ( t ) = h ( θ , x ( t , θ )) observables ε k ∼ N ( 0 , σ 2 ) , Measurements: y k = h ( θ , x ( t k , θ )) + ε k , k = 1 , . . . , n t ¯ Minimize the negative log likelihood function: � ¯ � 2 � � J ( θ ) = 1 y k − h ( θ , x ( t k , θ )) log ( 2 πσ 2 ) + min � 2 σ θ k Optimization problem with n θ parameters

Problem Statement ODE model: dx dt = f ( θ , x ( t , θ )) , x ( 0 , θ ) = x 0 ( θ ) dynamics y ( t ) = c · h ( θ , x ( t , θ )) observables Measurements that provide relative data: ε k ∼ N ( 0 , σ 2 ) , y k = c · h ( θ , x ( t k , θ )) + ε k , k = 1 , . . . , n t ¯ with unknown variance σ 2 of the measurement noise and unknown proportionality factor c

Standard Approach ODE model: dx dt = f ( θ , x ( t , θ )) , x ( 0 , θ ) = x 0 ( θ ) dynamics y ( t ) = c · h ( θ , x ( t , θ )) observables Measurements that provide relative data: ε k ∼ N ( 0 , σ 2 ) , y k = c · h ( θ , x ( t k , θ )) + ε k , k = 1 , . . . , n t ¯ with unknown variance σ 2 of the measurement noise and unknown proportionality factor c Minimize the negative log likelihood function: � ¯ � 2 � � J ( θ , c , σ 2 ) = 1 y k − c · h ( θ , x ( t k , θ )) log ( 2 πσ 2 ) + min � 2 σ θ , c , σ 2 k

Standard Approach ODE model: dx dt = f ( θ , x ( t , θ )) , x ( 0 , θ ) = x 0 ( θ ) dynamics y ( t ) = c · h ( θ , x ( t , θ )) observables Measurements that provide relative data: ε k ∼ N ( 0 , σ 2 ) , y k = c · h ( θ , x ( t k , θ )) + ε k , k = 1 , . . . , n t ¯ with unknown variance σ 2 of the measurement noise and unknown proportionality factor c Minimize the negative log likelihood function: � ¯ � 2 � � J ( θ , c , σ 2 ) = 1 y k − c · h ( θ , x ( t k , θ )) log ( 2 πσ 2 ) + min � 2 σ θ , c , σ 2 k Number of parameters: n θ + number of proportionality factors + number of variances

Hierarchical Approach Hierarchical optimization problem: � ¯ � 2 �� � � J ( θ , c , σ 2 ) = 1 y k − c · h ( θ , x ( t k , θ )) log ( 2 πσ 2 ) + min min � 2 σ θ c , σ 2 k

Hierarchical Approach Hierarchical optimization problem: � ¯ � 2 �� � � J ( θ , c , σ 2 ) = 1 y k − c · h ( θ , x ( t k , θ )) log ( 2 πσ 2 ) + min min � 2 σ θ c , σ 2 k In each step of the optimization: min 1. Calculate optimal proportionality current θ θ factors and variances analytically for a given θ min c , σ 2 2. Use analytical results to do the update step in the outer optimiza- update compute optimal tion to estimate the remaining c and σ 2 θ dynamical parameters

Hierarchical Approach Hierarchical optimization problem: � ¯ � 2 �� � � J ( θ , c , σ 2 ) = 1 y k − c · h ( θ , x ( t k , θ )) log ( 2 πσ 2 ) + min min � 2 σ θ c , σ 2 k In each step of the optimization: min 1. Calculate optimal proportionality current θ θ factors and variances analytically for a given θ min c , σ 2 2. Use analytical results to do the update step in the outer optimiza- update compute optimal tion to estimate the remaining c and σ 2 θ dynamical parameters Advantage: Outer optimization problem has n θ parameters

Analytical Derivation of the Proportionality Factors and Variances Necessary first order optimality condition: 2 ) T a local minimum of J, with J continuously differentiable, then Is (ˆ θ , ˆ c , ˆ σ 2 ) = 0 . � J (ˆ θ , ˆ c , ˆ σ

Analytical Derivation of the Proportionality Factors and Variances Necessary first order optimality condition: 2 ) T a local minimum of J, with J continuously differentiable, then Is (ˆ θ , ˆ c , ˆ σ 2 ) = 0 . � J (ˆ θ , ˆ c , ˆ σ � ¯ � 2 J ( θ , c , σ 2 ) = 1 y k − c · h ( θ , x ( t k , θ )) log ( 2 πσ 2 ) + � 2 σ k

Analytical Derivation of the Proportionality Factors and Variances Necessary first order optimality condition: 2 ) T a local minimum of J, with J continuously differentiable, then Is (ˆ θ , ˆ c , ˆ σ 2 ) = 0 . � J (ˆ θ , ˆ c , ˆ σ � ¯ � 2 J ( θ , c , σ 2 ) = 1 y k − c · h ( θ , x ( t k , θ )) log ( 2 πσ 2 ) + � 2 σ k ∂ J � ! = 0 � ∂ c � σ 2 ) ( θ , ˆ c , ˆ � − 1 c · h ( θ , x ( t k , θ )) 2 = 0 y k h ( θ , x ( t k , θ )) − ˆ � ¯ 2 σ ˆ k h ( θ , x ( t k , θ )) 2 � y k h ( θ , x ( t k , θ )) = ˆ c � ¯ k k

Analytical Derivation of the Proportionality Factors and Variances Necessary first order optimality condition: 2 ) T a local minimum of J, with J continuously differentiable, then Is (ˆ θ , ˆ c , ˆ σ 2 ) = 0 . � J (ˆ θ , ˆ c , ˆ σ � ¯ � 2 J ( θ , c , σ 2 ) = 1 y k − c · h ( θ , x ( t k , θ )) log ( 2 πσ 2 ) + � 2 σ k ∂ J � ! = 0 � ∂ c � σ 2 ) ( θ , ˆ c , ˆ � y k h ( θ , x ( t k , θ )) ¯ � − 1 c · h ( θ , x ( t k , θ )) 2 = 0 k y k h ( θ , x ( t k , θ )) − ˆ � c ( θ ) = ¯ ˆ 2 h ( θ , x ( t k , θ )) 2 σ � ˆ k k h ( θ , x ( t k , θ )) 2 � y k h ( θ , x ( t k , θ )) = ˆ c � ¯ k k

Analytical Derivation of the Proportionality Factors and Variances Necessary first order optimality condition: 2 ) T a local minimum of J, with J continuously differentiable, then Is (ˆ θ , ˆ c , ˆ σ 2 ) = 0 . � J (ˆ θ , ˆ c , ˆ σ � ¯ � 2 J ( θ , c , σ 2 ) = 1 y k − c · h ( θ , x ( t k , θ )) log ( 2 πσ 2 ) + � 2 σ k

Analytical Derivation of the Proportionality Factors and Variances Necessary first order optimality condition: 2 ) T a local minimum of J, with J continuously differentiable, then Is (ˆ θ , ˆ c , ˆ σ 2 ) = 0 . � J (ˆ θ , ˆ c , ˆ σ � ¯ � 2 J ( θ , c , σ 2 ) = 1 y k − c · h ( θ , x ( t k , θ )) log ( 2 πσ 2 ) + � 2 σ k ∂ J � ! = 0 � ∂ σ 2 � σ 2 ) ( θ , ˆ c , ˆ � � 2 y k − ˆ c · h ( θ , x ( t k , θ )) 1 � ¯ � 1 − = 0 2 2 2 ˆ σ σ ˆ k 1 = 1 � 2 � � y k − ˆ c · h ( θ , x ( t k , θ )) � ¯ 2 σ ˆ k k