# Efficient Parameter Estimation for ODE Models from Relative Data - PowerPoint PPT Presentation

## MASAMB 2016 Efficient Parameter Estimation for ODE Models from Relative Data Using Hierarchical Optimization Sabrina Krause, Carolin Loos, Jan Hasenauer Helmholtz Zentrum Mnchen Institute of Computational Biology Data-driven

1.   MASAMB 2016 Efficient Parameter Estimation for ODE Models from   Relative Data Using Hierarchical Optimization Sabrina Krause, Carolin Loos, Jan Hasenauer   Helmholtz Zentrum München Institute of Computational Biology   Data-driven Computational Modelling Cambridge, 02/10/16

2. Parameter Estimation

3. Parameter Estimation ODE model: dx dt = f ( θ , x ( t , θ )) , x ( 0 , θ ) = x 0 ( θ ) dynamics y ( t ) = h ( θ , x ( t , θ )) observables

4. Parameter Estimation ODE model: dx dt = f ( θ , x ( t , θ )) , x ( 0 , θ ) = x 0 ( θ ) dynamics y ( t ) = h ( θ , x ( t , θ )) observables ε k ∼ N ( 0 , σ 2 ) , Measurements: y k = h ( θ , x ( t k , θ )) + ε k , k = 1 , . . . , n t ¯

5. Parameter Estimation ODE model: dx dt = f ( θ , x ( t , θ )) , x ( 0 , θ ) = x 0 ( θ ) dynamics y ( t ) = h ( θ , x ( t , θ )) observables ε k ∼ N ( 0 , σ 2 ) , Measurements: y k = h ( θ , x ( t k , θ )) + ε k , k = 1 , . . . , n t ¯ Maximize the likelihood function: � ¯ � 2 �� � � 1 − 1 y k − h ( θ , x ( t k , θ )) max p ( D| θ ) = 2 πσ 2 exp � √ 2 σ θ k

6. Parameter Estimation ODE model: dx dt = f ( θ , x ( t , θ )) , x ( 0 , θ ) = x 0 ( θ ) dynamics y ( t ) = h ( θ , x ( t , θ )) observables ε k ∼ N ( 0 , σ 2 ) , Measurements: y k = h ( θ , x ( t k , θ )) + ε k , k = 1 , . . . , n t ¯ Minimize the negative log likelihood function: � ¯ � 2 � � J ( θ ) = 1 y k − h ( θ , x ( t k , θ )) log ( 2 πσ 2 ) + min � 2 σ θ k

7. Parameter Estimation ODE model: dx dt = f ( θ , x ( t , θ )) , x ( 0 , θ ) = x 0 ( θ ) dynamics y ( t ) = h ( θ , x ( t , θ )) observables ε k ∼ N ( 0 , σ 2 ) , Measurements: y k = h ( θ , x ( t k , θ )) + ε k , k = 1 , . . . , n t ¯ Minimize the negative log likelihood function: � ¯ � 2 � � J ( θ ) = 1 y k − h ( θ , x ( t k , θ )) log ( 2 πσ 2 ) + min � 2 σ θ k Optimization problem with n θ parameters

8. Multi-Start Optimization output parameter 2 time parameter 1

9. Multi-Start Optimization output parameter 2 time parameter 1

10. Multi-Start Optimization output parameter 2 time parameter 1

11. Multi-Start Optimization output parameter 2 time parameter 1

12. Multi-Start Optimization output parameter 2 time parameter 1

13. Multi-Start Optimization output parameter 2 time parameter 1

14. Multi-Start Optimization output parameter 2 time parameter 1

15. Multi-Start Optimization output parameter 2 time parameter 1

16. Multi-Start Optimization output parameter 2 time parameter 1

17. Multi-Start Optimization output parameter 2 time parameter 1

18. Multi-Start Optimization output parameter 2 time negative log-likelihood 1. local optimum global optimum optimizer runs parameter 1

19. https://github.com/ICB-DCM/PESTO Multi-Start Optimization https://github.com/ICB-DCM/AMICI output parameter 2 time negative log-likelihood 1. local optimum global optimum optimizer runs parameter 1

20. Parameter Estimation ODE model: dx dt = f ( θ , x ( t , θ )) , x ( 0 , θ ) = x 0 ( θ ) dynamics y ( t ) = h ( θ , x ( t , θ )) observables ε k ∼ N ( 0 , σ 2 ) , Measurements: y k = h ( θ , x ( t k , θ )) + ε k , k = 1 , . . . , n t ¯ Minimize the negative log likelihood function: � ¯ � 2 � � J ( θ ) = 1 y k − h ( θ , x ( t k , θ )) log ( 2 πσ 2 ) + min � 2 σ θ k Optimization problem with n θ parameters

21. Problem Statement ODE model: dx dt = f ( θ , x ( t , θ )) , x ( 0 , θ ) = x 0 ( θ ) dynamics y ( t ) = c · h ( θ , x ( t , θ )) observables Measurements that provide relative data: ε k ∼ N ( 0 , σ 2 ) , y k = c · h ( θ , x ( t k , θ )) + ε k , k = 1 , . . . , n t ¯ with unknown variance σ 2 of the measurement noise and unknown proportionality factor c

22. Standard Approach ODE model: dx dt = f ( θ , x ( t , θ )) , x ( 0 , θ ) = x 0 ( θ ) dynamics y ( t ) = c · h ( θ , x ( t , θ )) observables Measurements that provide relative data: ε k ∼ N ( 0 , σ 2 ) , y k = c · h ( θ , x ( t k , θ )) + ε k , k = 1 , . . . , n t ¯ with unknown variance σ 2 of the measurement noise and unknown proportionality factor c Minimize the negative log likelihood function: � ¯ � 2 � � J ( θ , c , σ 2 ) = 1 y k − c · h ( θ , x ( t k , θ )) log ( 2 πσ 2 ) + min � 2 σ θ , c , σ 2 k

23. Standard Approach ODE model: dx dt = f ( θ , x ( t , θ )) , x ( 0 , θ ) = x 0 ( θ ) dynamics y ( t ) = c · h ( θ , x ( t , θ )) observables Measurements that provide relative data: ε k ∼ N ( 0 , σ 2 ) , y k = c · h ( θ , x ( t k , θ )) + ε k , k = 1 , . . . , n t ¯ with unknown variance σ 2 of the measurement noise and unknown proportionality factor c Minimize the negative log likelihood function: � ¯ � 2 � � J ( θ , c , σ 2 ) = 1 y k − c · h ( θ , x ( t k , θ )) log ( 2 πσ 2 ) + min � 2 σ θ , c , σ 2 k Number of parameters: n θ + number of proportionality factors + number of variances

24. Hierarchical Approach Hierarchical optimization problem: � ¯ � 2 �� � � J ( θ , c , σ 2 ) = 1 y k − c · h ( θ , x ( t k , θ )) log ( 2 πσ 2 ) + min min � 2 σ θ c , σ 2 k

25. Hierarchical Approach Hierarchical optimization problem: � ¯ � 2 �� � � J ( θ , c , σ 2 ) = 1 y k − c · h ( θ , x ( t k , θ )) log ( 2 πσ 2 ) + min min � 2 σ θ c , σ 2 k In each step of the optimization: min 1. Calculate optimal proportionality current θ θ factors and variances analytically for a given θ min c , σ 2 2. Use analytical results to do the update step in the outer optimiza- update compute optimal tion to estimate the remaining c and σ 2 θ dynamical parameters

26. Hierarchical Approach Hierarchical optimization problem: � ¯ � 2 �� � � J ( θ , c , σ 2 ) = 1 y k − c · h ( θ , x ( t k , θ )) log ( 2 πσ 2 ) + min min � 2 σ θ c , σ 2 k In each step of the optimization: min 1. Calculate optimal proportionality current θ θ factors and variances analytically for a given θ min c , σ 2 2. Use analytical results to do the update step in the outer optimiza- update compute optimal tion to estimate the remaining c and σ 2 θ dynamical parameters Advantage: Outer optimization problem has n θ parameters

27. Analytical Derivation of the Proportionality Factors and Variances Necessary first order optimality condition: 2 ) T a local minimum of J, with J continuously differentiable, then Is (ˆ θ , ˆ c , ˆ σ 2 ) = 0 . � J (ˆ θ , ˆ c , ˆ σ

28. Analytical Derivation of the Proportionality Factors and Variances Necessary first order optimality condition: 2 ) T a local minimum of J, with J continuously differentiable, then Is (ˆ θ , ˆ c , ˆ σ 2 ) = 0 . � J (ˆ θ , ˆ c , ˆ σ � ¯ � 2 J ( θ , c , σ 2 ) = 1 y k − c · h ( θ , x ( t k , θ )) log ( 2 πσ 2 ) + � 2 σ k

29. Analytical Derivation of the Proportionality Factors and Variances Necessary first order optimality condition: 2 ) T a local minimum of J, with J continuously differentiable, then Is (ˆ θ , ˆ c , ˆ σ 2 ) = 0 . � J (ˆ θ , ˆ c , ˆ σ � ¯ � 2 J ( θ , c , σ 2 ) = 1 y k − c · h ( θ , x ( t k , θ )) log ( 2 πσ 2 ) + � 2 σ k ∂ J � ! = 0 � ∂ c � σ 2 ) ( θ , ˆ c , ˆ � − 1 c · h ( θ , x ( t k , θ )) 2 = 0 y k h ( θ , x ( t k , θ )) − ˆ � ¯ 2 σ ˆ k h ( θ , x ( t k , θ )) 2 � y k h ( θ , x ( t k , θ )) = ˆ c � ¯ k k

30. Analytical Derivation of the Proportionality Factors and Variances Necessary first order optimality condition: 2 ) T a local minimum of J, with J continuously differentiable, then Is (ˆ θ , ˆ c , ˆ σ 2 ) = 0 . � J (ˆ θ , ˆ c , ˆ σ � ¯ � 2 J ( θ , c , σ 2 ) = 1 y k − c · h ( θ , x ( t k , θ )) log ( 2 πσ 2 ) + � 2 σ k ∂ J � ! = 0 � ∂ c � σ 2 ) ( θ , ˆ c , ˆ � y k h ( θ , x ( t k , θ )) ¯ � − 1 c · h ( θ , x ( t k , θ )) 2 = 0 k y k h ( θ , x ( t k , θ )) − ˆ � c ( θ ) = ¯ ˆ 2 h ( θ , x ( t k , θ )) 2 σ � ˆ k k h ( θ , x ( t k , θ )) 2 � y k h ( θ , x ( t k , θ )) = ˆ c � ¯ k k

31. Analytical Derivation of the Proportionality Factors and Variances Necessary first order optimality condition: 2 ) T a local minimum of J, with J continuously differentiable, then Is (ˆ θ , ˆ c , ˆ σ 2 ) = 0 . � J (ˆ θ , ˆ c , ˆ σ � ¯ � 2 J ( θ , c , σ 2 ) = 1 y k − c · h ( θ , x ( t k , θ )) log ( 2 πσ 2 ) + � 2 σ k

32. Analytical Derivation of the Proportionality Factors and Variances Necessary first order optimality condition: 2 ) T a local minimum of J, with J continuously differentiable, then Is (ˆ θ , ˆ c , ˆ σ 2 ) = 0 . � J (ˆ θ , ˆ c , ˆ σ � ¯ � 2 J ( θ , c , σ 2 ) = 1 y k − c · h ( θ , x ( t k , θ )) log ( 2 πσ 2 ) + � 2 σ k ∂ J � ! = 0 � ∂ σ 2 � σ 2 ) ( θ , ˆ c , ˆ � � 2 y k − ˆ c · h ( θ , x ( t k , θ )) 1 � ¯ � 1 − = 0 2 2 2 ˆ σ σ ˆ k 1 = 1 � 2 � � y k − ˆ c · h ( θ , x ( t k , θ )) � ¯ 2 σ ˆ k k