Efficient Parameter Estimation for ODE Models from Relative Data - - PowerPoint PPT Presentation

▶
efficient parameter estimation for ode models from
SMART_READER_LITE
LIVE PREVIEW

Efficient Parameter Estimation for ODE Models from Relative Data - - PowerPoint PPT Presentation

Sep 26, 2023 587 likes •1.03k views

MASAMB 2016 Efficient Parameter Estimation for ODE Models from Relative Data Using Hierarchical Optimization Sabrina Krause, Carolin Loos, Jan Hasenauer Helmholtz Zentrum Mnchen Institute of Computational Biology Data-driven

slide-1
SLIDE 1

Efficient Parameter Estimation for ODE Models from 
 Relative Data Using Hierarchical Optimization

Sabrina Krause, Carolin Loos, Jan Hasenauer
 Helmholtz Zentrum München Institute of Computational Biology
 
 Data-driven Computational Modelling

Cambridge, 02/10/16

MASAMB 2016

slide-2
SLIDE 2

Parameter Estimation

slide-3
SLIDE 3

Parameter Estimation

ODE model: dx dt = f(θ, x(t, θ)), x(0, θ) = x0(θ) dynamics y(t) = h(θ, x(t, θ))

  • bservables
slide-4
SLIDE 4

Parameter Estimation

ODE model: dx dt = f(θ, x(t, θ)), x(0, θ) = x0(θ) dynamics y(t) = h(θ, x(t, θ))

  • bservables

Measurements: ¯ yk = h(θ, x(tk, θ)) + εk, εk ∼ N(0, σ2), k = 1, . . . , nt

slide-5
SLIDE 5

Parameter Estimation

Measurements: Maximize the likelihood function: ODE model: dx dt = f(θ, x(t, θ)), x(0, θ) = x0(θ) dynamics y(t) = h(θ, x(t, θ))

  • bservables

¯ yk = h(θ, x(tk, θ)) + εk, εk ∼ N(0, σ2), k = 1, . . . , nt max

θ

  • p(D|θ) =
  • k

1 √ 2πσ2 exp

  • −1

2 ¯ yk − h(θ, x(tk, θ)) σ 2

slide-6
SLIDE 6

Parameter Estimation

Minimize the negative log likelihood function: Measurements: ODE model: dx dt = f(θ, x(t, θ)), x(0, θ) = x0(θ) dynamics y(t) = h(θ, x(t, θ))

  • bservables

¯ yk = h(θ, x(tk, θ)) + εk, εk ∼ N(0, σ2), k = 1, . . . , nt min

θ

  • J(θ) = 1

2

  • k

log(2πσ2) + ¯ yk − h(θ, x(tk, θ)) σ 2

slide-7
SLIDE 7

Parameter Estimation

Minimize the negative log likelihood function: Measurements: ODE model: dx dt = f(θ, x(t, θ)), x(0, θ) = x0(θ) dynamics y(t) = h(θ, x(t, θ))

  • bservables

Optimization problem with nθ parameters min

θ

  • J(θ) = 1

2

  • k

log(2πσ2) + ¯ yk − h(θ, x(tk, θ)) σ 2 ¯ yk = h(θ, x(tk, θ)) + εk, εk ∼ N(0, σ2), k = 1, . . . , nt

slide-8
SLIDE 8

parameter 1 parameter 2 time

  • utput

Multi-Start Optimization

slide-9
SLIDE 9

parameter 1 parameter 2 time

  • utput

Multi-Start Optimization

slide-10
SLIDE 10

parameter 1 parameter 2 time

  • utput

Multi-Start Optimization

slide-11
SLIDE 11

parameter 1 parameter 2 time

  • utput

Multi-Start Optimization

slide-12
SLIDE 12

parameter 1 parameter 2 time

  • utput

Multi-Start Optimization

slide-13
SLIDE 13

parameter 1 parameter 2 time

  • utput

Multi-Start Optimization

slide-14
SLIDE 14

parameter 1 parameter 2 time

  • utput

Multi-Start Optimization

slide-15
SLIDE 15

parameter 1 parameter 2 time

  • utput

Multi-Start Optimization

slide-16
SLIDE 16

parameter 1 parameter 2 time

  • utput

Multi-Start Optimization

slide-17
SLIDE 17

parameter 1 parameter 2 time

  • utput

Multi-Start Optimization

slide-18
SLIDE 18

parameter 1 parameter 2 time

  • utput
  • ptimizer runs

negative log-likelihood

global

  • ptimum
  • 1. local
  • ptimum

Multi-Start Optimization

slide-19
SLIDE 19

parameter 1 parameter 2 time

  • utput
  • ptimizer runs

negative log-likelihood

global

  • ptimum
  • 1. local
  • ptimum

https://github.com/ICB-DCM/PESTO https://github.com/ICB-DCM/AMICI

Multi-Start Optimization

slide-20
SLIDE 20

Parameter Estimation

Minimize the negative log likelihood function: Measurements: ODE model: dx dt = f(θ, x(t, θ)), x(0, θ) = x0(θ) dynamics y(t) = h(θ, x(t, θ))

  • bservables

Optimization problem with nθ parameters min

θ

  • J(θ) = 1

2

  • k

log(2πσ2) + ¯ yk − h(θ, x(tk, θ)) σ 2 ¯ yk = h(θ, x(tk, θ)) + εk, εk ∼ N(0, σ2), k = 1, . . . , nt

slide-21
SLIDE 21

Problem Statement

ODE model: Measurements that provide relative data: with unknown variance σ2 of the measurement noise and unknown proportionality factor c dx dt = f(θ, x(t, θ)), x(0, θ) = x0(θ) dynamics y(t) = c·h(θ, x(t, θ))

  • bservables

¯ yk = c·h(θ, x(tk, θ)) + εk, εk ∼ N(0, σ2), k = 1, . . . , nt

slide-22
SLIDE 22

Standard Approach

Minimize the negative log likelihood function: ODE model: Measurements that provide relative data: with unknown variance σ2 of the measurement noise and unknown proportionality factor c dx dt = f(θ, x(t, θ)), x(0, θ) = x0(θ) dynamics y(t) = c·h(θ, x(t, θ))

  • bservables

¯ yk = c·h(θ, x(tk, θ)) + εk, εk ∼ N(0, σ2), k = 1, . . . , nt min

θ,c,σ2

  • J(θ, c, σ2) = 1

2

  • k

log(2πσ2) + ¯ yk − c·h(θ, x(tk, θ)) σ 2

slide-23
SLIDE 23

Standard Approach

Minimize the negative log likelihood function: ODE model: Measurements that provide relative data: with unknown variance σ2 of the measurement noise and unknown proportionality factor c dx dt = f(θ, x(t, θ)), x(0, θ) = x0(θ) dynamics y(t) = c·h(θ, x(t, θ))

  • bservables

min

θ,c,σ2

  • J(θ, c, σ2) = 1

2

  • k

log(2πσ2) + ¯ yk − c·h(θ, x(tk, θ)) σ 2 Number of parameters: nθ+ number of proportionality factors + number of variances ¯ yk = c·h(θ, x(tk, θ)) + εk, εk ∼ N(0, σ2), k = 1, . . . , nt

slide-24
SLIDE 24

Hierarchical Approach

Hierarchical optimization problem: min

θ

  • min

c,σ2

  • J(θ, c, σ2) = 1

2

  • k

log(2πσ2) + ¯ yk − c · h(θ, x(tk, θ)) σ 2

slide-25
SLIDE 25

Hierarchical Approach

In each step of the optimization:

  • 1. Calculate optimal proportionality

factors and variances analytically for a given θ

  • 2. Use analytical results to do the

update step in the outer optimiza- tion to estimate the remaining dynamical parameters min

c,σ2

min

θ

update θ current θ compute optimal c and σ2

Hierarchical optimization problem: min

θ

  • min

c,σ2

  • J(θ, c, σ2) = 1

2

  • k

log(2πσ2) + ¯ yk − c · h(θ, x(tk, θ)) σ 2

slide-26
SLIDE 26

Hierarchical Approach

In each step of the optimization:

  • 1. Calculate optimal proportionality

factors and variances analytically for a given θ

  • 2. Use analytical results to do the

update step in the outer optimiza- tion to estimate the remaining dynamical parameters min

c,σ2

min

θ

update θ current θ compute optimal c and σ2

Hierarchical optimization problem: Advantage: Outer optimization problem has nθ parameters min

θ

  • min

c,σ2

  • J(θ, c, σ2) = 1

2

  • k

log(2πσ2) + ¯ yk − c · h(θ, x(tk, θ)) σ 2

slide-27
SLIDE 27

Analytical Derivation of the Proportionality Factors and Variances

Necessary first order optimality condition: Is (ˆ θ, ˆ c, ˆ σ

2)T a local minimum of J, with J continuously differentiable, then

J(ˆ θ, ˆ c, ˆ σ

2) = 0.

slide-28
SLIDE 28

Analytical Derivation of the Proportionality Factors and Variances

Necessary first order optimality condition: Is (ˆ θ, ˆ c, ˆ σ

2)T a local minimum of J, with J continuously differentiable, then

J(ˆ θ, ˆ c, ˆ σ

2) = 0.

J(θ, c, σ2) = 1 2

  • k

log(2πσ2) + ¯ yk − c · h(θ, x(tk, θ)) σ 2

slide-29
SLIDE 29

Analytical Derivation of the Proportionality Factors and Variances

Necessary first order optimality condition: Is (ˆ θ, ˆ c, ˆ σ

2)T a local minimum of J, with J continuously differentiable, then

J(ˆ θ, ˆ c, ˆ σ

2) = 0.

J(θ, c, σ2) = 1 2

  • k

log(2πσ2) + ¯ yk − c · h(θ, x(tk, θ)) σ 2 ∂J ∂c

  • (θ,ˆ

c,ˆ σ2) !

= 0 − 1 ˆ σ

2

  • k

¯ ykh(θ, x(tk, θ)) − ˆ c · h(θ, x(tk, θ))2 = 0

  • k

¯ ykh(θ, x(tk, θ)) = ˆ c

  • k

h(θ, x(tk, θ))2

slide-30
SLIDE 30

Analytical Derivation of the Proportionality Factors and Variances

Necessary first order optimality condition: Is (ˆ θ, ˆ c, ˆ σ

2)T a local minimum of J, with J continuously differentiable, then

J(ˆ θ, ˆ c, ˆ σ

2) = 0.

∂J ∂c

  • (θ,ˆ

c,ˆ σ2) !

= 0 − 1 ˆ σ

2

  • k

¯ ykh(θ, x(tk, θ)) − ˆ c · h(θ, x(tk, θ))2 = 0

  • k

¯ ykh(θ, x(tk, θ)) = ˆ c

  • k

h(θ, x(tk, θ))2 J(θ, c, σ2) = 1 2

  • k

log(2πσ2) + ¯ yk − c · h(θ, x(tk, θ)) σ 2 ˆ c(θ) =

  • k

¯ ykh(θ, x(tk, θ))

  • k

h(θ, x(tk, θ))2

slide-31
SLIDE 31

Analytical Derivation of the Proportionality Factors and Variances

Necessary first order optimality condition: Is (ˆ θ, ˆ c, ˆ σ

2)T a local minimum of J, with J continuously differentiable, then

J(ˆ θ, ˆ c, ˆ σ

2) = 0.

J(θ, c, σ2) = 1 2

  • k

log(2πσ2) + ¯ yk − c · h(θ, x(tk, θ)) σ 2

slide-32
SLIDE 32

Analytical Derivation of the Proportionality Factors and Variances

Necessary first order optimality condition: Is (ˆ θ, ˆ c, ˆ σ

2)T a local minimum of J, with J continuously differentiable, then

J(ˆ θ, ˆ c, ˆ σ

2) = 0.

J(θ, c, σ2) = 1 2

  • k

log(2πσ2) + ¯ yk − c · h(θ, x(tk, θ)) σ 2 ∂J ∂σ2

  • (θ,ˆ

c,ˆ σ2) !

= 0 1 2ˆ σ

2

  • k

1 − ¯ yk − ˆ c · h(θ, x(tk, θ)) 2 ˆ σ

2

= 0

  • k

1 = 1 ˆ σ

2

  • k

¯ yk − ˆ c · h(θ, x(tk, θ)) 2

slide-33
SLIDE 33

Analytical Derivation of the Proportionality Factors and Variances

Necessary first order optimality condition: Is (ˆ θ, ˆ c, ˆ σ

2)T a local minimum of J, with J continuously differentiable, then

J(ˆ θ, ˆ c, ˆ σ

2) = 0.

J(θ, c, σ2) = 1 2

  • k

log(2πσ2) + ¯ yk − c · h(θ, x(tk, θ)) σ 2 ∂J ∂σ2

  • (θ,ˆ

c,ˆ σ2) !

= 0 1 2ˆ σ

2

  • k

1 − ¯ yk − ˆ c · h(θ, x(tk, θ)) 2 ˆ σ

2

= 0

  • k

1 = 1 ˆ σ

2

  • k

¯ yk − ˆ c · h(θ, x(tk, θ)) 2 ˆ σ

2(θ) = 1

nt

  • k

¯ yk − ˆ c · h(θ, x(tk, θ)) 2

slide-34
SLIDE 34

Several experiments, observables and replicates

Proportionality factors cil and variances σ2

il for each observable and

replicate, i = 1, . . . , ny, l = 1, . . . , nr Analytical solutions for the proportionality factors and the variances: ˆ σ

2 il(θ) =

  • j∈Ei

ntjil

  • k=1

¯ yjilk − ˆ cil(θ)hji(θ, x(tk, θ)) 2

  • j∈Ei

ntjil ˆ cil(θ) =

  • j∈Ei

ntjil

  • k=1

¯ yjilkhji(θ, x(tk, θ))

  • j∈Ei

ntjil

  • k=1

hji(θ, x(tk, θ))2 J(θ, c, σ2) = 1 2

ne

  • j=1
  • i∈Ij

nrji

  • l=1

ntjil

  • k=1
  • log(2πσ2

il) +

¯ yjilk − cil · hji(θ, x(tk, θ)) 2 σ2

il

slide-35
SLIDE 35

JAK-STAT Signaling Pathway

pEpoR

CYTOPLASM NUCLEUS

tSTAT pSTAT

phospho- rylation dimerization nuclear import dissociation + delayed export

p1 p2 p3 p4

R1 R2 R3 R4 R5 R6 R7 R8 R9 R1 R2 R3 R4 R5 R6 R7 R8 R9

nSTAT5 nSTAT4 nSTAT3 nSTAT2 2nSTAT1 npSTAT:npSTAT pSTAT:pSTAT pSTAT STAT nSTAT4 nSTAT3 nSTAT2 nSTAT1 nSTAT5 STAT 2pSTAT pSTAT:pSTAT npSTAT:npSTAT

p1 p2 p2 p4 p4 p4 p4 p4 p4

Fröhlich F et al. (2016) PLoS Comput Biol 12(7)

slide-36
SLIDE 36

Fitting and Convergence

20 40 60 0.5 1

Fitting

20 40 60 0.5 1

  • utput

20 40 60

time [min]

0.5 1

hierarchical approach standard approach measurements

Data from: Swaneye et al. (2003) Proc. Natl. Acad.

  • Sci. USA, 10.1073/pnas.0237333100
slide-37
SLIDE 37

Fitting and Convergence

20 40 60 0.5 1

Fitting

20 40 60 0.5 1

  • utput

20 40 60

time [min]

0.5 1

hierarchical approach standard approach measurements

Fits of same quality

slide-38
SLIDE 38

Fitting and Convergence

20 40 60 0.5 1

Fitting

20 40 60 0.5 1

  • utput

20 40 60

time [min]

0.5 1

hierarchical approach standard approach measurements

Fits of same quality

10 20 30 40 50 60

  • ptimizer runs
  • 100
  • 50

50 100

negative log-likelihood Convergence

hierarchical appoach standard approach

slide-39
SLIDE 39

Fitting and Convergence

20 40 60 0.5 1

Fitting

20 40 60 0.5 1

  • utput

20 40 60

time [min]

0.5 1

hierarchical approach standard approach measurements

Fits of same quality

10 20 30 40 50 60

  • ptimizer runs
  • 100
  • 50

50 100

negative log-likelihood Convergence

hierarchical appoach standard approach

Better convergence

slide-40
SLIDE 40

Comparison of Computation Times

standard hierarchical 20 40 60 80 100 120 140 160 180 200 cpu time [s] per converged starts Cpu times per converged starts

15 fold

10-1 100 101 102 103 104 105

log

10(cpu time single starts)[s]

2 4 6 8 10 12 14 16 18 20

frequency Cpu time per optimization

standard approach hierarchical approach

slide-41
SLIDE 41

Comparison of Computation Times

standard hierarchical 20 40 60 80 100 120 140 160 180 200 cpu time [s] per converged starts Cpu times per converged starts

15 fold

10-1 100 101 102 103 104 105

log

10(cpu time single starts)[s]

2 4 6 8 10 12 14 16 18 20

frequency Cpu time per optimization

standard approach hierarchical approach

Speed up in computation time

slide-42
SLIDE 42

Summary

  • Development of an hierarchical approach to parameter estimation for 


models with relative data


  • Analytical derivation of equations for proportionality factors and


variances


  • Implementation of the method

  • Evaluation of the method for JAK-STAT signaling pathway with


better convergence results and a substantial speed up in computation
 time

slide-43
SLIDE 43

Acknowledgements

Institute of Computational Biology Jan Hasenauer Carolin Loos Data-driven Computational Modelling

This project has received funding through the European Union's Horizon 2020 research and innovation programme under grant agreement no. 686282