Structural Identifiability of Biological Models Nikki Meshkat - - PowerPoint PPT Presentation

Structural Identifiability of Biological Models

Nikki Meshkat Santa Clara University Joint work with Zvi Rosen and Seth Sullivant

Symbolic/Numeric Seminar at CUNY August 31, 2017

Motivation: Unidentifiable models

• Model 1:
• Model 2:

Motivation: Unidentifiable models

• Model 1: No ID scaling reparametrization!
• Model 2: ID scaling reparametrization:

Outline

• Identifiable reparametrizations of linear

compartment models

– Scaling reparametrizations – Necessary and sufficient conditions

• Finding identifiable functions, in general

– Using Gröbner Bases – Linear algebra techniques

Structural Identifiability Analysis

• Model:

– 𝑦 state variable – 𝑣 input – 𝑧 output – 𝑞 parameter

• Finding which unknown parameters of a

model can be quantified from given input-

• utput data

Loss from blood Loss from organ Drug input Measured drug concentration Drug exchange

Motivation: biological models

Example: Linear 2- Compartment Model

x1 x2

y u1

𝑦1 = − 𝑏01 + 𝑏21 𝑦1 + 𝑏12𝑦2 + 𝑣1 𝑦2 = 𝑏21𝑦1 − (𝑏02 + 𝑏12)𝑦2 𝑧 = 𝑦1

Can we determine parameters 𝑏01, 𝑏02, 𝑏12, 𝑏21 from input-

• utput data?

𝑏01 𝑏02 𝑏12 𝑏21

Linear Compartment Models

• Let 𝐻 = (𝑊, 𝐹) be a directed graph with 𝑛

edges and 𝑜 vertices

• Model

𝑦(𝑢) = 𝐵(𝐻)𝑦(𝑢) + 𝑣(𝑢) 𝑧(𝑢) = 𝑦1(𝑢)

• where 𝑦 ∈ ℝ𝑜 is the state variable

𝑣 is the input 𝑧 is the output

Assumption #1: Single input/output in first compartment

where

Linear Compartment Models

• Let 𝐻 = (𝑊, 𝐹) be a directed graph with 𝑛

edges and 𝑜 vertices

• Model

𝑦(𝑢) = 𝐵(𝐻)𝑦(𝑢) + 𝑣(𝑢) 𝑧(𝑢) = 𝑦1(𝑢)

• where 𝐵(𝐻) has the form:

𝐵(𝐻)𝑗𝑘 = −𝑏0𝑗 −

𝑙: 𝑗→𝑙 ∈𝐹

𝑏𝑙𝑗 𝑗𝑔 𝑗 = 𝑘 𝑏𝑗𝑘 𝑗𝑔 𝑘 → 𝑗 𝑗𝑡 𝑏𝑜 𝑓𝑒𝑕𝑓 𝑝𝑔 𝐻 𝑝𝑢ℎ𝑓𝑠𝑥𝑗𝑡𝑓

Assumption #2: Leaks from every compartment, so 𝑏0𝑗 ≠ 0 for all 𝑗

Linear Compartment Models

• Let 𝐻 = (𝑊, 𝐹) be a directed graph with 𝑛

edges and 𝑜 vertices

• Model

𝑦(𝑢) = 𝐵(𝐻)𝑦(𝑢) + 𝑣(𝑢) 𝑧(𝑢) = 𝑦1(𝑢)

• where 𝐵(𝐻) has the form:

𝐵(𝐻)𝑗𝑘 = 𝑏𝑗𝑗 𝑗𝑔 𝑗 = 𝑘 𝑏𝑗𝑘 𝑗𝑔 𝑘 → 𝑗 𝑗𝑡 𝑏𝑜 𝑓𝑒𝑕𝑓 𝑝𝑔 𝐻 𝑝𝑢ℎ𝑓𝑠𝑥𝑗𝑡𝑓

Assumption #3: Strongly connected graph, with 𝑛 ≤ 2𝑜 − 2

Our class of models

• Assumptions:

– I/O in first compartment – Leaks from every compartment – 𝐻 strongly connected with at most 2𝑜 − 2 edges

𝑦 = 𝐵𝑦 + 𝑣 𝑧 = 𝑦1

• Identifiability: Which parameters of model can

be determined from given input-output data?

– Must first determine input-output equation

Find Input-Output Equation

• Rewrite system eqns as 𝜖𝐽 − 𝐵 𝑦 = 𝑣
• Cramer’s Rule:

𝐵 = 𝑏11 ⋯ 𝑏1𝑜 ⋮ ⋱ ⋮ 𝑏𝑜1 ⋯ 𝑏𝑜𝑜 𝐵1 = 𝑏22 ⋯ 𝑏2𝑜 ⋮ ⋱ ⋮ 𝑏𝑜2 ⋯ 𝑏𝑜𝑜 𝑦1 = 𝑒𝑓𝑢 𝜖𝐽 − 𝐵1 𝑣1 𝑒𝑓𝑢(𝜖𝐽 − 𝐵)

• I/O eqn: 𝑒𝑓𝑢 𝜖𝐽 − 𝐵 𝑧 = 𝑒𝑓𝑢 𝜖𝐽 − 𝐵1 𝑣1

𝑧 𝑜 + 𝑑1𝑧 𝑜−1 + ⋯ + 𝑑𝑜𝑧 = 𝑣1

𝑜−1 + 𝑑𝑜+1𝑣1 𝑜−2 + ⋯ + 𝑑2𝑜−1𝑣1

Identifiability

• Can recover coefficients from data [Soderstrom &

Stoica 1998]

• Identifiability: is it possible to recover the parameters
• f the original system, from the coefficients of I/O

eqn? [Ljung & Glad 1994] – Two sets of parameter values yield same coefficient values? – Is coeff map 1-to-1? 𝑧 𝑜 + 𝑑1𝑧 𝑜−1 + ⋯ + 𝑑𝑜𝑧 = 𝑣1

𝑜−1 + 𝑑𝑜+1𝑣1 𝑜−2 + ⋯ + 𝑑2𝑜−1𝑣1

Identifiability from I/O eqns

• Question of injectivity of the coefficient map

𝑑: ℝ𝑛+𝑜 → ℝ2𝑜−1

• If c is one-to-one: globally identifiable

finite-to-one: locally identifiable infinite-to-one: unidentifiable

• Concerned with generic local identifiability

– Check dimension of image of coefficient map

• If dim im c = m+n, then locally identifiable
• If dim im c < m+n, then unidentifiable

Local identifiability vs. Unidentifiability

• Ex: 2-comp model

I/O eqn:

𝑧 − 𝑏11 + 𝑏22 𝑧 + 𝑏11𝑏22 − 𝑏12𝑏21 𝑧 = 𝑣1 − 𝑏22𝑣1 Coefficient map 𝑑: ℝ4 → ℝ3

(𝑏11, 𝑏12, 𝑏21, 𝑏22) ⟼ (− 𝑏11 + 𝑏22 , 𝑏11𝑏22 − 𝑏12𝑏21, −𝑏22)

Jacobian has rank 3:

⇒ Unidentifiable! 𝑦1 𝑦2 = 𝑏11 𝑏12 𝑏21 𝑏22 𝑦1 𝑦2 + 𝑣1 0 , 𝑧 = 𝑦1

Unidentifiable models

• Cannot determine all parameters, but can we

determine some combination of the parameters? Ex: 𝑏12 + 𝑏21 or 𝑏12𝑏21

• A function 𝑔: ℝ𝑛+𝑜 → ℝ is called identifiable

from c if ℝ(𝑔, 𝑑1, … , 𝑑2𝑜−1)/ℝ(𝑑1, … , 𝑑2𝑜−1) is an algebraic field extension

Unidentifiable models

• Cannot determine all parameters, but can we

determine some combination of the parameters? Ex: 𝑏12 + 𝑏21 or 𝑏12𝑏21

• A function 𝑔: ℝ𝑛+𝑜 → ℝ is called identifiable

from c if 𝑔 = Φ(𝑑)

Identifiable functions

• Coefficients:
• Identifiable functions:
• What do we do with identifiable functions?
• Any special meaning in original model?

2-compartment model as graph

Model: Graph: 1 2

• Cycle:
• “Self” cycles:

𝑦1 𝑦2 = 𝑏11 𝑏12 𝑏21 𝑏22 𝑦1 𝑦2 + 𝑣1 𝑧 = 𝑦1

𝑏12𝑏21 𝑏11, 𝑏22

Coeff map factors through cycles

• Coeff map c in terms of cycles of graph G
• Due to cyclic permutations in determinant

(Leibniz formula)

• Strongly connected?
• G is strongly connected: m+1 independent cycles

(= n self-cycles + m-n+1 cycles) Thus dim im c ≤ m+1

Why are cycles identifiable?

• Recall dim im c = m+1 = 2+1 = 3
• Let 𝑕: ℝ4 → ℝ3 be the “cycle map”

(𝑏11, 𝑏22, 𝑏12, 𝑏21) ⟼ (𝑏11, 𝑏22, 𝑏12𝑏21)

• Commutative diagram:

𝑑 ℝ4 ℝ3 ℝ3 𝑕 Φ 𝑕 = Φ(𝑑)

Thus

Unidentifiable model

• Model
• Identifiable functions

i.e.

• Reparametrize: 4 independent parameters

3 independent parameters?

𝑦1 𝑦2 = 𝑏11 𝑏12 𝑏21 𝑏22 𝑦1 𝑦2 + 𝑣1 𝑧 = 𝑦1

𝑏11, 𝑏22, 𝑏12𝑏21 − 𝑏01 + 𝑏21 , − 𝑏02 + 𝑏12 , 𝑏12𝑏21

Identifiable reparametrization

Let 𝑑: ℝ𝑛+𝑜 → ℝ2𝑜−1 be our coefficient map An identifiable reparametrization of a model is a map 𝑟: ℝ𝑙 → ℝ𝑛+𝑜 such that:

• 𝑑 ∘ 𝑟: ℝ𝑙 → ℝ2𝑜−1 has the same image as

𝑑

• 𝑑 ∘ 𝑟 is identifiable (finite-to-one)

Scaling reparametrization

• Choice of functions 𝑔

1, … , 𝑔 𝑜: ℝ𝑛+𝑜 → ℝ where we

replace 𝑦1, … , 𝑦𝑜 with

𝑌𝑗 = 𝑔

𝑗 𝐵 𝑦𝑗

• Set 𝑔

1 = 1 since 𝑧 = 𝑦1 is observed

• Since model is

𝑦 = 𝐵𝑦 + 𝑣, each parameter 𝑏𝑗𝑘 is replaced with 𝑏𝑗𝑘𝑔

𝑗 𝐵

𝑔

𝑘 𝐵

• Only graphs with at most 2n-2 edges

Identifiable scaling reparametrization

• Use scaling: 𝑌1 = 𝑦1, 𝑌2 = 𝑏12𝑦2
• Re-write:
• Map 𝑑 ∘ 𝑟 has same image as 𝑑 and is 1-to-1

(𝑟11, 𝑟21, 𝑟22) ⟼ (− 𝑟11 + 𝑟22 , 𝑟11𝑟22 − 𝑟21, −𝑟22)

𝑌1 𝑌2 = 𝑏11 1 𝑏12𝑏21 𝑏22 𝑌1 𝑌2 + 𝑣1 𝑧 = 𝑌1 𝑧 = 𝑌1 𝑌1 𝑌2 = 𝑟11 1 𝑟21 𝑟22 𝑌1 𝑌2 + 𝑣1

Identifiable scaling reparametrization

• Use scaling: 𝑌1 = 𝑦1, 𝑌2 = 𝑏12𝑦2
• Why useful?

– Nondimensionalization of original model – New model: know qualitative behavior of 𝑌1, 𝑌2

• Worked for 2-comp model. Does this always

work?

𝑌1 𝑌2 = 𝑏11 1 𝑏12𝑏21 𝑏22 𝑌1 𝑌2 + 𝑣1 𝑧 = 𝑌1

Motivation: Unidentifiable models

• Model 1: No ID scaling reparametrization!
• Model 2: ID scaling reparametrization:

1 2 3 1 2 3

Main question:

Which models admit an identifiable scaling reparametrization?

Main result:

The model has an identifiable scaling reparametrization by monomial functions of the original parameters All the monomial cycles in G are identifiable functions dim im c = m+1 Theorem (M-Sullivant): The following are equivalent: The model has an identifiable scaling reparametrization

Non-Example: Model 1

Model: dim im c = 4, so no ID scaling reparametrization! 1 2 3

Example: Model 2

Model: Identifiable cycles: 1 2 3

Algorithm to find identifiable reparametrization

1) Form a spanning tree T 2) Form the directed incidence matrix E(T): 3) Let E be E(T) with first row removed 4) Columns of E-1 are exponent vectors of monomials 𝑔

𝑗(𝐵) in scaling 𝑌𝑗 = 𝑔 𝑗 𝐵 𝑦𝑗

Identifiable reparametrization

• Spanning tree
• Rescaling:
• Identifiable scaling reparametrization

1 2 3

Which graphs have this property?

• Inductively strongly connected graphs when

m=2n-2 Good:

• Not complete characterization:

1 2 3 4 1 2 3 4 1 2 3 4 Bad:

Problem: Finding identifiable functions

• What if we have a different class of models?

– How to find simplest identifiable functions?

• Problem: Given a field extension

ℝ(𝑑1, … , 𝑑𝑛)/ℝ find a “nice” transcendence basis

• In other words:

– Let 𝑒 = number of algebraically independent functions among 𝑑1, … , 𝑑𝑛 – Find algebraically independent functions 𝑔

1, … , 𝑔 𝑒 that

are algebraic over ℝ(𝑑1, … , 𝑑𝑛).

Gröbner Basis Heuristic to Find Identifiable Functions

• Consider the ideal

𝐾 = 𝑑1 𝑞 − 𝑑1 𝑞∗ , … , 𝑑𝑛 𝑞 − 𝑑𝑛 𝑞∗ ∈ ℝ(𝑞∗)[𝑞]

• Proposition: If 𝑔 𝑞 − 𝑔 𝑞∗ ∈ 𝐾, then 𝑔 is

generically identifiable from 𝑑

• Try to find sparse polynomials like this in 𝐾 by

using elimination orderings

M, Eisenberg, and DiStefano 2009

Gröbner Basis Heuristic to Find Identifiable Functions

• Example: 𝑑 𝑞 − 𝑑(𝑞∗) =

(−𝑏11 − 𝑏22 − (−𝑏11

∗ − 𝑏22 ∗ ),

𝑏11𝑏22 − 𝑏12𝑏21 − (𝑏11

∗ 𝑏22 ∗ − 𝑏12 ∗ 𝑏21 ∗ ),

−𝑏22 − (−𝑏22

∗ ))

Gröbner basis for ordering (𝑏11, 𝑏12, 𝑏21, 𝑏22) is (𝒃𝟐𝟐 − 𝑏11

∗ , 𝒃𝟐𝟑𝒃𝟑𝟐 − 𝑏12 ∗ 𝑏21 ∗ , 𝒃𝟑𝟑 − 𝑏22 ∗ )

So 𝑏11, 𝑏22, 𝑏12𝑏21 are identifiable functions

Gröbner Basis Heuristic to Find Identifiable Functions

• Model 2:
• Gröbner basis of 𝑑 𝑞 − 𝑑(𝑞∗) for ordering

(𝑏23, 𝑏31, 𝑏12, 𝑏21, 𝑏33, 𝑏22, 𝑏11) is

(𝒃𝟐𝟐 − 𝑏11

∗ ,

𝒃𝟑𝟑

𝟑 − 𝒃𝟑𝟑𝑏22 ∗ − 𝒃𝟑𝟑𝑏33 ∗ + 𝑏22 ∗ 𝑏33 ∗ ,

𝒃𝟑𝟑 + 𝒃𝟒𝟒 − 𝑏22

∗ − 𝑏33 ∗ ,

𝒃𝟐𝟑𝒃𝟑𝟐 − 𝑏12

∗ 𝑏21 ∗ ,

𝑏12

∗ 𝒃𝟑𝟐𝑏21 ∗ 𝒃𝟑𝟑 − 𝑏12 ∗ 𝒃𝟑𝟐𝑏21 ∗ 𝑏22 ∗ + 𝑏12 ∗ 𝑏21 ∗ 𝒃𝟑𝟒𝒃𝟒𝟐

− 𝑏12

∗ 𝒃𝟑𝟐𝑏23 ∗ 𝑏31 ∗ ,

𝑏12

∗ 𝑏21 ∗ 𝒃𝟑𝟑 − 𝑏12 ∗ 𝑏21 ∗ 𝑏22 ∗ + 𝒃𝟐𝟑𝒃𝟑𝟒𝒃𝟒𝟐 − 𝑏12 ∗ 𝑏23 ∗ 𝑏31 ∗ )

Testing for local identifiability

• Let 𝐾 𝑑 be the Jacobian matrix:

𝐾 = 𝜖𝑑1 𝜖𝑞1 ⋯ 𝜖𝑑1 𝜖𝑞𝑜 ⋮ ⋱ ⋮ 𝜖𝑑𝑛 𝜖𝑞1 ⋯ 𝜖𝑑𝑛 𝜖𝑞𝑜

• rank 𝐾(𝑑) (at a generic point) gives the dimension of

image of 𝑑

• Proposition: Let 𝑜 be the dimension of the parameter
• space. The model is locally identifiable if and only if

rank 𝐾 𝑑 = 𝑜.

M and Sullivant 2014

Testing for local identifiability of functions

• Proposition: A function 𝑔 is locally identifiable from 𝑑 if

and only if 𝛼𝑔 ∈ 𝑠𝑝𝑥𝑡𝑞𝑏𝑜 𝐾(𝑑)

• Example:

𝑑 𝑏11, 𝑏12, 𝑏21, 𝑏22 = (−𝑏11 − 𝑏22, 𝑏11𝑏22 − 𝑏12𝑏21, −𝑏22)

• Consider the function

𝑔 𝑏11, 𝑏12, 𝑏21, 𝑏22 = 𝑏12𝑏21

• Then 𝛼𝑔 = (0 𝑏21 𝑏12 0)

𝐾 𝑑 = −1 𝑏22 −𝑏21 −1 −𝑏12 𝑏11 −1

M and Sullivant 2014

Linear Algebra Heuristic to Find Identifiable Functions

• Proposition: Let 𝑑: ℝ𝑜 → ℝ𝑛 with 𝑑1, … , 𝑑𝑛 homogeneous. Let 𝑤

be a vector in the row span of 𝐾(𝑑) over ℝ(𝑑1, … , 𝑑𝑛). Then 𝑤 ∙ 𝑞 is a locally identifiable function.

• Example:

𝑑 𝑏11, 𝑏12, 𝑏21, 𝑏22 = (−𝑏11 − 𝑏22, 𝑏11𝑏22 − 𝑏12𝑏21, −𝑏22)

• We have:

𝐾 𝑑 = −1 𝑏22 −𝑏21 −1 −𝑏12 𝑏11 −1 → Apply Gaussian Elimination

• ver the field ℝ(𝑑1, … , 𝑑𝑛) →

1 𝑏21 𝑏12 1

• Thus 𝑏11, 2𝑏12𝑏21, 𝑏22 are identifiable functions
• Nonhomogeneous?

M, Rosen, and Sullivant, in press, 2017

Linear Algebra Heuristic to Find Identifiable Functions

• Model 1:

𝑑 𝑏11, 𝑏12, 𝑏22. 𝑏23, 𝑏31, 𝑏32, 𝑏33 = (−𝑏11 − 𝑏22 − 𝑏33, 𝑏11𝑏22 − 𝑏23𝑏32 + 𝑏11𝑏33 + 𝑏22𝑏33, −𝑏12𝑏23𝑏31 + 𝑏11𝑏23𝑏32 − 𝑏11𝑏22𝑏33, −𝑏22 − 𝑏33, 𝑏22𝑏33 − 𝑏23𝑏32)

• With respect to parameter ordering

𝑏11, 𝑏22, 𝑏23, 𝑏32, 𝑏33, 𝑏12, 𝑏31 , we have:

𝐾 𝑑 = −1 −1 𝑏22 + 𝑏33 𝑏11 + 𝑏33 −𝑏32 𝑏23𝑏32 − 𝑏22𝑏33 −𝑏11𝑏33 −1 𝑏33 −𝑏12𝑏31 + 𝑏11𝑏32 −𝑏32 −1 −𝑏23 𝑏11 + 𝑏22 𝑏11𝑏23 −𝑏23 −𝑏11𝑏22 −1 𝑏22 −𝑏23𝑏31 −𝑏12𝑏23

Linear Algebra Heuristic to Find Identifiable Functions

• Apply Gaussian Elimination over ℝ(𝑑1, … , 𝑑𝑛) →

1 −1 𝑏33 −𝑏32 −𝑏12𝑏31 −1 −𝑏23 𝑏22 −𝑏23𝑏31 −𝑏12𝑏23

• Thus 𝑏11,

𝑏22 + 𝑏33, 𝑏22𝑏33 − 𝑏23𝑏32, 𝑏12𝑏23𝑏31 are identifiable functions

Linear Algebra Heuristic to Find Identifiable Functions

• With respect to different parameter ordering

𝑏11, 𝑏31, 𝑏12, 𝑏32, 𝑏23, 𝑏22, 𝑏33 , we have:

𝐾 𝑑 = −1 𝑏22 + 𝑏33 𝑏23𝑏32 − 𝑏22𝑏33 −𝑏12𝑏23 −𝑏23𝑏31 −1 −𝑏23 −𝑏32 𝑏11 + 𝑏33 𝑏11𝑏23 −𝑏23 −𝑏12𝑏31 + 𝑏11𝑏32 −𝑏32 −𝑏11𝑏33 −1 𝑏33 −1 𝑏11 + 𝑏22 −𝑏11𝑏22 −1 𝑏22

Different column ordering

Linear Algebra Heuristic to Find Identifiable Functions

• Apply Gaussian Elimination over ℝ(𝑑1, … , 𝑑𝑛) →

1 −𝑏12𝑏23 −𝑏23𝑏31 −𝑏12𝑏31 −𝑏23 −𝑏32 𝑏11 − 𝑏22 −1 𝑏11 − 𝑏33 −1

• So 𝑏11,

𝑏12𝑏23𝑏31, − 𝑏22

2

2 − 𝑏33

2

2 − 𝑏23𝑏32 + 𝑏11𝑏22 2 + 𝑏11𝑏33 2 , 𝑏22 + 𝑏33 are identifiable functions

Linear Algebra Heuristic to Find Identifiable Functions

• Apply Gaussian Elimination over ℝ(𝑑1, … , 𝑑𝑛) →

1 −𝑏12𝑏23 −𝑏23𝑏31 −𝑏12𝑏31 −𝑏23 −𝑏32 𝑏11 − 𝑏22 −1 𝑏11 − 𝑏33 −1

• So 𝑏11,

𝑏12𝑏23𝑏31, − 𝑏22

2

2 − 𝑏33

2

2 − 𝑏23𝑏32 + 𝑏11𝑏22 2 + 𝑏11𝑏33 2 , 𝑏22 + 𝑏33 are identifiable functions

References

• L. Ljung and T. Glad, On global identifiability for arbitrary model parametrizations,

Automatica 30(2) (1994): 265-276.

• N. Meshkat, M. Eisenberg, and J. J. DiStefano III, An algorithm for finding globally

identifiable parameter combinations of nonlinear ODE models using Gröbner Bases, Math. Biosci. 222 (2009) 61-72.

• N. Meshkat, Z. Rosen, and S. Sullivant, Algebraic Tools for the Analysis of State

Space Models, to appear in Proceedings of the 8th MSJ SI (2017)

• N. Meshkat and S. Sullivant, Identifiable reparametrizations of linear

compartment models, Journal of Symbolic Computation 63 (2014) 46-67.

• T. Soderstrom and P. Stoica, System Identification, Prentice-Hall, Upper Saddle

River, NJ, 1988.

Thank you for your attention!