Controlling Systemic Inflammation Using Nonlinear Model Predictive - - PowerPoint PPT Presentation

Controlling Systemic Inflammation Using Nonlinear Model Predictive Control with State Estimation

Gregory Zitelli, Judy Day

University of Tennessee, Knoxville

July 2013 With generous support from the NSF, Award 1122462

Controlling Systemic Inflammation Using NMPC University of Tennessee, Knoxville

Motivation

◮ We’re going to be interested in the acute inflammatory

response to biological stress in the form of a bacterial pathogen.

◮ The inflammatory response aims to remove the presence of

the pathogen.

◮ However, an excessive response may lead to collateral tissue

damage, organ failure, or worse.

Controlling Systemic Inflammation Using NMPC University of Tennessee, Knoxville

Motivation

◮ To recover from a high-inflammation state, it is necessary for

the inflammatory mechanisms to downregulate itself through the use of some anti-inflammatory mediator.

◮ We will consider a mathematical formulation of these ideas in

terms of a highly coupled, nonlinear ODE model, as well as control applied to both the pro and anti-inflammatory influences within the system.

Controlling Systemic Inflammation Using NMPC University of Tennessee, Knoxville

Interaction Diagram

Controlling Systemic Inflammation Using NMPC University of Tennessee, Knoxville

Nonlinear Model

◮ P represents the bacterial pathogen population. ◮ N∗ represents the concentration of pro-inflammatory

mediators, such as activated phagocytes and their produced cytokines.

◮ D acts as a marker of tissue damage. ◮ CA represents the concentration of anti-inflammatory

mediators.

Controlling Systemic Inflammation Using NMPC University of Tennessee, Knoxville

Nonlinear Model

dP dt = kpgP

• 1 − P

P∞

• −

kpmsmP µm + kmpP − kpnf(N∗)P dN∗ dt = snrR(P, N ∗, D) µnr + R(P, N ∗, D) − µnN∗ dD dt = kdn f(N∗)6 x6

dn + f(N∗)6 − µdD

dCA dt = sc + kcn f(N∗ + kcndD) 1 + f(N∗ + kcndD) − µcCA

Controlling Systemic Inflammation Using NMPC University of Tennessee, Knoxville

Nonlinear Model

The system (P, N ∗, D, CA) has three fixed points corresponding to the three biologically relevant scenarios:

◮ P = N = D = 0 and CA = sc µc , where the patient is healthy. ◮ All states elevated, where the patient is septic. ◮ P = 0, and N∗, D, CA > 0, where the patient is aseptic.

For values of kpg (the pathogen growth rate) in the interval (0.5137, 1.755), all three states are stable. (Reynolds et al 2006)

Controlling Systemic Inflammation Using NMPC University of Tennessee, Knoxville

???

So what does this model actually look like?

Controlling Systemic Inflammation Using NMPC University of Tennessee, Knoxville

Septic Simulation

50 100 150 200 2 4 6 8 10 P (state (−)) Time 50 100 150 200 0.5 1 1.5 Time N* (state (−)) 50 100 150 200 5 10 15 20 D (state (−)) Time 50 100 150 200 0.1 0.2 0.3 0.4 0.5 0.6 Ca (state (−)) Time

Controlling Systemic Inflammation Using NMPC University of Tennessee, Knoxville

Aseptic Simulation

50 100 150 200 1 2 3 4 5 6 P (state (−),est(−−)) Time 50 100 150 200 0.2 0.4 0.6 0.8 1 Time N* (state (−),est(−−)) 50 100 150 200 5 10 15 20 D (state (−),est(−−)) Time 50 100 150 200 0.1 0.2 0.3 0.4 0.5 Ca (state (−),est(−−)) Time

Controlling Systemic Inflammation Using NMPC University of Tennessee, Knoxville

Healthy Simulation

50 100 150 200 0.2 0.4 0.6 0.8 P (state (−)) Time 50 100 150 200 0.05 0.1 0.15 0.2 Time N* (state (−)) 50 100 150 200 0.5 1 1.5 D (state (−)) Time 50 100 150 200 0.1 0.2 0.3 0.4 0.5 0.6 Ca (state (−)) Time

Controlling Systemic Inflammation Using NMPC University of Tennessee, Knoxville

Goals

◮ We would like to apply some kind of control to this model, to

direct septic and aseptic patients towards the healthy fixed state.

◮ This is accomplished using Nonlinear Model Predictive

Control, or NMPC.

◮ The NMPC is applied to the pro and anti-inflammatory

mediators, N∗ and CA.

◮ We only apply positive control, and there are restrictions on

how much can be introduced within certain time frames.

◮ We assume that the level of pro and anti-inflammatory

mediators, N∗ and CA, are measurable with Gaussian noise.

Controlling Systemic Inflammation Using NMPC University of Tennessee, Knoxville

Overview of NMPC

Controlling Systemic Inflammation Using NMPC University of Tennessee, Knoxville

Overview of NMPC

◮ Our reference trajectory is zero levels of P and D, and

minimal amounts of control. J = min

AI(t),PI(t) ΓDD2 2 + ΓP P2 2 + ΓAIAI2 2 + ΓPIPI2 2

Controlling Systemic Inflammation Using NMPC University of Tennessee, Knoxville

Note on Observations

◮ We take noisy measurements of N∗ and CA at each time

interval which update our predictive model.

◮ The levels of P and D are not measured directly, and are

instead estimated from the predictive model.

◮ Being unable to measure P often leads the NMPC to be too

• aggressive. Many virtual patients were unnecessarily harmed

under this scheme.

◮ Since there are biologically relevant scenarios for loose

measurements of P (indicators like body temperature or blood pressure give an idea whether or not the infection persists), a pathogen update is done every four timesteps.

Controlling Systemic Inflammation Using NMPC University of Tennessee, Knoxville

Pathogen Update

◮ If the predictive model reads low (< 0.05) but the pathogen

levels are high (P > 0.5) then the level in the predictive model is reset to 0.5.

◮ If the predictive model reads high (> 0.5) but the pathogen

levels are low (P < 0.05) then the level in the predictive model is reset to 0.

Controlling Systemic Inflammation Using NMPC University of Tennessee, Knoxville

Virtual Patient Pool

◮ 1,000 patients are randomly generated with different

parameters, including unique values of the pathogen growth rate kpg.

◮ 620 acquire elevated P values to suggest treatment. ◮ Of the 620, 251 (40%) will return to a healthy state on their

• wn.

Controlling Systemic Inflammation Using NMPC University of Tennessee, Knoxville

Initial Results (Patient Snapshot)

50 100 150 200 0.5 1 1.5 P (state (−),est(−−)) Time 50 100 150 200 0.2 0.4 0.6 0.8 Time N* (state (−),est(−−)) 50 100 150 200 1 2 3 4 D (state (−),est(−−)) Time 50 100 150 200 0.2 0.4 0.6 0.8 Ca (state (−),est(−−)) Time

Controlling Systemic Inflammation Using NMPC University of Tennessee, Knoxville

Initial Results (Day et al 2010)

Therapy Type: Placebo Mismatch kpg = 0.52 Mismatch kpg = 0.6 Mismatch kpg = 0.8 Percentage Healthy: 40% (251) 60% (369) 82% (510) 83% (513) Percentage Aseptic: 37% (228) 19% (120) 8% (49) 17% (107) Percentage Septic: 23% (141) 21% (131) 10% (61) 0% (0) Percentage Harmed: na 0% (0/251) 1% (2/251) 6% (16/251) Percentage Rescued: na 32% (118/369) 71% (261/369) 75% (278/369)

Controlling Systemic Inflammation Using NMPC University of Tennessee, Knoxville

Particle Filter

◮ One way to go about improving these results is to impliment

robust state estimation for the unobserved variables P and D.

◮ This is accomplished using a particle filter, which tracks its

accuracy by comparing its predictions of N∗ and CA to the

• bserved values.

Controlling Systemic Inflammation Using NMPC University of Tennessee, Knoxville

Particle Filter

◮ The particle filter is initialized with 1,000 particles randomized

near our measurements for N and CA and initial predictions for P and D.

◮ Each particle pi = (Pi, N∗ i , CA,i, Di) is simulated for one time

step.

◮ At the next time step, the particles pi are assigned weights qi

depending on how close N∗

i , CA,i are to the new

measurements for N∗ and CA.

Controlling Systemic Inflammation Using NMPC University of Tennessee, Knoxville

Particle Filter

◮ At the end of the time step, each of the 1,000 slots holding a

particle is randomly assigned a new particle pi according to their weights qi.

◮ This causes bad particles with low weights qi to die off, while

good particle with high weights replicate.

Controlling Systemic Inflammation Using NMPC University of Tennessee, Knoxville

Particle Filter Results

Therapy Type: Original kpg = 0.52 Original kpg = 0.6 Original kpg = 0.8 Percentage Healthy: 60% (369) 80% (497) 84% (519) Percentage Aseptic: 19% (121) 10% (60) 16% (101) Percentage Septic: 21% (130) 10% (63) 0% (0) Therapy Type: Particles kpg = 0.52 Particles kpg = 0.6 Particles kpg = 0.8 Percentage Healthy: 60% (369) 80% (493) 82% (511) Percentage Aseptic: 19% (120) 10% (66) 18% (109) Percentage Septic: 21% (131) 10% (61) 0% (0)

Controlling Systemic Inflammation Using NMPC University of Tennessee, Knoxville

Particle Filter Results

◮ The particle filter does slightly worse than our original

mismatched predictive model.

◮ On the other hand, the particle filter is self correcting, so it

needs no pathogen update to correct a misled pathogen prediction.

Controlling Systemic Inflammation Using NMPC University of Tennessee, Knoxville

Controllability

Consider the general nonlinear system with affine control ˙ y = F(y) +

m

• i=1

gi(x)ui w = h(y) where f : Rn → Rn and h : Rn → Rk. We typically take u : [0, ∞) → Rm locally integrable.

Controlling Systemic Inflammation Using NMPC University of Tennessee, Knoxville

Controllability Rank Condition

˙ y = F(y) +

m

• i=1

gi(x)ui w = h(y)

Definition (Controllability Rank Condition)

Given the same nonlinear system with affine control, we say that the system satisfies the controllability rank condition if a finite sub-matrix of the following matrix has rank n

• g1

. . . gm [F, g1] . . . [F, [F, g1]] . . .

• Controlling Systemic Inflammation Using NMPC

University of Tennessee, Knoxville

Controllability

◮ If the system is linear,

Controllability Rank Condition = ⇒ Locally Controllable

◮ If the system is nonlinear,

Controllability Rank Condition = ⇒ Locally Accessible

◮ If the system is nonlinear and has symmetric control (for each

control u there is a u′ such that F(y, u) = −F(y, u′) for every y ∈ Rn), then Controllability Rank Condition = ⇒ Locally Controllable

Controlling Systemic Inflammation Using NMPC University of Tennessee, Knoxville

Controllability

◮ It can be shown that our system does satisfy the

controllability rank condition in the first octant.

◮ This means that the system is locally accessible there. ◮ Unfortunately, our control does not satisfy the symmetry

property, so this does not imply that the system is locally controllable.

Controlling Systemic Inflammation Using NMPC University of Tennessee, Knoxville

What’s Next?

◮ Djouadi and Bara are working on the use of adaptive control

for this model, to help overcome the mismatch between the predictive model and patient model parameters like kpg.

◮ Although the pathogen update was necessary for the variable

P, it appears that the damage variable D is well estimated by

• ur predictive model under many different initial conditions

and model parameters. It would be nice to make this notion precise.

◮ Are there elements of this model that can be exploited to

increase the effectiveness of our filter?

Controlling Systemic Inflammation Using NMPC University of Tennessee, Knoxville

The End?

Thank you very much!

Controlling Systemic Inflammation Using NMPC University of Tennessee, Knoxville

Model

dP dt = kpgP

• 1 − P

P∞

• −

kpmsmP µm + kmpP − kpnf(N∗)P dN∗ dt = snrR(P, N ∗, D) µnr + R(P, N ∗, D) − µnN∗ dD dt = kdn f(N∗)6 x6

dn + f(N∗)6 − µdD

dCA dt = sc + kcn f(N∗ + kcndD) 1 + f(N∗ + kcndD) − µcCA

Controlling Systemic Inflammation Using NMPC University of Tennessee, Knoxville

Model

Where R(P, N ∗, D) = f(knpP + knnN∗ + kndD) f(x) = x 1 +

• CA

c∞

2 c∞ is chosen so that when CA is at its highest, f(x) ≈ 1

4x.

Controlling Systemic Inflammation Using NMPC University of Tennessee, Knoxville

Pathogen

◮ The bacterial pathogen follows logistic growth. ◮ Healthy individuals have a baseline, non-specific, local

immune response. We assume through the initial analysis of this interaction that the local response reaches a quasi-steady state value of

kpm µm+kmpP . ◮ The pathogen are directly attacked by the phagocytic immune

cells N∗, which may be inhibited by the presence of the anti-inflammatory CA. dP dt = kpgP

• 1 − P

P∞

• −

kpmsmP µm + kmpP − kpnf(N∗)P

Controlling Systemic Inflammation Using NMPC University of Tennessee, Knoxville

Pro-Inflammatory Mediator

◮ The first term comes from the assumed quasi-steady state of

a subsystem involving resting and activated phagocytes.

◮ The second term represents the gradual decay in the

concentration of pro-inflammatory mediators. dN∗ dt = snrR(P, N ∗, D) µnr + R(P, N ∗, D) − µnN∗

Controlling Systemic Inflammation Using NMPC University of Tennessee, Knoxville

Damage

◮ Collateral tissue damage from activated phagocytes motivated

the first term. The damage saturates, with the large Hill coefficient necessary to produce a more realistic basin of attraction for the healthy fixed state.

◮ The second term represents tissue regeneration.

dD dt = kdn f(N∗)6 x6

dn + f(N∗)6 − µdD

Controlling Systemic Inflammation Using NMPC University of Tennessee, Knoxville

Anti-Inflammatory Mediators

◮ The initial biological source term sc is augmented by the

Michaelis-Menten term.

◮ The second term represents the gradual decay in the

concentration of anti-inflammatory mediators. dCA dt = sc + kcn f(N∗ + kcndD) 1 + f(N∗ + kcndD) − µcCA

Controlling Systemic Inflammation Using NMPC University of Tennessee, Knoxville

Essential Components of NMPC

(I) Reference Trajectory Desired trajectory we want the system to approach. Our reference trajectory will be P = D = 0. (II) Prediction of Process Output A predictive model is used to provide an estimation of the states which are unobserved. (III) Objective Function An objective function will describe how are current predictions deviate from the reference trajectory. Our objective function is a weighted sum of squares for the undesireable variables P and D, as well as for the control levels. J = min

AI(t),PI(t) ΓDD2 2 + ΓP P2 2 + ΓAIAI2 2 + ΓPIPI2 2

Controlling Systemic Inflammation Using NMPC University of Tennessee, Knoxville

Essential Components of NMPC

(IV) Sequence of Control Moves at Each Time Step A discrete control is designed for k future steps to minimize the objective function over each time interval. (V) Error Prediction Update The measured values of N and CA from the patient model are compared to the predictive model after the control has been applied to both. It is at this step that implementations of the particle filter can be refined through this comparison.

Controlling Systemic Inflammation Using NMPC University of Tennessee, Knoxville

Controllability

Consider the general nonlinear system with affine control ˙ y = F(y) +

m

• i=1

gi(x)ui w = h(y) where f : Rn → Rn and h : Rn → Rk. We typically take u : [0, ∞) → Rm locally integrable. For us, F(y) =        kpgy1

• 1 − y1

P∞

• −

kpmsmy1 µm+kmpy1 − kpnf(y2)y1 snrR(y1,y2,y3) µnr+R(y1,y2,y3) − µny2

kdn

f(y2)6 x6

dn+f(y2)6 − µdy3

sc + kcn

f(y2+kcndy3) 1+f(y2+kcndy3) − µcy4

      

Controlling Systemic Inflammation Using NMPC University of Tennessee, Knoxville

Controllability

Definition

We say that a point a ∈ Rn is attainable from y if there exists an appropriate control uy (locally integrable) such that the trajectory given by ˙ y = F(y) +

m

• i=1

gi(x)uy

i

y(0) = y is such that y(T) = a for some finite trajectory [0, T].

Controlling Systemic Inflammation Using NMPC University of Tennessee, Knoxville

Controllability

Definition

If y ∈ Rn and O is an open set containing y, we let AO( y, T) denote the set of all points in O which are attainable from y in time T, such that y(t) ∈ O for all t ∈ [0, T].

Definition

We say that the nonlinear system is locally accessible at a point

• y ∈ Rn if the sets AO(

y, T) has nonempty interior for every open set O = ∅ and every T > 0.

Controlling Systemic Inflammation Using NMPC University of Tennessee, Knoxville

Controllability

Definition

Given the same nonlinear system with affine control, we say that the system satisfies the controllability rank condition if a finite sub-matrix of the following matrix has rank n

• g1

. . . gm [F, g1] . . . [F, [F, g1]] . . .

• Controlling Systemic Inflammation Using NMPC

University of Tennessee, Knoxville

Controllability

If our system was linear, then it turns out that the controllability rank condition implies that the system is controllable. In the nonlinear case, we have the following results.

Theorem

Given the same nonlinear system with affine control, if the system satisfies the controllability rank condition at a point y ∈ Rn then it is locally accessible there.

Theorem

If for a nonlinear system ˙ y = F(y, u) we have that for any locally integrable control u there exists another locally integrable control u′ such that F(y, u) = −F(y, u′) for all y ∈ Rn and the system satisfies the controllability rank condition at a point y ∈ Rn, then the system is locally controllable at y.

Controlling Systemic Inflammation Using NMPC University of Tennessee, Knoxville

Controllability

Recall our system is in the following form ˙ y =        kpgy1

• 1 − y1

P∞

• −

kpmsmy1 µm+kmpy1 − kpnf(y2)y1 snrR(y1,y2,y3) µnr+R(y1,y2,y3) − µny2 + u1

kdn

f(y2)6 x6

dn+f(y2)6 − µdy3

sc + kcn

f(y2+kcndy3) 1+f(y2+kcndy3) − µcy4 + u2

       +     u1     +     u2    

Controlling Systemic Inflammation Using NMPC University of Tennessee, Knoxville