System Modeling, part 2 Marc Claesen February 18, 2015 Marc - - PowerPoint PPT Presentation

Nonlinear systems & linearization System identification (cont)

System Modeling, part 2

Marc Claesen February 18, 2015

Marc Claesen System Modeling, part 2

Nonlinear systems & linearization System identification (cont)

1

Nonlinear systems & linearization

2

System identification (cont) Grey box identification Black box identification

Marc Claesen System Modeling, part 2

Nonlinear systems & linearization System identification (cont)

Outline

1

Nonlinear systems & linearization

2

System identification (cont) Grey box identification Black box identification

Marc Claesen System Modeling, part 2

Nonlinear systems & linearization System identification (cont)

Nonlinear systems

In this course we focus on the linear state-space representation: ˙ x(t) = Ax(t) + Bu(t), y(t) = Cx(t) + Du(t). x[k + 1] = Ax[k] + Bu[k], y[k] = Cx[k] + Du[k].

Marc Claesen System Modeling, part 2

Nonlinear systems & linearization System identification (cont)

Nonlinear systems

In this course we focus on the linear state-space representation: ˙ x(t) = Ax(t) + Bu(t), y(t) = Cx(t) + Du(t). x[k + 1] = Ax[k] + Bu[k], y[k] = Cx[k] + Du[k]. Most real life systems involve nonlinearity: ˙ x(t) = f

• x(t), u(t)
• ,

y(t) = g

• x(t), u(t)
• ,

Marc Claesen System Modeling, part 2

Nonlinear systems & linearization System identification (cont)

Nonlinear systems

In this course we focus on the linear state-space representation: ˙ x(t) = Ax(t) + Bu(t), y(t) = Cx(t) + Du(t). x[k + 1] = Ax[k] + Bu[k], y[k] = Cx[k] + Du[k]. Most real life systems involve nonlinearity: ˙ x(t) = f

• x(t), u(t)
• ,

y(t) = g

• x(t), u(t)
• ,

where f and/or g contain some nonlinearity, such as:

Marc Claesen System Modeling, part 2

Nonlinear systems & linearization System identification (cont)

Nonlinear systems

In this course we focus on the linear state-space representation: ˙ x(t) = Ax(t) + Bu(t), y(t) = Cx(t) + Du(t). x[k + 1] = Ax[k] + Bu[k], y[k] = Cx[k] + Du[k]. Most real life systems involve nonlinearity: ˙ x(t) = f

• x(t), u(t)
• ,

y(t) = g

• x(t), u(t)
• ,

where f and/or g contain some nonlinearity, such as: powers: e.g. ˙ x(t) = Ax(t) + Bu(t)+γu(t)2,

Marc Claesen System Modeling, part 2

Nonlinear systems & linearization System identification (cont)

Nonlinear systems

In this course we focus on the linear state-space representation: ˙ x(t) = Ax(t) + Bu(t), y(t) = Cx(t) + Du(t). x[k + 1] = Ax[k] + Bu[k], y[k] = Cx[k] + Du[k]. Most real life systems involve nonlinearity: ˙ x(t) = f

• x(t), u(t)
• ,

y(t) = g

• x(t), u(t)
• ,

where f and/or g contain some nonlinearity, such as: powers: e.g. ˙ x(t) = Ax(t) + Bu(t)+γu(t)2, interactions: e.g. ˙ x(t) = Ax(t) + Bu(t)+γx(t)u(t),

Marc Claesen System Modeling, part 2

Nonlinear systems & linearization System identification (cont)

Nonlinear systems

In this course we focus on the linear state-space representation: ˙ x(t) = Ax(t) + Bu(t), y(t) = Cx(t) + Du(t). x[k + 1] = Ax[k] + Bu[k], y[k] = Cx[k] + Du[k]. Most real life systems involve nonlinearity: ˙ x(t) = f

• x(t), u(t)
• ,

y(t) = g

• x(t), u(t)
• ,

where f and/or g contain some nonlinearity, such as: powers: e.g. ˙ x(t) = Ax(t) + Bu(t)+γu(t)2, interactions: e.g. ˙ x(t) = Ax(t) + Bu(t)+γx(t)u(t), clipping: e.g. α ≤x(t)≤ β.

Marc Claesen System Modeling, part 2

Nonlinear systems & linearization System identification (cont)

Linearization around equilibrium point

Marc Claesen System Modeling, part 2

Nonlinear systems & linearization System identification (cont)

Linearization around equilibrium point

Nonlinear systems have (several) equilibrium points xe, ue, ye: ˙ xe = f

• xe, ue
• = 0,

ye = g

• xe, ue
• .

Marc Claesen System Modeling, part 2

Nonlinear systems & linearization System identification (cont)

Linearization around equilibrium point

Nonlinear systems have (several) equilibrium points xe, ue, ye: ˙ xe = f

• xe, ue
• = 0,

ye = g

• xe, ue
• .

Linearizing in the region of (xe, ue, ye): x = xe + ∆x, u = ue + ∆u, y = ye + ∆y,

Marc Claesen System Modeling, part 2

Nonlinear systems & linearization System identification (cont)

Linearization around equilibrium point

Nonlinear systems have (several) equilibrium points xe, ue, ye: ˙ xe = f

• xe, ue
• = 0,

ye = g

• xe, ue
• .

Linearizing in the region of (xe, ue, ye): x = xe + ∆x, u = ue + ∆u, y = ye + ∆y, with ∆x, ∆u and ∆y sufficiently small.

Marc Claesen System Modeling, part 2

Nonlinear systems & linearization System identification (cont)

Linearization around equilibrium point

Nonlinear systems have (several) equilibrium points xe, ue, ye: ˙ xe = f

• xe, ue
• = 0,

ye = g

• xe, ue
• .

Linearizing in the region of (xe, ue, ye): x = xe + ∆x, u = ue + ∆u, y = ye + ∆y, with ∆x, ∆u and ∆y sufficiently small. Linearizing is done via first order Taylor expansions: dx

dt = d∆x dt

= f (x, u) = f (xe + ∆x, ue + ∆u), ye + ∆y = g(x, u) = g(xe + ∆x, ue + ∆u).

Marc Claesen System Modeling, part 2

Nonlinear systems & linearization System identification (cont)

Example: decalcification plant

Used to reduce concentration of calcium hydroxide in water: chemical reaction: Ca(OH)2 + CO2 → CaCO3 + H2O reaction speed: r = c[Ca(OH)2][CO2] rate of change of concentration: d[Ca(OH)2] dt = k V − r V , d[CO2] dt = u V − r V , with inflow rates k and u in mol/s and tank volume V in L. input u: inflow of CO2, output: [Ca(OH)2]

Marc Claesen System Modeling, part 2

Nonlinear systems & linearization System identification (cont)

Nonlinear model and equilibrium point

Nonlinear model for the given reactor:

Marc Claesen System Modeling, part 2

Nonlinear systems & linearization System identification (cont)

Nonlinear model and equilibrium point

Nonlinear model for the given reactor: d[Ca(OH)2] dt = k V − c V [Ca(OH)2][CO2], d[CO2] dt = u V − c V [Ca(OH)2][CO2], y = [Ca(OH)2], with two state variables: x1 = [Ca(OH)2] and x2 = [CO2].

Marc Claesen System Modeling, part 2

Nonlinear systems & linearization System identification (cont)

Nonlinear model and equilibrium point

Nonlinear model for the given reactor: d[Ca(OH)2] dt = k V − c V [Ca(OH)2][CO2], d[CO2] dt = u V − c V [Ca(OH)2][CO2], y = [Ca(OH)2], with two state variables: x1 = [Ca(OH)2] and x2 = [CO2]. The equilibrium point (keq, ueq, x1,eq, x2,eq, yeq) of this system is:

Marc Claesen System Modeling, part 2

Nonlinear systems & linearization System identification (cont)

Nonlinear model and equilibrium point

Nonlinear model for the given reactor: d[Ca(OH)2] dt = k V − c V [Ca(OH)2][CO2], d[CO2] dt = u V − c V [Ca(OH)2][CO2], y = [Ca(OH)2], with two state variables: x1 = [Ca(OH)2] and x2 = [CO2]. The equilibrium point (keq, ueq, x1,eq, x2,eq, yeq) of this system is: keq V − c V [Ca(OH)2]eq[CO2]eq = 0, ueq V − c V [Ca(OH)2]eq[CO2]eq = 0.

Marc Claesen System Modeling, part 2

Nonlinear systems & linearization System identification (cont)

Linearization of the decalcification plant

For small deviations near the equilibrium: d∆x1 dt = − c V [CO2]eq∆x1 − c V [Ca(OH)2]eq∆x2, d∆x2 dt = − c V [CO2]eq∆x1 − c V [Ca(OH)2]eq∆x2 + ∆u V , ∆y = ∆x1.

Marc Claesen System Modeling, part 2

Nonlinear systems & linearization System identification (cont)

Linearization of the decalcification plant

For small deviations near the equilibrium: d∆x1 dt = − c V [CO2]eq∆x1 − c V [Ca(OH)2]eq∆x2, d∆x2 dt = − c V [CO2]eq∆x1 − c V [Ca(OH)2]eq∆x2 + ∆u V , ∆y = ∆x1. The resulting linear state-space model is ˙ x(t) = Ax(t) + Bu(t) y(t) = Cx(t) + Du(t):

Marc Claesen System Modeling, part 2

Nonlinear systems & linearization System identification (cont)

Linearization of the decalcification plant

For small deviations near the equilibrium: d∆x1 dt = − c V [CO2]eq∆x1 − c V [Ca(OH)2]eq∆x2, d∆x2 dt = − c V [CO2]eq∆x1 − c V [Ca(OH)2]eq∆x2 + ∆u V , ∆y = ∆x1. The resulting linear state-space model is ˙ x(t) = Ax(t) + Bu(t) y(t) = Cx(t) + Du(t):

• d[Ca(OH)2]

dt d[CO2] dt

• = −

c

V [CO2]eq c V [Ca(OH)2]eq c V [CO2]eq c V [Ca(OH)2]eq

[Ca(OH)2] [CO2]

• +

1 V

• u(t)

y(t) = [Ca(OH)2]

Marc Claesen System Modeling, part 2

Nonlinear systems & linearization System identification (cont) Grey box identification Black box identification

Outline

1

Nonlinear systems & linearization

2

System identification (cont) Grey box identification Black box identification

Marc Claesen System Modeling, part 2

Nonlinear systems & linearization System identification (cont) Grey box identification Black box identification

Main classes of identification methods

White box modeling: based on first principles.

Marc Claesen System Modeling, part 2

Nonlinear systems & linearization System identification (cont) Grey box identification Black box identification

Main classes of identification methods

White box modeling: based on first principles. → known equations (structure) & parameters (coefficients).

Marc Claesen System Modeling, part 2

Nonlinear systems & linearization System identification (cont) Grey box identification Black box identification

Main classes of identification methods

White box modeling: based on first principles. → known equations (structure) & parameters (coefficients). Grey box identification: first principles & experimentation.

Marc Claesen System Modeling, part 2

Nonlinear systems & linearization System identification (cont) Grey box identification Black box identification

Main classes of identification methods

White box modeling: based on first principles. → known equations (structure) & parameters (coefficients). Grey box identification: first principles & experimentation. → known equations, unknown/uncertain parameters.

Marc Claesen System Modeling, part 2

Nonlinear systems & linearization System identification (cont) Grey box identification Black box identification

Main classes of identification methods

White box modeling: based on first principles. → known equations (structure) & parameters (coefficients). Grey box identification: first principles & experimentation. → known equations, unknown/uncertain parameters. Black box identification: based on experimentation.

Marc Claesen System Modeling, part 2

Nonlinear systems & linearization System identification (cont) Grey box identification Black box identification

Main classes of identification methods

White box modeling: based on first principles. → known equations (structure) & parameters (coefficients). Grey box identification: first principles & experimentation. → known equations, unknown/uncertain parameters. Black box identification: based on experimentation. → unknown equations & unknown parameters.

Marc Claesen System Modeling, part 2

Nonlinear systems & linearization System identification (cont) Grey box identification Black box identification

Main classes of identification methods

White box modeling: based on first principles. → known equations (structure) & parameters (coefficients). Grey box identification: first principles & experimentation. → known equations, unknown/uncertain parameters. Black box identification: based on experimentation. → unknown equations & unknown parameters. Most popular approaches are forms of black box identification.

Marc Claesen System Modeling, part 2

Nonlinear systems & linearization System identification (cont) Grey box identification Black box identification

Outline

1

Nonlinear systems & linearization

2

System identification (cont) Grey box identification Black box identification

Marc Claesen System Modeling, part 2

Nonlinear systems & linearization System identification (cont) Grey box identification Black box identification

Grey box identification: conceptual

Marc Claesen System Modeling, part 2

Nonlinear systems & linearization System identification (cont) Grey box identification Black box identification

Grey box identification: conceptual

Grey box identification starts from a known model structure but with unknown/uncertain parameters ↔ parametric statistics.

Marc Claesen System Modeling, part 2

Nonlinear systems & linearization System identification (cont) Grey box identification Black box identification

Grey box identification: conceptual

Grey box identification starts from a known model structure but with unknown/uncertain parameters ↔ parametric statistics. We assume linear, continuous time state space representation: ˙ x(t) = Ax(t) + Bu(t), y(t) = Cx(t) + Du(t).

Marc Claesen System Modeling, part 2

Nonlinear systems & linearization System identification (cont) Grey box identification Black box identification

Grey box identification: conceptual

Grey box identification starts from a known model structure but with unknown/uncertain parameters ↔ parametric statistics. We assume linear, continuous time state space representation: ˙ x(t) = Ax(t) + Bu(t), y(t) = Cx(t) + Du(t). Given: states, inputs, outputs and guesstimates of ˜ A, ˜ B, ˜ C & ˜ D.

Marc Claesen System Modeling, part 2

Nonlinear systems & linearization System identification (cont) Grey box identification Black box identification

Grey box identification: conceptual

Grey box identification starts from a known model structure but with unknown/uncertain parameters ↔ parametric statistics. We assume linear, continuous time state space representation: ˙ x(t) = Ax(t) + Bu(t), y(t) = Cx(t) + Du(t). Given: states, inputs, outputs and guesstimates of ˜ A, ˜ B, ˜ C & ˜ D. Task: estimate ˆ A, ˆ B, ˆ C and ˆ D adequately via experiments.

Marc Claesen System Modeling, part 2

Nonlinear systems & linearization System identification (cont) Grey box identification Black box identification

Grey box identification: conceptual

Grey box identification starts from a known model structure but with unknown/uncertain parameters ↔ parametric statistics. We assume linear, continuous time state space representation: ˙ x(t) = Ax(t) + Bu(t), y(t) = Cx(t) + Du(t). Given: states, inputs, outputs and guesstimates of ˜ A, ˜ B, ˜ C & ˜ D. Task: estimate ˆ A, ˆ B, ˆ C and ˆ D adequately via experiments. “All models are wrong, but some are useful.”

• George E. P. Box

Marc Claesen System Modeling, part 2

Nonlinear systems & linearization System identification (cont) Grey box identification Black box identification

Linear regression

Consider input matrix X, output vector y and residuals ǫ: Xθ = y + ǫ. The parameter vector θ must be estimated, given the observations.

Marc Claesen System Modeling, part 2

Nonlinear systems & linearization System identification (cont) Grey box identification Black box identification

Linear regression

Consider input matrix X, output vector y and residuals ǫ: Xθ = y + ǫ. The parameter vector θ must be estimated, given the observations. A common estimation approach is ordinary least squares (OLS): (XTX)ˆ θOLS = XTy, ˆ θOLS = (XTX)−1XTy.

Marc Claesen System Modeling, part 2

Nonlinear systems & linearization System identification (cont) Grey box identification Black box identification

Linear regression

Consider input matrix X, output vector y and residuals ǫ: Xθ = y + ǫ. The parameter vector θ must be estimated, given the observations. A common estimation approach is ordinary least squares (OLS): (XTX)ˆ θOLS = XTy, ˆ θOLS = (XTX)−1XTy. The OLS estimate minimizes the sum-of-squares of errors, i.e.: ˆ θOLS = arg min

θ N

• i=1
• y(i) −

d

• j=1

X(i, j)θ(j) 2

Marc Claesen System Modeling, part 2

Nonlinear systems & linearization System identification (cont) Grey box identification Black box identification

Linear regression with ordinary least squares

Image taken from http://freakonometrics.hypotheses.org/2348. Marc Claesen System Modeling, part 2

Nonlinear systems & linearization System identification (cont) Grey box identification Black box identification

Maximum likelihood estimation

The maximum likelihood estimate ˆ θML is the parameter vector that maximizes the likelihood L(·) of observing the (known) outputs y, given the (known) inputs X: ˆ θML = arg max

θ

L

• y, X | θ
• Marc Claesen

System Modeling, part 2

Nonlinear systems & linearization System identification (cont) Grey box identification Black box identification

Maximum likelihood estimation

The maximum likelihood estimate ˆ θML is the parameter vector that maximizes the likelihood L(·) of observing the (known) outputs y, given the (known) inputs X: ˆ θML = arg max

θ

L

• y, X | θ
• For some structures, ML estimate can be obtained in closed form.

Marc Claesen System Modeling, part 2

Nonlinear systems & linearization System identification (cont) Grey box identification Black box identification

Maximum likelihood estimation

The maximum likelihood estimate ˆ θML is the parameter vector that maximizes the likelihood L(·) of observing the (known) outputs y, given the (known) inputs X: ˆ θML = arg max

θ

L

• y, X | θ
• For some structures, ML estimate can be obtained in closed form.

Example: least squares estimators are the maximum likelihood estimators if the associated residuals ǫ are normally distributed.

Marc Claesen System Modeling, part 2

Nonlinear systems & linearization System identification (cont) Grey box identification Black box identification

Maximum a posteriori (MAP) estimation

Bayesian: maximum likelihood estimation with a prior p(θ). → MAP estimation is a regularization of ML estimation

Marc Claesen System Modeling, part 2

Nonlinear systems & linearization System identification (cont) Grey box identification Black box identification

Maximum a posteriori (MAP) estimation

Bayesian: maximum likelihood estimation with a prior p(θ). → MAP estimation is a regularization of ML estimation Bayes’ theorem: P(A | B) = P(B | A) · P(A) / P(B).

Marc Claesen System Modeling, part 2

Nonlinear systems & linearization System identification (cont) Grey box identification Black box identification

Maximum a posteriori (MAP) estimation

Bayesian: maximum likelihood estimation with a prior p(θ). → MAP estimation is a regularization of ML estimation Bayes’ theorem: P(A | B) = P(B | A) · P(A) / P(B). If a prior distribution p(·) is available for θ, then the posterior distribution for θ becomes: θ → L(θ | y, X) = L(y, X | θ)p(θ)

• ϑ L(y, X | ϑ)p(ϑ)dϑ.

Marc Claesen System Modeling, part 2

Nonlinear systems & linearization System identification (cont) Grey box identification Black box identification

Maximum a posteriori (MAP) estimation

Bayesian: maximum likelihood estimation with a prior p(θ). → MAP estimation is a regularization of ML estimation Bayes’ theorem: P(A | B) = P(B | A) · P(A) / P(B). If a prior distribution p(·) is available for θ, then the posterior distribution for θ becomes: θ → L(θ | y, X) = L(y, X | θ)p(θ)

• ϑ L(y, X | ϑ)p(ϑ)dϑ.

The MAP estimate is the mode of the posterior distribution of θ: ˆ θMAP = arg max

θ

L(y, X | θ)p(θ).

Marc Claesen System Modeling, part 2

Nonlinear systems & linearization System identification (cont) Grey box identification Black box identification

Errors-in-variables approach

Additionally accounts for measurement errors in inputs. ↔ standard regression only accounts for errors in outputs

Marc Claesen System Modeling, part 2

Nonlinear systems & linearization System identification (cont) Grey box identification Black box identification

Errors-in-variables approach

Additionally accounts for measurement errors in inputs. ↔ standard regression only accounts for errors in outputs Typically described via latent variables:    x = x⋆+η, y = y⋆+ǫ, y⋆ = g(x⋆ | θ), with x, y the observed inputs, outputs and latent variables x⋆, y⋆.

Marc Claesen System Modeling, part 2

Nonlinear systems & linearization System identification (cont) Grey box identification Black box identification

Errors-in-variables approach

Additionally accounts for measurement errors in inputs. ↔ standard regression only accounts for errors in outputs Typically described via latent variables:    x = x⋆+η, y = y⋆+ǫ, y⋆ = g(x⋆ | θ), with x, y the observed inputs, outputs and latent variables x⋆, y⋆. Assumption: latent variables x⋆ and y⋆ exist which follow the true functional relationship g(·).

Marc Claesen System Modeling, part 2

Nonlinear systems & linearization System identification (cont) Grey box identification Black box identification

Errors-in-variables approach

Additionally accounts for measurement errors in inputs. ↔ standard regression only accounts for errors in outputs Typically described via latent variables:    x = x⋆+η, y = y⋆+ǫ, y⋆ = g(x⋆ | θ), with x, y the observed inputs, outputs and latent variables x⋆, y⋆. Assumption: latent variables x⋆ and y⋆ exist which follow the true functional relationship g(·). Task: estimate θ.

Marc Claesen System Modeling, part 2

Nonlinear systems & linearization System identification (cont) Grey box identification Black box identification

Outline

1

Nonlinear systems & linearization

2

System identification (cont) Grey box identification Black box identification

Marc Claesen System Modeling, part 2

Nonlinear systems & linearization System identification (cont) Grey box identification Black box identification

Black box identification

Start from unknown equations & unknown parameters. → related to machine learning and nonparametric statistics.

Marc Claesen System Modeling, part 2

Nonlinear systems & linearization System identification (cont) Grey box identification Black box identification

Black box identification

Start from unknown equations & unknown parameters. → related to machine learning and nonparametric statistics. If we assume a linear state space system: ˙ x(t) = Ax(t) + Bu(t), y(t) = Cx(t) + Du(t). x[k + 1] = Ax[k] + Bu[k], y[k] = Cx[k] + Du[k].

Marc Claesen System Modeling, part 2

Nonlinear systems & linearization System identification (cont) Grey box identification Black box identification

Black box identification

Start from unknown equations & unknown parameters. → related to machine learning and nonparametric statistics. If we assume a linear state space system: ˙ x(t) = Ax(t) + Bu(t), y(t) = Cx(t) + Du(t). x[k + 1] = Ax[k] + Bu[k], y[k] = Cx[k] + Du[k]. Black box identification deals with:

Marc Claesen System Modeling, part 2

Nonlinear systems & linearization System identification (cont) Grey box identification Black box identification

Black box identification

Start from unknown equations & unknown parameters. → related to machine learning and nonparametric statistics. If we assume a linear state space system: ˙ x(t) = Ax(t) + Bu(t), y(t) = Cx(t) + Du(t). x[k + 1] = Ax[k] + Bu[k], y[k] = Cx[k] + Du[k]. Black box identification deals with: unknown states, both in number & physical interpretation

Marc Claesen System Modeling, part 2

Nonlinear systems & linearization System identification (cont) Grey box identification Black box identification

Black box identification

Start from unknown equations & unknown parameters. → related to machine learning and nonparametric statistics. If we assume a linear state space system: ˙ x(t) = Ax(t) + Bu(t), y(t) = Cx(t) + Du(t). x[k + 1] = Ax[k] + Bu[k], y[k] = Cx[k] + Du[k]. Black box identification deals with: unknown states, both in number & physical interpretation → dimensions of A, B & C unknown

Marc Claesen System Modeling, part 2

Nonlinear systems & linearization System identification (cont) Grey box identification Black box identification

Black box identification

Start from unknown equations & unknown parameters. → related to machine learning and nonparametric statistics. If we assume a linear state space system: ˙ x(t) = Ax(t) + Bu(t), y(t) = Cx(t) + Du(t). x[k + 1] = Ax[k] + Bu[k], y[k] = Cx[k] + Du[k]. Black box identification deals with: unknown states, both in number & physical interpretation → dimensions of A, B & C unknown unknown parameters (values in A, B, C, D)

Marc Claesen System Modeling, part 2

Nonlinear systems & linearization System identification (cont) Grey box identification Black box identification

Time series: Santa Fe laser

Marc Claesen System Modeling, part 2

Nonlinear systems & linearization System identification (cont) Grey box identification Black box identification

Modelling the Santa Fe laster

This laser can be treated as an autonomous discrete time system: x[k + 1] = f

• x[k − N + 1], . . . , x[k]
• ,

y[k] = x[k]. The output depends on the past N states & no inputs.

Marc Claesen System Modeling, part 2

Nonlinear systems & linearization System identification (cont) Grey box identification Black box identification

Modelling the Santa Fe laster

This laser can be treated as an autonomous discrete time system: x[k + 1] = f

• x[k − N + 1], . . . , x[k]
• ,

y[k] = x[k]. The output depends on the past N states & no inputs. → how large is N? → unknown structure

Marc Claesen System Modeling, part 2

Nonlinear systems & linearization System identification (cont) Grey box identification Black box identification

Modelling the Santa Fe laster

This laser can be treated as an autonomous discrete time system: x[k + 1] = f

• x[k − N + 1], . . . , x[k]
• ,

y[k] = x[k]. The output depends on the past N states & no inputs. → how large is N? → unknown structure Treat it as a regression problem with N inputs: y = f

• X1, . . . , XN
• .

Marc Claesen System Modeling, part 2

Nonlinear systems & linearization System identification (cont) Grey box identification Black box identification

Modelling the Santa Fe laster

This laser can be treated as an autonomous discrete time system: x[k + 1] = f

• x[k − N + 1], . . . , x[k]
• ,

y[k] = x[k]. The output depends on the past N states & no inputs. → how large is N? → unknown structure Treat it as a regression problem with N inputs: y = f

• X1, . . . , XN
• .

→ lets say linear, i.e. y = Xθ → unknown parameters θ ∈ RN.

Marc Claesen System Modeling, part 2

Nonlinear systems & linearization System identification (cont) Grey box identification Black box identification

Modelling the Santa Fe laster

This laser can be treated as an autonomous discrete time system: x[k + 1] = f

• x[k − N + 1], . . . , x[k]
• ,

y[k] = x[k]. The output depends on the past N states & no inputs. → how large is N? → unknown structure Treat it as a regression problem with N inputs: y = f

• X1, . . . , XN
• .

→ lets say linear, i.e. y = Xθ → unknown parameters θ ∈ RN. → for given N, we can estimate θ via grey box methods.

Marc Claesen System Modeling, part 2

Nonlinear systems & linearization System identification (cont) Grey box identification Black box identification

Modelling the Santa Fe laster

This laser can be treated as an autonomous discrete time system: x[k + 1] = f

• x[k − N + 1], . . . , x[k]
• ,

y[k] = x[k]. The output depends on the past N states & no inputs. → how large is N? → unknown structure Treat it as a regression problem with N inputs: y = f

• X1, . . . , XN
• .

→ lets say linear, i.e. y = Xθ → unknown parameters θ ∈ RN. → for given N, we can estimate θ via grey box methods. Nonlinear models can be obtained via machine learning methods. → neural networks, support vector machine, random forest, ...

Marc Claesen System Modeling, part 2

Nonlinear systems & linearization System identification (cont) Grey box identification Black box identification

Predictions of a least-squares support vector machine

Marc Claesen System Modeling, part 2

Nonlinear systems & linearization System identification (cont) Grey box identification Black box identification

Predictions of an artificial neural network

Marc Claesen System Modeling, part 2

Nonlinear systems & linearization System identification (cont) Grey box identification Black box identification

Neural network: biological

Image taken from http://www.extremetech.com/wp-content/uploads/2013/09/340.jpg. Marc Claesen System Modeling, part 2

Nonlinear systems & linearization System identification (cont) Grey box identification Black box identification

Structure of a single neuron

Image taken from http://en.wikipedia.org/wiki/File:Blausen_0657_MultipolarNeuron.png. Marc Claesen System Modeling, part 2

Nonlinear systems & linearization System identification (cont) Grey box identification Black box identification

Neural network: artificial

Marc Claesen System Modeling, part 2

Nonlinear systems & linearization System identification (cont) Grey box identification Black box identification Marc Claesen System Modeling, part 2