[PPT] - MSc in Computer Engineering, Cybersecurity and Artificial PowerPoint Presentation

SLIDE 1

MSc in Computer Engineering, Cybersecurity and Artificial Intelligence Course FDE , a.a. 2019/2020, Lecture 16 Review of stochastic processes and systems

Prof. Mauro Franceschelli
Dept. of Electrical and Electronic Engineering

University of Cagliari, Italy

Thursday, 7th May 2020

1 / 28

SLIDE 2

Outline

Modeling measurement and process noise Brief review of stochastic models Noise model

2 / 28

SLIDE 3

Modeling measurement and process noise

Introduction

So far, we have introduced the concept of measurement noise as an issue to

take care in the control system and state observer design problem without a formal characterization

Sensing and actuation equipment are inevitably affected by noise, uncertainty

and small modeling inaccuracies.

These issues can not be eliminated in practice but their existence can be

formally characterized and taken care of during the design process.

In particular, we ask the question: can we reduce the effect of measurement and

process noise on state estimation errors?

3 / 28

SLIDE 4

Modeling measurement and process noise

Introduction

So far, we have introduced the concept of measurement noise as an issue to

take care in the control system and state observer design problem without a formal characterization

Sensing and actuation equipment are inevitably affected by noise, uncertainty

and small modeling inaccuracies.

These issues can not be eliminated in practice but their existence can be

formally characterized and taken care of during the design process.

In particular, we ask the question: can we reduce the effect of measurement and

process noise on state estimation errors?

3 / 28

SLIDE 5

Modeling measurement and process noise

Introduction

So far, we have introduced the concept of measurement noise as an issue to

take care in the control system and state observer design problem without a formal characterization

Sensing and actuation equipment are inevitably affected by noise, uncertainty

and small modeling inaccuracies.

These issues can not be eliminated in practice but their existence can be

formally characterized and taken care of during the design process.

In particular, we ask the question: can we reduce the effect of measurement and

process noise on state estimation errors?

3 / 28

SLIDE 6

Modeling measurement and process noise

Introduction

So far, we have introduced the concept of measurement noise as an issue to

take care in the control system and state observer design problem without a formal characterization

Sensing and actuation equipment are inevitably affected by noise, uncertainty

and small modeling inaccuracies.

These issues can not be eliminated in practice but their existence can be

formally characterized and taken care of during the design process.

In particular, we ask the question: can we reduce the effect of measurement and

process noise on state estimation errors?

3 / 28

SLIDE 7

Modeling measurement and process noise

Measurement and process noise

Our objective is to study a state estimation problem for the next discrete-time, time-varying, linear system: ①(k + 1) = ❆(k)①(k) + ❇(k)✉(k) + ✇(k) ②(k) = ❈(k)①(k) + ✈(k) where ❆(k) is an n × n matrix; ❇(k) is an n × r matrix; ❈(k) is an n × p matrix;

The dependence of matrix A, B, C on the discrete time k could be inherent to

the model or be due to the linearization of a nonlinear model at time k around the current state ①(k).

4 / 28

SLIDE 8

Modeling measurement and process noise

Measurement and process noise

①(k + 1) = ❆(k)①(k) + ❇(k)✉(k) + ✇(k) ②(k) = ❈(k)①(k) + ✈(k)

✇(k) is an n element vector which represents process noise, i.e., an element of

uncertainty in the update of the state variables to external factors or modeling inaccuracies of the dynamical system (the process). Process noise could also model actuation noise if ✇(k) = ❇ ˆ ✇(k) without lack of generality.

✈(k) is a p element vector which represents measurement noise, i.e., an element
f unavoidable uncertainty in measurement of the system outputs by the sensing

equipment.

5 / 28

SLIDE 9

Modeling measurement and process noise

Measurement and process noise

①(k + 1) = ❆(k)①(k) + ❇(k)✉(k) + ✇(k) ②(k) = ❈(k)①(k) + ✈(k)

Both ✇ and ✈ are stochastic processes
If we fix the time k then ✇(k) and ✈(k) are vectors whose elements are random

variables.

6 / 28

SLIDE 10

Modeling measurement and process noise

Measurement and process noise

①(k + 1) = ❆(k)①(k) + ❇(k)✉(k) + ✇(k) ②(k) = ❈(k)①(k) + ✈(k)

Both ✇ and ✈ are stochastic processes
If we fix the time k then ✇(k) and ✈(k) are vectors whose elements are random

variables.

6 / 28

SLIDE 11

Outline

Modeling measurement and process noise Brief Review of stochastic models Noise model

7 / 28

SLIDE 12

Brief review of stochastic models

Random variable

A discrete random variable X (variabile aleatoria) takes a random value among

a discrete set of possible values according to a probability distribution.

For instance, the discrete random variable representing the outcome of a dice

throw has possible values S = {1, 2, 3, 4, 5, 6]}.

Let p(X = x) with x ∈ S, also simply denoted as p(x) be the probability

distribution of the discrete random variable X.

In the case of the dice, p(1) = 1/6, p(2) = 1/6, p(3) = 1/6, p(4) = 1/6,

p(5) = 1/6, p(6) = 1/6 and p(x) = 0 for x ∈ S. Throwing the dice is called a realization of the random variable, or its outcome.

8 / 28

SLIDE 13

Brief review of stochastic models

Random variable

A discrete random variable X (variabile aleatoria) takes a random value among

a discrete set of possible values according to a probability distribution.

For instance, the discrete random variable representing the outcome of a dice

throw has possible values S = {1, 2, 3, 4, 5, 6]}.

Let p(X = x) with x ∈ S, also simply denoted as p(x) be the probability

distribution of the discrete random variable X.

In the case of the dice, p(1) = 1/6, p(2) = 1/6, p(3) = 1/6, p(4) = 1/6,

p(5) = 1/6, p(6) = 1/6 and p(x) = 0 for x ∈ S. Throwing the dice is called a realization of the random variable, or its outcome.

8 / 28

SLIDE 14

Brief review of stochastic models

Random variable

A discrete random variable X (variabile aleatoria) takes a random value among

a discrete set of possible values according to a probability distribution.

For instance, the discrete random variable representing the outcome of a dice

throw has possible values S = {1, 2, 3, 4, 5, 6]}.

Let p(X = x) with x ∈ S, also simply denoted as p(x) be the probability

distribution of the discrete random variable X.

In the case of the dice, p(1) = 1/6, p(2) = 1/6, p(3) = 1/6, p(4) = 1/6,

p(5) = 1/6, p(6) = 1/6 and p(x) = 0 for x ∈ S. Throwing the dice is called a realization of the random variable, or its outcome.

8 / 28

SLIDE 15

Brief review of stochastic models

Random variable

A discrete random variable X (variabile aleatoria) takes a random value among

a discrete set of possible values according to a probability distribution.

For instance, the discrete random variable representing the outcome of a dice

throw has possible values S = {1, 2, 3, 4, 5, 6]}.

Let p(X = x) with x ∈ S, also simply denoted as p(x) be the probability

distribution of the discrete random variable X.

In the case of the dice, p(1) = 1/6, p(2) = 1/6, p(3) = 1/6, p(4) = 1/6,

p(5) = 1/6, p(6) = 1/6 and p(x) = 0 for x ∈ S. Throwing the dice is called a realization of the random variable, or its outcome.

8 / 28

SLIDE 16

Brief review of stochastic models

Continuous random variable

A continuous random variable X takes a random value within a given

continuous interval of admissible values according to a probability distribution.

Let X take values in a continuous interval [x1, x2] where x1, x2 ∈ R
Let p(X = x) with x ∈ R, also simply denoted as p(x) be the probability

density function of the random variable X

9 / 28

SLIDE 17

Brief review of stochastic models

Continuous random variable

A continuous random variable X takes a random value within a given

continuous interval of admissible values according to a probability distribution.

Let X take values in a continuous interval [x1, x2] where x1, x2 ∈ R
Let p(X = x) with x ∈ R, also simply denoted as p(x) be the probability

density function of the random variable X

9 / 28

SLIDE 18

Brief review of stochastic models

Continuous random variable

A continuous random variable X takes a random value within a given

continuous interval of admissible values according to a probability distribution.

Let X take values in a continuous interval [x1, x2] where x1, x2 ∈ R
Let p(X = x) with x ∈ R, also simply denoted as p(x) be the probability

density function of the random variable X

9 / 28

SLIDE 19

Brief review of stochastic models

Continuous random variable

The probability that a given realization of the random variable belongs to an

interval [a, b] is Pr(X ∈ (a, b)) = b

a

p(x)dx

Clearly, Pr(X ∈ (−∞, ∞)) =

∞

−∞ p(x)dx = 1

10 / 28

SLIDE 20

Brief review of stochastic models

Continuous random variable

The probability that a given realization of the random variable belongs to an

interval [a, b] is Pr(X ∈ (a, b)) = b

a

p(x)dx

Clearly, Pr(X ∈ (−∞, ∞)) =

∞

−∞ p(x)dx = 1

10 / 28

SLIDE 21

Brief review of stochastic models

Expected value and variance

The expected value µ of a random variable generalizes the concept of average
value. It is defined as

E[X] = ∞

−∞

xp(x)dx = µ

The variance of a random variable around its expected value is denoted as σ2

and is computed as σ2 = E[(X − E[x])2] = E[(X − µ)2] = ∞

−∞

(x − µ)2p(x)dx

The variance of a random variable is the square of the so-called standard

deviation σ. The variance is a measure of the dispersion of the possible outcomes

f a random variable around its expected value.

11 / 28

SLIDE 22

Brief review of stochastic models

Expected value and variance

The expected value µ of a random variable generalizes the concept of average
value. It is defined as

E[X] = ∞

−∞

xp(x)dx = µ

The variance of a random variable around its expected value is denoted as σ2

and is computed as σ2 = E[(X − E[x])2] = E[(X − µ)2] = ∞

−∞

(x − µ)2p(x)dx

The variance of a random variable is the square of the so-called standard

deviation σ. The variance is a measure of the dispersion of the possible outcomes

f a random variable around its expected value.

11 / 28

SLIDE 23

Brief review of stochastic models

Expected value and variance

The expected value µ of a random variable generalizes the concept of average
value. It is defined as

E[X] = ∞

−∞

xp(x)dx = µ

The variance of a random variable around its expected value is denoted as σ2

and is computed as σ2 = E[(X − E[x])2] = E[(X − µ)2] = ∞

−∞

(x − µ)2p(x)dx

The variance of a random variable is the square of the so-called standard

deviation σ. The variance is a measure of the dispersion of the possible outcomes

f a random variable around its expected value.

11 / 28

SLIDE 24

Brief review of stochastic models

Expected value and variance

The expected value µ of a random variable generalizes the concept of average
value. It is defined as

E[X] = ∞

−∞

xp(x)dx = µ

The variance of a random variable around its expected value is denoted as σ2

and is computed as σ2 = E[(X − E[x])2] = E[(X − µ)2] = ∞

−∞

(x − µ)2p(x)dx

The variance of a random variable is the square of the so-called standard

deviation σ. The variance is a measure of the dispersion of the possible outcomes

f a random variable around its expected value.

11 / 28

SLIDE 25

Brief review of stochastic models

Random vectors

A random vector X = (x1, x2, . . . , xn)T is a vector whose elements are random

variables.

Its expected value is simply

E[X] =      E[x1] E[x2] . . . E[xn]     

12 / 28

SLIDE 26

Brief review of stochastic models

Random vectors

A random vector X = (x1, x2, . . . , xn)T is a vector whose elements are random

variables.

Its expected value is simply

E[X] =      E[x1] E[x2] . . . E[xn]     

12 / 28

SLIDE 27

Brief review of stochastic models

Covariance matrix

The covariance matrix Σ generalizes the concept of variance of a random

variable for a random vector. It is defined as Σ = E[(X − E[X])(X − E[X])T]

If vector X has a single element, it is equal to the definition of variance.
Its generic ith diagonal element is the variance of the i-th element of vector X.
The generic i, j off-diagonal element represents the correlation between the

random variables xi and xj of vector X.

If the random vector X has for elements uncorrelated random variables, then

the covariance matrix Σ is diagonal.

13 / 28

SLIDE 28

Brief review of stochastic models

Covariance matrix

The covariance matrix Σ generalizes the concept of variance of a random

variable for a random vector. It is defined as Σ = E[(X − E[X])(X − E[X])T]

If vector X has a single element, it is equal to the definition of variance.
Its generic ith diagonal element is the variance of the i-th element of vector X.
The generic i, j off-diagonal element represents the correlation between the

random variables xi and xj of vector X.

If the random vector X has for elements uncorrelated random variables, then

the covariance matrix Σ is diagonal.

13 / 28

SLIDE 29

Brief review of stochastic models

Covariance matrix

The covariance matrix Σ generalizes the concept of variance of a random

variable for a random vector. It is defined as Σ = E[(X − E[X])(X − E[X])T]

If vector X has a single element, it is equal to the definition of variance.
Its generic ith diagonal element is the variance of the i-th element of vector X.
The generic i, j off-diagonal element represents the correlation between the

random variables xi and xj of vector X.

If the random vector X has for elements uncorrelated random variables, then

the covariance matrix Σ is diagonal.

13 / 28

SLIDE 30

Brief review of stochastic models

Covariance matrix

The covariance matrix Σ generalizes the concept of variance of a random

variable for a random vector. It is defined as Σ = E[(X − E[X])(X − E[X])T]

If vector X has a single element, it is equal to the definition of variance.
Its generic ith diagonal element is the variance of the i-th element of vector X.
The generic i, j off-diagonal element represents the correlation between the

random variables xi and xj of vector X.

If the random vector X has for elements uncorrelated random variables, then

the covariance matrix Σ is diagonal.

13 / 28

SLIDE 31

Brief review of stochastic models

Covariance matrix

The covariance matrix Σ generalizes the concept of variance of a random

variable for a random vector. It is defined as Σ = E[(X − E[X])(X − E[X])T]

If vector X has a single element, it is equal to the definition of variance.
Its generic ith diagonal element is the variance of the i-th element of vector X.
The generic i, j off-diagonal element represents the correlation between the

random variables xi and xj of vector X.

If the random vector X has for elements uncorrelated random variables, then

the covariance matrix Σ is diagonal.

13 / 28

SLIDE 32

Brief review of stochastic models

Gaussian random variable

A random variable is said to be Gaussian (or normal) if its probability density is

p(X) = 1 √ 2πσ2 e− 1

2 (X−µ)2 σ2

A Gaussian random variable is completely defined by its expectation µ and

variance σ2.

Standard notation to denote a Gaussian (normal) distribution is N(µ, σ2),

14 / 28

SLIDE 33

Brief review of stochastic models

Gaussian random variable

A random variable is said to be Gaussian (or normal) if its probability density is

p(X) = 1 √ 2πσ2 e− 1

2 (X−µ)2 σ2

A Gaussian random variable is completely defined by its expectation µ and

variance σ2.

Standard notation to denote a Gaussian (normal) distribution is N(µ, σ2),

14 / 28

SLIDE 34

Brief review of stochastic models

Gaussian random variable

A random variable is said to be Gaussian (or normal) if its probability density is

p(X) = 1 √ 2πσ2 e− 1

2 (X−µ)2 σ2

A Gaussian random variable is completely defined by its expectation µ and

variance σ2.

Standard notation to denote a Gaussian (normal) distribution is N(µ, σ2),

14 / 28

SLIDE 35

Brief review of stochastic models

Gaussian random variable

Examples of Gaussian probability density functions:

15 / 28

SLIDE 36

Brief review of stochastic models

Gaussian random variable

Probability that the outcome of a random Guassian variable is within an integer

number of standard deviations σ

16 / 28

SLIDE 37

Brief review of stochastic models

Gaussian random variable

Collected data points form a Gaussian (normal) random variable.

17 / 28

SLIDE 38

Brief review of stochastic models

Gaussian random vector

A random vector is said to be Gaussian (or normal) if its probability density is

p(X) = 1

(2π)2det(Σ)

e− 1

2 (X−µ)T Σ(X−µ)

Notice that p(x) is now a function p(x) : Rn → R.

18 / 28

SLIDE 39

Brief review of stochastic models

Stochastic process

A stochastic process is a sequence of random variables for k = 1, . . . , ∞
If we fix a time instant, the stochastic process is a random variable.
If each such random variable has probability distribution p(x) independent from

the time k then the stochastic process is said to be stationary.

A stationary Gaussian stochastic process is a process each random variable is

independent and identically distributed (i.i.d.) and has Gaussian probability distribution.

19 / 28

SLIDE 40

Brief review of stochastic models

Stochastic process

A stochastic process is a sequence of random variables for k = 1, . . . , ∞
If we fix a time instant, the stochastic process is a random variable.
If each such random variable has probability distribution p(x) independent from

the time k then the stochastic process is said to be stationary.

A stationary Gaussian stochastic process is a process each random variable is

independent and identically distributed (i.i.d.) and has Gaussian probability distribution.

19 / 28

SLIDE 41

Brief review of stochastic models

Stochastic process

A stochastic process is a sequence of random variables for k = 1, . . . , ∞
If we fix a time instant, the stochastic process is a random variable.
If each such random variable has probability distribution p(x) independent from

the time k then the stochastic process is said to be stationary.

A stationary Gaussian stochastic process is a process each random variable is

independent and identically distributed (i.i.d.) and has Gaussian probability distribution.

19 / 28

SLIDE 42

Brief review of stochastic models

Stochastic process

A stochastic process is a sequence of random variables for k = 1, . . . , ∞
If we fix a time instant, the stochastic process is a random variable.
If each such random variable has probability distribution p(x) independent from

the time k then the stochastic process is said to be stationary.

A stationary Gaussian stochastic process is a process each random variable is

independent and identically distributed (i.i.d.) and has Gaussian probability distribution.

19 / 28

SLIDE 43

Outline

Modeling measurement and process noise Brief Review of stochastic models Noise model

20 / 28

SLIDE 44

Noise model

Model of measurement and process noise

Additive white Gaussian noise (AWGN) is an example of stationary Gaussian

stochastic process where the expected value of each Gaussian distribution has µ = 0.

AWGN is a good approximation of electrical noise and thus measurement noise

in many physical processes.

Going back to our dynamical system model, we consider the case where the

stochastic processes of the process noise ✇(k) and measurement noise ✈(k) are AWGN.

Thus ✇(k) and ✈(k) have zero expected value

E[w(k)] = µ = 0, and E[v(k)] = µ = 0, ∀k

21 / 28

SLIDE 45

Noise model

Model of measurement and process noise

Additive white Gaussian noise (AWGN) is an example of stationary Gaussian

stochastic process where the expected value of each Gaussian distribution has µ = 0.

AWGN is a good approximation of electrical noise and thus measurement noise

in many physical processes.

Going back to our dynamical system model, we consider the case where the

stochastic processes of the process noise ✇(k) and measurement noise ✈(k) are AWGN.

Thus ✇(k) and ✈(k) have zero expected value

E[w(k)] = µ = 0, and E[v(k)] = µ = 0, ∀k

21 / 28

SLIDE 46

Noise model

Model of measurement and process noise

Additive white Gaussian noise (AWGN) is an example of stationary Gaussian

stochastic process where the expected value of each Gaussian distribution has µ = 0.

AWGN is a good approximation of electrical noise and thus measurement noise

in many physical processes.

Going back to our dynamical system model, we consider the case where the

stochastic processes of the process noise ✇(k) and measurement noise ✈(k) are AWGN.

Thus ✇(k) and ✈(k) have zero expected value

E[w(k)] = µ = 0, and E[v(k)] = µ = 0, ∀k

21 / 28

SLIDE 47

Noise model

Model of measurement and process noise

Since µ = 0, ✇(k) and ✈(k) have covariance matrix

E[w(k)w(k)T] = Q(k), and E[v(k)v(k)T] = R(k), ∀k

Also, since the stochastic process is stationary and i.i.d, if we take two instants

k1 = k2 it means E[w(k1)w(k2)T] = 0, and E[v(k1)v(k2)T] = 0.

22 / 28

SLIDE 48

Noise model

Model of measurement and process noise

Since µ = 0, ✇(k) and ✈(k) have covariance matrix

E[w(k)w(k)T] = Q(k), and E[v(k)v(k)T] = R(k), ∀k

Also, since the stochastic process is stationary and i.i.d, if we take two instants

k1 = k2 it means E[w(k1)w(k2)T] = 0, and E[v(k1)v(k2)T] = 0.

22 / 28

SLIDE 49

Noise model

Model of measurement and process noise

Since ✇(k) and ✈(k) are uncorrelated because they are independent process, it

also holds ∀k1, k2 E[w(k1)v(k2)T] = 0

23 / 28

SLIDE 50

Noise model

Model of measurement and process noise

Now, consider again our dynamical system

①(k + 1) = ❆(k)①(k) + ❇(k)✉(k) + ✇(k) ②(k) = ❈(k)①(k) + ✈(k) where ❆(k) is an n × n matrix; ❇(k) is an n × r matrix; ❈(k) is an n × p matrix;

Assume that at the initial instant of time k the covariance matrices of ✇(k) and

✈(k) are diagonal, i.e., noise on different state/output variables is uncorrelated.

24 / 28

SLIDE 51

Noise model

Model of measurement and process noise

At each time step, now also ①(k) can be seen as a random vector
The convariance matrix of vector ①(k) though contains in general off-diagonal

terms which are different from zero because the system dynamics introduces correlation among the state variables.

25 / 28

SLIDE 52

Noise model

Model of measurement and process noise

At each time step, now also ①(k) can be seen as a random vector
The convariance matrix of vector ①(k) though contains in general off-diagonal

terms which are different from zero because the system dynamics introduces correlation among the state variables.

25 / 28

SLIDE 53

Noise model

Covariance matrices

Thus, covariance matrices can help us understand how errors in different state

variables are related and how uncertainty propagates in the time-evolution of a dynamical system.

Note that now, ①(k + 1) and ①(k) are random vectors the dynamics of their

expectation is E[①(k + 1)] = E[❆(k)①(k) + ❇(k)✉(k) + ✇(k)] = ❆(k)E[①(k)] + ❇(k)✉(k) + E[✇(k)] = ❆(k)E[①(k)] + ❇(k)✉(k) The covariance dynamics can be computed by substituting into Σ(k) = E[(x(k) − E[x(k)])(x(k) − E[x(k)])T]

26 / 28

SLIDE 54

Noise model

Covariance matrices

Thus, covariance matrices can help us understand how errors in different state

variables are related and how uncertainty propagates in the time-evolution of a dynamical system.

Note that now, ①(k + 1) and ①(k) are random vectors the dynamics of their

expectation is E[①(k + 1)] = E[❆(k)①(k) + ❇(k)✉(k) + ✇(k)] = ❆(k)E[①(k)] + ❇(k)✉(k) + E[✇(k)] = ❆(k)E[①(k)] + ❇(k)✉(k) The covariance dynamics can be computed by substituting into Σ(k) = E[(x(k) − E[x(k)])(x(k) − E[x(k)])T]

26 / 28

SLIDE 55

Noise model

Covariance matrices

Let P =

σ2

x1

σ2

x2

=

4 9

be a diagonal covariance matrix

corresponding to two independent random variables. Its 2-dimensional probability distribution is

27 / 28

SLIDE 56

Noise model

Covariance matrices

Now consider P =
σ2

x1

σ2

x1,x2

σ2

x2,x1

σ2

x2

=
4

1 1 9

be the covariance matrix

corresponding to two correlated random variables. Its 2-dimensional probability distribution is

28 / 28