Models for LTI systems LTI system stands for linear time invariant - PowerPoint PPT Presentation

Models for LTI systems LTI system stands for linear time invariant system Model describing LTI system using an input-output relation y k = G ( q ) u k + H ( q ) e k with output (time) series y = { y k : k = 0 , 1 , . . . , N } input (time) series u = { u k : k = 0 , 1 , . . . , N } and disturbances (e.g. uncontrollable input, noise with distribution f e ( · ) , . . . ) e = { e k : k = 0 , 1 , . . . , N } Lecture 3: Elementary Models for LTI Systems 1 / 91

Models for LTI systems Forward delay operator q qu k = u k + 1 q − 1 u k = u k − 1 Transfer functions G ( q ) and H ( q ) associate with impulse responses g k and h k ∞ � g k q − k G ( q ) = k = 1 ∞ � h k q − k H ( q ) = 1 + k = 1 Lecture 3: Elementary Models for LTI Systems 2 / 91

Modeling disturbances for LTI systems matlab demo ∞ � h k q − k H ( q ) = 1 + k = 1 Example: Butterworth lowpass filter of degree 2 and cutoff frequency 1/10 of the Nyquist frequency, with transfer function H ( q ) = 0 . 0201 + 0 . 0402 q − 1 + 0 . 0201 q − 2 1 . 0000 − 1 . 5610 q − 1 + 0 . 6414 q − 2 Consider two cases 1. � with probability 1 − γ 0 e k = r with probability γ with γ small (0.02) and r normally distributed 2. e k normally distributed Lecture 3: Elementary Models for LTI Systems 3 / 91

Modeling disturbances for LTI systems matlab demo 0 magnitude (dB/Hz) −20 −40 −60 −80 −100 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5 normalized frequency (T s = 1) 0.15 impulse response 0.1 0.05 0 −0.05 0 5 10 15 20 25 30 35 time (samples) Lecture 3: Elementary Models for LTI Systems 4 / 91

Modeling disturbances for LTI systems matlab demo Case 1: 4 0.6 0.4 2 0.2 0 0 −2 −0.2 −4 −0.4 0 500 1000 1500 2000 0 500 1000 1500 2000 time (samples) time (samples) 2000 2000 1500 1500 1000 1000 500 500 0 0 −2 0 2 −2 0 2 For? Disturbances such as peaks, trends and offsets 1 1 For trends and offsets use lowpass filter with very low cutoff frequencies Lecture 3: Elementary Models for LTI Systems 5 / 91

Modeling disturbances for LTI systems matlab demo Case 2: 4 1.5 1 2 0.5 0 0 −0.5 −2 −1 −4 −1.5 0 500 1000 1500 2000 0 500 1000 1500 2000 time (samples) time (samples) 600 600 400 400 200 200 0 0 −4 −2 0 2 4 −4 −2 0 2 4 For? Measurement noise Lecture 3: Elementary Models for LTI Systems 6 / 91

Modeling disturbances for LTI systems matlab demo Do we need the noise distribution function f e ( · ) ? Often we work with some statistical characteristics such as the first and second order moments � E [ e k ] = xf e ( x ) dx � σ 2 = E � ( e k − E [ e k ]) 2 � ( x − E [ e k ]) 2 f e ( x ) dx = Often we assume e k to be white noise with mean zero and unknown variance σ . Lecture 3: Elementary Models for LTI Systems 7 / 91

System identification for LTI systems How to estimate G ( q ) and H ( q ) ? Parametrized model with unknown (to be estimated) parameters θ y k = G ( q , θ ) u k + H ( q , θ ) e k Structure choice for transfer functions G ( q ) and H ( q ) ? A ( q ) y k = B ( q ) F ( q ) u k + C ( q ) D ( q ) e k Very general, simplifications possible, why? Occam’s razor, avoid overfitting, easier to solve, . . . Lecture 3: Elementary Models for LTI Systems 8 / 91

Short overview of commonly used models “equation error” (ARX) model structure Input-output relation described by a linear difference equation y k + a 1 y k − 1 + . . . + a n a y k − n a = b 1 u k + . . . + b n b u k − n b + 1 + e k White noise term e k serves to quantify the direct error in the equations. The parameter vector is θ = � � t a 1 a 2 . . . a n a b 1 b 2 . . . b n b Transfer function A ( q ) y k = B ( q ) u k + e k with A ( q ) = 1 + a 1 q − 1 + . . . + a n a q − n a B ( q ) = b 1 + . . . + b n b q − n b + 1 or with G ( q ) = B ( q ) 1 A ( q ) and H ( q ) = A ( q ) : y k = B ( q ) 1 A ( q ) u k + A ( q ) e k ARX models possess an autoregressive (AR) part A ( q ) y k , ‘X’ refers to the input term B ( q ) u k (eXogeneous variable) Lecture 3: Elementary Models for LTI Systems 9 / 91

Short overview of commonly used models ARMAX model structure ARX models do not adequately describe disturbances, a moving average (MA) model for the equation error provides more flexibility: y k + a 1 y k − 1 + . . . + a n a y k − n a = b 1 u k + . . . + b n b u k − n b + 1 + e k + c 1 e k − 1 + c n c e k − n c � �� � moving average where e k is white noise. The parameter vector is θ = � � t a 1 . . . a n a b 1 . . . b n b c 1 . . . c n c Now, C ( q ) = 1 + c 1 q − 1 + . . . + c n c q − n c , and y k = B ( q ) A ( q ) u k + C ( q ) A ( q ) e k G ( q ) = B ( q ) A ( q ) H ( q ) = C ( q ) A ( q ) Lecture 3: Elementary Models for LTI Systems 10 / 91

Short overview of commonly used models “output error” (OE) model structure Transfer functions G ( q ) and H ( q ) in ARX and ARMAX models share the same denominator A ( q ) . From a physical point of view, it would more sense to decouple and parameterize the functions separately. In “output error” models we model relation the relation between input and output as a linear difference equation, and assume additive white noise is added: y k = B ( q ) F ( q ) u k + e k Note that this is an ARMAX model where A ( q ) = C ( q ) = F ( q ) Lecture 3: Elementary Models for LTI Systems 11 / 91

Short overview of commonly used models Box-Jenkins model structure Just as ARX models have been extended to ARMAX models by modeling the “equation error” more extensively, so can OE models be extended to y k = B ( q ) F ( q ) u k + C ( q ) D ( q ) e k These are called Box Jenkins models. General model structure A ( q ) y k = B ( q ) F ( q ) u k + C ( q ) D ( q ) e k While this model unifies all previous ones, we often use the more compact special cases, for which specialized and more efficient algorithms exist. Important: ARMAX, OE and Box-Jenkins models are nonlinear in the parameters! Lecture 3: Elementary Models for LTI Systems 12 / 91

AutoRegressive modeling (AR) Autoregressive model for a sequence of scalar data y k : k = 0 , 1 , . . . , N : y k + a 1 y k − 1 + . . . + a n a y k − n a = e k Assumption: sequence e k : k = 0 , 1 , . . . , N is zero mean white noise E [ e k ] = 0 ∀ k E [ e k e l ] = δ kl σ 2 ∀ k , l ◮ AR-model can be considered a whitening filter, converting y k into white noise e k ◮ or y k generated by internal feedback mechanism combined with white noise − a 1 y k − 1 − . . . − a n a y k − n a = y k − e k Lecture 3: Elementary Models for LTI Systems 13 / 91

AR transfer function Transfer function given by (all-pole representation, G ( q ) = 0) 1 1 H ( q ) = A ( q , a ) = 1 + a 1 q − 1 + . . . + a n a q − n a where θ = a . Solve using linear system         yna ena − a 1 yna − 1 yna − 2 y 1 y 0 . . . − a 2 yna + 1 ena + 1 yna yna − 1 y 2 y 1 . . .       . . .  =  −   . . . . .     . . . . . . . − ana − 1 yN − 1 eN − 1 yN − 1 yN − 2 yN − na + 1 yN − na . . . − ana yN eN Two problems ◮ Consistency not guaranteed ⇒ least squares solution ◮ Real time measurements y k allow updates to a ⇒ recursive least squares solution Lecture 3: Elementary Models for LTI Systems 14 / 91

AR error covariance No knowledge of residuals e and noise variance σ 2 known beforehand, but can be estimated using E � e t e � = ( N − n a + 1 ) σ 2 (1) Solve the least squares problem over x in R n � b = Ax � 2 min (2) In the assumption that rank ( A ) = n a , we find x = ( A t A ) − 1 A t b ˆ (3) Estimate of the variance can be obtained from the norm of the residuals σ 2 est = � b − A ( A t A ) − 1 A t b � 2 2 / ( N − n a + 1 ) (4) while the error covariance of the estimate x is given by E � x ) t � = σ 2 ( A t A ) − 1 ( x − ˆ x )( x − ˆ (5) Lecture 3: Elementary Models for LTI Systems 15 / 91

AR modeling matlab demo How to interpret vector of residuals e ? Vector of residuals e considered as a perturbation of the right hand side may not be interpreted as measurement noise: noise data y k in the right hand side side also appears as noisefree data in the matrix A :       yna ¯ − a 1 ¯ yna − 1 yna − 2 ¯ y 1 ¯ ¯ y 0 . . . ¯ − a 2 yna + 1 ¯ yna yna − 1 ¯ y 2 ¯ y 1 ¯ . . .     .  = .   . . . .    . . . . . . − ana − 1 ¯ yN − 1 yN − 1 ¯ ¯ yN − 2 yN − na + 1 ¯ yN − na ¯ . . . − ana ¯ yN The residuals should be considered as an ‘input’ to the system instead, consider two signals described by 1. Data y k are observed exactly without noise but are generated by driving a white noise sequence through an all-pole system y k + a 1 y k − 1 + . . . + a n a y k − n a = e k 2. Data y k are generated by an autonomous linear deterministic system but observed with measurement errors e k y k + a 1 y k − 1 + . . . + a n a y k − n a = 0 ¯ y k = y k + e k Lecture 3: Elementary Models for LTI Systems 16 / 91

AR modeling matlab demo 5 0 −5 0 50 100 150 200 250 300 5 0 −5 0 50 100 150 200 250 300 Lecture 3: Elementary Models for LTI Systems 17 / 91