[PPT] - Order Recursion Introduction Introduction Rc = d Order versus Time PowerPoint Presentation

SLIDE 1

Order Updates Rmcm = dm

values

efficiently solve for cm+1?

notation

Order Recursion

Time Updates R(n)c(n) = d(n)

segment, x(n) for n ∈ {0, 1, . . . , N − 1}

normal equations more efficiently by using the solution for the first N points?

Introduction Rc = d

– Numeric stability – Order of computation – Usefulness of intermediate quantities: partial autocorrelation coefficients – Studying properties of solution: minimum phase,

SLIDE 2

Linear Combiner: Notation In this set of notes and m subscript is used to indicate the size of matrices and vectors Rm+1 = E

xm+1 xH

x∗

Rm rbm rH

ρbm

xm xm+1

dm dm+1

rbm E

Scope of Order Updates Rmcm = dm

– Any linear estimation problem – Optimum FIR filters – Optimum FIR filters for stationary processes

no more efficient computationally

Inversion of Partitioned Hermitian Matrices

so that we can solve the normal equations Rm+1cm+1 = dm+1

Qm+1 R−1

Qm qm qH

qm

Rm+1Qm+1 = Rm rbm rH

ρbm Qm qm qH

qm

Im 0m 0H

1

rH

Rmqm + rbmqm = 0m rH

Stationary Case Rc = d

– Invert the matrix in O(M 2) operations – Perform an LDLH decomposition in O(M 2) operations

linear combiner cases

SLIDE 3

Matrix Inversion by Partitioning Lemma qm = 1 αbm qm = bm αbm Qm = R−1

αbm We can then express R−1

R−1

Qm qm qH

qm

⎡ ⎣

m

m

⎤ ⎦ = R−1

0m 0H

1 αbm bm 1 bH

1

Inversion of Partitioned Hermitian Matrices Continued RmQm + rbmqH

rH

Rmqm + rbmqm = 0m rH

We only have three unknowns, so only three of these equations are needed to obtain the solutions. qm = −R−1

qm = 1 ρbm − rH

qm = −R−1

ρbm − rH

Qm = R−1

ρbm − rH

Matrix Inversion Counterpart Following similar steps, we can show R−1

rH

rfm Rfm −1 = 0H

0m R−1

1 αfm

am 1 aH

am −R−1

αfm ρfm + rH

Inversion of Partitioned Hermitian Matrices Continued qm = 1 ρbm − rH

qm = −R−1

ρbm − rH

Qm = R−1

ρbm − rH

Now if we define bm −R−1

αbm ρbm − rH

The we can simplify the expressions for the components of Qm+1 as qm = 1 αbm qm = bm αbm Qm = R−1

αbm

SLIDE 4

Order Updates of Parameter Vectors cm+1 = R−1

= R−1

0m 0H

dm dm+1

1 αbm

1 bH

1 dm dm+1

1 bH

αbm =

1

where kcm βcm αbm βcm bH

Here the c subscript is presumably an indicator that these coefficients are for updating the parameter vector c

Interpretations ebm = xm+1 − ˆ xm+1 = xm+1 + bH

x2 . . . xm T efm = x1 − ˆ x1 = x1 + aH

x2 x3 . . . xm+1 T and Pbm = ρbm − rH

Pfm = ρfm − rH

xm+1 from the first m elements

from the last m elements

x(n) x(n − 1) . . . x(n − m) , then these are the forward (FLP) and backward (BLP) linear predictors

Order Updates of Parameter Vectors Discussion bm = −R−1

αbm = ρbm + rH

βcm bH

kcm βcm αbm cm+1 = cm

bm 1

R−1

R−1

0m 0H

1 αbm bm 1 bH

1

requires O(m2) operations for the update to cm+1

for cm+1 = R−1

Hermitian Inversion Summary bm = −R−1

αbm = ρbm + rH

R−1

R−1

0m 0H

1 αbm

1 bH

1

inverse of Rm with a rank one update

every m of interest

SLIDE 5

Stationary Case: Parameter Updates cm+1 =

1

kcm βcm Pbm bm = −R−1

βcm bH

= (−R−1

= −rH

= −rH

bm and requires O(m2) operations

Stationary Case: Notation When x(n) and y(n) are jointly stationary and xm+1(n) =

x(n − 1) . . . x(n − m)

r(2) . . . r(m) T and Rm+1 is Toeplitz Rm+1 = ⎡ ⎢ ⎢ ⎢ ⎣ r(0) . . . r(m − 1) r(m) . . . ... . . . . . . r∗(m − 1) . . . r(0) r(1) r∗(m) . . . r∗(1) r(0) ⎤ ⎥ ⎥ ⎥ ⎦ =

rT

r∗

Rm

rH

rfm Rm

Rm Jrm rH

r(0)

Rm rbm rH

ρbm

dm+1

Stationary Case: The Trick Recall that in the stationary case the FLP and BLP are related bm = Ja∗

Pm Pbm = Pfm Hence, we can trivially determine bm from am and vice versa Suppose now that we solve for the FLP coefficients. In this case y(n) = x(n + 1) cm = am dm = r(1) . . . r(m − 1)T = rm The recursive expression for the optimal FLP coefficients is then am+1 =

1

1

where km − βm Pm βm bH

Cool!

Stationary Case: Autocorrelation Matrix Inversion From the matrix inversion by partitioning lemma, we have R−1

R−1

0m 0H

1 Pbm

1 bH

1

= −R−1

Pbm = αbm = ρbm + rH

= r(0) + rH

= Pfm

SLIDE 6

Levinson Algorithm Given {r(ℓ)}M

P−1 = r(0) β−1 = 0 k∗

a0 = ∅ For m = 0, 1, . . . , M − 1 rm = r(1) r(2) . . . r(m)T βm = aT

Pm = Pm−1 + βm−1k∗

km = − βm Pm am+1 =

1

βcm = −cH

kcm = βcm Pm cm+1 =

1

Pc(m+1) = Pcm + βcmk∗

See Table 7.2 in text for more orderly presentation of the algorithm.

Stationary Case: BLP/FLP MMSE Pm = r(0) + rH

Pm+1 = r(0) + rH

= r(0) +

r∗(m + 1) am

1

∗ = r(0) + rH

= Pm + βmk∗

= Pm + β∗

= Pm + (−kmPm)∗km = Pm(1 − |km|2) The BLP MMSE therefore is only decreased as we increase m in the stationary case

Levinson and Levinson-Durbin Algorithms

simplified

(see text for details)

coefficients under the same conditions

Stationary Case: FIR Filter MMSE Similarly, kcm βcm Pm Pc(m+1) = Pcm − β∗

Pm ≤ Pcm Thus the MMSE of the FIR filter can also only decrease as m increases!

SLIDE 7

Remaining

Partial Correlation pc(y, xm) = E

x(n − m)) (y(n) − ˆ y(n))∗ = E [ebm(n)e∗

= bmdm + dm+1 = βcm

between x(n − m) and y(n) after the “influence” of the intermediate variables {x(n), x(n − 1), . . . , x(n − m + 1)} has been removed

calculate the correlation of the residuals

Equivalent Solutions for Optimum Linear Prediction

predictors of a stationary process – Direct-form filter structure: {PM, a1, a2, . . . , aM} – Lattice filter structure: {PM, k0, k1, . . . , kM−1} – Autocorrelation sequence: {r(0), r(1), . . . , r(M)}