Analytic Interpolation on the Unit Disk History, Recent - - PowerPoint PPT Presentation

Analytic Interpolation on the Unit Disk

History, Recent Developments, and Applications

Hendra Ishwara Nurdin Department of Information Engineering, Research School of Information Sciences and Engineering (RSISE), Australian National University, Canberra ACT 0200, Australia

Analytic Interpolation on the Unit Disk – p. 1/39

Contents

• Orthogonal polynomials
• The Carathèodory and Nevanlinna-Pick

interpolation problem

• The Schur algorithm
• Rational interpolation with a degree bound
• The Kimura-Georgiou parametrization

Analytic Interpolation on the Unit Disk – p. 2/39

Contents (cont’d)

• Parametrization of all bounded degree

solutions

• Computing bounded degree solutions
• Some engineering applications: Circuit

theory, spectral estimation, and spectral factorization

Analytic Interpolation on the Unit Disk – p. 3/39

Orthogonal Polynomials

• A simple but powerful concept introduced by

Szëgo.

• Let C = {c0, c1, c2, ...} be a bonafide covari-

ance sequence of a stochastic process, i.e. Tn =         c0 c1 c2 . . . cn c1 c0 c1 ... . . . c2 c1 c0 ... c2 . . . ... ... ... c1 cn . . . c2 c1 c0         ≥ 0 for all n

Analytic Interpolation on the Unit Disk – p. 4/39

Orthogonal Polynomials (cont’d)

• {c0, c1, . . .} induces an inner product on the

space of polynomials with complex coefficients: <

u

• i=0

aizi,

v

• j=0

bjzj >=

u

• i=0

v

• j=0

aicj−ibj where c−t = ct.

• Is a norm (semi-norm) if sequence is positive

(non-negative).

Analytic Interpolation on the Unit Disk – p. 5/39

Orthogonal Polynomials (cont’d)

• By Gram-Schmidt, one can construct an
• rthogonal set of polynomials

{Φ0 = 1, Φ1, Φ2, . . .} from the standard basis {1, z, z2, . . .}, called orthogonal polynomials of the first kind.

• The Φi’s satisfy a certain recurrence relation:

Φk(z) = zΦk−1 − rkΦk−1(z)# k = 1, 2, ... where Φi(z)#, the reversed polynomial, is defined by Φi(z)# = zdeg(Φi)Φi(¯ z−1).

Analytic Interpolation on the Unit Disk – p. 6/39

Orthogonal Polynomials (cont’d)

• {r1, r2, . . . } are called reflection or Schur

coefficients, they satisfy |ri| ≤ 1.

• If C is positive then |ri| < 1 ∀i ≥ 1, while if C

is non-negative but not positive then |ri| < 1 for i = 1, . . . , s − 1 and |rs| = 1 with s = min{k||Tk| = 0}.

• Bijectivity exists between positive C’s and

pairs (c0, R) with c0 > 0 and R a sequence with terms having modulus less than 1.

Analytic Interpolation on the Unit Disk – p. 7/39

Orthogonal Polynomials (cont’d)

• Given C then (c0, R) can be computed and

vice versa. It is also true that if C is positive then the first n + 1 terms of C determine c0 and the first n terms of R and vice versa,

• There is also a bijectivity between

non-negative sequences C with s = min{k||Tk| = 0} < ∞ and pairs (c0, Rs) with c0 > 0 and Rs = {r1, . . . , rs} satisfying |rk| < 1 for k = 1, · · · , s − 1 and |rs| = 1. In this case, and in this case alone, c0, c1, . . . , cn has a unique extension which is rational.

Analytic Interpolation on the Unit Disk – p. 8/39

Orthogonal Polynomials (cont’d)

• If |ri| < 1 ∀i ≥ 1 then

1 Φn(z) converges as

n → ∞ to a spectral factor of the spectral density associated with c0, c1, . . ..

Analytic Interpolation on the Unit Disk – p. 9/39

Notation

• Let H(D) denote the class of all functions

holomorphic in D.

• Let C = {f ∈ H(D) : ℜf(z) ≥ 0 ∀z ∈ D} and

let CA

+ denote the subset of C containing

functions which are positive (> 0) on ∂D and with Taylor coefficients in A ⊂ C.

• C is typically known as the Carathèodory

class.

Analytic Interpolation on the Unit Disk – p. 10/39

The General Interpolation Problem

• Let there be given an indexed set

Z = {z1, . . . , zn} ⊂ D and W = {w1, . . . , wn} ⊂ C. Assume the indexing in {z1, . . . , zn} is such that all non-distinct points are ordered consecutively.

• Problem: Given (Z, W) find all f ∈ C such

that: f(zk) = wk if #zk = 1, or f (l)(zk+l) = wk+l l = 0, . . . , m−1 if #zk = m

Analytic Interpolation on the Unit Disk – p. 11/39

The Carathèodory Extension Problem

• Special case of the general problem if

z0 = . . . = zn = 0, w0 = 1

2c0, and wi = ci for

i = 1, . . . , n.

• It is also known as the covariance extension

problem.

• Has a solution if and only if the Toeplitz matrix

Tn = [cj−i]n

i,j=1 ≥ 0 where c−i = ci.

• Extension is unique and is a linear combina-

tion of sinusoids iff Tn is non-negative but singular.

Analytic Interpolation on the Unit Disk – p. 12/39

Nevanlinna-Pick Interpolation

• Special case of the general problem if

z0, . . . , zn are all distinct (i.e. each has multiplicity 1).

• Has a solution if and only if the Pick matrix

P = [wi+wj

1−zizj]n i,j=1 is nonnegative definite.

• Solution is unique and is a linear combination
• f sinusoids iff P is non-negative but singular.

Analytic Interpolation on the Unit Disk – p. 13/39

The Schur Class S

• S is the class of all functions S holomorphic in

D satisfying |S(z)| ≤ 1 ∀z ∈ D.

• There is an LFT which relates an element of

C to an element of S given by: S(z) = 1 2 C(z) − C(0) C(z) + C(0) where S ∈ S while C ∈ C.

• In fact, there is bijectivity between C and S.

Analytic Interpolation on the Unit Disk – p. 14/39

The Schur Algorithm

• A famous algorithm attributed to Schur.
• Idea: A function S0 is in S iff
• 1. |S0(0)|<1 and S1(z) = 1

z S0(z)−S0(0) 1−S0(0)S0(z) is in S, or

• 2. S0 is a constant function of modulus 1.

then repeat the iteration for S1, S2, . . . until Sn is a constant function of modulus one, or keep going forever.

• S1(0), S2(0), . . . are called the Schur coeffi-

cients.

Analytic Interpolation on the Unit Disk – p. 15/39

Solving the Carathèodory Extension Problem

• Translating the result of Schur to C: A function

C0 beginning with 1 + 2c(0)

1 z is in C iff

• 1. |c(0)

1 | < 1 and C1(z) = s0(z)C0(z)−q0(z) −r0(z)C0(z)+p0(z) is in C,

with p0(z) = (1 + z)(1 − c(0)

1 ),

q0(z) = (1 − z)(1 − c(0)

1 ),

r0(z) = (1 − z)(1 + c(0)

1 ), and

s0(z) = (1 + z)(1 + c(0)

1 ), or

• 2. |c(0)

1 | = 1 and C0(z) = 1+c(0)

1 z

1−c(0)

1 z.

Analytic Interpolation on the Unit Disk – p. 16/39

Solving the Carathèodory Extension Problem(cont’d)

• Compute c(i)

1 from i = 1 until i = n or until

some u ≤ n such that c(u)

1

= 1. Then proceed by:

• 1. If |c(i)

1 | < 1 for i = 1, . . . , n, do the backward

iteration: Ci(z) = pi(z)Ci+1(z)+qi(z)

ri(z)Ci+1(z)+si(z) for

i = n, . . . , 1, with Cn+1 is any function in C,

• therwise
• 2. Set Cu(z) = 1+c(u)

1 z

1−c(u)

1 z, proceed as in step (1)

for i = u − 1, . . . , 1.

Analytic Interpolation on the Unit Disk – p. 17/39

Solving the Carathèodory Extension Problem(cont’d)

• The sequence r1, . . . , rn obtained via the
• rthogonal polynomials is exactly the same

as the sequence c(1)

1 , . . . , c(n) 1 , hence also the

name Schur coefficients for r1, . . . , rn!

Analytic Interpolation on the Unit Disk – p. 18/39

Solving the Nevanlinna-Pick Problem

• It is solved by an algorithm akin to (or a

variant of) Schur’s which was independently developed by Pick and Nevanlinna.

• This algorithm is often referred to as the

Nevanlinna-Schur (NS) algorithm.

• NS recursion in S: If fk ∈ S then

fk+1(z) = 1 − zkz z − zk fk(z) − fk(zk) 1 − fk(zk)fk(z) ∈ S

Analytic Interpolation on the Unit Disk – p. 19/39

Introducing a Degree Bound

• Dewilde & Dym in 1981: In a particular

situation one can always construct a rational function in C of (McMillan) degree ≤ n satisfying the general interpolation constraints.

• Q: Is this existence more general? If so, can
• ne parametrize all such solutions in a

system theoretically “meaningful” way?

• Q: How can one explicitly compute all such

rational solutions?

Analytic Interpolation on the Unit Disk – p. 20/39

Introducing a Degree Bound (cont’d)

• Thus investigation into rational solutions of

the covariance extension problem with degree ≤ n started.

• Name of new problem: Rational covariance

extension problem (RCEP).

• The constraint on the degree of the solution

also known as "complexity constraint". Degree = complexity.

Analytic Interpolation on the Unit Disk – p. 21/39

The Kimura-Georgiou Parametrization

• Kimura & Georgiou: All rational covariance

interpolators in H(D) (but not necessarily in C) of degree at most n has the form (with c0 normalized to 1): f(z) = 1 2 Ψn + α1Ψn−1 + ... + αnΨ0 Φn + α1Φn−1 + ... + αnΦ0 with αi ∈ C and {Ψi}i≥1 another set of

• rthogonal polynomials (of the second kind).
• Hence all rational covariance extensions are

proper, but never strictly proper!

Analytic Interpolation on the Unit Disk – p. 22/39

Notation

• Let Q+(n, A) denote the set of all complex

functions of the form: f(z) = a0 +

n

• i=1
• aizi + aiz−i

, with (a0, a1, . . . , an) ∈ R × An and A ⊂ C, satisfying f(z) > 0 ∀z ∈ D.

• Elements of Q+(n, A) are called (symmetric)

pseudopolynomials.

• A topology is induced on Q+(n, A) via the

norm ||f||∞ = max

z∈∂Df(z).

Analytic Interpolation on the Unit Disk – p. 23/39

Parametrization of Bounded Degree Solutions

• Characterizing the set of all αi’s in the

Kimura-Georgiou parametrization such that f is in C proved to be too difficult a task.

• The papers of Georgiou (1983), Byrnes et al

(1995), and Georgiou (1999) proved the important result: There is a bijectivity between elements of Ψ ∈ Q+(n, C) and pairs

• f polynomials (π, λ), each of degree at most

n with roots in Dc, such that f = π

λ solves the

general interpolation problem. Luckily so!

Analytic Interpolation on the Unit Disk – p. 24/39

Computing Bounded Degree Solutions

• Byrnes et al (1998): Given Ψ ∈ Q+(n, R), the

corresponding rational solution of the RCEP uniquely solves the optimization problem (P1): maximize

f∈CR

+

IΨ =

π

• −π

Ψ(eiθ) log Φ(f)(eiθ)dθ subject to 1 2π

π

• −π

Φ(f)(eiθ)e−kiθdθ = ck k = 1, . . . , n

Analytic Interpolation on the Unit Disk – p. 25/39

Computing Bounded Degree Solutions (cont’d)

• (P1) is an infinite dimensional convex optimi-

zation problem.

• The dual problem to (P1) is a finite dimensio-

nal convex optimization problem (D1): minimize

q∈Q−1(Q+(n,R))

JΨ(q) =

π

• −π

Ψ(eiθ) log Q(q)(eiθ)dθ where Q : (q0, . . . , qn) ∈ Rn+1 → 1

2q0 + n

• i=1

qi(zi + z−1).

Analytic Interpolation on the Unit Disk – p. 26/39

Computing Bounded Degree Solutions (cont’d)

• The technique in Byrnes et al (1998) can be

extended to solve the general interpolation problem.

• Consequence: Gone are the days of

unnecessarily high order interpolators!

• However, Byrnes et al (1998) assumes the

unessential restriction that Ψ ∈ Q+(n, R).

Analytic Interpolation on the Unit Disk – p. 27/39

Computing All Bounded Degree Solutions in H∞(D)

• I recently showed: Given Ψ ∈ Q+(n, R), as

long as the corresponding rational solution of the RCEP is in H∞(D), it uniquely solves problem (P2): maximize

f∈CR

+∩H∞(D)

IΨ =

π

• −π

Ψ(eiθ) log Φ(f)(eiθ)dθ subject to 1 2π

π

• −π

Φ(f)(eiθ)e−kiθdθ = ck k = 1, . . . , n

Analytic Interpolation on the Unit Disk – p. 28/39

Computing All Bounded Degree Solutions in H∞(D) (cont’d)

• (P2) is also an infinite-dimensional convex
• ptimization problem. It has the same dual

problem as (P1).

• Interesting new result: Given Ψ ∈ Q+(n, R), a

solution of the RCEP is bounded (i.e. in H∞(D)) iff the minimizer of JΨ is a stationary point (which can lie in the interior or at the boundary).

• Interpretation: The minimizer of JΨ is a

certificate for the boundedness of a solution.

Analytic Interpolation on the Unit Disk – p. 29/39

Computing All Bounded Degree Solutions in H∞(D) (cont’d)

• I proposed a systematic procedure for com-

puting all bounded solutions, which for a given Ψ ∈ Q+(n, R) either:

• 1. Returns the corresponding rational,

bounded solution of the RCEP , or

• 2. Concludes that the rational solution has a

pole on ∂D (i.e. it is unbounded).

• More detail and discussion on these new

results in a (possible) future talk ...

Analytic Interpolation on the Unit Disk – p. 30/39

Generalizations

• Most of the interpolation results and algori-

thms which I have described today have been generalized to the matrix case.

Analytic Interpolation on the Unit Disk – p. 31/39

Application: Circuit Theory

• Lossless inverse scattering (LIS) problem:

S(z) A1(z) B1(z) A1(z) B1(z)

Σ(z)

SL(z)

A2(z) B2(z)

• Given S(z) = A1(z)

B1(z), find Σ(z) and SL(z) to

model S(z)... (more on next slide)

Analytic Interpolation on the Unit Disk – p. 32/39

Application: Circuit Theory (cont’d)

• ... S1(z) is passive, Σ is a rational lossless

passive 2 × 2 scattering matrix with Σ22 = 0, while SL is passive.

• Solved by a matrix NS algorithm, by extrac-

ting a cascade of chain scattering matrices

• ne at a time (Dewilde & Dym).

θ1(z)

θ i(z)

SL(z) A1(z) B1(z) A2(z) Ai(z) Ai+1(z) B2(z) Bi(z) Bi+1(z) AL(z) BL(z)

Analytic Interpolation on the Unit Disk – p. 33/39

Application: Spectral Estimation

• Given samples {Y0, Y1, . . . , YN} of a WSS

process, estimate the spectral density.

• “Modern” method: Compute covariance lag

estimates ˆ ck =

1 N−k+1 N−k

• l=0

YlYl+k for k = 0, ..., n with n << N. Then extend it to a full covari- ance sequence.

• Construct estimate by choosing an “appro-

priate” Ψ and computing the corresponding spectral density.

Analytic Interpolation on the Unit Disk – p. 34/39

Application: Spectral Estimation (cont’d)

• Prior to 1998, only two extensions were

known: the maximum entropy extension (i.e. Ψ = 1), and the (deterministic) Pisarenko extension when the sequence is non-negative but not positive.

• A better Nevanlinna-Pick interpolation based

method has been developed that can give sharp spectral estimates even for low N, called THREE (Tunable High-Resolution Spectral Estimator).

Analytic Interpolation on the Unit Disk – p. 35/39

Application: Spectral Factorization

• The (matrix version of the) Schur and NS

algorithms can produce approximate spectral factors of certain scalar and matrix valued spectral densities (rational and non-rational).

• In the rational case, a right spectral

numerator G of f = ND−1 ∈ Sn×n (N, D are co-prime matrix polynomials) is defined by G∗G = D∗D − N∗N (Note: I − f∗f is a spec- tral density, and f ∈ Sn×n iff 1) f(z) ∈ Cn×n, 2) fij ∈ H(D) and 3) f(z)f(z)∗ ≤ I ∀z ∈ D).

Analytic Interpolation on the Unit Disk – p. 36/39

Application: Spectral Factorization (cont’d)

• Let {zk}k≥0 be such that

∞

• k=0

(1 − |zk|) = ∞. Let f0 = N0D−1 ∈ Sn×n and fk = NkD−1

k

∈ Sn×n for k = 1, 2, . . . be obtained from f0 via a Schur or NS recursion. Then lim

k→∞Dk = UG0,

where U is a constant unitary matrix, and G0 is a right spectral factor of f.

• An approximate spectral factor of I − f∗f is

given by: DkD−1 for some suitably large k.

Analytic Interpolation on the Unit Disk – p. 37/39

Application: Spectral Factorization (cont’d)

• Advantage of the interpolation based spectral

factorization algorithm: Really fast convergen- ce can be obtained by a good choice of inter- polation points z1, z2, . . ..

Analytic Interpolation on the Unit Disk – p. 38/39

The End (Finally! ...) Thank you for listening! Slides are available at: http:\\rsise.anu.edu.au\~hendra

Analytic Interpolation on the Unit Disk – p. 39/39