Optimal Rank-1 Hankel Approximation of Matrices Gerlind Plonka - - PowerPoint PPT Presentation

Optimal Rank-1 Hankel Approximation of Matrices

Gerlind Plonka

University of Göttingen

CodEx Seminar November 3, 2020

Gerlind Plonka (University of Göttingen) Rank-1 Hankel Approximation CodEx Seminar 2020 1 / 31

Outline

Introduction: Low Rank Hankel Approximation

◮ Known Solution Strategies ◮ Applications of Low Rank Hankel Matrices ◮ Low Rank Hankel Operators

Rank-1 Hankel Matrices Optimal Rank-1 Hankel Approximation in the Frobenius Norm Optimal Rank-1 Hankel Approximation in the Spectral Norm Comparison to Cadzow’s Algorithm Collaborations Hanna Knirsch Markus Petz

Gerlind Plonka (University of Göttingen) Rank-1 Hankel Approximation CodEx Seminar 2020 2 / 31

Before We Start: Theorem (Eckhart-Young-Mirsky Theorem)

Let A ∈ CM×N with M ≥ N and let A = U D V∗ be the singular value decomposition (SVD) of A, where U = (u0, u1, . . . , uM−1) ∈ CM×M and V = (v0, v1, . . . , vN−1) ∈ CN×N are unitary matrices and D ∈ CM×N is a diagonal matrix with entries σ0 ≥ σ1 ≥ . . . ≥ σN−1 ≥ 0. Then for r ≤ rank A, Ar =

r−1

• i=0

σi ui vT

i

is the optimal rank-r approximation of A, and we have A − Ar2 = σr, A − Ar2

F = N−1

• i=r

σ2

i .

Gerlind Plonka (University of Göttingen) Rank-1 Hankel Approximation CodEx Seminar 2020 3 / 31

Optimal low-rank Hankel approximation

Let A ∈ CM×N. Hankel matrices are of the form

H := (hℓ+k)M−1,N−1

k,ℓ=0

=        h0 h1 h2 · · · hN−1 h1 h2 hN h2 . . . . . . hM−1 hM hM+1 · · · hM+N−2        ∈ CM×N.

We want to solve min

Hr∈CM×N A − Hr2 F

• r

min

Hr∈CM×N A − Hr2 2,

under the restriction that Hr is a Hankel matrix of rank r.

Gerlind Plonka (University of Göttingen) Rank-1 Hankel Approximation CodEx Seminar 2020 4 / 31

Low-rank Hankel Approximation: Solution Strategies

1 Heuristic approaches: Cadzow Algorithm

Cadzow ’88, Chu et al. ’03, . . .

2 Consider the problem as a nonlinear structured least squares

problem (NSLSP)

de Moor ’93, Lemmerling et al. ’00, ’01, Gillard et al. ’11, Ishteva et al. ’14, Markovsky ’05, ’08, ’18, Usevich et al ’14, . . .

3 Consider the problem as a nonlinear eigenvalue problem

Bresler et al. ’86, Osborne et al. 95, Zhang et al. ’19

4 Relaxation of the optimization problem

Fazel et al. ’13, Grussler et al. ’18, Andersson et al. ’19

5 Application of subspace methods

Van Overschee et al. ’96, Liu et al. ’09

6 Application of AAK-Theory from complex analysis, transfer from

infinite Hankel matrices to finite Hankel matrices

Beylkin et al. ’05, ’10, Andersson et al. ’11, Plonka & Pototskaia ’16, ’19

7 Algebraic methods Ottaviani ’14 Gerlind Plonka (University of Göttingen) Rank-1 Hankel Approximation CodEx Seminar 2020 5 / 31

Low Rank Hankel Approximation

Non-convex optimization problem: For a given matrix A ∈ CM×N and r < min{M, N}, find Hr := argmin

H Hankel rank H≤r

A − H, Numerous applications: linear system theory (minimal partial realizations) system identification problems approximation with finite rate of innovation signals Prony’s method for parameter estimation and approximation Singular Spectrum Analysis (SSA)

Gerlind Plonka (University of Göttingen) Rank-1 Hankel Approximation CodEx Seminar 2020 6 / 31

Application 1: Parameter identification

Assume, we have the signal structure f (x) =

M

• j=1

cj eαjx We have f (ℓ), ℓ = 0, . . . , L, L ≥ 2M − 1. We want M, cj ∈ C \ {0}, eαj ∈ C, j = 1, . . . , M.

Gerlind Plonka (University of Göttingen) Rank-1 Hankel Approximation CodEx Seminar 2020 7 / 31

Application 1: Parameter identification

Assume, we have the signal structure f (x) =

M

• j=1

cj eαjx We have f (ℓ), ℓ = 0, . . . , L, L ≥ 2M − 1. We want M, cj ∈ C \ {0}, eαj ∈ C, j = 1, . . . , M. Consider the Prony polynomial P(z) :=

M

• j=1

(z − eαj) =

M

• ℓ=0

pℓ zℓ with unknown parameters αj and pM = 1.

M

• ℓ=0

pℓf (ℓ + m) =

M

• ℓ=0

pℓ

M

• j=1

cjeαj(ℓ+m) =

M

• j=1

cj eαjm M

• ℓ=0

pℓ eαjℓ =

M

• j=1

cjeαjmP(eαj) = 0, m = 0, . . . , L − M.

Gerlind Plonka (University of Göttingen) Rank-1 Hankel Approximation CodEx Seminar 2020 7 / 31

Reconstruction Algorithm

Input: f (ℓ), ℓ = 0, . . . , L Solve the Hankel system

     

f (0) f (1) . . . f (M) f (1) f (2) . . . f (M + 1) . . . . . . . . . f (L − M) f (M) . . . f (L)

           

p0 p1 . . . pM

     

=

     

. . .

     

Compute the zeros zj = eαj, j = 1, . . . , M of the Prony polynomial P(z) = M

ℓ=0 pℓzℓ.

Compute cj solving the linear system f (ℓ) =

M

• j=1

cjeαjℓ, ℓ = 0, . . . , L. Output: Parameters αj and cj, j = 1, . . . , M.

Gerlind Plonka (University of Göttingen) Rank-1 Hankel Approximation CodEx Seminar 2020 8 / 31

Application 2: Low rank Hankel matrices for regularization

Parallel MRI

1 Calibrationless parallel imaging reconstruction based on structured

low-rank matrix completion (SAKE) (2013) Shin et al.

2 ESPIRiT- An eigenvalue approach to autocalibrating parallel MRI:

Where SENSE meets GRAPPA (2014) Uecker et al.

Gerlind Plonka (University of Göttingen) Rank-1 Hankel Approximation CodEx Seminar 2020 9 / 31

Application 2: Low rank Hankel matrices for regularization

Parallel MRI

1 Calibrationless parallel imaging reconstruction based on structured

low-rank matrix completion (SAKE) (2013) Shin et al.

2 ESPIRiT- An eigenvalue approach to autocalibrating parallel MRI:

Where SENSE meets GRAPPA (2014) Uecker et al. Signal denoising

1 Sparse and low-rank decomposition of a Hankel structured matrix

for impulse noise removal (ALOHA) (2017) Jin et al.

2 Grid-free localization algorithm using low-rank Hankel matrix for

super-resolution microscopy (2018) Min et al.

3 Low-rank seismic denoising with optimal rank selection for Hankel

matrices (2019) Wang et al.

Gerlind Plonka (University of Göttingen) Rank-1 Hankel Approximation CodEx Seminar 2020 9 / 31

Is there hope to solve the problem exactly?

Let (fj)∞

j=0 ∈ ℓ2 be a complex sequence and

Γf :=

     

f0 f1 f2 . . . f1 f2 f3 . . . f2 f3 f4 . . . . . . . . . . . . ...

     

Gerlind Plonka (University of Göttingen) Rank-1 Hankel Approximation CodEx Seminar 2020 10 / 31

Is there hope to solve the problem exactly?

Let (fj)∞

j=0 ∈ ℓ2 be a complex sequence and

Γf :=

     

f0 f1 f2 . . . f1 f2 f3 . . . f2 f3 f4 . . . . . . . . . . . . ...

     

Nehari: Γf is bounded on ℓ2, if there exists a periodic function ψ ∈ L∞([0, 2π)) with Fourier coefficients cj(ψ) = fj, j ≥ 0.

Gerlind Plonka (University of Göttingen) Rank-1 Hankel Approximation CodEx Seminar 2020 10 / 31

Is there hope to solve the problem exactly?

Let (fj)∞

j=0 ∈ ℓ2 be a complex sequence and

Γf :=

     

f0 f1 f2 . . . f1 f2 f3 . . . f2 f3 f4 . . . . . . . . . . . . ...

     

Nehari: Γf is bounded on ℓ2, if there exists a periodic function ψ ∈ L∞([0, 2π)) with Fourier coefficients cj(ψ) = fj, j ≥ 0. Peller: Γf has finite rank N if and only if f (z) :=

∞

• j=0

fjzj defines a rational function of type (N − 1, N). For example,

∞

• j=0

fj zj =

N

• k=1

ck (1 − zkz) =

∞

• j=0

N

• k=1

ck zj

k zj

Gerlind Plonka (University of Göttingen) Rank-1 Hankel Approximation CodEx Seminar 2020 10 / 31

Is there hope to solve the problem exactly?

Aram, Arov and Krein: Let s0(Γf ) ≥ s1(Γf ) ≥ ... denote the decreasing sequence of singular values of the bounded Γf . Then there exists a Hankel operator ΓN of rank at most N such that Γf − ΓNℓ2→ℓ2 = sN(Γf ).

Gerlind Plonka (University of Göttingen) Rank-1 Hankel Approximation CodEx Seminar 2020 11 / 31

Is there hope to solve the problem exactly?

Aram, Arov and Krein: Let s0(Γf ) ≥ s1(Γf ) ≥ ... denote the decreasing sequence of singular values of the bounded Γf . Then there exists a Hankel operator ΓN of rank at most N such that Γf − ΓNℓ2→ℓ2 = sN(Γf ). If Γf ,M has already finite rank M > N, then we have an algorithm to get the optimal low-rank matrix ΓN satisfying Γf ,M − ΓNℓ2→ℓ2 = sN(Γf ,M).

Beylkin & Monzon (2005), Plonka & Pototskaia (2016,19).

Gerlind Plonka (University of Göttingen) Rank-1 Hankel Approximation CodEx Seminar 2020 11 / 31

Optimal Rank-1 Hankel Approximation

We want to solve min

H1∈CM×N A − H12 F

• r

min

H1∈CM×N A − H12 2,

under the restriction that H1 is a Hankel matrix of rank 1.

Gerlind Plonka (University of Göttingen) Rank-1 Hankel Approximation CodEx Seminar 2020 12 / 31

Characterization of Rank-1 Hankel Matrices Theorem

A complex rank-1 matrix H1 ∈ CM×N with min{M, N} ≥ 2 has Hankel structure if and only if it is of the form H1 = c zM zT

N = c

     

1 z z2 . . . zN−1 z z2 z3 . . . zN . . . . . . zM−1 zM . . . zM+N−3 zM+N−2

     

.

• r

H1 = c eM eT

N = c

• 1
• ,

where c ∈ C \ {0}, z ∈ C, zN(z) = zN := (1, z, z2, . . . , zN−1)T, eM := (0, . . . , 0, 1)T ∈ CM.

Gerlind Plonka (University of Göttingen) Rank-1 Hankel Approximation CodEx Seminar 2020 13 / 31

Optimal Rank-1 Hankel Approximation (Frobenius Norm)

Problem: Solve min

c,z∈C A − c zM zT N2 F.

Gerlind Plonka (University of Göttingen) Rank-1 Hankel Approximation CodEx Seminar 2020 14 / 31

Optimal Rank-1 Hankel Approximation (Frobenius Norm)

Problem: Solve min

c,z∈C A − c zM zT N2 F.

Theorem (Knirsch, Plonka, Petz ’20)

Let A = (ajk)M−1,N−1

j,k=0

∈ CM×N with M, N ≥ 2 and |a0,0| > |aM−1,N−1|. Let rank (A) ≥ 1. Then the optimal rank-1 Hankel approximation ˜ c ˜ zM ˜ zT

N of A is determined by

˜ z := argmax

z∈C

|z∗

M A zN|

zM2 zN2 , ˜ c := ˜ z∗

M A ˜

zN zM2

2 zN2 2

, where the vectors ˜ zM and ˜ zN are defined by ˜ z and ˜ z∗ := ˜ zT. The optimal error is given by A − ˜ c ˜ zM ˜ zT

N2 F = A2 F −

|˜ z∗

MA˜

zN|2 ˜ zM2

2 ˜

zN2

2

.

Gerlind Plonka (University of Göttingen) Rank-1 Hankel Approximation CodEx Seminar 2020 14 / 31

Optimal Rank-1 Hankel Approximation (Frobenius Norm) Theorem (Knirsch, Plonka, Petz ’20)

The optimal rank-1 Hankel approximation error satisfies A − ˜ c ˜ zM˜ zT

N2 F = A2 F − σ2 0 = A2 F − A2 2,

if and only if the two singular vectors u and v of A, corresponding to the largest singular value σ0 (i.e., AA∗u = σ2

0u and A∗Av = σ2 0v) are either

• f the form

v = 1 ˜ zN2 ˜ zN and u = 1 ˜ zM2 ˜ zM, where ˜ z and ˜ c are given by ˜ z := argmax

z∈C

|z∗

MAzN|

zM2 zN2 , ˜ c := ˜ z∗

MA˜

zN zM2

2 zN2 2

,

• r v = eN and u = eM.

Gerlind Plonka (University of Göttingen) Rank-1 Hankel Approximation CodEx Seminar 2020 15 / 31

Optimal Rank-1 Hankel Approximation (Example)

Let A =

• 1

1 1 1 1

T

. We get the optimal Hankel approximations

HFrob = ±0.4469 0.4670 ±0.4881 0.5101 ±0.5331 0.4670 ±0.4881 0.5101 ±0.5331 0.5571 T , HCadzow =

• 1

T .

For the Frobenius norm, we obtain two optimal solutions,

(˜ z, ˜ c) = (1.0451, 0.4469), (˜ z, ˜ c) = (−1.0451, −0.4469)

with A − ˜ c ˜ z ˜ zTF = 1.577594, A − HCadzowF = 2.

Gerlind Plonka (University of Göttingen) Rank-1 Hankel Approximation CodEx Seminar 2020 16 / 31

Optimal Real Rank-1 Hankel Approximation (Frobenius Norm) Theorem (Knirsch, Plonka, Petz ’20)

Let A ∈ RM×N with M, N ≥ 2, |a0,0| > |aM−1,N−1|, and rank (A) ≥ 1. If H1 = ˜ c zMzT

N is an optimal rank-1 Hankel approximation of A, then

a′(˜ z) p(˜ z) − a(˜ z) p′(˜ z) = 0, with a(z) := zT

MAzN = M−1

• j=0

N−1

• k=0

ajkzj+k, p(z) := zM2 zN2 =

M−1

• k=0

z2k1/2 N−1

• k=0

z2k1/2 ≥ 1. Here, a′(z) and p′(z) denote the first derivatives of a(z) and p(z).

Gerlind Plonka (University of Göttingen) Rank-1 Hankel Approximation CodEx Seminar 2020 17 / 31

Optimal Real Rank-1 Hankel Approximation (Frobenius Norm) Theorem (Knirsch, Plonka, Petz ’20)

Let A ∈ RN×N and a(z) := zT

MAzN = 2N−2

• ℓ=0

aℓ zℓ with aℓ ≥ 0 ∀ ℓ = 0, . . . , 2N − 2. Assume that the two sequences (a2ℓ)N−1

ℓ=0 and (a2ℓ+1)N−2 ℓ=0 are

monotonically decreasing with a0 > a2N−2 and a1 > 0. Then, ˜ z determining the optimal (real) rank-1 Hankel approximation of A satisfies ˜ z := argmax

z∈R

(zT

NAzN)2

zN4

2

= argmax

z∈R

a(z)

p(z)

2

∈ (0, 1). Moreover, ˜ z is the only non-negative zero of Q(z) := a′(z) p(z) − a(z) p′(z), where p(z) = zT

NzN = N−1

• ℓ=0

z2ℓ.

Gerlind Plonka (University of Göttingen) Rank-1 Hankel Approximation CodEx Seminar 2020 18 / 31

Optimal Real Rank-1 Hankel Approximation (Frobenius Norm)

Can we assume that real matrices A can be optimally approximated by real rank-1 Hankel matrices?

Gerlind Plonka (University of Göttingen) Rank-1 Hankel Approximation CodEx Seminar 2020 19 / 31

Optimal Real Rank-1 Hankel Approximation (Frobenius Norm)

Can we assume that real matrices A can be optimally approximated by real rank-1 Hankel matrices? No! Let A =

  

1 −1

2

−1 −1

2

−1 −1

2

−1 −1

2

1

  

with eigenvalues 2.0, −1.3660, 0.3660, and AF = 2.449490. Optimal real parameters are given by (˜ z, ˜ c) = (−0.1291, 1.0458)

• r

(˜ z, ˜ c) = (−7.7438, 0.0003). Then A − ˜ c ˜ z ˜ zTF = 2.2066 (for both solutions).

Gerlind Plonka (University of Göttingen) Rank-1 Hankel Approximation CodEx Seminar 2020 19 / 31

Optimal Real Rank-1 Hankel Approximation (Frobenius Norm)

Can we assume that real matrices A can be optimally approximated by real rank-1 Hankel matrices? No! Let A =

  

1 −1

2

−1 −1

2

−1 −1

2

−1 −1

2

1

  

with eigenvalues 2.0, −1.3660, 0.3660, and AF = 2.449490. Optimal real parameters are given by (˜ z, ˜ c) = (−0.1291, 1.0458)

• r

(˜ z, ˜ c) = (−7.7438, 0.0003). Then A − ˜ c ˜ z ˜ zTF = 2.2066 (for both solutions). If we allow ˜ c and z to be complex, we obtain with (˜ z, ˜ c) = (i, 5

9) as well

as with (˜ z, ˜ c) = (−i, 5

9) the error A − ˜

c ˜ z ˜ zTF =

√ 261 9

= 1.7950.

Gerlind Plonka (University of Göttingen) Rank-1 Hankel Approximation CodEx Seminar 2020 19 / 31

Optimal Real Rank-1 Hankel Approximation (Frobenius Norm)

Can we assume that real matrices A can be optimally approximated by real rank-1 Hankel matrices? No! Let A =

  

1 −1

2

−1 −1

2

−1 −1

2

−1 −1

2

1

  

with eigenvalues 2.0, −1.3660, 0.3660, and AF = 2.449490. Optimal real parameters are given by (˜ z, ˜ c) = (−0.1291, 1.0458)

• r

(˜ z, ˜ c) = (−7.7438, 0.0003). Then A − ˜ c ˜ z ˜ zTF = 2.2066 (for both solutions). If we allow ˜ c and z to be complex, we obtain with (˜ z, ˜ c) = (i, 5

9) as well

as with (˜ z, ˜ c) = (−i, 5

9) the error A − ˜

c ˜ z ˜ zTF =

√ 261 9

= 1.7950. The Cadzow Algorithm completely fails!

Gerlind Plonka (University of Göttingen) Rank-1 Hankel Approximation CodEx Seminar 2020 19 / 31

Optimal Real Rank-1 Hankel Approximation (Spectral Norm)

Let A ∈ RN×N be symmetric with eigenvalues ordered by modulus λ0 = |λ0| > |λ1| ≥ |λ2| . . . ≥ |λN−1| and corresponding orthonormal basis of real eigenvectors {v0, . . . , vN−1}. Let z := zN = (1, z, . . . , zN−1)T for z ∈ R. Consider f (z, λ2) := 1 zTz

• k∈Sz

(vT

k z)2

λ2

k − λ2 = zT(A2 − λ2I)−1z

zTz , where Sz := {k : vT

k z = 0} ⊂ {0, . . . , N − 1}.

Then f (z, λ2) is well-defined for λ2 ∈ (λ2

1, λ2 0).

f (z, λ2

1) :=

lim

λ2→+λ2

1

f (z, λ2) is bounded if vT

k z = 0 for all k with λ2 k = λ2 1.

Gerlind Plonka (University of Göttingen) Rank-1 Hankel Approximation CodEx Seminar 2020 20 / 31

Optimal Real Rank-1 Hankel Approximation (Spectral Norm) Theorem (Knirsch, Plonka, Petz ’20 (special case in Antoulas ’97))

Let A = (ajk)N−1,N−1

j,k=0

∈ RN×N be symmetric with N ≥ 2. Assume that rank (A) ≥ 1 and λ0 = A2 > |λ1|. Then the optimal rank-1 Hankel approximation error A − ˜ c ˜ z ˜ zT2

2 = λ2 1

is achieved if and only if there exists ˜ z ∈ R such the vector ˜ z satisfies vT

k ˜

z = 0, for all k with |λk| = |λ1| and f (˜ z, λ2

1) ≥ 0,

and if ˜ c is chosen such that

• k∈S

(vT

k ˜

z)2 λk + |λ1| ≤ 1 ˜ c ≤

• k∈S

(vT

k ˜

z)2 λk − |λ1|, with S = {k : vT

k ˜

z = 0} ⊂ {0, . . . , N − 1}.

Gerlind Plonka (University of Göttingen) Rank-1 Hankel Approximation CodEx Seminar 2020 21 / 31

Optimal Real Rank-1 Hankel Approximation (Spectral Norm) Theorem (Knirsch, Plonka, Petz ’20)

Let A = (ajk)N−1,N−1

j,k=0

∈ RN×N be symmetric with N ≥ 2. Assume that rank (A) ≥ 1 and λ0 = A2 > |λ1|. If the condition of the last theorem is not satisfied, then the optimal rank-1 Hankel approximation of A possesses the error ˜ λ := A − ˜ c ˜ z ˜ zT2 ∈ (|λ1|, λ0), where ˜ λ is the minimal number in (|λ1|, λ0) satisfying the relation f (˜ z, ˜ λ2) = max

z∈R f (z, ˜

λ2) = 0 and then ˜ z := argmax

z∈R

f (z, ˜ λ2). Further, ˜ c :=

N−1

• k=0

(vT

k ˜

z)2 λk − ˜ λ

−1

=

• ˜

zT A − ˜ λI

−1 ˜

z

−1

> 0.

Gerlind Plonka (University of Göttingen) Rank-1 Hankel Approximation CodEx Seminar 2020 22 / 31

Optimal Rank-1 Hankel Approximation (Example)

Let A :=

    

3 2 1 1 2 1 1 2 1 1 2 5 1 2 5 2

    

with eigenvalues λ0 = 8.4211, λ1 = −3.1551, λ2 = 3.0092, λ3 = −0.2752. Optimal parameters: zFr = 1.2256, cFr = 1.0203 zSp = 1.1431, cSp = 1.5952 Achieved errors: A − cFrzFrzT

FrF

= 4.5685, A − cFr zFrzT

Fr2 = 3.2085,

A − cSpzSpzT

SpF

= 4.9327, A − cSpzSpzT

Sp2 = 3.1595,

A − H1,CadzowF = 4.5748, A − H1,Cadzow2 = 3.2397.

Gerlind Plonka (University of Göttingen) Rank-1 Hankel Approximation CodEx Seminar 2020 23 / 31

Rank-1 Hankel Approximation (Cadzow Algorithm)

Define the operator P : CM×N → CM×N of counter diagonal averaging: For A = (ak,ℓ)M,N−1

k,ℓ=0

let P(A) := (hℓ+k)M,N−1

k,ℓ=0

∈ CM×N by hℓ :=

                

1 ℓ+1 ℓ

• r=0

ar,ℓ−r for ℓ = 0, . . . , M − 1,

1 M M−1

• r=0

ar,ℓ−r for ℓ = M, . . . , N − 1,

1 M+N−1−ℓ M−1

• r=ℓ+1−N

ar,ℓ−r for ℓ = N, . . . , N + M − 2.

Gerlind Plonka (University of Göttingen) Rank-1 Hankel Approximation CodEx Seminar 2020 24 / 31

Rank-1 Hankel Approximation (Cadzow Algorithm)

Example: A =

  

1 −1 2 1 3 2 6 −1 8 −2 3

  

⇒ P(A) =

  

1 1 1 3 1 1 3 2 1 3 2 3

  

Gerlind Plonka (University of Göttingen) Rank-1 Hankel Approximation CodEx Seminar 2020 25 / 31

Rank-1 Hankel Approximation (Cadzow Algorithm)

Example: A =

  

1 −1 2 1 3 2 6 −1 8 −2 3

  

⇒ P(A) =

  

1 1 1 3 1 1 3 2 1 3 2 3

  

Lemma

For a ∈ CM, b ∈ CN we have P(a b∗)2 ≤ P(a b∗)F ≤ a b∗F = a b∗2 = a2 b2, and equality only holds iff a = zM and b = zN for some z ∈ C or a = eM and b = eN.

Gerlind Plonka (University of Göttingen) Rank-1 Hankel Approximation CodEx Seminar 2020 25 / 31

Rank-1 Hankel Approximation (Cadzow Algorithm)

Cadzow Algoritm: Input: A ∈ CM×N with rank A ≥ 1 and single largest singular value.

1 Compute best rank-1 approximation A0 := σ0 u0 v∗

0.

2 Iterate 1

˜ Aj := P(Aj−1)

2

Compute best rank-1 approximation Aj := σj uj v∗

j

Output: u := lim

j→∞ uj, v := lim j→∞ vj, σ := lim j→∞ σj (hopefully!)

Obviously, the Cadzow algorithm is an alternating projection algorithm. But the underlying optimization problem is not convex!

Gerlind Plonka (University of Göttingen) Rank-1 Hankel Approximation CodEx Seminar 2020 26 / 31

Rank-1 Hankel Approximation (Cadzow Algorithm) Theorem (Knirsch, Plonka, Petz ’20)

Let A ∈ CM×N with M, N ≥ 2 and rank (A) ≥ 1. Then the sequences (uj)∞

j=0, (vj)∞ j=0 and (σj)∞ j=0 in the Cadzow algorithm converge.

If σ = lim

j→∞ σj > 0, then the Cadzow algorithm provides an approximation

σ u v∗ of Hankel structure, i.e., there exists a z ∈ C such that u := lim

j→∞ uj =

1 zM2 zM and v := lim

j→∞ vj =

1 zN2 zN,

• r u = eM, v = eN.

If σ = lim

j→∞ σj = 0, then Cadzow’s algorithm converges to the zero

matrix, while the matrix u v∗ generated u := lim

j→∞ uj and v := lim j→∞ vj

may not be a Hankel matrix.

Gerlind Plonka (University of Göttingen) Rank-1 Hankel Approximation CodEx Seminar 2020 27 / 31

Rank-1 Hankel Approximation (Cadzow Algorithm, Example)

Let A :=

  

1 1/2 1/2 1/2 1

  

with eigenvalues 3/2, 1/2, 1/2. We find u0 = v0 =

1 √ 2 (1, 0, 1)T and

P(3 2 u0 v∗

0) = 3

4

  

1 2/3 2/3 2/3 1

  

with eigenvalues 5/4, 1/2, 1/4. Further iterations yield uj = vj =

1 √ 2(1, 0, 1)T,

σj = 3

2 ·

• 5

6

j.

Gerlind Plonka (University of Göttingen) Rank-1 Hankel Approximation CodEx Seminar 2020 28 / 31

Rank-1 Hankel Approximation (Cadzow Algorithm, Example)

Let A :=

  

1 1/2 1/2 1/2 1

  

with eigenvalues 3/2, 1/2, 1/2. We find u0 = v0 =

1 √ 2 (1, 0, 1)T and

P(3 2 u0 v∗

0) = 3

4

  

1 2/3 2/3 2/3 1

  

with eigenvalues 5/4, 1/2, 1/4. Further iterations yield uj = vj =

1 √ 2(1, 0, 1)T,

σj = 3

2 ·

• 5

6

j.

Thus, (uj)∞

j=0 and (vj)∞ j=0 are constant sequences, and lim j→∞ σj = 0.

Thus, the Cadzow algorithm completely fails!

Gerlind Plonka (University of Göttingen) Rank-1 Hankel Approximation CodEx Seminar 2020 28 / 31

Rank-1 Hankel Approximation (Cadzow Algorithm, Example)

Comparison: For A :=

  

1 1/2 1/2 1/2 1

  

with eigenvalues 3/2, 1/2, 1/2. we find two optimal solutions with respect to Frobenius and spectral norm (zFr, cFr) = (1, 7/18), (zFr, cFr) = (−1, 7/18) (zSp, cSp) = (1, 2/3), (zSp, cSp) = (−1, 2/3) with errors A − cFrzFrzT

FrF

= 1.3889, A − cFr zFrzT

Fr2 = 1.0458,

A − cSpzSpzT

SpF

= 1.4434, A − cSpzSpzT

Sp2 = 0.9574.

Gerlind Plonka (University of Göttingen) Rank-1 Hankel Approximation CodEx Seminar 2020 29 / 31

Summary

We have new numerical methods to compute the optimal rank-1 Hankel approximation of a given matrix A ∈ CM×N with regard to the Frobenius norm, and for symmetric square matrices with regard to the spectral norm. The optimal solutions with respect to the two norms usually not coincide! The rank-1 Hankel approximation may not be unique but the achieved error is unique. Cadzow’s algorithm always converges for rank-1 Hankel approximation. But: It usually converges neither to the optimal solution with regard to the Frobenius nor the spectral norm. The solutions for the three considered algorithms only coincide in the trivial case when the singular vectors to the largest singular value of A already provide a rank-1 Hankel matrix.

Gerlind Plonka (University of Göttingen) Rank-1 Hankel Approximation CodEx Seminar 2020 30 / 31

Paper

• H. Knirsch, G. Plonka, M. Petz:

Optimal Rank-1 Hankel Approximation of Matrices: Frobenius Norm and Spectral Norm and Cadzow’s Algorithm. preprint, arXiv:2004.11099.

Gerlind Plonka (University of Göttingen) Rank-1 Hankel Approximation CodEx Seminar 2020 31 / 31