Designs of Orthogonal Filter Banks and Orthogonal Cosine-Modulated - - PowerPoint PPT Presentation

Designs of Orthogonal Filter Banks and Orthogonal Cosine-Modulated Filter Banks

Jie Yan

Department of Electrical and Computer Engineering University of Victoria

April 16, 2010

1 / 45

OUTLINE

1

INTRODUCTION

2

LS DESIGN OF ORTHOGONAL FILTER BANKS AND WAVELETS

3

MIMINAX DESIGN OF ORTHOGONAL FILTER BANKS AND WAVELETS

4

DESIGN OF ORTHOGONAL COSINE-MODULATED FILTER BANKS

5

CONCLUSIONS AND FUTURE RESEARCH

2 / 45

• 1. INTRODUCTION

A two-channel conjugate quadrature (CQ) filter bank H1(z) = −z−(N−1)H0(−z−1) G0(z) = H1(−z) G1(z) = −H0(−z) where H0(z) = ∑N−1

n=0 hnz−n

0( )

H z

1( )

H z

2 2 2 2

0( )

G z

1( )

G z

}

Analysis Filter Bank

}

Synthesis Filter Bank

3 / 45

Two-Channel Orthogonal Filter Banks Perfect reconstruction (PR) condition

N−1−2m

∑

n=0

hn ⋅ hn+2m = 훿m for m = 0, 1, ..., (N − 2)/2 Vanishing moment (VM) requirement: A CQ filter has L vanishing moments if

N−1

∑

n=0

(−1)n ⋅ nl ⋅ hn = 0 for l = 0, 1, ..., L − 1

4 / 45

Two-Channel Orthogonal Filter Banks Cont’d A least squares (LS) design of CQ lowpass filter H0(z) having L VMs minimize ∫ 휋

휔a

∣H0(ej휔)∣2d휔 subject to: PR condition and VM requirement The LS problem above can be expressed as minimize hTQh subject to:

N−1−2m

∑

n=0

hn ⋅ hn+2m = 훿m for m = 0, 1, ..., (N − 2)/2

N−1

∑

n=0

(−1)n ⋅ nl ⋅ hn = 0 for l = 0, 1, ..., L − 1

5 / 45

Two-Channel Orthogonal Filter Banks Cont’d A minimax design minimizes the maximum instantaneous power

• f H0(z) over its stopband

minimize maximize

휔a≤휔≤휋

∣H0(ej휔)∣ subject to: PR condition and VM requirement The minimax problem can be further cast as minimize 휂 subject to: ∥T(휔) ⋅ h∥ ≤ 휂 for 휔 ∈ Ω

N−1−2m

∑

n=0

hn ⋅ hn+2m = 훿m for m = 0, 1, ..., (N − 2)/2

N−1

∑

n=0

(−1)n ⋅ nl ⋅ hn = 0 for l = 0, 1, ..., L − 1

6 / 45

Orthogonal Cosine-Modulated Filter Banks An orthogonal cosine-modulated (OCM) filter bank hk(n) = 2h(n) cos [ 휋 M ( k + 1 2 ) ( n − D 2 ) + (−1)k 휋 4 ] fk(n) = 2h(n) cos [ 휋 M ( k + 1 2 ) ( n − D 2 ) − (−1)k 휋 4 ] for 0 ≤ k ≤ M − 1 and 0 ≤ n ≤ N − 1

x(n) y(n) H0(z) HM-1(z) M M M M M M

. . . . . .

H1(z) F

0(z)

F

M-1(z)

F

1(z)

+

}

• }
• 7 / 45

Orthogonal Cosine-Modulated Filter Banks Cont’d An M-channel OCM filter bank is uniquely characterized by its prototype filter (PF) The design of the PF of an OCM filter bank can be formulated as minimize ∫ 휋

휔s

∣H0(ej휔)∣2d휔 subject to: PR condition As the PF has linear phase, h is symmetrical. The design problem can be reduced to minimize e2(ˆ h) = ˆ hT ˆ Pˆ h subject to: al,n(ˆ h) = ˆ hT ˆ Ql,nˆ h − cn = 0 for 0 ≤ n ≤ m − 1 and 0 ≤ l ≤ M/2 − 1 where the design variables are reduced by half to ˆ h = [h0 h1 ⋅ ⋅ ⋅ hN/2−1]T.

8 / 45

Overview and Contribution of the Thesis Overview We have formulated three nonconvex optimization problems

LS design of CQ filter banks Minimax design of CQ filter banks Design of OCM filter banks

Contribution of the thesis Several improved local design methods for the three problems Several strategies proposed for potentially GLOBAL solutions of the three problems

9 / 45

Global Design Method at a Glance Multiple local solutions exist for a nonconvex problem Algorithms in finding a locally optimal solution are available Start the local design algorithm from a good initial point How do we secure such a good initial point?

10 / 45

• 2. LS DESIGN OF ORTHOGONAL FILTER BANKS AND WAVELETS

A least squares (LS) design of a conjugate quadrature (CQ) filter

• f length-N with L vanishing moments (VMs) can be cast as

minimize hTQh subject to:

N−1−2m

∑

n=0

hn ⋅ hn+2m = 훿m for m = 0, 1, ..., (N − 2)/2

N−1

∑

n=0

(−1)n ⋅ nl ⋅ hn = 0 for l = 0, 1, ..., L − 1

11 / 45

Local LS Design of CQ Filter Banks An effective direct design method is recently proposed by W.-S. Lu and T. Hinamoto Based on the direct design technique, we develop two local methods

Sequential convex-programming (SCP) method Sequential quadratic-programming (SQP) method

Both methods produce improved local designs than the direct method

12 / 45

Local LS Design of CQ Filter Banks Cont’d Sequential Convex-Programming Method

Suppose we are in the kth iteration to compute 휹h so that hk+1 = hk + 휹h reduces the filter’s stopband energy and better satisfies the constraints, then hT

k+1Qhk+1 = 휹T hQ휹h + 2휹T hQhk + hT k Qhk N−1

∑

n=0

(−1)n ⋅ nl ⋅ (휹h)n = −

N−1

∑

n=0

(−1)n ⋅ nl ⋅ (hk)n

N−1−2m

∑

n=0

(hk)n(휹h)n+2m +

N−1−2m

∑

n=0

(hk)n+2m(휹h)n ≈ 훿m −

N−1−2m

∑

n=0

(hk)n(hk)n+2m

13 / 45

Local LS Design of CQ Filter Banks Cont’d

With h bounded to be small, the kth iteration assumes the form minimize 휹T

hQ휹h + 휹T hgk

subject to: Ak휹h = −ak C휹h ≤ b By using SVD to remove the equality constraint, the problem is reduced to minimize xT ˆ Qx + xT ˆ gk subject to: ˆ Cx ≤ ˆ b We modify the problem to make it always feasible as minimize xT ˆ Qx + xT ˆ gk subject to: Fx ≤ a which is a convex QP problem.

14 / 45

Local LS Design of CQ Filter Banks Cont’d Sequential Quadratic-Programming Method

The design problem is a general nonlinear optimization problem minimize f(h) subject to: ai(h) = 0 for i = 1, 2, ..., p By using the first-order necessary conditions of a local minimizer, the problem can be reduced to minimize 1 2휹T

hWk휹h + 휹T hgk

subject to: Ak휹h = −ak ∣∣휹h∣∣ is small

15 / 45

Local LS Design of CQ Filter Banks Cont’d

where Wk = ∇2

hf(hk) − p

∑

i=1

(흀k)i∇2

hai(hk)

(13a) Ak = [ ∇ha1(hk) ∇ha2(hk) ⋅ ⋅ ⋅ ∇hap(hk) ]T (13b) gk = ∇hf(hk) (13c) ak = [ a1(hk) a2(hk) ⋅ ⋅ ⋅ ap(hk) ]T (13d) By removing the equality constraint using the SVD or QR decomposition, the problem assumes the form of a QP problem. Once the minimizer 휹∗

h is found,

the next iterate is set to hk+1 = hk + 휹∗

h, 흀k+1 = (AkAT k )−1Ak(Wk휹∗ h + gk)

16 / 45

Global LS Design of Low-Order CQ Filter Banks The LS design problem is a polynomial optimization problem (POP) Two recent breakthroughs in solving POPs

Global solutions of POPs are made available by Lasserre’s method Sparse SDP relaxation is proposed for global solutions of POPs of relatively larger scales

MATLAB toolbox SparsePOP and GloptiPoly can be used to find global solutions of POPs, but only for POPs of limited sizes

17 / 45

Global LS Design of Low-Order CQ Filter Banks Cont’d Example: Design a globally optimal LS CQ filter with N = 6, L = 2 and 휔a = 0.56휋 MATLAB toolbox GloptiPoly and SparsePOP are utilized to produce the globally optimal solution h(6,2)

LS

= ⎡ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎣ 0.33268098788629 0.80689591454849 0.45986215652386 −0.13501431772967 −0.08543638600240 0.03522516035714 ⎤ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎦ However, GloptiPoly and SparsePOP fail to work as long as the filter length N is greater than or equal to 18

18 / 45

Global LS Design of High-Order CQ Filter Banks A common pattern shared among globally optimal low-order impulse responses.

0.2 0.4 0.6 0.8 1 −0.2 −0.1 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 N = 6, L = 2 N = 8, L = 2 N = 10, L = 2

19 / 45

Global LS Design of High-Order CQ Filter Banks Cont’d h6: Globally optimal impulse response when N = 6 hzp

8 : Impulse response generated by zero-padding h6

h8: Globally optimal impulse response when N = 8

0.2 0.4 0.6 0.8 1 −0.2 −0.1 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 h6 (N=6, L=2) h8

zp

h8 (N=8, L=2)

Generate initial point by zero-padding!

20 / 45

Global LS Design of High-Order CQ Filter Banks Cont’d Global design strategy in brief:

1

Design a globally optimal CQ filter of short length, say 4, using e.g. GloptiPoly

2

Generating an impulse response for higher order design by zero-padding

3

Apply the SCP or SQP method with the zero-padded impulse response as the initial point to obtain the optimal impulse response of higher order

4

Follow this concept in an iterative way, until desired filter length is reached

21 / 45

Global LS Design of High-Order CQ Filter Banks Cont’d The designs obtained are quite likely to be globally optimal because:

1

Zero-padded initial point sufficiently close to the global minimizer.

2

The local design methods are known to converge to a nearby minimizer.

22 / 45

Design Examples Potentially globally optimal design of an LS CQ filter with N = 96, L = 3 and 휔 = 0.56휋

0.2 0.4 0.6 0.8 1 −120 −100 −80 −60 −40 −20 Normalized frequency

23 / 45

Design Examples Cont’d Comparisons Global design Global design based on SCP Energy in stopband 1.18103e-9 Largest eq. error 1e-14 Local design Local design based on SCP Energy in stopband 3.15564e-9 Largest eq. error 1e-14

24 / 45

Design Examples Cont’d Zero-pole plots

−1 −0.5 0.5 1 −1 −0.8 −0.6 −0.4 −0.2 0.2 0.4 0.6 0.8 1

4 95

Real Part Imaginary Part

−1.5 −1 −0.5 0.5 1 1.5 −1 −0.5 0.5 1

3 95

Real Part Imaginary Part

Global design Local design

The globally optimal LS CQ filter possesses minimum phase

25 / 45

• 3. MIMINAX DESIGN OF ORTHOGONAL FILTER BANKS AND

WAVELETS A minimax design of a conjugate quadrature (CQ) filter of length-N with L vanishing moments (VMs) can be cast as minimize 휂 subject to: ∥T(휔) ⋅ h∥ ≤ 휂 for 휔 ∈ Ω

N−1−2m

∑

n=0

hn ⋅ hn+2m = 훿m for m = 0, 1, ..., (N − 2)/2

N−1

∑

n=0

(−1)n ⋅ nl ⋅ hn = 0 for l = 0, 1, ..., L − 1

26 / 45

Local Minimax Design of CQ Filter Banks Like the LS design, an effective direct design method is recently proposed by W.-S. Lu and T. Hinamoto Based on the direct design technique, we develop an improved method named the SCP-GN method The SCP-GN method can achieve convergence at a small tolerance 휀, by implementing two techniques

1

Constructing Ω by locating magnitude-response peaks

2

A Gauss-Newton method with adaptively controlled weights

27 / 45

Local Minimax Design of CQ Filter Banks Cont’d

In the kth iteration, the problem assumes the form minimize 휂 subject to: ∥T(휔)(hk + 휹h)∥ ≤ 휂 for 휔 ∈ Ω Ak휹h = −ak C휹h ≤ b By using SVD of matrix Ak to remove the equality constraints, the problem can be reduced to minimize 휂 subject to: ∥Tk(휔)x + ek(휔)∥ ≤ 휂 for 휔 ∈ Ω ˆ Cx ≤ ˆ b As a technical remedy to make the above problem to be always feasible, we modify the problem as minimize 휂 subject to: ∥Tk(휔)x + ek(휔)∥ ≤ 휂 for 휔 ∈ Ω Fx ≤ a which is a second-order cone programming (SOCP) problem for which efficient solvers such as SeDuMi exist.

28 / 45

Global Minimax Design of Low-Order CQ Filter Banks Example: Design a globally optimal minimax CQ filter with N = 4, L = 1 and 휔a = 0.56휋 Since the Minimax design problem is a POP , GloptiPoly and SparsePOP can be used to produce the globally optimal solution h(4,1)

minimax =

⎡ ⎢ ⎢ ⎣ 0.48296282173531 0.83651623138234 0.22414405492402 −0.12940935473280 ⎤ ⎥ ⎥ ⎦ However, GloptiPoly and SparsePOP fail to work as long as the filter length N is greater than or equal to 6

29 / 45

Global Minimax Design of High-Order CQ Filter Banks Method 1 Globally optimal minimax impulse responses appear to exhibit a pattern similar to that in the LS case Thus, we proposed method 1 in spirit similar to that utilized in the global LS designs by passing the zero-padded impulse response as the initial point for the SCP-GN local method in each round of iteration Method 2 We simply pass the impulse response of the globally optimal LS filter as an initial point for the SCP-GN method to design an

• ptimal minimax filter with the same design specifications

30 / 45

Design Examples Potentially globally optimal design of a minimax CQ filter with N = 96, L = 3 and 휔 = 0.56휋

0.2 0.4 0.6 0.8 1 −100 −80 −60 −40 −20 Normalized frequency

31 / 45

Design Examples Cont’d Comparisons Global design Global design based on Method 1 Maximum instantaneous energy in stopband 6.75750e-9 Largest eq. error <1e-15 Local design Local design based on SCP-GN Maximum instantaneous energy in stopband 1.81165e-8 Largest eq. error 2.9e-14

32 / 45

Design Examples Cont’d Zero-pole plots

−1 −0.5 0.5 1 −1 −0.8 −0.6 −0.4 −0.2 0.2 0.4 0.6 0.8 1

4 95

Real Part Imaginary Part −1 −0.5 0.5 1 1.5 −1 −0.5 0.5 1

4 95

Real Part Imaginary Part

Global design Local design

The globally optimal minimax CQ filter possesses minimum phase

33 / 45

• 4. DESIGN OF ORTHOGONAL COSINE-MODULATED FILTER

BANKS We have formulated the design of the prototype filter (PF) of an

• rthogonal cosine-modulated (OCM) filter bank as

minimize e2(ˆ h) = ˆ hT ˆ Pˆ h subject to: al,n(ˆ h) = ˆ hT ˆ Ql,nˆ h − cn = 0 for 0 ≤ n ≤ m − 1 and 0 ≤ l ≤ M/2 − 1

34 / 45

Local Design of OCM Filter Banks We improved an effective direct design method proposed by W.-S. Lu, T. Saramäki and R. Bregovi´ c Gauss-Newton method with adaptively controlled weights was applied for the algorithm to converge to a highly accurate solution

35 / 45

Local Design of OCM Filter Banks Cont’d

Suppose we are in the kth iteration to compute 휹 so that ˆ hk+1 = ˆ hk + 휹 reduces the PF’s stopband energy and better satisfies the PR conditions. Then, ˆ hT

k+1ˆ

Pˆ hk+1 = 휹T ˆ P휹 + 2휹T ˆ Pˆ hk + ˆ hT

k ˆ

Pˆ hk al,n(ˆ hk + 휹) ≈ al,n(ˆ hk) + gT

l,n(ˆ

hk)휹 = 0 for 0 ≤ n ≤ m − 1 and 0 ≤ l ≤ M/2 − 1 And the kth iteration assumes the form minimize 휹T ˆ P휹 + 휹Tbk subject to: Gk휹 = −ak ∥휹∥ is small

36 / 45

Local Design of OCM Filter Banks Cont’d

The equality constraint can be eliminated via SVD of Gk = UΣV as 휹 = Ve흓 + 휹s (18) Thus, the problem can be cast as minimize 흓T ˜ Pk흓 + 흓T˜ bk subject to: ∣∣흓∣∣ is small The Gauss-Newton technique with adaptively controlled weights is used as a post-processing step to achieve convergence at a small tolerance.

37 / 45

Global Design of Low-Order OCM Filter Banks Example: Design a globally optimal OCM filter bank with M = 2, m = 1 and 휌 = 1 GloptiPoly and SparsePOP can be used to produce the globally

• ptimal solution

h(2,1) = ⎡ ⎢ ⎢ ⎣ 0.235923416966353 0.440840267366581 0.440840267366581 0.235923416966353 ⎤ ⎥ ⎥ ⎦ The software was found to work only for the following cases: a) M = 2, 1 ≤ m ≤ 5; b) M = 4, 1 ≤ m ≤ 3; c) M = 6, m = 1; d) M = 8, m = 1.

38 / 45

Global Design of High-Order OCM Filter Banks Two observations:

1

For a fixed M, the impulse responses with different m exhibit a similar pattern and are close to each other

2

For m = 1, the impulse responses with different M also exhibit a similar shape.

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 −0.05 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 m=1,M=2 m=2,M=2 m=3,M=2 m=4,M=2 m=1,M=4 m=2,M=4

39 / 45

Global Design of High-Order OCM Filter Banks Cont’d

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 −0.05 0.05 0.1 0.15 0.2 0.25 0.3 0.35 m=1,M=4 h0

zp

m=2,M=4 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5 m=1,M=2 h0

int

m=1,M=4

Effect of zero-padding when M = 4 Effect of linear interpolation when m = 1

40 / 45

Global Design of High-Order OCM Filter Banks Cont’d An improvement in initial point when m = 1, by downshifting hint

0 by a

constant value d computed using the Gauss-Newton method with adaptively controlled weights.

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5 h0

int

h0 h

d

41 / 45

Global Design of High-Order OCM Filter Banks Cont’d An order-recursive algorithm in brief

1

Obtaining a low-order global design;

2

Using zero-padding/linear interpolation in conjuction of the G-N method with adaptively controlled weights of the impulse response to produce a desirable initial point for PF of slightly increased

• rder, and carrying out the design by a locally optimal method;

3

Repeating step 2 until the filter order reaches the targeted value.

42 / 45

Design Examples Design of an OCM filter bank with m = 20, M = 4 and 휌 = 1. Shown below are the impulse responses of the PF from global and local design, respectively.

0.2 0.4 0.6 0.8 1 −160 −140 −120 −100 −80 −60 −40 −20 20

Normalized frequency

0.2 0.4 0.6 0.8 1 −200 −180 −160 −140 −120 −100 −80 −60 −40 −20 20 Normalized frequency

Global design Local design

43 / 45

Design Examples Cont’d Performance comparison for OCM filter banks with m = 20, M = 4 and 휌 = 1 Global design Local design Energy in stopband 8.226e-13 6.585e-10 Largest eq. error 1.839e-15 2.297e-10 By comparing the OCM filter banks reported in the literature, the OCM filter bank designed using our proposed algorithm offers the BEST performance, because it is a globally optimal design.

44 / 45

• 5. CONCLUSIONS AND FUTURE RESEARCH

We have investigated three design problems,

1

LS design of orthogonal filter banks and wavelets

2

Minimax design of orthogonal filter banks and wavelet

3

Design of OCM filter banks

Improved local design methods for the three problems Several strategies proposed for GLOBAL designs of the three design scenarios Future research

Theoretical proof of the global optimality of our proposed method

45 / 45