Constrained optimal discrimination designs for Fourier regression - - PowerPoint PPT Presentation

Title Page ◭◭ ◮◮ ◭ ◮ Page 1 of 37 Go Back Full Screen Close Quit

Constrained optimal discrimination designs for Fourier regression models

• S. Biedermann, School of Mathematics, University of Southampton

joint work with

• H. Dette and P. Hoffmann, Department of Mathematics, Bochum

June, 7th, 2007 mODa 8, June 4-8, 2007, Almagro, Spain

Title Page ◭◭ ◮◮ ◭ ◮ Page 2 of 37 Go Back Full Screen Close Quit

• 0. Contents
• The Fourier model
• The design problem: constrained optimal discrimination

designs

• Canonical moments
• Results

Title Page ◭◭ ◮◮ ◭ ◮ Page 3 of 37 Go Back Full Screen Close Quit

• 1. The Fourier Regression Model

n independent observations Y1, . . . , Yn at x1, . . . , xn where Yi ∼ N(g2d(xi), τ 2) and g2d(x) = a0 +

d

• j=1

aj sin(jx) +

d

• j=1

bj cos(jx), x ∈ [−π, π] a0, . . . , ad, b1, . . . , bd ∈ I R unknown model parameters, d ∈ I N

Title Page ◭◭ ◮◮ ◭ ◮ Page 4 of 37 Go Back Full Screen Close Quit

Used to model periodic phenomena, e.g., in the engineering, physical, biological and medical sciences, and in two-dimensional shape analysis, . . .

Title Page ◭◭ ◮◮ ◭ ◮ Page 5 of 37 Go Back Full Screen Close Quit

• 2. The Design problem

Do we really need the full model? Goal: Model identification Successive F-tests with hypotheses H(2d) : bd = 0, H(2d−1) : ad = 0, H(2d−2) : bd−1 = 0, H(2d−3) : ad−1 = 0, . . . , H(0) : a0 = 0 in the models g2d, g2d−1, . . . , g0 until H(k) is rejected

Title Page ◭◭ ◮◮ ◭ ◮ Page 6 of 37 Go Back Full Screen Close Quit

Is there a way to influence (maximise) the power of these tests? The noncentrality parameter of the test for H(k) is: δk(σ) = (eT

k M −1 k (σ)ek)−1

k = 1, . . . , 2d where ek = (0, 0, . . . , 0, 1)T ∈ I Rk+1 and M −1

k (σ) is the covari-

ance matrix for estimating the full parameter vector in model gk and σ is the design of the experiment.

Title Page ◭◭ ◮◮ ◭ ◮ Page 7 of 37 Go Back Full Screen Close Quit

Example: σ =   x1 x2 . . . xm ω1 ω2 . . . ωm   M2(σ) =

m

• i=1

ωi       1 sin(xi) cos(xi) sin(xi) sin2(xi) sin(xi) cos(xi) cos(xi) sin(xi) cos(xi) cos2(xi)       = π

−π

      1 sin(x) cos(x) sin(x) sin2(x) sin(x) cos(x) cos(x) sin(x) cos(x) cos2(x)       dσ(x)

Title Page ◭◭ ◮◮ ◭ ◮ Page 8 of 37 Go Back Full Screen Close Quit

Recall: The non-centrality parameter δk(σ) (and therefore the power of the F-test for H(k)

0 ) depends on the design σ

For one test only: Maximise δk(σ) with respect to σ ֒ → D(k)

1 -optimality

(Equivalent to optimal design for estimating the highest order parameter in model gk) Problem: It is impossible to maximise all δk(σ)’s simultan- eously

Title Page ◭◭ ◮◮ ◭ ◮ Page 9 of 37 Go Back Full Screen Close Quit

The Constrained Optimality Criterion Define the efficiency of a design σ for discriminating between models gk and gk−1 as effk(σ) := δk(σ) δk(σ∗

k),

k = 1, . . . , 2d where σ∗

k is the D(k) 1 -optimal design.

Assume that testing H(2d) is most important, and assign lower boundaries γk to each efficiency effk(σ) according to the relative importance of the corresponding discrimination problem.

Title Page ◭◭ ◮◮ ◭ ◮ Page 10 of 37 Go Back Full Screen Close Quit

A constrained optimal discriminating design σ∗ maximises eff2d(σ) subject to effk(σ) ≥ γk, k = 2d − 1, 2d − 2, . . . , 2d − 2j − 1 for some j ∈ {0, . . . , d − 1}.

Title Page ◭◭ ◮◮ ◭ ◮ Page 11 of 37 Go Back Full Screen Close Quit

Constrained optimisation problem: ֒ → only numerical solutions possible? ֒ → idea: rewrite criterion in terms of canonical moments to find analytical results

• 1. Show that a symmetric design is optimal
• 2. Transform σ into a design ξσ on [−1, 1] by

ξσ(cos x) =    2σ(x) = 2σ(−x) if 0 < x ≤ π σ(0) if x = 0

• 3. Express matrices Mk(σ) in terms of ξσ

Title Page ◭◭ ◮◮ ◭ ◮ Page 12 of 37 Go Back Full Screen Close Quit

Example: M2(σ) = π

−π

      1 sin(x) cos(x) sin(x) sin2(x) sin(x) cos(x) cos(x) sin(x) cos(x) cos2(x)       dσ(x) = 1

−1

      1 √ 1 − z2 z √ 1 − z2 1 − z2 z √ 1 − z2 z z √ 1 − z2 z2       dξσ(z) Mk is now “almost” a moment matrix, and the efficiencies effk

• f matrices of such a form can be expressed in terms of canonical

moments.

Title Page ◭◭ ◮◮ ◭ ◮ Page 13 of 37 Go Back Full Screen Close Quit

• 3. Canonical Moments (I)

Canonical Moments p1, p2, . . . are transformations of the ordin- ary moments c1, c2, . . . of a probability measure. Definition: M class of probability measures with moments c1, . . . , ck−1, c+

k = max µ∈M ck(µ),

c−

k = min µ∈M ck(µ)

ξ probability measure on [−1, 1] with moments c1, . . . , ck, . . .. Then the kth canonical moment pk of ξ is defined as pk = ck − c−

k

c+

k − c− k

Title Page ◭◭ ◮◮ ◭ ◮ Page 14 of 37 Go Back Full Screen Close Quit

pk = ck − c−

k

c+

k − c− k

Properties:

• pk ∈ [0, 1] for k = 1, 2, . . .
• pk gives the relative position of ck in its moment space

given ck−1, . . . , c1

• moment space for p1, . . . , pk is [0, 1]k

Title Page ◭◭ ◮◮ ◭ ◮ Page 15 of 37 Go Back Full Screen Close Quit

Example: The moment space for the second moment c2 on [−1, 1]. c+

2 = 1, c− 2 = c2 1

Title Page ◭◭ ◮◮ ◭ ◮ Page 16 of 37 Go Back Full Screen Close Quit

Example: The moment space for the second canonical moment p2 on [−1, 1]. p+

2 = 1, p− 2 = 0 (do not depend on the value of p1)

Title Page ◭◭ ◮◮ ◭ ◮ Page 17 of 37 Go Back Full Screen Close Quit

Advantage of canonical moments: Maximisation of a function with respect to m canonical moments is maximisation on the m-dimensional cube [0, 1]m.

Title Page ◭◭ ◮◮ ◭ ◮ Page 18 of 37 Go Back Full Screen Close Quit

Properties:

• If m is the first index for which pm ∈ {0, 1} then the

sequence of canonical moments terminates at pm and the measure is supported at a finite number of points.

• If a sequence p1, . . . , p2d is given, and there is no such

m then the measure is not unique. One can always find a measure with these canonical moments by adding arbitrary pi’s.

• Given the canonical moments, the measure can be found

by evaluating certain orthogonal polynomials (see section 5).

Title Page ◭◭ ◮◮ ◭ ◮ Page 19 of 37 Go Back Full Screen Close Quit

• 4. Results

Theorem 1: If there exists a constrained optimal discrimin- ating design for (γ2d−2j−1, . . . , γ2d−1), the canonical moments up to the order 2d of ξσ∗ are given by (qk = 1 − pk)

Title Page ◭◭ ◮◮ ◭ ◮ Page 20 of 37 Go Back Full Screen Close Quit

• 4. Results

Theorem 1: If there exists a constrained optimal discrimin- ating design for (γ2d−2j−1, . . . , γ2d−1), the canonical moments up to the order 2d of ξσ∗ are given by (qk = 1 − pk) p2n−1 = 1 2, n = 1, . . . , d,

Title Page ◭◭ ◮◮ ◭ ◮ Page 21 of 37 Go Back Full Screen Close Quit

• 4. Results

Theorem 1: If there exists a constrained optimal discrimin- ating design for (γ2d−2j−1, . . . , γ2d−1), the canonical moments up to the order 2d of ξσ∗ are given by (qk = 1 − pk) p2n−1 = 1 2, n = 1, . . . , d, p2n = 1 2, n = 1, . . . , d − j − 1

Title Page ◭◭ ◮◮ ◭ ◮ Page 22 of 37 Go Back Full Screen Close Quit

p2d−2j+2n =                                          1 − max{1 2, γ2d−2j+2n−1 22n d−j+n−1

l=d−j

p2lq2l }, if γ2d−2j+2n−1 > γ2d−2j+2n max{1 2, γ2d−2j+2n 22n d−j+n−1

l=d−j

p2lq2l }, if γ2d−2j+2n ≥ γ2d−2j+2n−1, n = 0, . . . , j − 1

Title Page ◭◭ ◮◮ ◭ ◮ Page 23 of 37 Go Back Full Screen Close Quit

p2d−2j+2n =                                          1 − max{1 2, γ2d−2j+2n−1 22n d−j+n−1

l=d−j

p2lq2l }, if γ2d−2j+2n−1 > γ2d−2j+2n max{1 2, γ2d−2j+2n 22n d−j+n−1

l=d−j

p2lq2l }, if γ2d−2j+2n ≥ γ2d−2j+2n−1, n = 0, . . . , j − 1 p2d = 1 − γ2d−1 22j d−1

l=d−j p2lq2l

Title Page ◭◭ ◮◮ ◭ ◮ Page 24 of 37 Go Back Full Screen Close Quit

Sketch of Proof: 1) Express the criterion in terms of canonical moments: effk(σ) =

• 24j−2 j

ℓ=1 q2ℓ−2p2ℓ−1q2ℓ−1p2ℓ

if k = 2j 24j−2 j

ℓ=1 p2ℓ−2q2ℓ−1p2ℓ−1q2ℓ

if k = 2j − 1

Title Page ◭◭ ◮◮ ◭ ◮ Page 25 of 37 Go Back Full Screen Close Quit

Sketch of Proof: 1) Express the criterion in terms of canonical moments: effk(σ) =    24j−2 j

ℓ=1 q2ℓ−2 p2ℓ−1q2ℓ−1

• p2ℓ

if k = 2j 24j−2 j

ℓ=1 p2ℓ−2 q2ℓ−1p2ℓ−1

• q2ℓ

if k = 2j − 1

Title Page ◭◭ ◮◮ ◭ ◮ Page 26 of 37 Go Back Full Screen Close Quit

Sketch of Proof: 1) Express the criterion in terms of canonical moments: effk(σ) =    24j−2 j

ℓ=1 q2ℓ−2 p2ℓ−1q2ℓ−1

• p2ℓ

if k = 2j 24j−2 j

ℓ=1 p2ℓ−2 q2ℓ−1p2ℓ−1

• q2ℓ

if k = 2j − 1 ⇒ p2ℓ−1 = 1 2, ℓ = 1, . . . , j

Title Page ◭◭ ◮◮ ◭ ◮ Page 27 of 37 Go Back Full Screen Close Quit

Sketch of Proof: 1) Express the criterion in terms of canonical moments: effk(σ) =    24j−2 j

ℓ=1 q2ℓ−2 p2ℓ−1q2ℓ−1

• p2ℓ

if k = 2j 24j−2 j

ℓ=1 p2ℓ−2 q2ℓ−1p2ℓ−1

• q2ℓ

if k = 2j − 1 ⇒ p2ℓ−1 = 1 2, ℓ = 1, . . . , j ⇒ effk(σ) =

• 22j−2p2j

j−1

ℓ=1 q2ℓp2ℓ

if k = 2j 22j−2q2j j−1

ℓ=1 q2ℓp2ℓ

if k = 2j − 1

Title Page ◭◭ ◮◮ ◭ ◮ Page 28 of 37 Go Back Full Screen Close Quit

The rest of the proof is as straightforward as the first part! . . . but I’m not going to prove the above assertion in this talk

Title Page ◭◭ ◮◮ ◭ ◮ Page 29 of 37 Go Back Full Screen Close Quit

• 5. Canonical moments (II)

How can we find designs from given sequences of canonical moments? Stieltjes Transform of a measure ξ on [a, b] with finite support: S(z, ξ) = dξ(x) z − x =

m

• i=1

ωi z − xi = Pm−1(z) Qm(z) Continued fraction expansion of S(z, ξ) where ξi = pi qi−1: S(z, ξ) = 1 | | z − a − ξ1(b − a) − ξ1ξ2(b − a)2 | | z − a − (ξ2 + ξ3)(b − a) − . . . − ξ2m−3ξ2m−2(b − a)2 | | z − a − (ξ2m−2 + ξ2m−1)(b − a)

Title Page ◭◭ ◮◮ ◭ ◮ Page 30 of 37 Go Back Full Screen Close Quit

Properties:

• The support points of the measure are the roots of the

polynomial Qm(z)

• The weights follow from

ωk = lim

z→xk(z − xk)S(z, ξ) =

Pm−1(xk)

d d zQm(z) |z=xk

• Theory of continued fractions gives recursive formulae for

the polynomials Pm−1(z) and Qm(z)

Title Page ◭◭ ◮◮ ◭ ◮ Page 31 of 37 Go Back Full Screen Close Quit

Example: d = 2, γ2 = 0.6, γ3 = 0.5, no constraint on eff1(σ) From Theorem 1: p1 = 0.5, p3 = 0.5, p2 = 0.6, p4 = 23

48

The optimal design is not unique. Choose arbitrary canonical moments, e.g., p5 = 0.5, p6 = 0 S(z, ξ) = 1 | | z − 0.6 | | z − 23/120 | | z = 1 z − 0.6 z − 23/120 z = z2 − 23/120 z3 − 19/24 z

Title Page ◭◭ ◮◮ ◭ ◮ Page 32 of 37 Go Back Full Screen Close Quit

The support points xk, k = 1, 2, 3, are given by the roots of Q3(z) = z3 − 19/24 z, i.e. 0, −

• 19/24 and
• 19/24.

The weights are calculated from the formula ωk = (z2 − 23/120) |z=xk

d d z(z3 − 19/24 z) |z=xk

= (z2 − 23/120) |z=xk (3z2 − 19/24) |z=xk =              −23/120 −19/24 ≈ 0.242 if xk = 0 19/24 − 23/120 3 · 19/24 − 19/24 ≈ 0.379 if xk = −

• 19/24

19/24 − 23/120 3 · 19/24 − 19/24 ≈ 0.379 if xk =

• 19/24

Title Page ◭◭ ◮◮ ◭ ◮ Page 33 of 37 Go Back Full Screen Close Quit

The design ξσ on [−1, 1] is then given by ξσ =   −

• 19/24
• 19/24

0.379 0.242 0.379   and the transformation yields σ =   −2.668 −1.571 −0.474 0.474 1.571 2.668 0.1895 0.121 0.1895 0.1895 0.121 0.1895  

Title Page ◭◭ ◮◮ ◭ ◮ Page 34 of 37 Go Back Full Screen Close Quit

The design ξσ on [−1, 1] is then given by ξσ =   −

• 19/24
• 19/24

0.379 0.242 0.379   and the transformation yields σ =   −2.668 −1.571 −0.474 0.474 1.571 2.668 0.1895 0.121 0.1895 0.1895 0.121 0.1895   Efficiencies of σ: eff1 eff2 eff3 eff4 0.4 0.6 0.5 0.46

Title Page ◭◭ ◮◮ ◭ ◮ Page 35 of 37 Go Back Full Screen Close Quit

Summary:

• We have applied the theory of canonical moments to de-

rive constrained optimal designs for discriminating between Fourier models of different degree.

• Choice of the lower bounds γl for the efficiencies: according

to the experimenter’s interest in a specific testing problem

• Necessary condition for existence of an optimal design:

γ2d−2j + γ2d−2j−1 ≤ 1

• Optimal designs are usually not unique.
• For special cases (wrt the γk’s) we have found explicit for-

mulae for the polynomials Pm−1 and Qm

Title Page ◭◭ ◮◮ ◭ ◮ Page 36 of 37 Go Back Full Screen Close Quit

Thank you!

Title Page ◭◭ ◮◮ ◭ ◮ Page 37 of 37 Go Back Full Screen Close Quit

References Biedermann, S., Dette, H. and Hoffmann, P. (2007). Con- strained optimal discriminating designs for Fourier re- gression models. Accepted in: Annals of the Institute of Statistical Mathematics. Dette, H. and Studden, W.J. (1997). The Theory of Canonical Moments with Applications in Statistics, Probability and

• Analysis. Wiley, New York.