Iterative Methods for Symmetric Quasi-Definite Linear Systems Mario - - PowerPoint PPT Presentation

iterative methods for symmetric quasi definite linear
SMART_READER_LITE
LIVE PREVIEW

Iterative Methods for Symmetric Quasi-Definite Linear Systems Mario - - PowerPoint PPT Presentation

Iterative Methods for Symmetric Quasi-Definite Linear Systems Mario Arioli 1 Dominique Orban 2 1 Rutherford Appleton Laboratory, mario.arioli@stfc.ac.uk 2 GERAD and Ecole Polytechnique de Montr eal, dominique.orban@gerad.ca Sparse Days,


slide-1
SLIDE 1

Iterative Methods for Symmetric Quasi-Definite Linear Systems

Mario Arioli1 Dominique Orban2

1Rutherford Appleton Laboratory, mario.arioli@stfc.ac.uk 2GERAD and ´

Ecole Polytechnique de Montr´ eal, dominique.orban@gerad.ca

slide-2
SLIDE 2

Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban

Overview of talk

◮ Symmetric Quasi Positive Definite matrices

2 / 28

slide-3
SLIDE 3

Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban

Overview of talk

◮ Symmetric Quasi Positive Definite matrices ◮ Why SQD are important?

2 / 28

slide-4
SLIDE 4

Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban

Overview of talk

◮ Symmetric Quasi Positive Definite matrices ◮ Why SQD are important? ◮ Main properties

2 / 28

slide-5
SLIDE 5

Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban

Overview of talk

◮ Symmetric Quasi Positive Definite matrices ◮ Why SQD are important? ◮ Main properties ◮ Generalized singular values and minimization problem ◮ G-K bidiagonalization ◮ Generalized LSQR and Craig (Stopping criteria)

2 / 28

slide-6
SLIDE 6

Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban

Overview of talk

◮ Symmetric Quasi Positive Definite matrices ◮ Why SQD are important? ◮ Main properties ◮ Generalized singular values and minimization problem ◮ G-K bidiagonalization ◮ Generalized LSQR and Craig (Stopping criteria) ◮ Numerical examples

2 / 28

slide-7
SLIDE 7

Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban

Symmetric Quasi-Definite Systems

M A AT −N x y

  • =

f g

  • where

M = MT ≻ 0, N = NT ≻ 0.

◮ Interior-point methods for LP, QP, NLP, SOCP, SDP, . . . ◮ Regularized/stabilized PDE problems ◮ Regularized least squares ◮ How to best take advantage of the structure?

3 / 28

slide-8
SLIDE 8

Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban

Main Property

Theorem (Vanderbei, 1995) If K is SQD, it is strongly factorizable, i.e., for any permutation matrix P, there exists a unit lower triangular L and a diagonal D such that PTKP = LDLT.

◮ Cholesky-factorizable ◮ Used to speed up factorization in regularized least-squares

(Saunders) and interior-point methods (Friedlander and O.)

◮ Stability analysis by Gill, Saunders, Shinnerl (1996).

4 / 28

slide-9
SLIDE 9

Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban

Centered preconditioning

  • M− 1

2

N− 1

2

M A AT −N M− 1

2

N− 1

2

ˆ x ˆ y

  • =
  • M− 1

2 f

N− 1

2 g

  • which is equivalent to

A

  • Im

M− 1

2 AN− 1 2

N− 1

2 ATM− 1 2

−In ˆ x ˆ y

  • =
  • M− 1

2 f

N− 1

2 g

  • 5 / 28
slide-10
SLIDE 10

Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban

Centered preconditioning

  • M− 1

2

N− 1

2

M A AT −N M− 1

2

N− 1

2

ˆ x ˆ y

  • =
  • M− 1

2 f

N− 1

2 g

  • which is equivalent to

A

  • Im

M− 1

2 AN− 1 2

N− 1

2 ATM− 1 2

−In ˆ x ˆ y

  • =
  • M− 1

2 f

N− 1

2 g

  • Theorem (Saunders (1995))

Suppose ˜ A = M− 1

2 AN− 1 2 has rank p ≤ m with nonzero singular

values σ1, . . . , σp. The eigenvalues of A are +1, −1 and ±√1 + σk, k = 1, . . . , p.

5 / 28

slide-11
SLIDE 11

Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban

Symmetric spectrum and Iterative methods

A symmetric matrix with a symmetric spectrum can be transform preserving the symmetry of the spectrum in a SQD one. Moreover, Fischer (Theorem 6.9.9 in “Polynomial based iteration methods for symmetric linear systems”) Freund (1983), Freund Golub Nachtigal (1992), and Ramage Silvester Wathen (1995) give different poofs that MINRES and CG perform redundant iterations.

6 / 28

slide-12
SLIDE 12

Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban

Iterative Methods I

Facts: SQD systems are symmetric, non-singular, square and indefinite.

7 / 28

slide-13
SLIDE 13

Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban

Iterative Methods I

Facts: SQD systems are symmetric, non-singular, square and indefinite.

◮ MINRES ◮ SYMMLQ ◮ (F)GMRES?? ◮ QMRS????

7 / 28

slide-14
SLIDE 14

Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban

Iterative Methods I

Facts: SQD systems are symmetric, non-singular, square and indefinite.

◮ MINRES ◮ SYMMLQ ◮ (F)GMRES?? ◮ QMRS????

Fact: . . . none exploits the SQD structure and they are doing redundant iterations

7 / 28

slide-15
SLIDE 15

Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban

Related Problems: an example

M A AT −N x y

  • =

b

  • 8 / 28
slide-16
SLIDE 16

Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban

Related Problems: an example

M A AT −N x y

  • =

b

  • are the optimality conditions of

min

y∈I

R

m

1 2

  • A

I

  • y −

b

  • 2

E −1

+

≡ min

y∈I

R

m

1 2

  • M− 1

2

N

1 2

A I

  • y −

b

  • 2

2

8 / 28

slide-17
SLIDE 17

Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban

Normal equations for SQD

Let assume that M = RTR and N = UTU

9 / 28

slide-18
SLIDE 18

Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban

Normal equations for SQD

Let assume that M = RTR and N = UTU We observe that the normal equations A2 ˆ x ˆ y

  • = A

R−1b

  • have a very interesting structure because

9 / 28

slide-19
SLIDE 19

Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban

Normal equations for SQD

Let assume that M = RTR and N = UTU We observe that the normal equations A2 ˆ x ˆ y

  • = A

R−1b

  • have a very interesting structure because

A2 = Im−n + ˜ A˜ AT In + ˜ AT ˜ A

  • = D

and DA = AD

9 / 28

slide-20
SLIDE 20

Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban

Normal equations for SQD

Let assume that M = RTR and N = UTU A2 = Im−n + ˜ A˜ AT In + ˜ AT ˜ A

  • = D

and DA = AD The Krylov space Kk(A, h0) = Range

  • Ajh0, . . . , Ah0, h0
  • is

Kk(A, h0) = K⌊k/2⌋(D, h0) + K⌊(k+1)/2⌋(D, Ah0)

9 / 28

slide-21
SLIDE 21

Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban

A−1

A−1 = D−1A = AD−1

10 / 28

slide-22
SLIDE 22

Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban

Linear operators

Let M ∈ I Rm×m and N ∈ I Rn×n be symmetric positive definite matrices, and let A ∈ I Rm×n be a full rank matrix. M = {v ∈ I Rm; u2

M = vTMv}, N = {q ∈ I

Rn; q2

N = qTNq}

M′ = {w ∈ I Rm; w2

M−1 = wTM−1w},

N ′ = {y ∈ I Rn; y2

N−1 = yTN−1y}

11 / 28

slide-23
SLIDE 23

Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban

Linear operators

Let M ∈ I Rm×m and N ∈ I Rn×n be symmetric positive definite matrices, and let A ∈ I Rm×n be a full rank matrix. M = {v ∈ I Rm; u2

M = vTMv}, N = {q ∈ I

Rn; q2

N = qTNq}

M′ = {w ∈ I Rm; w2

M−1 = wTM−1w},

N ′ = {y ∈ I Rn; y2

N−1 = yTN−1y}

v, AqM,M′ = vTAq, Aq ∈ M′ ∀q ∈ N.

11 / 28

slide-24
SLIDE 24

Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban

Linear operators

Let M ∈ I Rm×m and N ∈ I Rn×n be symmetric positive definite matrices, and let A ∈ I Rm×n be a full rank matrix. M = {v ∈ I Rm; u2

M = vTMv}, N = {q ∈ I

Rn; q2

N = qTNq}

M′ = {w ∈ I Rm; w2

M−1 = wTM−1w},

N ′ = {y ∈ I Rn; y2

N−1 = yTN−1y}

v, AqM,M′ = vTAq, Aq ∈ M′ ∀q ∈ N. The adjoint operator A⋆ of A can be defined as A⋆g, fN ′,N = fTATg, ATg ∈ N ′ ∀g ∈ M.

11 / 28

slide-25
SLIDE 25

Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban

Generalized SVD

Given q ∈ M and v ∈ N, the critical points for the functional vTAq qN vM are the “elliptic singular values and singular vectors’’ of A.

12 / 28

slide-26
SLIDE 26

Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban

Generalized SVD

Given q ∈ M and v ∈ N, the critical points for the functional vTAq qN vM are the “elliptic singular values and singular vectors’’ of A. The saddle-point conditions are Aqi = σiMvi vT

i Mvj = δij

ATvi = σiNqi qT

i Nqj = δij

σ1 ≥ σ2 ≥ · · · ≥ σn > 0

12 / 28

slide-27
SLIDE 27

Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban

Generalized SVD

Given q ∈ M and v ∈ N, the critical points for the functional vTAq qN vM are the “elliptic singular values and singular vectors’’ of A. The saddle-point conditions are Aqi = σiMvi vT

i Mvj = δij

ATvi = σiNqi qT

i Nqj = δij

σ1 ≥ σ2 ≥ · · · ≥ σn > 0

The elliptic singular values are the standard singular values of ˜ A = M−1/2AN−1/2. The elliptic singular vectors qi and vi, i = 1, . . . , n are the transformation by M−1/2 and N−1/2 respectively of the left and right standard singular vector of ˜ A.

12 / 28

slide-28
SLIDE 28

Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban

Generalized Golub-Kahan bidiagonalization

In Golub Kahan (1965), Paige Saunders (1982), several algorithms for the bidiagonalization of a m × n matrix are presented. All of them can be theoretically applied to ˜ A and their generalization to A is straightforward as shown by Bembow (1999). Here, we want specifically to analyse one of the variants known as the ”Craig”-variant (see Paige Saunders (1982), Saunders (1995,1997)).

13 / 28

slide-29
SLIDE 29

Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban

Generalized Golub-Kahan bidiagonalization

     A˜ Q = M˜ V ˜ B

  • ˜

VTM˜ V = Im AT ˜ V = N˜ Q

  • ˜

BT; 0

  • ˜

QTN˜ Q = In where ˜ B =           ˜ α1 · · · ˜ β2 ˜ α2 ... . . . ... ... ... ... · · · ˜ βn−1 ˜ αn−1 · · · ˜ βn ˜ αn · · · ˜ βn+1           .

13 / 28

slide-30
SLIDE 30

Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban

Generalized Golub-Kahan bidiagonalization

   AQ = MV B

  • VTMV = Im

ATV = NQ

  • BT; 0
  • QTNQ = In

where B =         α1 β1 · · · α2 β2 ... . . . ... ... ... ... · · · αn−1 βn−1 · · · αn         .

13 / 28

slide-31
SLIDE 31

Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban

Algorithm

Thus, we can compute the first column of B and of V: α1Mv1 = Aq1, such as w = M−1Aq1 α1 = √ wTMw = √wAq1 v1 = w/α1.

14 / 28

slide-32
SLIDE 32

Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban

Algorithm

Thus, we can compute the first column of B and of V: α1Mv1 = Aq1, such as w = M−1Aq1 α1 = √ wTMw = √wAq1 v1 = w/α1. Finally, knowing q1 and v1 we can start the recursive relations gi+1 = N−1 ATvi − αiNqi

  • βi+1 =
  • gTNg

qi+1 = g βi+1 w = M−1 (Aqi+1 − βi+1Mvi) αi+1 = √ wTMw vi+1 = w/αi+1.

14 / 28

slide-33
SLIDE 33

Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban

Generalized Least Squares

Normal equations: (ATM−1A + N)y = ATM−1b.

15 / 28

slide-34
SLIDE 34

Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban

Generalized Least Squares

Normal equations: (ATM−1A + N)y = ATM−1b. At k-th iteration, seek y ≈ yk := ˜ Vk¯ yk: (˜ BT

k ˜

Bk + I)¯ yk = ˜ BT

k β1e1

15 / 28

slide-35
SLIDE 35

Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban

Generalized Least Squares

Normal equations: (ATM−1A + N)y = ATM−1b. At k-th iteration, seek y ≈ yk := ˜ Vk¯ yk: (˜ BT

k ˜

Bk + I)¯ yk = ˜ BT

k β1e1

i.e.: min

¯ y∈I

R

k

1 2

  • ˜

Bk I

  • ¯

y − β1e1

  • 2

2

15 / 28

slide-36
SLIDE 36

Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban

Generalized Least Squares

Normal equations: (ATM−1A + N)y = ATM−1b. At k-th iteration, seek y ≈ yk := ˜ Vk¯ yk: (˜ BT

k ˜

Bk + I)¯ yk = ˜ BT

k β1e1

i.e.: min

¯ y∈I

R

k

1 2

  • ˜

Bk I

  • ¯

y − β1e1

  • 2

2

  • r:

I ˜ Bk ˜ BT

k

−I ¯ xk ¯ yk

  • =

β1e1

  • .

15 / 28

slide-37
SLIDE 37

Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban

Generalized LSQR

Solve min

¯ y∈I

R

k

1 2

  • ˜

Bk I

  • ¯

y − β1e1

  • 2

2

by specialized Givens Rotations (Eliminate I first and ˜ Rk will be upper bidiagonal)

16 / 28

slide-38
SLIDE 38

Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban

Generalized LSQR

Solve min

¯ y∈I

R

k

1 2

  • ˜

Bk I

  • ¯

y − β1e1

  • 2

2

by specialized Givens Rotations (Eliminate I first and ˜ Rk will be upper bidiagonal) min

¯ y∈I

R

k

1 2

  • ˜

Rk

  • ¯

y − φk

  • 2

2

.

16 / 28

slide-39
SLIDE 39

Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban

Generalized LSQR

Solve min

¯ y∈I

R

k

1 2

  • ˜

Bk I

  • ¯

y − β1e1

  • 2

2

by specialized Givens Rotations (Eliminate I first and ˜ Rk will be upper bidiagonal) min

¯ y∈I

R

k

1 2

  • ˜

Rk

  • ¯

y − φk

  • 2

2

. As in Paige-Saunders ’82 we can build recursive expressions of yk yk+1 = yk + dkφk

  • Dk = ˜

Vk ˜ R−1

k

  • and we have that

||¯ y||2

N+AT M−1A = m

  • j=1

φ2

j

and ||¯ y − yk||2

N+AT M−1A = m

  • j=k+1

φ2

j

16 / 28

slide-40
SLIDE 40

Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban

Error bound

Lower bound We can estimate ||¯ y − yk||2

N+AT M−1A by the lower

bound ξ2

k,d = k+d+1

  • j=k+1

φ2

j < ||¯

y − yk||2

N+AT M−1A.

and ||¯ y||2

N+AT M−1A by the lower bound k j=1 φ2 j .

Given a threshold τ < 1 and an integer d, we can stop the iterations when ξ2

k,d ≤ τ k+d+1

  • j=1

φ2

j < τ k

  • j=1

φ2

j < τ||¯

y||2

N+AT M−1A.

17 / 28

slide-41
SLIDE 41

Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban

Error bound

Lower bound We can estimate ||¯ y − yk||2

N+AT M−1A by the lower

bound ξ2

k,d = k+d+1

  • j=k+1

φ2

j < ||¯

y − yk||2

N+AT M−1A.

and ||¯ y||2

N+AT M−1A by the lower bound k j=1 φ2 j .

Given a threshold τ < 1 and an integer d, we can stop the iterations when ξ2

k,d ≤ τ k+d+1

  • j=1

φ2

j < τ k

  • j=1

φ2

j < τ||¯

y||2

N+AT M−1A.

Upper bound Despite being very inexpensive, the previous estimator is still a lower bound of the error. We can use an approach inspired by the Gauss-Radau quadrature algorithm and similar to the one described in Golub-Meurant (2010).

17 / 28

slide-42
SLIDE 42

Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban

Generalized CRAIG

min

y,x 1 2(y2 N + x2 M)

s.t. Ay + Mx = b.

18 / 28

slide-43
SLIDE 43

Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban

Generalized CRAIG

min

y,x 1 2(y2 N + x2 M)

s.t. Ay + Mx = b. At step k of GK bidiagonalization, we seek x ≈ zk := Uk¯ xk, and y ≈ yk := Vk¯ yk.

18 / 28

slide-44
SLIDE 44

Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban

Generalized CRAIG

min

y,x 1 2(y2 N + x2 M)

s.t. Ay + Mx = b. At step k of GK bidiagonalization, we seek x ≈ zk := Uk¯ xk, and y ≈ yk := Vk¯ yk. min

¯ y,¯ x 1 2(¯

y2 + ¯ x2) s.t. Bk¯ yk + ¯ xk = β1e1

18 / 28

slide-45
SLIDE 45

Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban

Generalized CRAIG

min

y,x 1 2(y2 N + x2 M)

s.t. Ay + Mx = b. At step k of GK bidiagonalization, we seek x ≈ zk := Uk¯ xk, and y ≈ yk := Vk¯ yk. min

¯ y,¯ x 1 2(¯

y2 + ¯ x2) s.t. Bk¯ yk + ¯ xk = β1e1

  • r:

min

¯ y∈I

R

k

1 2

  • Bk

I

  • ¯

y − β1e1

  • 2

2

.

18 / 28

slide-46
SLIDE 46

Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban

Generalized CRAIG

By contrast with generalized LSQR, we solve the SQD subsystem Ik Bk BT

k

−Ik ¯ xk ¯ yk

  • =

β1e1

  • 19 / 28
slide-47
SLIDE 47

Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban

Generalized CRAIG

By contrast with generalized LSQR, we solve the SQD subsystem Ik Bk BT

k

−Ik ¯ xk ¯ yk

  • =

β1e1

  • Following Saunders (1995) and Paige (1974), we compute an LQ

factorization to the k-by-2k matrix

  • Bk

Ik

  • by applying 2k − 1

Givens rotations that zero out the identity block.

19 / 28

slide-48
SLIDE 48

Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban

Generalized CRAIG

By contrast with generalized LSQR, we solve the SQD subsystem Ik Bk BT

k

−Ik ¯ xk ¯ yk

  • =

β1e1

  • Bk

Ik

  • QT

k =

ˆ Bk

  • QT

k Qk = I

where ˆ Bk :=      ˆ α1 ˆ β2 ˆ α2 ... ... ˆ βk ˆ αk      .

19 / 28

slide-49
SLIDE 49

Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban

Generalized CRAIG

β1e1 = Bk¯ yk + ¯ xk =

  • Bk

Ik ¯ yk ¯ xk

  • =

ˆ Bk

  • Qk

¯ yk ¯ xk

  • =

ˆ Bk ¯ zk

  • = ˆ

Bk¯ zk, for some ¯ zk ∈ I Rk: ¯ zk = (ζ1, . . . , ζk)

20 / 28

slide-50
SLIDE 50

Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban

Generalized CRAIG

β1e1 = Bk¯ yk + ¯ xk =

  • Bk

Ik ¯ yk ¯ xk

  • =

ˆ Bk

  • Qk

¯ yk ¯ xk

  • =

ˆ Bk ¯ zk

  • = ˆ

Bk¯ zk, for some ¯ zk ∈ I Rk: ¯ zk = (ζ1, . . . , ζk) ζ1 = β1/ˆ α1, ζi+1 = −ˆ βi+1ζi/ˆ αi+1, (i = 1, . . . , k − 1).

20 / 28

slide-51
SLIDE 51

Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban

Generalized CRAIG: errors bound

Moreover, for k = 1, . . . , p := min(m, n), xk2

M + yk2 N = k

  • i=1

ζ2

i ,

x∗ − xk2

M + y∗ − yk2 N = p

  • i=k+1

ζ2

i .

21 / 28

slide-52
SLIDE 52

Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban

Generalized CRAIG: errors bound

Moreover, for k = 1, . . . , p := min(m, n), xk2

M + yk2 N = k

  • i=1

ζ2

i ,

x∗ − xk2

M + y∗ − yk2 N = p

  • i=k+1

ζ2

i .

As for generalized LSQR, we can estimate the error using the windowing technique and we can give a lower bound of the error by ξ2

k,d = k+d+1

  • j=k+1

ζ2

i ≤ x∗ − xk2 M + y∗ − yk2 N

and xk2

M + yk2 N by the lower bound k j=1 ζ2 j .

21 / 28

slide-53
SLIDE 53

Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban

Generalized CRAIG: errors bound

Moreover, for k = 1, . . . , p := min(m, n), xk2

M + yk2 N = k

  • i=1

ζ2

i ,

x∗ − xk2

M + y∗ − yk2 N = p

  • i=k+1

ζ2

i .

As for GLSQR. If we know a lower bound of singular values we can use an approach inspired by the Gauss-Radau quadrature algorithm and similar to the one described in Golub-Meurant (2010).

21 / 28

slide-54
SLIDE 54

Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban

Numerical experiments

We will focus on optimization problems: min

x∈I

R

n gTx + 1

2xTQx

  • s. t.

Dx = d, x ≥ 0, where g ∈ I Rn and Q = QT ∈ I Rn×n is positive semi-definite, and result in linear systems with coefficient matrix Q + X 1Z + ρI DT D −δI

  • where ρ > 0 and δ > 0 are regularization parameters.

22 / 28

slide-55
SLIDE 55

Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban

Numerical experiments MINRES

This is a blow-up of some iterations

23 / 28

slide-56
SLIDE 56

Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban

Numerical experiments GLSQR

20 40 60 80 100 120 101 102 103 104 105

Regularized Least-Squares Objective G-LSQR G-LSMR

20 40 60 80 100 120 10-7 10-5 10-3 10-1 101 103 105 107 109

Residual of Normal Equations

Figure: Problem DUAL1 (255, 171).

24 / 28

slide-57
SLIDE 57

Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban

Numerical experiments GLSQR

50 100 150 200 101 102 103

Regularized Least-Squares Objective G-LSQR G-LSMR

50 100 150 200 10-8 10-7 10-6 10-5 10-4 10-3 10-2 10-1 100 101 102 103 104

Residual of Normal Equations

Figure: Problem MOSARQP1 (5700, 3200).

25 / 28

slide-58
SLIDE 58

Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban

Numerical experiments GCraig

d = 5, 10

50 100 150 200 10-4 10-3 10-2 10-1 100 101 102 103

Direct Error

50 100 150 200 10-4 10-3 10-2 10-1 100 101 102 103

Direct Error

26 / 28

slide-59
SLIDE 59

Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban

Numerical experiments GCraig

d = 10, 15

50 100 150 200 10-4 10-3 10-2 10-1 100 101 102 103

Direct Error

50 100 150 200 10-4 10-3 10-2 10-1 100 101 102 103

Direct Error

27 / 28

slide-60
SLIDE 60

Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban

Conclusions

◮ Preconditioning −

→ Norms i.e. different topologies!!

28 / 28

slide-61
SLIDE 61

Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban

Conclusions

◮ Preconditioning −

→ Norms i.e. different topologies!!

◮ Nice relation between the algebraic error and the

approximation error

28 / 28

slide-62
SLIDE 62

Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban

Conclusions

◮ Preconditioning −

→ Norms i.e. different topologies!!

◮ Nice relation between the algebraic error and the

approximation error

◮ Dominique Orban and I are analysing GLSMR and the

numerical results validate the theory.

28 / 28

slide-63
SLIDE 63

Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban

Conclusions

◮ Preconditioning −

→ Norms i.e. different topologies!!

◮ Nice relation between the algebraic error and the

approximation error

◮ Dominique Orban and I are analysing GLSMR and the

numerical results validate the theory.

◮ A. and Orban ”Iterative methods for symmetric quasi definite

systems” in preparation. WORK IN PROGRESS

28 / 28