Evolution equations and Tikhonov regularization Juan PEYPOUQUET - - PowerPoint PPT Presentation

Tikhonov regularization

Evolution equations and Tikhonov regularization

Juan PEYPOUQUET Universidad T´ ecnica Federico Santa Mar´ ıa joint work with R. Cominetti & S. Sorin JOURN´

EES FRANCO-CHILIENNES D’OPTIMISATION

Toulon, 20 mai 2008

Tikhonov regularization

Outline

1

Generalized gradient method and Tikhonov regularization

2

Coupling evolution and regularization in convex minimization: strong and weak convergence

3

The case of differential inclusions governed by maximal monotone operators

Tikhonov regularization

Generalized gradient method

Let f be a proper lower-semicontinuous convex function on a Hilbert space H and let S = Argmin f. Functions u satisfying − ˙ u(t) ∈ ∂f(u(t)) are minimizing. Moreover, if S = ∅ they converge weakly as t → ∞ to some x∞ ∈ S.

Tikhonov regularization

Generalized gradient method

Let f be a proper lower-semicontinuous convex function on a Hilbert space H and let S = Argmin f. Functions u satisfying − ˙ u(t) ∈ ∂f(u(t)) are minimizing. Moreover, if S = ∅ they converge weakly as t → ∞ to some x∞ ∈ S.

Tikhonov regularization

Tikhonov regularization

For ε > 0 consider the strongly convex function fε(x) = f(x) + ε 2x2. The solutions of − ˙ u(t) ∈ ∂fε(u(t)) converge strongly as t → ∞ to xε, the unique minimizer of fε. If S = ∅ then xε converges strongly to the least-norm element x∗ of S.

Tikhonov regularization

Tikhonov regularization

For ε > 0 consider the strongly convex function fε(x) = f(x) + ε 2x2. The solutions of − ˙ u(t) ∈ ∂fε(u(t)) converge strongly as t → ∞ to xε, the unique minimizer of fε. If S = ∅ then xε converges strongly to the least-norm element x∗ of S.

Tikhonov regularization

Tikhonov regularization

For ε > 0 consider the strongly convex function fε(x) = f(x) + ε 2x2. The solutions of − ˙ u(t) ∈ ∂fε(u(t)) converge strongly as t → ∞ to xε, the unique minimizer of fε. If S = ∅ then xε converges strongly to the least-norm element x∗ of S.

Tikhonov regularization

Coupling Tikhonov regularization and the gradient method

Tikhonov regularization

Coupling

Let ε be a positive function on [0, ∞) such that lim

t→∞ ε(t) → 0

and let u : [0, ∞) → H satisfy − ˙ u(t) ∈ ∂fε(t)(u(t)) = ∂f(u(t)) + ε(t)u(t). Theorem (Cominetti, P . & Sorin) (i) If ∞

0 ε(t) dt = ∞ then u(t) → x∗.

(ii) If ∞

0 ε(t) dt < ∞ then u(t) ⇀ x∞ for some x∞ ∈ S.

Tikhonov regularization

Coupling

Let ε be a positive function on [0, ∞) such that lim

t→∞ ε(t) → 0

and let u : [0, ∞) → H satisfy − ˙ u(t) ∈ ∂fε(t)(u(t)) = ∂f(u(t)) + ε(t)u(t). Theorem (Cominetti, P . & Sorin) (i) If ∞

0 ε(t) dt = ∞ then u(t) → x∗.

(ii) If ∞

0 ε(t) dt < ∞ then u(t) ⇀ x∞ for some x∞ ∈ S.

Tikhonov regularization

Proof

(i) Let θ(t) = 1

2|u(t) − x∗|2. One easily proves

˙ θ(t) + ε(t)θ(t) ≤ 1

2ε(t)

• |x∗|2 − |xε(t)|2

. A Gronwall-like inequality then gives 0 ≤ lim sup

t→∞

θ(t) ≤ lim sup

t→∞

1 2

• |x∗|2 − |xε(t)|2

= 0. (ii) Will be done later.

Tikhonov regularization

Previous attempts

Under additional assumptions on ε and xε: (i)

• Attouch-Cominetti 1996.
• Baillon-Cominetti 2001.
• Cabot 2004.
• Reich 1976.

(ii)

• Cominetti-Alemany 1999.
• Cabot 2004.
• Furuya-Miyashiba-Kenmochi 1986.
• Alvarez-P

. 2007

Tikhonov regularization

Tikhonov regularization and differential inclusions governed by monotone operators

Tikhonov regularization

Monotone operators

A (possibly multi-valued) map A : H → 2H is monotone if x∗ − y∗, x − y ≥ 0 for all y ∈ Ax, y∗ ∈ Ax∗ and maximal if its graph is not properly contained in the graph

• f any other monotone operator.

Tikhonov regularization

Examples

Examples: A = ∂f; with f closed, proper and convex. A = I − F; with F nonexpansive. The solution set S = A−10 coincides with the minimizers of f if A = ∂f the fixed points of F if A = I − F

Tikhonov regularization

Examples

Examples: A = ∂f; with f closed, proper and convex. A = I − F; with F nonexpansive. The solution set S = A−10 coincides with the minimizers of f if A = ∂f the fixed points of F if A = I − F

Tikhonov regularization

Examples

Examples: A = ∂f; with f closed, proper and convex. A = I − F; with F nonexpansive. The solution set S = A−10 coincides with the minimizers of f if A = ∂f the fixed points of F if A = I − F

Tikhonov regularization

Examples

Examples: A = ∂f; with f closed, proper and convex. A = I − F; with F nonexpansive. The solution set S = A−10 coincides with the minimizers of f if A = ∂f the fixed points of F if A = I − F

Tikhonov regularization

Asymptotic behavior of − ˙ u ∈ Au

In general, S = ∅ does not imply that functions satisfying − ˙ u(t) ∈ Au(t) converge (although it does imply they converge in average). Counterexample For A(x, y) = (−y, x)

• ne gets

u(t) = r0

• cos(2t0 − t), sin(2t0 − t)
• ,

which does not converge unless r0 = 0.

Tikhonov regularization

Asymptotic behavior of − ˙ u ∈ Au

In general, S = ∅ does not imply that functions satisfying − ˙ u(t) ∈ Au(t) converge (although it does imply they converge in average). Counterexample For A(x, y) = (−y, x)

• ne gets

u(t) = r0

• cos(2t0 − t), sin(2t0 − t)
• ,

which does not converge unless r0 = 0.

Tikhonov regularization

Tikhonov regularization

In what follows we assume ε is a positive function on [0, ∞) such that lim

t→∞ ε(t) → 0 and study the behavior as t → ∞ of

functions u : [0, ∞) → H satisfying − ˙ u(t) ∈ Au(t) + ε(t)u(t) according to whether the function ε is in L1 or not.

Tikhonov regularization

ε ∈ L1

Theorem A perturbation ε ∈ L1(0, ∞; R) makes no difference in the asymptotic behavior of the system. There is no loss − and no gain! − in applying the Tikhonov regularization in this case.

Tikhonov regularization

Proof I

Let A(·) be a family of maximal monotone operators and let ε ∈ L1(0, ∞; R). Lemma (Alvarez & P .) If every function u : [0, ∞) → H satisfying − ˙ u(t) ∈ A(t)u(t) converges strongly (weakly) as t → ∞, so does every function v : [0, ∞) → H satisfying − ˙ v(t) ∈ A(t)v(t) + ε(t)v(t).

Tikhonov regularization

Proof I

Let A(·) be a family of maximal monotone operators and let ε ∈ L1(0, ∞; R). Lemma (Alvarez & P .) If every function u : [0, ∞) → H satisfying − ˙ u(t) ∈ A(t)u(t) converges strongly (weakly) as t → ∞, so does every function v : [0, ∞) → H satisfying − ˙ v(t) ∈ A(t)v(t) + ε(t)v(t).

Tikhonov regularization

Proof II

Let U(t, s)x = u(t), where − ˙ u(t) ∈ Au(t) u(s) = x and let y satisfy −y(t) ∈ Ay(t) + ε(t)y(t). First on proves lim

t→∞

• sup

h≥0

y(t + h) − U(t + h, t)y(t)

• = 0.

Tikhonov regularization

Proof II

Let U(t, s)x = u(t), where − ˙ u(t) ∈ Au(t) u(s) = x and let y satisfy −y(t) ∈ Ay(t) + ε(t)y(t). First on proves lim

t→∞

• sup

h≥0

y(t + h) − U(t + h, t)y(t)

• = 0.

Tikhonov regularization

Proof III

Next, lim

t→∞

• τ − lim

h→∞ U(t + h, t)y(t)

• = ζ.

Finally one writes y(t + h) − ζ = [y(t + h) − U(t + h, t)y(t)] + [U(t + h, t)y(t) − ζ] and concludes that τ − lim

t→∞ y(t) = ζ.

Tikhonov regularization

Proof III

Next, lim

t→∞

• τ − lim

h→∞ U(t + h, t)y(t)

• = ζ.

Finally one writes y(t + h) − ζ = [y(t + h) − U(t + h, t)y(t)] + [U(t + h, t)y(t) − ζ] and concludes that τ − lim

t→∞ y(t) = ζ.

Tikhonov regularization

ε / ∈ L1 and bounded total variation

Assume S = ∅ and ε / ∈ L1. Theorem (Cominetti, P . & Sorin) If ∞

0 | ˙

ε(t)| dt < ∞ then any function u : [0, ∞) → H satisfying − ˙ u(t) ∈ Au(t) + ε(t)u(t) converges strongly as t → ∞ to the least-norm element of S.

Tikhonov regularization

Idea of the proof

One first proves that lim

t→∞ ˙

u(t) = 0.1 Next we verify that, as a consequence, all weak cluster points of u(t) for t → ∞ belong to S. Finally, the latter implies u(t) → x∗ strongly.

• 1Actually we prove something weaker

Tikhonov regularization

Idea of the proof

One first proves that lim

t→∞ ˙

u(t) = 0.1 Next we verify that, as a consequence, all weak cluster points of u(t) for t → ∞ belong to S. Finally, the latter implies u(t) → x∗ strongly.

• 1Actually we prove something weaker

Tikhonov regularization

Idea of the proof

One first proves that lim

t→∞ ˙

u(t) = 0.1 Next we verify that, as a consequence, all weak cluster points of u(t) for t → ∞ belong to S. Finally, the latter implies u(t) → x∗ strongly.

• 1Actually we prove something weaker

Tikhonov regularization

The case of infinite total variation

If ∞ | ˙ ε(t)| dt = ∞ the solutions of − ˙ u(t) ∈ Au(t) + ε(t)u(t) need not converge (not even weakly) as t → ∞. Let us build a counterexample!

Tikhonov regularization

The case of infinite total variation

If ∞ | ˙ ε(t)| dt = ∞ the solutions of − ˙ u(t) ∈ Au(t) + ε(t)u(t) need not converge (not even weakly) as t → ∞. Let us build a counterexample!

Tikhonov regularization

The operator A

Let A be the π

2-counterclockwise rotation around p = (1, 1); i.e.

A(x, y) = (2 − y, x). We consider the system − ˙ u(t) = Au(t) + ε(t)u(t).

Tikhonov regularization

The parameter function ε

Let εn be a sequence of positive real numbers with εn → 0 and εn = ∞. Take a0 = 0 and let bn = an + τn, an+1 = bn + σn with τn > 0, σn > 0 to be fixed later on, and consider the step function ε(t) = εn if an ≤ t < bn if bn ≤ t < an+1. Clearly ε(t) → 0 and we get ∞

0 ε(t)dt = ∞ provided τn is

bounded away from zero.

Tikhonov regularization

Global behavior

p

1

i u(a ) u(b )

n n

*

Tikhonov regularization

Regularity

The lack of continuity of the function ε is not the problem, nor is it the fact that ε vanishes in some intervals. In fact, one can find a strictly positive, infinitely differentiable function η such that η / ∈ L1 while ε − η ∈ L1. The method also yields nonconvergent trajectories with this new parameter function.

Tikhonov regularization

Thank you THANK YOU !