Proof mining in convex optimization Andrei Sipos , (joint work - - PowerPoint PPT Presentation

Proof mining in convex optimization

Andrei Sipos

,

(joint work with Laurent ¸iu Leus ¸tean and Adriana Nicolae)

Institute of Mathematics of the Romanian Academy University of Bucharest

May 2, 2017 PhDs in Logic IX Bochum, Deutschland

Proof mining

Proof mining (introduced and developed by U. Kohlenbach) aims to obtain quantitative information from proofs of theorems (from various areas of mathematics) of a nature which is not (fully)

• constructive. A comprehensive reference is:
• U. Kohlenbach, Applied proof theory: Proof interpretations and

their use in mathematics, Springer, Berlin/Heidelberg, 2008. An extensive survey detailing the intervening research can be found in:

• U. Kohlenbach, “Recent progress in proof mining in nonlinear

analysis”, preprint 2016, to appear in forthcoming special issue of IFCoLog Journal of Logic and its Applications with invited articles by recipients of a G¨

• del Centenary Research Prize Fellowship.

Andrei Sipos

,

Proof mining in convex optimization

The general question

Proof theory is one of the four main branches of logic and has as its scope of study proofs themselves (inside given logical systems), with a special aim upon consistency results, structural and substructural transformations, proof-theoretic ordinals et al. The driving question of proof mining / interpretative proof theory is the following: “What more do we know if we have proved a theorem by restricted means than if we merely know that it is true?” (posed by G. Kreisel in the 1950s)

Andrei Sipos

,

Proof mining in convex optimization

Kinds of information

By analysing a specific proof of a mathematical theorem, one could

• btain, in addition:

Terms coding effective algorithms for witnesses and bounds for existentially quantified variables; Independence of certain parameters or at least continuity of the dependency; Weakening of premises. In order for this to work, we must impose well-behavedness conditions upon the logical system and upon the complexity of the statement of the theorem.

Andrei Sipos

,

Proof mining in convex optimization

The logical systems

We generally use systems of arithmetic in all finite types, intuitionistic or classical, augmented by restricted non-constructive rules (such as choice principles) and by types referring to (metric/normed/Hilbert) spaces and functionals involving them. Two such systems are denoted by Aω

i [X, ·, ·, C] (intuitionistic)

and Aω[X, ·, ·, C] (classical). One typically uses proof interpretations to extract the necessary quantitative information. Metatheorems guaranteeing this fact were developed by Gerhardy and Kohlenbach in the 2000s. A sample metatheorem is the following, for classical logic, which uses G¨

• del’s functional interpretation, in its “monotone” variant

introduced by Kohlenbach, combined with the negative translation.

Andrei Sipos

,

Proof mining in convex optimization

Proof interpretations

We have mainly two proof interpretations at our disposal: monotone modified realizability, which:

can extract bounds for all kinds of formulas; does not permit the use of excluded middle;

monotone functional interpretation (combined with negative translation), which:

can extract bounds only for Π2 (that is, ∀∃) formulas; permits the use of excluded middle.

These “interpretations” have corresponding metatheorems, which can be used to extract the required quantitative information. In some cases, where no set of restrictions is met, the two may be used in conjunction – see, e.g., Leus ¸tean (2014), A.S. (2016).

Andrei Sipos

,

Proof mining in convex optimization

What does “quantitative” mean?

Let us see what kind of information we might hope to extract. An example from nonlinear analysis would be a limit statement of the form: ∀ε > 0∃Nε∀N ≥ Nε(xn − Anxn < ε). What we want to get is a “formula” for Nε in terms of (obviously) ε and of some other arguments parametrizing our situation. Such a function is called a rate of convergence for the sequence. As the formula above is not in a Π2/∀∃ form, and we generally work with the monotone functional interpretation, in some cases we are forced to only quantify its Herbrand normal form and obtain its so-called rate of metastability (in the sense of T. Tao).

Andrei Sipos

,

Proof mining in convex optimization

Our setting

Let us describe our first tentative object of study. Set H to be a Hilbert space. We say that a multi-valued operator A : H → 2H is monotone if for all x, y, u, v ∈ H with u ∈ A(x) and v ∈ A(y) we have that x − y, u − v ≥ 0. We call it maximally monotone if it is maximal among monotone

• perators considered as subsets of H × H.

The proximal point algorithm’s goal is to find zeroes of A, i.e. points x ∈ H s.t. 0 ∈ A(x).

Andrei Sipos

,

Proof mining in convex optimization

Resolvents

The main tool to use is the resolvent of A – that is, the mapping defined by: JA := (id + A)−1. We have the following classical results: if A is maximally monotone, then JA is single-valued; for all x ∈ H, x is a zero of A iff x is a fixed point of JA. Now we can state the PPA theorem.

Andrei Sipos

,

Proof mining in convex optimization

The PPA theorem

Theorem (Proximal Point Algorithm) Let A : H → 2H be a maximally monotone operator that has at least one zero and let (γn)n∈N ⊆ (0, ∞) be such that

∞

n=0 γ2 n = ∞. Let x ∈ H. Set x0 := x and for all n ∈ N,

xn+1 := JγnAxn. Then the sequence (xn)n∈N converges weakly to a zero of A. A particular case of it, which we shall analyse first is: if the set of zeroes of A has nonempty interior, then the convergence is strong.

Andrei Sipos

,

Proof mining in convex optimization

Metastability

This is where we are going to use the idea of metastability announced before. Metastability can be formulated as: ∀k ∈ N∀g : N → N∃n∀i, j ∈ [n, n + g(n)]d(xi, xj) ≤ 1 k + 1, which can be seen to be a Π2 statement in the extended system. A rate of metastability will be a bound Ψ(k, g) on n.

Andrei Sipos

,

Proof mining in convex optimization

The “nonempty interior” case – quantitatively

Theorem (L.L., A.N., A.S.) For any g : N → N, define χg : N → N recursively, as follows: χg(0) := 0 χg(n + 1) := χg(n) + g(χg(n)) Set now, for any k ∈ N and g : N → N, Φb,r(k, g) := χg

• b2(k + 1)

2r

• .

Then, in this case, Φb,r is a rate of metastability for (xn)n∈N. This is a simple application of Tao’s “finite convergence principle”.

Andrei Sipos

,

Proof mining in convex optimization

The uniform case

The most interesting case for bound extraction will be when the

• perator is “uniformly monotone”, i.e. satisfies the stronger

inequality: x − y, u − v ≥ φ(x − y) with respect to an increasing function φ : [0, ∞) → [0, ∞) which vanishes only at 0. In this case, it is known that the zero is unique and the convergence is necessarily strong.

Andrei Sipos

,

Proof mining in convex optimization

Briseid’s work

Following Briseid’s work in his PhD thesis, one might try to quantify the following uniqueness statement for a fixed point: ∀x∀y(Tx = x ∧ Ty = y → x = y). For that, we exploit the implementation of the equality sign in our system, i.e. we write the above as: ∀x∀y(∀δ(d(Tx, x) ≤ δ ∧ d(Ty, y) ≤ δ) → ∀ε(d(x, y) ≤ ε).

Andrei Sipos

,

Proof mining in convex optimization

The modulus of uniqueness

The quantified version is: ∀x∀y∀ε((d(Tx, x) ≤ δ(ε) ∧ d(Ty, y) ≤ δ(ε)) → (d(x, y) ≤ ε), where δ(ε) is the quantity extracted by proof mining, called the “modulus of uniqueness”. Eyvind Briseid has observed in his PhD thesis that this modulus together with an associated asymptotic regularity lemma might help in obtaining the rate of convergence directly, regardless of the principles used in the proof (except in said lemma).

Andrei Sipos

,

Proof mining in convex optimization

Our case

In our case, given that we use a whole family of mappings for which the zero sought is a fixed point, we had to modify the idea, so that we use the modulus of uniqueness only implicitly. The relevant lemma is the following: Lemma In this framework, if x ∈ H and z is a zero of A, we have that: γφ(d(JγAx, z)) ≤ d(x, JγAx)d(JγAx, z). The corresponding asymptotic regularity result will be: Lemma Let Σb : N → N be defined, for all k, by Σb,θ(k) := θ(b2(k + 1)2). For all k ∈ N and all n ≥ Σb,θ(k), we have that d(xn, xn+1) γn ≤ 1 k + 1.

Andrei Sipos

,

Proof mining in convex optimization

The rate of convergence

Putting all this together, we obtain our main quantitative result, which is: Theorem (L.L., A.N., A.S.) Set, for any k ∈ N, Ψb,θ,φ(k) := Σb,θ

     

2b φ

• 1

k+1

• 

     + 1,

where Σb,θ is the one from the lemma before. Then Ψb,θ,φ is a rate of convergence for (xn)n∈N.

Andrei Sipos

,

Proof mining in convex optimization

Convergence in finitely many steps

A final quantitative result that we have obtained claims that in some cases, the algorithm converges in a computed finite number

• f steps.

Theorem (L.L., A.N., A.S.) Suppose that there is an η > 0 such that for all n ∈ N and for all w ∈ C with w − z ≤ η we have that Tnw = z. Then for all n ≥ ∆b

• 2

η

• + 1, we have that xn = z, where ∆b is defined, for

any k, by ∆b(k) := b2(k + 1)2.

Andrei Sipos

,

Proof mining in convex optimization

The greater picture

The name “proximal point algorithm” denotes a whole family of such iterative schemes, involving not only Hilbert spaces, but more general ones like CAT(0) spaces. Also, it may be used to find minima of convex functions and fixed points of nonexpansive mappings. Remember that we started with a sequence (γn)n∈N ⊆ (0, ∞) and we associated at each step n the resolvent of order γn of our maximally monotone operator. All known PPAs use this schema, which raises the question whether this could be naturally generalized.

Andrei Sipos

,

Proof mining in convex optimization

Joint families

Definition (L.L., A.N., A.S.) Let X be a CAT(0) space, (Tn : X → X)n∈N be a family of mappings and (γn)n∈N ⊆ (0, ∞). We say that the family (Tn)n∈N is jointly firmly nonexpansive with respect to the sequence (γn)n∈N if for all n, m ∈ N, x, y ∈ X and all α, β ∈ [0, 1] such that (1 − α)γn = (1 − β)γm we have that: d(Tnx, Tmy) ≤ d((1 − α)x + αTnx, (1 − β)y + βTmy). Definition (L.L., A.N., A.S.) Let H be a Hilbert space, (Tn : H → H)n∈N be a family of mappings and (γn)n∈N ⊆ (0, ∞). We say that the family (Tn)n∈N is jointly (P2) with respect to the sequence (γn)n∈N if for all n, m ∈ N and all x, y ∈ H we have that: 1 γn Tnx − Tmy, x − Tnx ≥ 1 γm Tnx − Tmy, y − Tmy.

Andrei Sipos

,

Proof mining in convex optimization

Technical issues

This “jointly (P2)” condition can also be expressed in general CAT(0) spaces by: 1 γn − − − − − → TnxTmy, − − − → xTnx ≥ 1 γm − − − − − → TnxTmy, − − − → yTmy, where ·, · is the quasi-linearization mapping introduced by Berg and Nikolaev. This condition, however, is strictly more general than the jointly firmly nonexpansive one.

Andrei Sipos

,

Proof mining in convex optimization

Joint families – the facts

The most striking facts are: all known instances of the PPA can be shown to yield jointly (P2) families; any jointly (P2) family is jointly firmly nonexpansive;

• ne may prove the convergence of the PPA just assuming

jointly firmly nonexpansive mappings in CAT(0) spaces. All quantitative results presented before may be expressed in this general framework, including the uniform ones (which unify, e.g., the problem of finding zeroes uniformly monotone mappings from before with the one for finding minimizers of uniformly convex mappings).

Andrei Sipos

,

Proof mining in convex optimization

Thank you for your attention.

Andrei Sipos

,

Proof mining in convex optimization