On projections onto polyhedral sets and applications to primal-dual - PowerPoint PPT Presentation

On projections onto polyhedral sets and applications to primal-dual projection algorithms for solving maximally monotone inclusion problem Krzysztof Rutkowski Faculty of Mathematics and Information Science Warsaw University of Technology Systems Research Institute Polish Academy of Sciences Paris 23.07.2018

Table of contents Problem statement 1 Operator inclusions problem Form of projection algorithms Projection onto polyhedral sets Projection algorithms 2 Fej´ er algorithm Best approximation algorithm Best approximation algorithms with memory Extension to systems of monotone inclusions 3 Extended Best Approximation Primal-Dual Algorithm with memory Block-Asynchronous Primal-Dual Algorithm with memory Application 4

Problem statement Projection algorithms Extension to systems of monotone inclusions Application Operator inclusions problem Let H,G be Hilbert spaces, A : H → H , B : G → G be maximally monotone operators and L : H → G be a linear, bounded continuous operator. We are interested in finding a point p ∈ H which solves the following inclusion problem 0 ∈ Ap + L ∗ BLp . (P) Within the generalized Fenchel - Rockafellar duality framework the dual inclusion problem to (P) is to find v ∗ ∈ G such that 0 ∈ − LA − 1 ( − L ∗ v ∗ ) + B − 1 v ∗ . (D) A point p ∈ H solves (P) if and only if v ∗ ∈ G solves (D) and ( p , v ∗ ) ∈ Z , where Z := { ( p , v ∗ ) ∈ H × G | − L ∗ v ∗ ∈ Ap Lp ∈ B − 1 v ∗ } . and Z is a closed convex subset of H × G . 3 / 22

Problem statement Projection algorithms Extension to systems of monotone inclusions Application Form of projection algorithms In this presentation we consider two types of primal-dual projection algorithms: Fejer-type algorithms Haugazeau-type algorithms Haugazeau-type Fejer-type 1: Let x 0 ∈ H × G 1: Let x 0 ∈ H × G 2: for n = 0 , 1 , . . . do 2: for n = 0 , 1 , . . . do Let C n be closed convex Let C n be closed convex 3: 3: such that Z ⊂ C n such that Z ⊂ C n 4: 4: x n +1 = P C n ( x n ) x n +1 = P H ( x 0 , x n ) ∩ C n ( x 0 ) 5: 5: where H ( a , b ) = { h ∈ H × G | � h − a | a − b � ≤ 0 } . Weak convergence of Fejer-type algorithims and strong convergence of Haugazeau-type algorithms to ¯ z ∈ Z can be shown under some assumptions on the choice of C n . When C n are polyhedral sets (i.e intersection of halfspaces) exact or closed-form expression formulas need to be used to calculate the projection P C n ( · ). 4 / 22

Problem statement Projection algorithms Extension to systems of monotone inclusions Application Projection onto polyhedral sets In the literature following formulas for projections are known: When C is a halfspace i.e C = { h ∈ H × G | � h | s � ≤ η } and x / ∈ C then 1 P C ( x ) = x − � x | s � − η s � s � 2 When C is an intersection of two halfspaces closed-form formulas are provided by 2 Bauschke Combettes Heinz H. Bauschke and Patrick L. Combettes. Convex analysis and monotone operator theory in Hilbert spaces . CMS Books in Mathematics/Ouvrages de Math´ ematiques de la SMC. With a foreword by H´ edy Attouch. Springer, New York, 2011, pp. xvi+468. isbn : 978-1-4419-9466-0. doi : 10.1007/978-1-4419-9467-7 . url : http://dx.doi.org/10.1007/978-1-4419-9467-7 General form of projection formula onto poyhedral set is given by Deutsch 3 m � P C ( x ) = x − λ i s i , i =1 where λ i ≥ 0, i = 1 , . . . , m satisfy m � � x − λ j s j | s i � − η i ≤ 0 , i = 1 , . . . , m , j =1 m � λ i ( � x − λ j s j | s i � − η i ) = 0 , i = 1 , . . . , m . j =1 5 / 22

Problem statement Projection algorithms Extension to systems of monotone inclusions Application In the paper Krzysztof E. Rutkowski. “Closed-Form Expressions for Projectors onto Polyhedral Sets in Hilbert Spaces”. In: SIAM Journal on Optimization 27.3 (2017), pp. 1758–1771. doi : 10.1137/16M1087540 . eprint: https://doi.org/10.1137/16M1087540 . url : https://doi.org/10.1137/16M1087540 we propose the closed-form formulas for general polyhedral in a form C n = { h ∈ H × G | � h | s i � ≤ η i , i = 1 , . . . , m } The formula requires to check at most 2 m − 1 conditions for finding an exact projection. The number of conditions to check can be reduced dependently on the structure of the polyhedral set. 6 / 22

Problem statement Projection algorithms Extension to systems of monotone inclusions Application Notation In the next presented algorithms we will use the following notation: For any monotone operator S : H → H the resolvent operator J S : H → H is given by J S ( · ) = ( Id − S ) − 1 ( · ). For any ( a , a ∗ ) ∈ gra A , ( b , b ∗ ) ∈ gra B , a , b := ( a ∗ + L ∗ b ∗ , b − La ) , s ∗ η a , b := � a | a ∗ � + � b | b ∗ � , H a , b := { x ∈ H × G | � x | s ∗ a , b � ≤ η a , b } . 7 / 22

Problem statement Projection algorithms Extension to systems of monotone inclusions Application Fej´ er algorithm Let x 0 = ( p 0 , v ∗ 0 ) ∈ H × G , ε ∈ (0 , 1) and iterate 1: for n = 0 , 1 , . . . do Pick ( γ n , µ n ) ∈ [ ε, 1 /ε ] 2 2: a n = J γ n A ( p n − γ n L ∗ v ∗ n ), a ∗ γ n ( p n − γ n L ∗ v ∗ 1 n = n − a n ) 3: b n = J µ n B ( Lp n + µ n v ∗ n ), b ∗ µ n ( Lp n + µ n v ∗ 1 n = n − b n ) 4: if s ∗ a n , b n = 0 then 5: v ∗ = b ∗ p = a n , ¯ ¯ 6: n terminate 7: else 8: ( p n +1 , v ∗ n +1 ) = P H an , bn ( p n , v ∗ n ) 9: Theorem Suppose, that set Z is nonempty. Let x 0 = ( p 0 , v ∗ 0 ) ∈ H × G, ε ∈ (0 , 1) . The following holds ( p n , v ∗ n ) n ∈ N is Fej´ er monotone with respect to Z, (i) n ∈ N � p n +1 − p n � 2 < ∞ and � n � 2 < ∞ , n ∈ N � v ∗ n +1 − v ∗ � (ii) v ∗ ( p n ) n ∈ N converges weakly to a point ¯ p, ( v n ) n ∈ N converges weakly to a point ¯ (iii) v ∗ ) ∈ Z. and (¯ p , ¯ A. Alotaibi, P. L. Combettes, and N. Shahzad. “Solving Coupled Composite Monotone Inclusions by Successive Fej´ er Approximations of their Kuhn-Tucker Set.”. In: SIAM Journal on Optimization 24.4 (2014), pp. 2076–2095. doi : 10.1137/130950616 . eprint: http://dx.doi.org/10.1137/130950616 . url : http://dx.doi.org/10.1137/130950616 8 / 22

Problem statement Projection algorithms Extension to systems of monotone inclusions Application Best approximation algorithm Algorithm: 1: for n = 0 , 1 , . . . do Find ( γ n , µ n ) ∈ [ ε, 1 /ε ] 2 2: a n = J γ n A ( p n − γ n L ∗ v ∗ n ), a ∗ γ n ( p n − γ n L ∗ v ∗ 1 n = n − a n ) 3: b n = J µ n B ( Lp n + µ n v ∗ n ), b ∗ µ n ( Lp n + µ n v ∗ 1 n = n − b n ) 4: if s ∗ a n , b n = 0 then 5: v ∗ = b ∗ p = a n , ¯ ¯ 6: n terminate 7: else 8: ( p n +1 , v ∗ n )) ∩ H an , bn ( p 0 , v ∗ n +1 ) = P H ( x 0 , ( p n , v ∗ 0 ) 9: Theorem Suppose, that set Z is nonempty. Let x 0 = ( p 0 , v ∗ 0 ) ∈ H × G, ε ∈ (0 , 1) . The following holds � ( p n +1 , v ∗ n +1 ) − x 0 � > � ( p n , v ∗ n ) − x 0 � for all n = 0 , 1 . . . , (i) n ∈ N � p n +1 − p n � 2 < ∞ and � n � 2 < ∞ . n ∈ N � v ∗ n +1 − v ∗ � (ii) v ∗ and (iii) ( p n ) n ∈ N converges to a point ¯ p, ( v n ) n ∈ N converges to a point ¯ v ∗ ) = P Z ( x 0 ) . (¯ p , ¯ Abdullah Alotaibi, Patrick L. Combettes, and Naseer Shahzad. “Best approximation from the Kuhn-Tucker set of composite monotone inclusions”. In: Numer. Funct. Anal. Optim. 36.12 (2015), pp. 1513–1532. issn : 0163-0563. doi : 10.1080/01630563.2015.1077864 . url : http://dx.doi.org/10.1080/01630563.2015.1077864 9 / 22

Problem statement Projection algorithms Extension to systems of monotone inclusions Application Best approximation algorithm with memory Algorithm 1 1: for n = 0 , 1 , . . . do Find ( γ n , µ n ) ∈ [ ε, 1 /ε ] 2 2: a n = J γ n A ( p n − γ n L ∗ v ∗ n ), a ∗ γ n ( p n − γ n L ∗ v ∗ 1 n − a n ) n = 3: 1 b n = J µ n B ( Lp n + µ n v ∗ n ), b ∗ µ n ( Lp n + µ n v ∗ n = n − b n ) 4: if s ∗ a n , b n = 0 then 5: v ∗ = b ∗ ¯ p = a n , ¯ 6: n terminate 7: else 8: Let C n be closed convex set such that Z ⊂ C n ⊂ H a n , b n 9: ( p n +1 , v ∗ n )) ∩ C n ( p 0 , v ∗ n +1 ) = P H ( x 0 , ( p n , v ∗ 0 ) 10: Let H n := H a n , b n . Possible C n choices: 2 C n := H n (without memory), 1 C n := H n ∩ H n − 1 for n ≥ 1 and C 0 = H , 2 C n := H n ∩ H ( x 0 , x n − 1 ) for n ≥ 1 and C 0 = H , 3 C n := H n ∩ H ( x 0 , τ n x n + (1 − τ n ) x n − 1 ) for τ n ∈ (0 , 1) , n ≥ 1 and C 0 = H . 4 E. M. Bednarczuk, A. Jezierska, and K. E. Rutkowski. “Proximal primal-dual best approximation algorithm with memory”. In: ArXiv e-prints (Oct. 2016). arXiv: 1610.08697 [math.OC] 10 / 22