Can One Genuinely Split m > 2 Monotone Operators? P . L. - PowerPoint PPT Presentation

Inclusions Can One Genuinely Split m > 2 Monotone Operators? P . L. Combettes Laboratoire Jacques-Louis Lions Facult´ e de Math´ ematiques Universit´ e Pierre et Marie Curie – Paris 6 75005 Paris, France Playa Blanca – 14 Octubre 2013 P . L. Combettes Monotone operator splitting 1/ 15

Inclusions Notation H , H i , G , G i : real Hilbert spaces. B ( H , G ) bounded linear operators from H to G . A : H → 2 H a set-valued operator. � � Graph of A : gra A = ( x , u ) ∈ H × H | u ∈ Ax . � � Zeros of A : zer A = x ∈ H | 0 ∈ Ax . Inverse of A : gra A − 1 = � � ( u , x ) ∈ H × H | u ∈ Ax . Resolvent of A : J A = ( Id + A ) − 1 . Parallel sum of A and B : A � B = ( A − 1 + B − 1 ) − 1 . P . L. Combettes Monotone operator splitting 2/ 15

Inclusions Monotone operators A : H → 2 H is monotone if ( ∀ ( x , u ) ∈ gra A )( ∀ ( y , v ) ∈ gra A ) � x − y | u − v � � 0 , and maximally monotone if there exists no monotone operator B : H → 2 H such that gra A ⊂ gra B � = gra A . If A is maximally monotone, its resolvent J A = ( Id + A ) − 1 is single- valued, defined everywhere (Minty), and firmly nonexpansive : � J A x − J A y � 2 + � ( Id − J A ) x − ( Id − J A ) y � 2 � � x − y � 2 . Moreover, J A + J A − 1 = Id and Fix J A = zer ( A ) . H. H. Bauschke and PLC, Convex Analysis and Monotone Operator The- ory in Hilbert Spaces, Springer, 2011. P . L. Combettes Monotone operator splitting 3/ 15

Inclusions The proximal point algorithm Many problems in nonlinear analysis can be reduced to where C : H → 2 H is maximally monotone. find x ∈ zer C , P . L. Combettes Monotone operator splitting 4/ 15

Inclusions The proximal point algorithm Many problems in nonlinear analysis can be reduced to where C : H → 2 H is maximally monotone. find x ∈ zer C , This inclusion can be solved by the proximal point algorithm x n + 1 = J γ n C x n , (1) n ∈ N γ 2 where ( γ n ) n ∈ N lies in ] 0 , + ∞ [ and � n = + ∞ . H. Br´ ezis and P .-L. Lions, Produits infinis de r´ esolvantes, Israel J. Math., vol. 29, pp. 329-345, 1978. P . L. Combettes Monotone operator splitting 4/ 15

Inclusions The proximal point algorithm Many problems in nonlinear analysis can be reduced to where C : H → 2 H is maximally monotone. find x ∈ zer C , This inclusion can be solved by the proximal point algorithm x n + 1 = J γ n C x n , (1) n ∈ N γ 2 where ( γ n ) n ∈ N lies in ] 0 , + ∞ [ and � n = + ∞ . H. Br´ ezis and P .-L. Lions, Produits infinis de r´ esolvantes, Israel J. Math., vol. 29, pp. 329-345, 1978. Unfortunately, in most situations, (1) is not implementable be- cause the resolvents of C are too hard to compute. P . L. Combettes Monotone operator splitting 4/ 15

Inclusions The proximal point algorithm Many problems in nonlinear analysis can be reduced to where C : H → 2 H is maximally monotone. find x ∈ zer C , This inclusion can be solved by the proximal point algorithm x n + 1 = J γ n C x n , (1) n ∈ N γ 2 where ( γ n ) n ∈ N lies in ] 0 , + ∞ [ and � n = + ∞ . H. Br´ ezis and P .-L. Lions, Produits infinis de r´ esolvantes, Israel J. Math., vol. 29, pp. 329-345, 1978. Unfortunately, in most situations, (1) is not implementable be- cause the resolvents of C are too hard to compute. Splitting methods: Decompose C in terms of operators which are simpler (i.e., they can be used explicitly or have easily com- putable resolvents), and devise an algorithm which employs these operators individually. P . L. Combettes Monotone operator splitting 4/ 15

Inclusions Splitting methods: Some hard facts of life One knows how to split only two operators: 0 ∈ Ax + Bx . P . L. Combettes Monotone operator splitting 5/ 15

Inclusions Splitting methods: Some hard facts of life One knows how to split only two operators: 0 ∈ Ax + Bx . There exist only only three splitting schemes. P . L. Combettes Monotone operator splitting 5/ 15

Inclusions Splitting methods: Some hard facts of life One knows how to split only two operators: 0 ∈ Ax + Bx . There exist only only three splitting schemes. Yet, we want to solve systems of monotone inclusions such as find x 1 ∈ H 1 , . . . , x m ∈ H m such that  K m � � ��  � L ∗ �  z 1 ∈ A 1 x 1 + ( B k � D k ) L ki x i − r k + C 1 x 1  k 1    k = 1 i = 1  . . . K m  � � ��  � � L ∗  z m ∈ A m x m + ( B k � D k ) L ki x i − r k + C m x m ,  km    k = 1 i = 1 for instance [inf-convolution: g k � ℓ k : y �→ inf t g k ( t ) + ℓ k ( y − t ) ] m K m m � � � � � � minimize f i ( x i )+ ( g k � ℓ k ) L ki x i − r k + h i ( x i ) −� x i | z i � x 1 ∈H 1 ,..., x m ∈H m i = 1 k = 1 i = 1 i = 1 P . L. Combettes Monotone operator splitting 5/ 15

Inclusions Early example: Legendre’s method of least squares Set m = 1, z 1 = 0, H 1 = R N , L k 1 = Id, A 1 = C 1 = 0, D k = Id, and  u k ∈ R N � span { u k } , if � x | u k � = ρ k ;   B k : x �→ where � u k � = 1 Ø , if � x | u k � � = ρ k ,  ρ k ∈ R .  Then the problem becomes m � |� x | u k � − ρ k | 2 , minimize x ∈ R N k = 1 which is precisely Legendre’s least squares method for solving the overdetermined system � x | u k � = ρ k , 1 � k � K . A. M. Legendre, Nouvelles M´ ethodes pour la D´ etermination de l’Orbite des Com` etes. Courcier, Paris, 1805. C. F. Gauss, Theoria Motus Corporum Coelestium. Perthes and Besser, Hamburg, 1809. P . L. Combettes Monotone operator splitting 6/ 15

Inclusions Basic splitting schemes for 0 ∈ Ax + Bx Douglas-Rachford algorithm: γ ∈ ] 0 , + ∞ [ . � � ��� 1 � zer ( A + B ) = J γ B Fix ( 2 J γ A − Id ) ◦ ( 2 J γ B − Id ) + Id . 2 Iterate � x n = J γ B y n (backward step) � � y n + 1 = J γ A 2 x n − y n + y n − x n (backward step) Then y n ⇀ y and z = J γ B y ∈ zer ( A + B ) (Lions&Mercier, 1979 ), and x n ⇀ z ∈ zer ( A + B ) . ADMM, method of partial inverses are essentially special cases. There are tricks to reduce m -operator problems to 2- operator problems in product spaces [Spingarn (1983), PLC (2009), Brice˜ no-PLC (2011)] and use Douglas-Rachford splitting. P . L. Combettes Monotone operator splitting 7/ 15

Inclusions Basic splitting schemes for 0 ∈ Ax + Bx Forward-Backward algorithm: γ ∈ ] 0 , + ∞ [ . B : H → H is β -cocoercive: � x − y | Bx − By � � β � Bx − By � 2 ; γ ∈ ] 0 , 2 β [ . � �� � zer ( A + B ) = Fix J γ A Id − γ B . Iterate � y n = x n − γ Bx n (forward step) x n + 1 = J γ A y n (backward step) Then x n ⇀ z ∈ zer ( A + B ) (Mercier, 1979 ) There are tricks to use the forward-backward algorithm (on the dual problem if the primal is strongly monotone, in primal-dual spaces, in renormed spaces) to solve m - operator problems; see [PLC&V˜ u, (2013)] P . L. Combettes Monotone operator splitting 8/ 15

Inclusions Basic splitting schemes for 0 ∈ Ax + Bx Forward-Backward-Forward algorithm: γ ∈ ] 0 , + ∞ [ . � �� � zer ( A + B ) = Fix J γ A Id − γ B . B : H → H is 1 /β -Lipschitzian; 0 < γ n < β . Iterate  y n = x n − γ Bx n (forward step)   p n = J γ A y n (backward step)   q n = p n − γ Bp n (forward step)  x n + 1 = x n − y n + q n Then x n ⇀ z ∈ zer ( A + B ) [Tseng (2000)] There are tricks to use the forward-backward-forward algo- rithm to obtain fully split algorithms for rather complex struc- tured monotone inclusion problems, such as... P . L. Combettes Monotone operator splitting 9/ 15

Inclusions Multivariate structured inclusion problem find x ∈ H such that z ∈ Ax + Bx (2) where: z ∈ H , A : H → 2 H is maximally monotone B : H → 2 H is maximally monotone P . L. Combettes Monotone operator splitting 10/ 15

Inclusions Multivariate structured inclusion problem find x ∈ H such that z ∈ Ax + L ∗ B ( Lx − r ) (2) where: z ∈ H , A : H → 2 H is maximally monotone B : G → 2 G is maximally monotone, r ∈ G , L ∈ B ( H , G ) P . L. Combettes Monotone operator splitting 10/ 15

Inclusions Multivariate structured inclusion problem find x ∈ H such that K � L ∗ z ∈ Ax + k B k ( L k x − r k ) (2) k = 1 where: z ∈ H , A : H → 2 H is maximally monotone B k : G k → 2 G k is maximally monotone, r k ∈ G k , L k ∈ B ( H , G k ) P . L. Combettes Monotone operator splitting 10/ 15

Inclusions Multivariate structured inclusion problem find x ∈ H such that K � L ∗ z ∈ Ax + k ( B k � D k )( L k − r k x ) (2) k = 1 where: z ∈ H , A : H → 2 H is maximally monotone B k : G k → 2 G k is maximally monotone, r k ∈ G k , L k ∈ B ( H , G k ) D k : G k → 2 G k is maximally monotone, D − 1 is ν k -Lipschitzian, k B k � D k = ( B − 1 + D − 1 k ) − 1 k P . L. Combettes Monotone operator splitting 10/ 15