Sherman-Morrison-Woodbury formula for Sylvester and T -Sylvester - PowerPoint PPT Presentation

Sherman-Morrison-Woodbury formula for Sylvester and T -Sylvester equation Ivana Kuzmanović Department of Mathematics, University of Osijek, Croatia I.Kuzmanović (University of Osijek, Croatia) 1 / 9

Problem We consider the Sylvester equation ( A 0 + U 1 C 1 V 1 ) X + X ( B 0 + U 2 C 2 V 2 ) = E , (1) 1 ∈ R m × r 1 , C 1 ∈ R r 1 × r 1 , B 0 ∈ R n × n , where A 0 ∈ R m × m , U 1 , V T U 2 , V T 2 ∈ R n × r 2 , C 2 ∈ R r 2 × r 2 , X , E ∈ R m × n . I.Kuzmanović (University of Osijek, Croatia) 2 / 9

Problem We consider the Sylvester equation ( A 0 + U 1 C 1 V 1 ) X + X ( B 0 + U 2 C 2 V 2 ) = E , (1) 1 ∈ R m × r 1 , C 1 ∈ R r 1 × r 1 , B 0 ∈ R n × n , where A 0 ∈ R m × m , U 1 , V T U 2 , V T 2 ∈ R n × r 2 , C 2 ∈ R r 2 × r 2 , X , E ∈ R m × n . We assume r 1 , r 2 ≪ m , n and that C 1 , C 2 are regular matrices I.Kuzmanović (University of Osijek, Croatia) 2 / 9

Problem We consider the Sylvester equation ( A 0 + U 1 C 1 V 1 ) X + X ( B 0 + U 2 C 2 V 2 ) = E , (1) 1 ∈ R m × r 1 , C 1 ∈ R r 1 × r 1 , B 0 ∈ R n × n , where A 0 ∈ R m × m , U 1 , V T U 2 , V T 2 ∈ R n × r 2 , C 2 ∈ R r 2 × r 2 , X , E ∈ R m × n . We assume r 1 , r 2 ≪ m , n and that C 1 , C 2 are regular matrices Goal: efficient method for solving equation (1) many times with different update matrices U 1 , C 1 , V 1 , U 2 , C 2 , V 2 . I.Kuzmanović (University of Osijek, Croatia) 2 / 9

Basic idea Using Kronecker product and vectorization operator, as well as equality vec ( ABC ) = ( C T ⊗ A ) vec ( B ) , equation (1) can be rewritten as mn × mn matrix equation ( I n ⊗ ( A 0 + U 1 C 1 V 1 ) + ( B 0 + U 2 C 2 V 2 ) T ⊗ I m ) vec ( X ) = vec ( E ) I.Kuzmanović (University of Osijek, Croatia) 3 / 9

Basic idea Using Kronecker product and vectorization operator, as well as equality vec ( ABC ) = ( C T ⊗ A ) vec ( B ) , equation (1) can be rewritten as mn × mn matrix equation ( I n ⊗ ( A 0 + U 1 C 1 V 1 ) + ( B 0 + U 2 C 2 V 2 ) T ⊗ I m ) vec ( X ) = vec ( E ) and further as equation ( L 0 + UCV ) vec ( X ) = vec ( E ) I.Kuzmanović (University of Osijek, Croatia) 3 / 9

Basic idea Using Kronecker product and vectorization operator, as well as equality vec ( ABC ) = ( C T ⊗ A ) vec ( B ) , equation (1) can be rewritten as mn × mn matrix equation ( I n ⊗ ( A 0 + U 1 C 1 V 1 ) + ( B 0 + U 2 C 2 V 2 ) T ⊗ I m ) vec ( X ) = vec ( E ) and further as equation ( L 0 + UCV ) vec ( X ) = vec ( E ) where I n ⊗ A 0 + B T 0 ⊗ I m ∈ R mn × mn , L 0 = I n ⊗ U 1 V T ∈ R mn × ( r 1 n + r 2 m ) , � � U = 2 ⊗ I m � I n ⊗ C 1 � 0 ∈ R ( r 1 n + r 2 m ) × ( r 1 n + r 2 m ) , C = C T 0 2 ⊗ I m � I n ⊗ V 1 � ∈ R ( r 1 n + r 2 m ) × mn . V = U T 2 ⊗ I m I.Kuzmanović (University of Osijek, Croatia) 3 / 9

Basic idea ( A 0 + U 1 C 1 V 1 ) X + X ( B 0 + U 2 C 2 V 2 ) = E � ( L 0 + UCV ) vec ( X ) = vec ( E ) I.Kuzmanović (University of Osijek, Croatia) 4 / 9

Basic idea ( A 0 + U 1 C 1 V 1 ) X + X ( B 0 + U 2 C 2 V 2 ) = E � ( L 0 + UCV ) vec ( X ) = vec ( E ) After applying the Sherman-Morrison-Woodbury formula on the right hand side of the previous equation, we obtain 0 U ( C − 1 + VL − 1 vec ( X ) = ( L − 1 − L − 1 0 U ) − 1 VL − 1 0 ) vec ( E ) . 0 I n ⊗ A 0 + B T 0 ⊗ I m ∈ R mn × mn , = L 0 ∈ R mn × ( r 1 n + r 2 m ) , � I n ⊗ U 1 V T � = 2 ⊗ I m U � I n ⊗ C 1 � 0 ∈ R ( r 1 n + r 2 m ) × ( r 1 n + r 2 m ) , C = C T 0 2 ⊗ I m � I n ⊗ V 1 � ∈ R ( r 1 n + r 2 m ) × mn . V = U T 2 ⊗ I m I.Kuzmanović (University of Osijek, Croatia) 4 / 9

Special case Special case that we have considered: optimization of the solution of the parameter-dependent Sylvester equation ( A 0 − vU 1 V 1 ) X ( v ) + X ( v )( B 0 − vU 2 V 2 ) = E . We want to optimize the solution X ( v ) of the above Sylvester equation (i.e. to determine optimal parameter v ∈ R ) with respect to some optimization criterion. I.Kuzmanović (University of Osijek, Croatia) 5 / 9

Special case Special case that we have considered: optimization of the solution of the parameter-dependent Sylvester equation ( A 0 − vU 1 V 1 ) X ( v ) + X ( v )( B 0 − vU 2 V 2 ) = E . We want to optimize the solution X ( v ) of the above Sylvester equation (i.e. to determine optimal parameter v ∈ R ) with respect to some optimization criterion. Optimization criteria: tr ( X ( v )) → min for m = n , � X ( v ) � F → min I.Kuzmanović (University of Osijek, Croatia) 5 / 9

Special case vec ( X ) = ( L − 1 − v L − 1 0 U ( I + v ∆) − 1 VL − 1 0 ) vec ( E ) 0 ∆ = VL − 1 0 U We need efficient methods for solving system L 0 x = y L − 1 given or A 0 ˆ X + ˆ XB 0 = ˆ Y where vec (ˆ X ) = x , vec ( ˆ Y ) = y . 0 matrix - vector product with U and V � � � � ( I n ⊗ V 1 ) vec ( ˆ vec ( V 1 ˆ � � Y ) Y ) I n ⊗ V 1 vec ( ˆ V y = Y ) = = U T 2 ⊗ I m ) vec ( ˆ vec ( ˆ ( U T , 2 ⊗ I m Y ) Y U 2 ) solving system with I + v ∆ Krylov subspace methods (e.g. Full Orthogonalization Method ) I.Kuzmanović (University of Osijek, Croatia) 6 / 9

Number of operations Let: O ( r 1 ) = O ( r 2 ) =: r , m = n Calculation vec ( X ( v )) for the first value of v : 8 ( rk + r + k + 2 ) m 2 + O ( rmk ) . I.Kuzmanović (University of Osijek, Croatia) 7 / 9

Number of operations Let: O ( r 1 ) = O ( r 2 ) =: r , m = n Calculation vec ( X ( v )) for the first value of v : 8 ( rk + r + k + 2 ) m 2 + O ( rmk ) . For every additional evaluation of vec ( X ( v )) ( 4 r + 8 ) m 2 + O ( rmk ) . I.Kuzmanović (University of Osijek, Croatia) 7 / 9

Number of operations Let: O ( r 1 ) = O ( r 2 ) =: r , m = n Calculation vec ( X ( v )) for the first value of v : 8 ( rk + r + k + 2 ) m 2 + O ( rmk ) . For every additional evaluation of vec ( X ( v )) ( 4 r + 8 ) m 2 + O ( rmk ) . Additional number of flops for vec ( X ′ ( v )) and vec ( X ′′ ( v )) is for each ( 4 r + 8 ) m 2 + O ( rmk ) where k ≪ m I.Kuzmanović (University of Osijek, Croatia) 7 / 9

Flops number growth for algorithms lyap, old, and new for the case of r=1,2,3,4 r=1 r=2 10 Number of flops 10 10 Number of flops 10 lyap lyap 5 10 5 old 10 old new new 0 1000 2000 3000 3500 0 1000 2000 3000 3500 m m r=3 r=4 10 Number of flops Number of flops 10 10 10 lyap lyap 5 5 10 10 old old new new 0 1000 2000 3000 3500 0 1000 2000 3000 3500 m m I.Kuzmanović (University of Osijek, Croatia) 8 / 9

T -Sylvester equation Similarly as for Sylvester equation, Sherman-Morrison-Woodbury formula can be used for updating the solution of equation ( A 0 + U 1 C 1 V 1 ) X + X T ( B 0 + U 2 C 2 V 2 ) = E . Corresponding matrices are I ⊗ A 0 + ( B T L ′ = 0 ⊗ I )Π 0 � � U ′ I ⊗ U 1 V T = 2 ⊗ I � � I ⊗ C 1 0 C ′ = C T 0 2 ⊗ I � � I ⊗ V 1 V ′ = ( U T 2 ⊗ I )Π , where Π vec ( X ) = vec ( X T ) . I.Kuzmanović (University of Osijek, Croatia) 9 / 9