Computing Nearby Non-Trivial Smith Forms Joseph Haraldson with - PowerPoint PPT Presentation

Introduction Theory SNF via Optimization Examples Conclusion Computing Nearby Non-Trivial Smith Forms Joseph Haraldson with Mark Giesbrecht and George Labahn David R. Cheriton School of Computer Science University of Waterloo July 18, 2018 1/19 Haraldson

Introduction Theory SNF via Optimization Examples Conclusion The Smith Normal Form Smith Normal Form (SNF) Any A ∈ R [ t ] n × n is unimodularily equivalent to S = diag ( s 1 , s 2 , . . . , s n ) where s j | s j + 1 and s j ∈ R [ t ] . That is, there exists U , V ∈ R [ t ] n × n such that UAV = S and det( U ) , det( V ) ∈ R \{ 0 } . • The { s j } n j = 1 are the invariant factors • Computing S is well understood in exact-arithmetic • Analyze the SNF as a symbolic-numeric optimization problem 2/19 Haraldson

Introduction Theory SNF via Optimization Examples Conclusion Smith Normal Forms Example (Boring SNF over R [ t ] 3 × 3 )      1  t 3 + 3 t + 1     1 t + 1              t 2 + 2 t + 2    A =    and SNF( A ) = 1    0 0         t 3 + 5 t + 1 t + 1 t + 1 det( A ) det( A ) = t 8 + 2 t 7 + 10 t 6 + 18 t 5 + 34 t 4 + 38 t 3 + 40 t 2 + 12 t . Example (Interesting SNF over R [ t ] 3 × 3 )      t − 1    t + 1 t + 1 1                 t 3     B = 0 t + 1  and SNF( B ) = t + 1            t 2 − 1 ( t + 1)( t 2 − 1) 0 0 3/19 Haraldson

Introduction Theory SNF via Optimization Examples Conclusion SNF Computation in a Floating Point Environment When does A have a non-trivial Smith Normal Form? • Small perturbations to A generically produce a trivial SNF • How far is A from a matrix polynomial � A with non-trivial SNF? • Is there a radius of triviality? • I.e., if A is perturbed by a small amount is the SNF still trivial? When is Computing the SNF Well-Posed? Is there a nearest matrix polynomial � A with an interesting SNF? • Is � A locally unique? • How do we compute � A ? • How do perturbations to A affect � A ? 4/19 Haraldson

Introduction Theory SNF via Optimization Examples Conclusion Nearby SNF via Optimization The McCoy Rank - Number of 1’s in the SNF Formally: McCoy rank of A ∈ R [ t ] n × n is min ω ∈ C rank ( A ( ω )) . Approximations Require a Norm �� �� 0 ≤ k ≤ deg A ij A 2 1 ≤ i , j ≤ n �A ij � 2 �A ij � 2 = and �A� = �A� F = 2 . ijk Main Problem: Nearby Interesting SNF Given A ∈ R [ t ] n × n of McCoy rank at most n − 1 , find � A ∈ R [ t ] n × n that (locally) solves the optimization problem   SNF( �  A ) = diag (ˆ s 1 , ˆ s 2 , . . . , ˆ s n − 1 , ˆ s n ) ,  min �A − � A� such that    deg( s n ) ≥ deg(ˆ s n − 1 ) ≥ 1 . 5/19 Haraldson

Introduction Theory SNF via Optimization Examples Conclusion Our Contributions 1. Tight lower bounds on the radius of triviality 2. Polynomial-time decision procedure for ill-posedness 3. Stability analysis on SNF via Optimization 4. Iterative algorithms with local quadratic convergence • Nearest matrix with reduced McCoy rank • Nearest matrix with McCoy rank at most n − r • Reasonable initial guess heuristics for both algorithms • Polynomial per-iteration cost 5. Implementation in Maple 6/19 Haraldson

Introduction Theory SNF via Optimization Examples Conclusion Previous Work on Floating Point SNF Computations Reduction to Degree One Every matrix polynomial A ∈ R [ t ] n × n can be linearized to for some P 0 , P 1 ∈ R nd × nd . P = P 0 + t P 1 • Extract the SNF from Kronecker’s Canonical Form • SNF( P ) = diag (1 , 1 , . . . , 1 , SNF( A )) Backward Stable: Finds the SNF of a nearby matrix. • Full Rank Case: QZ Algorithm • Wilkinson (1979) • Singular Case: Fast Staircase/Deflation Algorithms • Beelen and Van Dooren (1984,1988) • Current: GUPTRI • Demmel and Edelman (1995) 7/19 Haraldson

Introduction Theory SNF via Optimization Examples Conclusion Applications of Approximate Smith Form • Structured stability of polynomial eigenvalue problems • Matrix polynomial eigenvalue least squares problems • Occurs frequently in control systems engineering • Decide if the SNF can be inferred numerically Our goal is different: Find a nearby matrix with a non-trivial SNF. • Structured backward stability analysis of SNF computations • Detect irrecoverable failures of existing algorithms • SNF of a nearby matrix may be meaningless • Problem is not always continuous • We compute a nearby matrix with an interesting SNF 8/19 Haraldson

Introduction Theory SNF via Optimization Examples Conclusion Reduction to Approximate GCD Example (Find Nearest 2 × 2 matrix with a non-trivial SNF) � � t 2 − 2 t + 1 , t 2 + 2 t + 2 find a lower McCoy rank � C = diag C . Approximate GCD of C 11 and C 22 (Karmarkar and Lakshman ’96) � � � C 11 − � 2 + � C 22 − � gcd( � C 11 , � C 11 � 2 C 22 � 2 s.t. inf C 22 ) � 1 . 2 Assume: � � C 11 = ( c 11 t + c 10 )( h 1 t + 1) and C 22 = ( c 21 t + c 20 )( h 1 t + 1) . The distance to a matrix with a non-trivial SNF is 5 h 14 − 4 h 13 + 14 h 1 + 2 inf = 2 when h 1 = 0 . h 14 + h 12 + 1 h 1 ∈ R Thus gcd( � C 11 , � C 22 ) = 1 at the infima. 9/19 Haraldson

Introduction Theory SNF via Optimization Examples Conclusion Reducing Approximate SNF to Approximate GCD • We can define the SNF in terms of the minors δ j s j = where δ j = GCD { all j × j minors of A } δ j + 1 • Requiring δ j � 1 = ⇒ A has McCoy rank at most n − j − 1 • Use Sylvester matrices and approximate GCD techniques • δ j ’s are approximate GCD’s of several polynomials • Coefficient structure is multi-linear in the entries of A Lemma A has McCoy rank at most n − 2 iff entries of the adjoint matrix have a non-trivial GCD. We compute the adjoint matrix quickly and robustly! 10/19 Haraldson

Introduction Theory SNF via Optimization Examples Conclusion Distance lower bounds via unstructured SVDs • Embed matrix polynomials into scalar matrices over R Generalized Sylvester matrices Let a ∈ R [ t ] with deg a ≤ d .    a 0 · · · a d        ... ...    ∈ R r × ( r + d ) .   φ r ( a ) =      a 0 · · · a d • Let f = ( f 1 , . . . , f k ) ∈ R [ t ] k be ordered by decreasing degree • Take d = (deg( f 1 ) , . . . , deg( f k )) , r = deg f 1 and d = max { deg f j } k j = 2   φ r ( f 1 )             φ d ( f 2 )       ∈ R ( r + ( k − 1) d ) × ( r + d ) . Syl ( f ) = Syl d ( f )   = .     . � �������������� �� �������������� �    .      Generalized Sylvester Matrix φ d ( f k ) 11/19 Haraldson

Introduction Theory SNF via Optimization Examples Conclusion Generalized Sylvester Matrices Theorem gcd( f ) = 1 ⇐⇒ Syl ( f ) has full rank . Problem: What if the degrees of f can increase? � � • Degrees of f can be at-most d ′ = d ′ 1 , . . . , d ′ k • Spurious solutions can occur due to over-padding of zeros j ( f j ) = t d j f ( t − 1 ) • Define rev d ′ • Define rev d ′ ( f ) in the obvious way Theorem If Syl d ′ ( f ) is rank deficient then gcd( f ) = 1 iff Syl ( rev d ′ ( f )) has full rank. 12/19 Haraldson

Introduction Theory SNF via Optimization Examples Conclusion Approximate SNF via Sylvester Matrices Theorem A nearest rank at most e Sylvester matrix always exists. Theorem Suppose that d ′ = ( γ, γ . . . , γ ) and Syl d ′ (Adj( A )) has rank e . σ e ( Syl d ′ (Adj( A ))) ∞ n n / 2 ≤ �A − � A� F , where SNF( � A ) is non-trivial . γ n 3 ( d + 1) 3 / 2 �A� n • σ e ( Syl d ′ (Adj( A ))) is the distance to a nearest singular matrix Example (Same A as the First Example) A lower bound on the distance to non-triviality is 4 . 3556 e − 4 . 13/19 Haraldson

Introduction Theory SNF via Optimization Examples Conclusion Nearest Matrix Polynomial with an Interesting SNF Constrained Optimization Approach   Adj( �  A ) = F h ,       F ∈ R [ t ] n × n ,  min � A − � � 2 A such that  � � �� � �  F  h = h 0 + h 1 t + h 2 t 2 ,     ∆ A   h 2 2 + h 2 1 − 1 = 0 . • Assume the adjoint has a finite approximate GCD • Otherwise the reversal has a non-trivial GCD • Generically, the approximate GCD has degree 1 or 2 • h 2 2 + h 2 1 − 1 = 0 = ⇒ h has degree at least 1 • Solve with Lagrange Multipliers and Levenberg-Marquardt 14/19 Haraldson

Introduction Theory SNF via Optimization Examples Conclusion Levenberg-Marquardt Iteration Theorem The Levenberg-Marquardt iteration converges quadratically to the minimum value with a suitable initial guess. Corollary Under small perturbations: • Well-posed approximate SNF instances remain well-posed. • Ill-posed instances cannot be regularized to be well-posed. • Theory applies by induction to arbitrary McCoy rank • Applies to infinite eigenvalues: consider t d A ( t − 1 ) • This is why existing algorithms fail and cannot be saved 15/19 Haraldson