Subspace Regularization for Large Linear Discrete Ill-Posed Problems - PowerPoint PPT Presentation

Fachbereich 03 Zentrum für Technomathematik Mathematik/Informatik Subspace Regularization for Large Linear Discrete Ill-Posed Problems 8th GAMM Workshop Applied and Numerical Linear Algebra TU Hamburg-Harburg, September 11-12, 2008 Angelika Bunse-Gerstner Zentrum für Technomathematik, Fachbereich 3 - Mathematik und Informatik Universität Bremen and Valia Guerra Instituto de Cibernetica, Matematica y Fisica Havanna Linear Discrete Ill-Posed Problems Preconditioning Subspace Regularization Numerical Examples A. Bunse-Gerstner 1 / 17

Fachbereich 03 Zentrum für Technomathematik Mathematik/Informatik Overview Linear Discrete Ill-Posed Problems 1 Preconditioning 2 3 Subspace Regularization Numerical Examples 4 Linear Discrete Ill-Posed Problems Preconditioning Subspace Regularization Numerical Examples A. Bunse-Gerstner 2 / 17

Fachbereich 03 Zentrum für Technomathematik Mathematik/Informatik Linear Discrete Ill-Posed Problems ˆ x ex = arg min x � A ex x − b ex � 2 x = arg min x � Ax − b � 2 � A ex − A � 2 / � A ex � 2 , � b ex − b � 2 / � b ex � 2 with small Reconstruct x ex using the perturbed data A , b ! Problem: � x ex − ˆ x � 2 / � x ex � 2 can be huge ! Standard techniques to solve � Ax − b � 2 = min ! cannot be applied !! Linear Discrete Ill-Posed Problems Preconditioning Subspace Regularization Numerical Examples A. Bunse-Gerstner 3 / 17

Fachbereich 03 Zentrum für Technomathematik Mathematik/Informatik Characteristics of Linear Discrete Ill-Posed Problems Typical properties of the problem � Ax − b ex � 2 = min ! (inherited from source problem): (Perturbations in A ex neglected,i.e. A = A ex ) ◮ A dense, often very large ◮ A = U Σ V T with σ i ” i →∞ ” 0 , rapidly decreasing, no distinct gap, − → Often but not always: u i , v i ’ smooth ’ for small i , more ’ oscillations ’ in u i , v i with decreasing σ i u T i b ex ◮ solution x ex = P n v i i = 1 σ i ◮ Discrete Picard Condition: u T i b ex ” i →∞ ” 0 , − → σ i Therefore u T u T i b ex i b ex ❳ ❳ x ex = v i + v i σ i σ i σ i not small σ i small ⑤ ④③ ⑥ harmless ’ smooth ’, entries not large Linear Discrete Ill-Posed Problems Preconditioning Subspace Regularization Numerical Examples A. Bunse-Gerstner 4 / 17

Fachbereich 03 Zentrum für Technomathematik Mathematik/Informatik Characteristics of Linear Discrete Ill-Posed Problems In practice: measurement errors. Instead of b ex we have b = b ex + η, and we can only solve � Ax − b � = min ! ◮ A dense, often very large ◮ A = U Σ V T with σ i ” i →∞ ” 0 , rapidly decreasing, no distinct gap, − → Often but not always: u i , v i ’ smooth ’ for small i , more ’ oscillations ’ in u i , v i with decreasing σ i u T i b ex ◮ solution x ex = P n v i i = 1 σ i ◮ Discrete Picard Condition: u T i b ex ” i →∞ ” 0 , − → σ i Therefore u T u T i b ex i b ex ❳ ❳ x ex = v i + v i σ i σ i σ i not small σ i small ⑤ ④③ ⑥ harmless ’ smooth ’, entries not large Linear Discrete Ill-Posed Problems Preconditioning Subspace Regularization Numerical Examples A. Bunse-Gerstner 4 / 17

Fachbereich 03 Zentrum für Technomathematik Mathematik/Informatik Characteristics of Linear Discrete Ill-Posed Problems In practice: measurement errors. Instead of b ex we have b = b ex + η, and we can only solve � Ax − b � = min ! ◮ A dense, often very large ◮ A = U Σ V T with σ i ” i →∞ ” 0 , rapidly decreasing, no distinct gap, − → Often but not always: u i , v i ’ smooth ’ for small i , more ’ oscillations ’ in u i , v i with decreasing σ i u T u T u T i b i b ex i η ◮ solution x = P n σ i v i = P n v i + P n ˆ σ i v i i = 1 i = 1 i = 1 σ i ◮ Discrete Picard Condition: u T i b ex ” i →∞ ” 0 , − → σ i Therefore u T u T i b ex i b ex ❳ ❳ x ex = v i + v i σ i σ i σ i not small σ i small ⑤ ④③ ⑥ harmless ’ smooth ’, entries not large Linear Discrete Ill-Posed Problems Preconditioning Subspace Regularization Numerical Examples A. Bunse-Gerstner 4 / 17

Fachbereich 03 Zentrum für Technomathematik Mathematik/Informatik Characteristics of Linear Discrete Ill-Posed Problems In practice: measurement errors. Instead of b ex we have b = b ex + η, and we can only solve � Ax − b � = min ! ◮ A dense, often very large ◮ A = U Σ V T with σ i ” i →∞ ” 0 , rapidly decreasing, no distinct gap, − → Often but not always: u i , v i ’ smooth ’ for small i , more ’ oscillations ’ in u i , v i with decreasing σ i u T u T u T i b i b ex i η ◮ solution x = P n σ i v i = P n v i + P n ˆ σ i v i i = 1 i = 1 i = 1 σ i ◮ No Picard Condition: u T i b growing with decreasing σ i σ i Therefore u T u T i b ex i b ex ❳ ❳ x ex = v i + v i σ i σ i σ i not small σ i small ⑤ ④③ ⑥ harmless ’ smooth ’, entries not large Linear Discrete Ill-Posed Problems Preconditioning Subspace Regularization Numerical Examples A. Bunse-Gerstner 4 / 17

Fachbereich 03 Zentrum für Technomathematik Mathematik/Informatik Characteristics of Linear Discrete Ill-Posed Problems In practice: measurement errors. Instead of b ex we have b = b ex + η, and we can only solve � Ax − b � = min ! ◮ A dense, often very large ◮ A = U Σ V T with σ i ” i →∞ ” 0 , rapidly decreasing, no distinct gap, − → Often but not always: u i , v i ’ smooth ’ for small i , more ’ oscillations ’ in u i , v i with decreasing σ i u T u T u T i b i b ex ◮ solution x = P n σ i v i = P n v i + P n i η ˆ σ i v i i = 1 i = 1 i = 1 σ i ◮ No Picard Condition: u T i b growing with decreasing σ i σ i Therefore u T u T i b i b ❳ ❳ x = ˆ v i + v i σ i σ i σ i not small σ i small ⑤ ④③ ⑥ extremely harmful ’ highly oscillating ’, entries extremely large, dominated by perturbations Linear Discrete Ill-Posed Problems Preconditioning Subspace Regularization Numerical Examples A. Bunse-Gerstner 4 / 17

Fachbereich 03 Zentrum für Technomathematik Mathematik/Informatik Regularization Regularization methods use knowledge about the exact solution to find a good approximation. Example: Tychonov regularization. Instead of � Ax − b � 2 = min ! solve � Ax − b � 2 2 + λ 2 � Lx � 2 2 = min ! , λ small equivalently ✔ b ✌ ✔ ✕ ✕✌ A ✌ ✌ = min! x − ✌ ✌ 0 λ L ✌ ✌ 2 e. g. L = I or L discretized differential operator Need a good choice for the regularization paramter λ (and for L ) Linear Discrete Ill-Posed Problems Preconditioning Subspace Regularization Numerical Examples A. Bunse-Gerstner 5 / 17

Fachbereich 03 Zentrum für Technomathematik Mathematik/Informatik Numerical Methods for the Large Dimensional Case Iterative method needed ( A is dense!), e.g. CG or LSQR Advantage: Krylov methods have regularization properties (regularization parameter: iteration index) Problems: Good regularization parameter not really known Convergence maybe slow Additional regularization in general still needed. Linear Discrete Ill-Posed Problems Preconditioning Subspace Regularization Numerical Examples A. Bunse-Gerstner 6 / 17

Fachbereich 03 Zentrum für Technomathematik Mathematik/Informatik Work Done in This Field ◮ Ake Björck, Lars Eldén, Per Christian Hansen, Tommy Elfving ◮ Martin Hanke, Robert Plemmons ◮ Misha E. Kilmer, Dianne P ., Oleary, James Nagy ◮ Daniela Calvetti, Lothar Reichel, Brian Lewis ◮ Gene H. Golub, Urs von Matt ◮ Jerry Eriksson, Marten Gullikson, Per-Åke Wedin ◮ Uri Asher, Eldad Haber, Douglas Oldenburg ◮ Marielba Rojas, Trond Steihaug ◮ Tony Chan, Stanley Osher, Curtis R. Vogel ◮ Matlab Software (Toolboxen): ◮ Restore Tools (Nagy, 2002) ◮ MOORe Tools (Jacobsen,2005) Linear Discrete Ill-Posed Problems Preconditioning Subspace Regularization Numerical Examples A. Bunse-Gerstner 7 / 17

Fachbereich 03 Zentrum für Technomathematik Mathematik/Informatik Preconditioning Conventional preconditioning is not an option ! Only very few approaches exist for the general case. [e.g. Calvetti und Reichel 2002 ]: Smoothing of the solution Ideal: Process only the parts in the "right" (smooth) subspace V . Leave the remaining parts unchanged. ◮ Recall x k essentially determined by the dominant SVD-parts ◮ The leading singular vectors v 1 , v 2 , . . . are often "smooth", i.e. V "is smooth". ◮ for special structure (Toeplitz etc. image processing): Fourier basis vectors ◮ general A : use Fourier- or Wavelet basis vectors or approximations of the singular vectors [ [Hanke and Vogel 1999, Hansen and Jacobsen 2000, Hansen,Jacobsen and Saunders 2003 (Two Level Preconditioning PreLSQR) ] What about non-smooth solutions? Linear Discrete Ill-Posed Problems Preconditioning Subspace Regularization Numerical Examples A. Bunse-Gerstner 8 / 17