Notes Review LU Note that r 2 log(r) is NaN at r=0: Write A=LU in - PDF document

Notes Review LU � Note that r 2 log(r) is NaN at r=0: � Write A=LU in block partitioned form • By convention, L has all ones on diagonal instead smoothly extend to be 0 at r=0 � Equate blocks: pick order to compute them • “Up-looking”: compute a row at a time � Schedule a make-up lecture? (refer just to entries in A in rows 1 to i) • “Left-looking”: compute a column at a time (refer just to entries in A in columns 1 to j) • “Bordering”: row of L and column of U • “Right-looking”: column of L and row of U (note: outer-product update of remaining A) � Can do all of these “in-place” (overwrite A) cs542g-term1-2007 1 cs542g-term1-2007 2 Pivoting Row Partial-Pivoting � LU and LDL T can fail � Row partial-pivoting: PA=LU • Compute a column of L, swap rows to get biggest • Example: if A 11 =0 entry on diagonal � Go back to Gaussian Elimination ideas: reorder • Express as PA=LU where P is a permutation matrix the equations (rows) to get a nonzero entry • P is the identity with rows swapped (but store it as a permutation vector) � In fact, nearly zero entries still a problem • This is what LAPACK uses • Perhaps cancellation error => few significant digits � Guarantees entries of L bounded by 1 in • Dividing through will taint rest of calculation magnitude � Pivoting: reorder to get biggest entry on diagonal � No good guarantee on U – � but usually fine • Partial pivoting: just reorder rows (or columns) � If U doesn � t grow too much, comes very close to • Complete pivoting: reorder rows and columns optimal accuracy (expensive) cs542g-term1-2007 3 cs542g-term1-2007 4 Symmetric Pivoting Reconsidering RBF � RBF interpolation has advantages: � Problem: partial (or complete) pivoting destroys symmetry • Mesh-free • Optimal in some sense � How can we factor a symmetric indefinite matrix reliably but twice as fast as unsymmetric • Exponential convergence (each point extra matrices? data point improves fit everywhere) • Defined everywhere � One idea: symmetric pivoting PAP T =LDL T • Swap the rows the same as the columns � But some disadvantages: • It � s a global calculation � But let D have 2x2 as well as 1x1 blocks on the (even with compactly supported functions) diagonal • Big dense matrix to form and solve • Partial pivoting: Bunch-Kaufman (LAPACK) (though later we � ll revisit that… • Complete pivoting: Bunch-Parlett (safer) cs542g-term1-2007 5 cs542g-term1-2007 6

Gibbs Noise � Globally smooth � If data contains noise (errors), RBF strictly calculation also interpolates them makes for � If the errors aren � t spatially correlated, lots overshoot/ of discontinuities: RBF interpolant undershoot becomes wiggly (Gibbs phenomena) around discontinuities � Can � t easily control effect cs542g-term1-2007 7 cs542g-term1-2007 8 Linear Least Squares Rewriting � Idea: instead of interpolating data + noise, � Write it in matrix-vector form: approximate 2 � � n k � � f i � � j � j ( x i ) = b � Ax 2 2 � Pick our approximation from a space of � � � � i = 1 j = 1 functions we expect (e.g. not wiggly -- ( ) maybe low degree polynomials) to filter b = f 1 T � f 2 f n out the noise ( ) T x = � 1 � k � � Standard way of defining it: A ij = � j ( x i ) (a rectangular n � k matrix) k � f ( x ) = � i � i ( x ) i = 1 ( ) n � 2 � = argmin f j � f ( x j ) � j = 1 cs542g-term1-2007 9 cs542g-term1-2007 10 Normal Equations Normal Equations: Good Stuff � A T A is a square k � k matrix � First attempt at finding minimum: set the gradient equal to zero (k probably much smaller than n) (called “the normal equations”) � 2 = 0 Symmetric positive (semi-)definite � x b � Ax 2 � ( ) = 0 � x ( b � Ax ) T ( b � Ax ) � ( ) = 0 � x b T b � 2 x T A T b + x T A T Ax � 2 A T b + 2 A T Ax = 0 A T Ax = A T b cs542g-term1-2007 11 cs542g-term1-2007 12

Normal Equations: Problem � What if k=n? At least for 2-norm condition number, � (A T A)=k(A) 2 • Accuracy could be a problem… � In general, can we avoid squaring the errors? cs542g-term1-2007 13