Radial basis functions topics in 40 +1 slides Stefano De Marchi - PowerPoint PPT Presentation

Radial basis functions topics in 40 +1 slides Stefano De Marchi Department of Mathematics “Tullio Levi-Civita” University of Padova Napoli, 22nd March 2018

Outline 1 From splines to RBF Error estimates, conditioning and stability 2 Strategies for “controlling” errors 3 Stable basis approaches WSVD Basis Rescaled, Rational RBF and some applications 4 2 of 41

From cubic splines to RBF 3 of 41

Cubic splines Let S 3 ( X ) = { s ∈ C 2 [ a , b ] : s | [ x i , x i + 1 ] ∈ P 3 , 1 ≤ i ≤ N − 1 , s [ a , x 1 ] , s [ x N , b ] ∈ P 1 } be the space of natural cubic splines. S 3 ( X ) has the basis of truncated powers ( · − x j ) 3 + , 1 ≤ j ≤ N plus an arbitrary basis for P 3 ( R ) N 3 � � a j ( x − x j ) 3 b j x j , x ∈ [ a , b ] . s ( x ) = + + (1) j = 1 j = 0 + = ( | x | 3 + x 3 ) Using the identity x 3 and the fact that s [ a , x 1 ] , s [ x N , b ] ∈ P 1 2 4 of 41

Cubic splines Let S 3 ( X ) = { s ∈ C 2 [ a , b ] : s | [ x i , x i + 1 ] ∈ P 3 , 1 ≤ i ≤ N − 1 , s [ a , x 1 ] , s [ x N , b ] ∈ P 1 } be the space of natural cubic splines. S 3 ( X ) has the basis of truncated powers ( · − x j ) 3 + , 1 ≤ j ≤ N plus an arbitrary basis for P 3 ( R ) N 3 � � a j ( x − x j ) 3 b j x j , x ∈ [ a , b ] . s ( x ) = + + (1) j = 1 j = 0 + = ( | x | 3 + x 3 ) Using the identity x 3 and the fact that s [ a , x 1 ] , s [ x N , b ] ∈ P 1 2 Every natural cubic spline s has the representation N � s ( x ) = a j φ ( | x − x j | ) + p ( x ) , x ∈ R j = 1 where φ ( r ) = r 3 , r ≥ 0 and p ∈ P 1 ( R ) s.t. N N � � a j = a j x j = 0 . (2) j = 1 j = 1 4 of 41

Extension to R d Going to R d is straightforward: ”radiality” becomes more evident N a j φ ( � x − x j � 2 ) + p ( x ) , x ∈ R d , � s ( x ) = (3) i = 1 where φ : [ 0 , ∞ ) → R is a univariate fixed function and p ∈ P l − 1 ( R d ) is a low degree d -variate polynomial. The additional conditions in ( ?? ) become N � a j q ( x j ) = 0 , ∀ q ∈ P l − 1 ( R d ) . (4) i = 1 5 of 41

Extension to R d Going to R d is straightforward: ”radiality” becomes more evident N a j φ ( � x − x j � 2 ) + p ( x ) , x ∈ R d , � s ( x ) = (3) i = 1 where φ : [ 0 , ∞ ) → R is a univariate fixed function and p ∈ P l − 1 ( R d ) is a low degree d -variate polynomial. The additional conditions in ( ?? ) become N � a j q ( x j ) = 0 , ∀ q ∈ P l − 1 ( R d ) . (4) i = 1 Obs: We can omit the side conditions on the coefficients ( ?? ) (like using “B-splines”) and consider approximants of the form N � s ( x ) = a j φ ( � x − x j � 2 ) i = 1 and also omit the 2-norm symbol 5 of 41

Scattered data fitting:I Setting Given data X = { x j , j = 1 , . . . , N } , with x j ∈ Ω ⊂ R d and values Y = { y j ∈ R , j = 1 , ..., N } ( y j = f ( x j ) ), find a (continuous) function P f ∈ span { φ ( · − x i ) , x i ∈ X } s.t. N � P f = α j φ ( � · − x i � ) , s . t .. ( P f ) | X = Y ( interpolation ) j = 1 6 of 41

Scattered data fitting:I Setting Given data X = { x j , j = 1 , . . . , N } , with x j ∈ Ω ⊂ R d and values Y = { y j ∈ R , j = 1 , ..., N } ( y j = f ( x j ) ), find a (continuous) function P f ∈ span { φ ( · − x i ) , x i ∈ X } s.t. N � P f = α j φ ( � · − x i � ) , s . t .. ( P f ) | X = Y ( interpolation ) j = 1 Figure: Data points, data values and data function Obs: P f = � n i = j u j f j , with u j ( x i ) = δ ji . 6 of 41

Scattered data fitting:II Problem For which functions φ : [ 0 , ∞ ) → R such that for all d , N ∈ N and all pairwise distrinct x 1 , . . . , x n ∈ R d the matrix det ( A φ, X ) � 0 ? When the matrix A φ, X := ( φ ( � x i − x j � 2 ) 1 ≤ i , j ≤ N , is invertible? 7 of 41

Scattered data fitting:II Problem For which functions φ : [ 0 , ∞ ) → R such that for all d , N ∈ N and all pairwise distrinct x 1 , . . . , x n ∈ R d the matrix det ( A φ, X ) � 0 ? When the matrix A φ, X := ( φ ( � x i − x j � 2 ) 1 ≤ i , j ≤ N , is invertible? Answer The functions φ should be positive definite or conditionally positive definite of some order. 7 of 41

General kernel-based approximation φ , Conditionally Positive Definite (CPD) of order ℓ or Strictly Positive Definite (SPD) and radial name φ ℓ globally supported: e − ε 2 r 2 Gaussian C ∞ (GA) 0 Generalized Multiquadrics C ∞ (GM) ( 1 + r 2 /ε 2 ) 3 / 2 2 name φ ℓ locally supported: Wendland C 2 (W2) ( 1 − ε r ) 4 + ( 4 ε r + 1 ) 0 Buhmann C 2 (B2) 2 r 4 log r − 7 / 2 r 4 + 16 / 3 r 3 − 2 r 2 + 1 / 6 0 we often consider φ ( ε · ) , with ε called shape parameter 8 of 41

General kernel-based approximation φ , Conditionally Positive Definite (CPD) of order ℓ or Strictly Positive Definite (SPD) and radial name φ ℓ globally supported: e − ε 2 r 2 Gaussian C ∞ (GA) 0 Generalized Multiquadrics C ∞ (GM) ( 1 + r 2 /ε 2 ) 3 / 2 2 name φ ℓ locally supported: Wendland C 2 (W2) ( 1 − ε r ) 4 + ( 4 ε r + 1 ) 0 Buhmann C 2 (B2) 2 r 4 log r − 7 / 2 r 4 + 16 / 3 r 3 − 2 r 2 + 1 / 6 0 we often consider φ ( ε · ) , with ε called shape parameter kernel notation K ( x , y )(= K ε ( x , y )) = Φ ε ( x − y ) = φ ( ε � x − y � 2 ) 8 of 41

General kernel-based approximation φ , Conditionally Positive Definite (CPD) of order ℓ or Strictly Positive Definite (SPD) and radial name φ ℓ globally supported: e − ε 2 r 2 Gaussian C ∞ (GA) 0 Generalized Multiquadrics C ∞ (GM) ( 1 + r 2 /ε 2 ) 3 / 2 2 name φ ℓ locally supported: Wendland C 2 (W2) ( 1 − ε r ) 4 + ( 4 ε r + 1 ) 0 Buhmann C 2 (B2) 2 r 4 log r − 7 / 2 r 4 + 16 / 3 r 3 − 2 r 2 + 1 / 6 0 we often consider φ ( ε · ) , with ε called shape parameter kernel notation K ( x , y )(= K ε ( x , y )) = Φ ε ( x − y ) = φ ( ε � x − y � 2 ) native space N K (Ω) (where K is the reproducing kernel) finite subspace N K ( X ) = span { K ( · , x ) : x ∈ X } ⊂ N K (Ω) . 8 of 41

Error estimates, conditioning and stability 9 of 41

Separation distance, fill-distance and power function q X := 1 2 min i � j � x i − x j � 2 , ( separation distance ) h X , Ω := sup x j ∈ X � x − x j � 2 , min ( fill − distance ) x ∈ Ω � u ∗ vector of cardinal functions Φ ε ( 0 ) − ( u ∗ ( x )) T Au ∗ ( x ) , P Φ ε, X ( x ) := ( power function ) Figure: The fill-distance of 25 Halton points h ≈ 0 . 2667 Figure: Power function for the Gaussian kernel with ε = 6 on a grid of 81 uniform, Chebyshev and Halton points, 10 of 41 respectively.

Pointwise error estimates Theorem Let Ω ⊂ R d and K ∈ C (Ω × Ω) be PD on R d . Let X = { x 1 , . . . , , x n } be a set of distinct points. Take a function f ∈ N Φ (Ω) and denote with P f its interpolant on X. Then, for every x ∈ Ω | f ( x ) − P f ( x ) | ≤ P Φ ε , X ( x ) � f � N K (Ω) . (5) Theorem Let Ω ⊂ R d and K ∈ C 2 κ (Ω × Ω) be symmetric and positive definite, X = { x 1 , . . . , , x N } a set of distinct points. Consider f ∈ N K (Ω) and its interpolant P f on X. Then, there exist positive constants h 0 and C (independent of x , f and Φ ), with h X , Ω ≤ h 0 , such that � | f ( x ) − P f ( x ) | ≤ C h κ C K ( x ) � f � N K (Ω) . (6) X , Ω max w , z ∈ Ω ∪ B ( x , c 2 h X , Ω ) | D β and C K ( x ) = max | β | = 2 κ 2 Φ( w , z ) | . 11 of 41

Reducing the interpolation error Obs: The choice of the shape parameter ε in order to get the smallest (possible) interpolation error is crucial. Trial and Error Power function minimization Leave One Out Cross Validation (LOOCV) 12 of 41

Reducing the interpolation error Obs: The choice of the shape parameter ε in order to get the smallest (possible) interpolation error is crucial. Trial and Error Power function minimization Leave One Out Cross Validation (LOOCV) Trial and error strategy: interpolation of the 1-d sinc function with Gaussian for ε ∈ [ 0 , 20 ] , taking 100 values of ε and different equispaced data points. 12 of 41

Trade-off principle Consider the errors (RMSE and MAXERR) and the condition number µ 2 of A = (Φ ε ( � x i − x j � )) N i , j = 1 Figure: RMSE, MAXERR and Condition Number, µ 2 , with 30 values of ε ∈ [ 0 . 1 , 20 ] , for interpolation of the Franke function on a grid of 40 × 40 Chebyshev points Trade-off or uncertainty principle [Schaback 1995] Accuracy vs Stability Accuracy vs Efficiency Accuracy and stability vs Problem size 13 of 41

Accuracy vs stability The error bounds for non-stationary interpolation using infinitely smooth basis functions show that the error decreases (exponentially) as the fill distance decreases. For well-distributed data a decrease in the fill distance also implies a decrease of the separation distance But now the condition estimates above imply that the condition number of A grows exponentially This leads to numerical instabilities which make it virtually impossible to obtain the highly accurate results promised by the theoretical error bounds. 14 of 41

Stable basis approaches 15 of 41

Problem setting and questions Obs: the standard basis of translates (data-dependent) of N K ( X ) is unstable and not flexible 16 of 41