On the construction of minimax-distance (sub-)optimal designs Luc - PowerPoint PPT Presentation

On the construction of minimax-distance (sub-)optimal designs Luc Pronzato Université Côte d’Azur, CNRS, I3S, France

1) Introduction 1) Introduction & motivation Objective: Approximation/interpolation of a function f : x ∈ X ⊂ R d − → R , (with X compact: typically, X = [0 , 1] d ) ➠ Choose n points X n = { x 1 , . . . , x n } ∈ X n (the design) where to evaluate f (no repetition) Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 2 / 41

1) Introduction 1) Introduction & motivation Objective: Approximation/interpolation of a function f : x ∈ X ⊂ R d − → R , (with X compact: typically, X = [0 , 1] d ) ➠ Choose n points X n = { x 1 , . . . , x n } ∈ X n (the design) where to evaluate f (no repetition) Design criterion = minimax distance ➠ minimize Φ mM ( X n ) = max x ∈ X min i =1 ,..., n � x − x i � ( ℓ 2 -distance) = max x ∈ X d ( x , X n ) = d H ( X , X n ) (Hausdorff distance, ℓ 2 ) = dispersion of X n in X (Niederreiter, 1992, Chap. 6) Φ ∗ X ∗ n an optimal n -point design ➔ Φ mM -efficiency Eff mM ( X n ) = Φ mM ( X n ) ∈ (0 , 1] mM , n with Φ ∗ mM , n = Φ mM ( X ∗ n ) Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 2 / 41

1) Introduction d = 2, n = 7 Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 3 / 41

1) Introduction two good reasons (at least) to minimize Φ mM ( X n ) : Why Φ mM ? ① Suppose f ∈ RKHS H with kernel K ( x , y ) = C ( � x − y � ), then ∀ x ∈ X , | f ( x ) − ˆ η n ( x ) | ≤ � f � H ρ n ( x ) where η n ( x ) = BLUP based on the f ( x i ), i = 1 , . . . , n ˆ ρ 2 n ( x ) = “kriging variance" at x see, e.g., Vazquez and Bect (2011); Auffray et al. (2012) Schaback (1995) ➠ sup x ∈ X ρ n ( x ) ≤ S [Φ mM ( X n )] for some increasing function S [ · ] (depending on K ) Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 4 / 41

1) Introduction two good reasons (at least) to minimize Φ mM ( X n ) : Why Φ mM ? ① Suppose f ∈ RKHS H with kernel K ( x , y ) = C ( � x − y � ), then ∀ x ∈ X , | f ( x ) − ˆ η n ( x ) | ≤ � f � H ρ n ( x ) where η n ( x ) = BLUP based on the f ( x i ), i = 1 , . . . , n ˆ ρ 2 n ( x ) = “kriging variance" at x see, e.g., Vazquez and Bect (2011); Auffray et al. (2012) Schaback (1995) ➠ sup x ∈ X ρ n ( x ) ≤ S [Φ mM ( X n )] for some increasing function S [ · ] (depending on K ) ② X ∗ n has no (or few) points on the boundary of X Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 4 / 41

1) Introduction Evaluation of Φ mM ( X n ) ? Not considered here! Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 5 / 41

1) Introduction Evaluation of Φ mM ( X n ) ? Not considered here! To evaluate Φ mM ( X n ) = max x ∈ X min i =1 ,..., n � x − x i � = max x ∈ X d ( x , X n ) we need to find x ∗ = arg max x ∈ X d ( x , X n ) Key idea: replace arg max x ∈ X d ( x , X n ) by arg max x ∈ X Q d ( x , X n ) for a suitable finite X Q ∈ X Q Replacing X Q by a regular grid , or first Q points of a Low Discrepancy Sequence in X , is not accurate: ➠ Φ mM ( X n ; X Q ) ≤ Φ mM ( X n ) (optimistic result) requires Q = O (1 /ǫ d ) to have Φ mM ( X n ) < Φ mM ( X n ; X Q ) + ǫ For d � 5, use tools from algorithmic geometry (Delaunay triangulation or Voronoï tessellation) ➞ exact result For larger d , use MCMC with X Q = adaptive grid (LP, 2017a) Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 5 / 41

1) Introduction Bounds on Φ ∗ mM , n = Φ mM ( X ∗ n ) when X = [0 , 1] d Lower bound: the n balls B ( x i , Φ ∗ mM , n ) cover X mM , n ) d ≥ vol( X ) (= 1), with V d = vol[ B (0 , 1)] = π d / 2 / Γ( d / 2 + 1) ⇒ nV d (Φ ∗ n = ( nV d ) − 1 / d ≤ Φ ∗ R ∗ mM , n Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 6 / 41

1) Introduction Bounds on Φ ∗ mM , n = Φ mM ( X ∗ n ) when X = [0 , 1] d Lower bound: the n balls B ( x i , Φ ∗ mM , n ) cover X mM , n ) d ≥ vol( X ) (= 1), with V d = vol[ B (0 , 1)] = π d / 2 / Γ( d / 2 + 1) ⇒ nV d (Φ ∗ n = ( nV d ) − 1 / d ≤ Φ ∗ R ∗ mM , n Upper bound: use any design! m d -point regular grid in X : √ Φ ∗ d mM , m d ≤ 2 m : Take m = ⌊ n 1 / d ⌋ , so that m d ≤ n and Φ ∗ mM , n ≤ Φ ∗ mM , m d , therefore √ ∗ Φ ∗ mM , n ≤ R d n = 2 ⌊ n 1 / d ⌋ Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 6 / 41

1) Introduction d = 2 Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 7 / 41

1) Introduction d = 5 Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 7 / 41

1) Introduction d = 10 Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 7 / 41

1) Introduction d = 20 Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 7 / 41

1) Introduction ② Minimization of Φ mM ( X n ) with respect to X n ∈ X n for a given n Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 8 / 41

1) Introduction ② Minimization of Φ mM ( X n ) with respect to X n ∈ X n for a given n ③ n is not fixed ( n min ≤ n ≤ n max , we may stop before n max evaluations of f ) How to obtain good “anytime designs”, such that all nested designs X n have a high efficiency Eff mM ( X n ), n min ≤ n ≤ n max Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 8 / 41

1) Introduction ② Minimization of Φ mM ( X n ) with respect to X n ∈ X n for a given n ③ n is not fixed ( n min ≤ n ≤ n max , we may stop before n max evaluations of f ) How to obtain good “anytime designs”, such that all nested designs X n have a high efficiency Eff mM ( X n ), n min ≤ n ≤ n max ④ Design measures that minimize a regularized version of Φ mM Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 8 / 41

2) Minimization of Φ mM ( Xn ), Xn ∈ X n , n fixed 2) Minimization of Φ mM ( X n ), X n ∈ X n , n fixed Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 9 / 41

2) Minimization of Φ mM ( Xn ), Xn ∈ X n , n fixed 2) Minimization of Φ mM ( X n ), X n ∈ X n , n fixed General global optimization method (e.g., simulated annealing): not promising 2.1) k-means and centroids 2.2) Stochastic gradient Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 9 / 41

2) Minimization of Φ mM ( Xn ), Xn ∈ X n , n fixed 2.1/ k-means and centroids 2.1/ k-means and centroids Minimize the L 2 energy functional � n � � � n � � � x − x i � 2 d x I C i ( x ) � x − x i � 2 E 2 ( T n , X n ) = d x = X C i i =1 i =1 where T n = {C i , i = 1 , . . . , n } is a tessellation of X I C i = indicator function of C i Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 10 / 41

2) Minimization of Φ mM ( Xn ), Xn ∈ X n , n fixed 2.1/ k-means and centroids 2.1/ k-means and centroids Minimize the L 2 energy functional � n � � � n � � � x − x i � 2 d x I C i ( x ) � x − x i � 2 E 2 ( T n , X n ) = d x = X C i i =1 i =1 where T n = {C i , i = 1 , . . . , n } is a tessellation of X I C i = indicator function of C i Then (Du et al., 1999): C i = V ( x i ) = Voronoï region for the site x i , for all i � X d 2 ( x , X n ) d x ) ( ⇒ E 2 ( T n , X n ) = simultaneously x i = centroid of C i (center of gravity) for all i : � C i x d x ) / vol( C i ) x i = ( ➞ such a X n should thus perform reasonably well in terms of space-filling (Lekivetz and Jones, 2015) Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 10 / 41

2) Minimization of Φ mM ( Xn ), Xn ∈ X n , n fixed 2.1/ k-means and centroids Lloyd’s method (1982): (= fixed-point iterations) ➞ Move each x i to the centroid of its own Voronoï cell, repeat . . . ➠ Algorithmic geometry (Voronoï tessellation) if d very small, use a finite set X Q otherwise Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 11 / 41

2) Minimization of Φ mM ( Xn ), Xn ∈ X n , n fixed 2.1/ k-means and centroids 30 points from Sobol’ LDS Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 12 / 41

2) Minimization of Φ mM ( Xn ), Xn ∈ X n , n fixed 2.1/ k-means and centroids k-means clustering (30 clusters) of 1,000 point from Sobol’ LDS Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 12 / 41

2) Minimization of Φ mM ( Xn ), Xn ∈ X n , n fixed 2.1/ k-means and centroids However. . . minimax-optimal design is related to the construction of a centroidal tessellation for � n � � � � � n � x − x i � q d x E q ( T n , X n ) = I C i ( x ) � x − x i � q d x = X C i i =1 i =1 for q → ∞ ➠ use Chebyshev centers Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 13 / 41