MATH 590: Meshfree Methods Chapter 5: Completely Monotone and - - PowerPoint PPT Presentation

MATH 590: Meshfree Methods

Chapter 5: Completely Monotone and Multiply Monotone Functions Greg Fasshauer

Department of Applied Mathematics Illinois Institute of Technology

Fall 2010

fasshauer@iit.edu MATH 590 – Chapter 5 1

Outline

1

Completely Monotone Functions

2

Multiply Monotone Functions

fasshauer@iit.edu MATH 590 – Chapter 5 2

In Chapter 3 we saw that translation invariant (“stationary” in the statistics literature) strictly positive definite functions can be characterized via Fourier transforms. Since Fourier transforms are not always easy to compute, we now present two alternative criteria that allow us to decide whether a function is strictly positive definite and radial on Rs (“isotropic” in the statistics literature): complete monotonicity (for the case of all s), and multiple monotonicity (for only limited choices of s).

fasshauer@iit.edu MATH 590 – Chapter 5 3

Completely Monotone Functions

Definition A function ϕ : [0, ∞) → R that is in C[0, ∞) ∩ C∞(0, ∞) and satisfies (−1)ℓϕ(ℓ)(r) ≥ 0, r > 0, ℓ = 0, 1, 2, . . . , is called completely monotone on [0, ∞).

fasshauer@iit.edu MATH 590 – Chapter 5 5

Completely Monotone Functions

Example The following are completely monotone on [0, ∞): ϕ(r) = ε, ε ≥ 0, ϕ(r) = e−εr, ε ≥ 0, since for ℓ = 0, 1, 2, . . . (−1)ℓϕ(ℓ)(r) = εℓe−εr ≥ 0, ϕ(r) = 1 (1 + r)β , β ≥ 0, since for ℓ = 0, 1, 2, . . . (−1)ℓϕ(ℓ)(r) = (−1)2ℓβ(β + 1) · · · (β + ℓ − 1)(1 + r)−β−ℓ ≥ 0.

fasshauer@iit.edu MATH 590 – Chapter 5 6

Completely Monotone Functions

Properties of completely monotone functions

(see [Cheney and Light (1999), Feller (1966), Widder (1941)])

1

A non-negative finite linear combination of completely monotone functions is completely monotone.

2

The product of two completely monotone functions is completely monotone.

3

If ϕ is completely monotone and ψ is absolutely monotone (i.e., ψ(ℓ) ≥ 0 for all ℓ ≥ 0), then ψ ◦ ϕ is completely monotone.

4

If ϕ is completely monotone and ψ is a positive function such that its derivative is completely monotone, then ϕ ◦ ψ is completely monotone.

fasshauer@iit.edu MATH 590 – Chapter 5 7

Completely Monotone Functions

Remark ϕ(r) = e−εr and ϕ(r) = 1 (1 + r)β , β ≥ 0 are reminiscent of Gaussians and inverse multiquadrics (subject to transformation r → r 2). Question Is there a connection between completely monotone functions and strictly positive definite radial functions? Possible Answer Find an integral characterization of completely monotone functions.

fasshauer@iit.edu MATH 590 – Chapter 5 8

Completely Monotone Functions

Just as we recalled Fourier transforms and Fourier-Bessel transforms earlier, we now need to remember a third integral transform. In the following, the Laplace transform will be important: Definition Let f be a piecewise continuous function that satisfies |f(t)| ≤ Meat for some constants a and M. The Laplace transform of f is given by Lf(s) = ∞ f(t)e−stdt, s > a. Similarly, the Laplace transform of a Borel measure µ on [0, ∞) is given by Lµ(s) = ∞ e−stdµ(t). The Laplace transform is continuous at the origin if and only if µ is finite.

fasshauer@iit.edu MATH 590 – Chapter 5 9

Completely Monotone Functions

Theorem (Hausdorff-Bernstein-Widder) A function ϕ : [0, ∞) → R is completely monotone on [0, ∞) if and only if it is the Laplace transform of a finite non-negative Borel measure µ

• n [0, ∞), i.e., ϕ is of the form

ϕ(r) = Lµ(r) = ∞ e−rtdµ(t). Remark The HBW-Theorem shows that the functions ϕε(r) = e−εr can be viewed as the fundamental completely monotone functions.

fasshauer@iit.edu MATH 590 – Chapter 5 10

Completely Monotone Functions

Proof. Widder’s proof of this theorem can be found in [Widder (1941),

• p. 160], where he reduces the proof of this theorem to another

theorem by Hausdorff on completely monotone sequences. A detailed proof can also be found in the books [Cheney and Light (1999), Wendland (2005a)].

fasshauer@iit.edu MATH 590 – Chapter 5 11

Completely Monotone Functions

The following connection between positive definite radial and completely monotone functions was first pointed out by Schoenberg in 1938: Theorem A function ϕ is completely monotone on [0, ∞) if and only if Φ = ϕ( · 2) is positive definite and radial on Rs for all s. Remark Note that the function Φ is now defined via the square of the norm. This differs from our earlier definition of radial functions.

fasshauer@iit.edu MATH 590 – Chapter 5 12

Completely Monotone Functions

Proof

We prove only one direction (ϕ completely monotone = ⇒ Φ positive definite and radial on any Rs). Details for the other direction are in [Wendland (2005a)]. The HBW theorem implies ϕ(r) = ∞ e−rtdµ(t) with a finite non-negative Borel measure µ. Therefore, Φ(x) = ϕ(x2) has the representation Φ(x) = ∞ e−x2tdµ(t).

fasshauer@iit.edu MATH 590 – Chapter 5 13

Completely Monotone Functions

Now look at the quadratic form

N

• j=1

N

• k=1

cjckΦ(xj − xk) = ∞

N

• j=1

N

• k=1

cjcke−txj−xk2dµ(t). Since we saw earlier that the Gaussians are strictly positive definite and radial on any Rs it follows that the quadratic form is non-negative, and therefore Φ is positive definite on any Rs. Remark One could also have used a change of variables to combine Schoenberg’s characterization of functions that are positive definite and radial on any Rs with the HBW characterization of completely monotone functions.

fasshauer@iit.edu MATH 590 – Chapter 5 14

Completely Monotone Functions

Up to now we only have a connection between completely monotone functions and positive definite functions, but not with strictly positive definite ones! We can see from the previous proof that if the measure µ is not concentrated at the origin, then Φ is even strictly positive definite and radial on any Rs. This condition on the measure is equivalent with ϕ not being constant. With this additional restriction on ϕ we can connect completely monotone function with the scattered data interpolation problem.

fasshauer@iit.edu MATH 590 – Chapter 5 15

Completely Monotone Functions

The following interpolation theorem already appears in [Schoenberg (1938a), p. 823]. It provides a very simple test for verifying the well-posedness of many scattered data interpolation problems. Theorem A function ϕ : [0, ∞) → R is completely monotone but not constant if and only if ϕ( · 2) is strictly positive definite and radial on Rs for any s. Remark Schoenberg only showed “completely monotone and not constant = ⇒ strictly positive definite and radial”. A proof that the converse also holds can be found in [Wendland (2005a)].

fasshauer@iit.edu MATH 590 – Chapter 5 16

Completely Monotone Functions

Example

1

Gaussians

ϕ(r) = e−εr, ε > 0, is completely monotone on [0, ∞) and not constant. The Schoenberg interpolation theorem tells us that Gaussians Φ(x) = ϕ(x2) = e−ε2x2 are strictly positive definite and radial on Rs for all s.

2

Inverse multiquadrics

ϕ(r) = 1/(1 + r)β, β > 0, is completely monotone on [0, ∞) and not constant. The Schoenberg interpolation theorem tells us that inverse multiquadrics Φ(x) = ϕ(x2) = 1/(1 + x2)β are strictly positive definite and radial on Rs for all s.

Remark Not only is the test for complete monotonicity simpler than the Fourier transform, but we also are able to verify strict positive definiteness of the inverse multiquadrics without any dependence of s on β.

fasshauer@iit.edu MATH 590 – Chapter 5 17

Completely Monotone Functions

Remark For radial (or “isotropic”) strictly positive definite functions complete monotonicity is a simple test. As long as we have translation invariant (or “stationary”) strictly positive definite functions we can use Fourier transforms. If we don’t have either property, then we need to use the definition

• f general positive definite kernels:

Definition A complex-valued continuous function K : Rs × Rs → C is called positive definite on Rs if

N

• j=1

N

• k=1

cjckK(xj, xk) ≥ 0 (1) for any N pairwise different points x1, . . . , xN ∈ Rs, and c = [c1, . . . , cN]T ∈ CN.

fasshauer@iit.edu MATH 590 – Chapter 5 18

Multiply Monotone Functions

We can also use monotonicity to test for strict positive definiteness of radial functions on Rs for some fixed value of s. To this end we introduce Definition A function ϕ : (0, ∞) → R which is in Ck−2(0, ∞), k ≥ 2, and for which (−1)lϕ(l)(r) is non-negative, non-increasing, and convex for l = 0, 1, 2, . . . , k − 2 is called k-times monotone on (0, ∞). In case k = 1 we only require ϕ ∈ C(0, ∞) to be non-negative and non-increasing. Remark Since convexity of ϕ means that ϕ( r1+r2

2

) ≤ ϕ(r1)+ϕ(r2)

2

, or simply ϕ′′(r) ≥ 0 if ϕ′′ exists, a multiply monotone function is in essence just a completely monotone function whose monotonicity is “truncated”.

fasshauer@iit.edu MATH 590 – Chapter 5 20

Multiply Monotone Functions

Example The truncated power function ϕℓ(r) = (1 − r)ℓ

+

is ℓ-times monotone for any ℓ since (−1)lϕ(l)

ℓ (r) = ℓ(ℓ − 1) . . . (ℓ − l + 1)(1 − r)ℓ−l +

≥ 0, l = 0, 1, 2, . . . , ℓ. Remark We mentioned in Chapter 4 that the truncated power functions lead to radial functions that are strictly positive definite on Rs provided ℓ ≥ ⌊s/2⌋ + 1. We now want to come up with a multiple monotonicity criterion that will let us come to this conclusion much easier.

fasshauer@iit.edu MATH 590 – Chapter 5 21

Multiply Monotone Functions

Example If we define the integral operator I by (If)(r) = ∞

r

f(t)dt, r ≥ 0, (2) and f is ℓ-times monotone, then If is ℓ + 1-times monotone. This follows immediately from the fundamental theorem of calculus. Remark The operator I plays an important role in the construction of compactly supported radial basis functions.

fasshauer@iit.edu MATH 590 – Chapter 5 22

Multiply Monotone Functions

To make the connection to strictly positive definite radial functions we require an integral representation for the class of multiply monotone

• functions. This was given in [Williamson (1956)] but apparently already

known to Schoenberg in 1940. Theorem (Williamson) A continuous function ϕ : (0, ∞) → R is k-times monotone on (0, ∞) if and only if it is of the form ϕ(r) = ∞ (1 − rt)k−1

+

dµ(t), where µ is a non-negative Borel measure on (0, ∞).

fasshauer@iit.edu MATH 590 – Chapter 5 23

Multiply Monotone Functions

Proof. To see that a function of the form ϕ(r) = ∞ (1 − rt)k−1

+

dµ(t), is indeed multiply monotone we just need to differentiate under the integral (since derivatives up to order k − 2 of (1 − rt)k−1

+

are continuous and bounded). The other direction can be found in [Williamson (1956)]. Remark For k → ∞ the Williamson characterization corresponds to the HBW characterization of completely monotone functions (and is equivalent provided we extend Williamson’s work to include continuity at the

• rigin).

fasshauer@iit.edu MATH 590 – Chapter 5 24

Multiply Monotone Functions

We introduced multiply monotone functions to establish a connection to positive definite radial functions. Such a connection was first noted in [Askey (1973)] (and in the

• ne-dimensional case by Pólya) using the truncated power

functions. In the RBF literature the following theorem was stated in [Micchelli (1986)], and then refined in [Buhmann (1993a)]: Theorem (Micchelli) Let k = ⌊s/2⌋ + 2 be a positive integer. If ϕ : [0, ∞) → R, ϕ ∈ C[0, ∞), is k-times monotone on (0, ∞) but not constant, then ϕ is strictly positive definite and radial on Rs for any s such that ⌊s/2⌋ ≤ k − 2.

fasshauer@iit.edu MATH 590 – Chapter 5 25

Multiply Monotone Functions

Remark

1

This theorem allows us to verify the strict positive definiteness of truncated power functions without the use of Fourier transforms.

2

As for the Bochner functions ΦB(x) = eix·y, Gaussians and the Poisson radial functions earlier, use of a point evaluation measure with mass concentrated at t = ε > 0, shows that we can view the truncated power functions with different support sizes ϕε(r) = (1 − εr)k−1

+

as the fundamental compactly supported positive definite radial functions since any such function is given by an infinite linear combination from {ϕε}ε.

fasshauer@iit.edu MATH 590 – Chapter 5 26

Multiply Monotone Functions

Mismatch I

It is interesting to observe a certain lack of symmetry in the theory for completely monotone and multiply monotone functions. In the completely monotone case we can use

Schoenberg’s theorem to

conclude that if ϕ is completely monotone and not constant then ϕ(·2) is strictly positive definite and radial on Rs for any s.

In the multiply monotone case the square is missing.

We cannot expect the statement with a square to be true in the multiply monotone case.

fasshauer@iit.edu MATH 590 – Chapter 5 27

Multiply Monotone Functions

To see this, consider the truncated power function ϕ(r) = (1 − r)ℓ

+.

We know ϕ is ℓ-times multiply monotone for any ℓ. However, we can show that the function ψ(r) = (1 − r 2)ℓ

+

is not strictly positive definite and radial on Rs for any s since it is not even strictly positive definite and radial on R (and therefore even much less so on any higher-dimensional space).

fasshauer@iit.edu MATH 590 – Chapter 5 28

Multiply Monotone Functions

We can see this from the univariate radial Fourier transform of ψ F1ψ(r) = 1 √ r −1 ∞ (1 − t2)ℓ

+t

1 2 J− 1 2 (rt)dt

=

• 2

π 1 (1 − t2)ℓ cos(rt)dt = 2ℓΓ(ℓ + 1) Jℓ+ 1

2 (r)

r ℓ+ 1

2

. Here we used the compact support of ψ and the fact that J− 1

2 (r) =

• 2/πr cos r.

The function F1ψ is oscillatory, and therefore ψ cannot be strictly positive definite by one of our earlier theorems.

fasshauer@iit.edu MATH 590 – Chapter 5 29

Multiply Monotone Functions

Mismatch II

In the completely monotone case we have an equivalence between completely monotone and strictly positive definite functions that are radial on any Rs

Schoenberg’s theorem .

Again, we cannot expect such an equivalence to hold in the multiply monotone case, i.e., the converse of

• ur earlier theorem cannot be true.

This is clear since we have already seen a number of functions that are strictly positive definite and radial, but not monotone at all — namely the oscillatory Laguerre-Gaussians and the Poisson radial functions.

fasshauer@iit.edu MATH 590 – Chapter 5 30

Multiply Monotone Functions

Additional insights

It is interesting to combine the

Schoenberg Theorem and Williamson’s characterization .

If one starts with the strictly positive definite radial Gaussian ϕ(r) = e−ε2r 2, then Schoenberg’s theorem tells us that φ(r) = ϕ( √ r) = e−ε2r is completely monotone. Now, any function that is completely monotone is also multiply monotone of any order, so that we can use Williamson’s characterization to conclude that the function φ(r) = e−ε2r is also strictly positive definite and radial on Rs for all s.

fasshauer@iit.edu MATH 590 – Chapter 5 31

Multiply Monotone Functions

Of course, now we can repeat the argument and conclude that ψ(r) = e−ε2√r is strictly positive definite and radial on Rs for all s, and so on. This result was already known to Schoenberg (at least in the non-strict case). In fact, Schoenberg showed that φ(r) = e−ε2r α is positive definite for any α ∈ (0, 2].

fasshauer@iit.edu MATH 590 – Chapter 5 32

Multiply Monotone Functions

Remark We are a long way from having a complete characterization of (radial) functions for which the scattered data interpolation problem has a unique solution. As we will see later, such an (as of now unknown) characterization will involve also functions which are not strictly positive definite. For example, we will mention a result of Micchelli’s according to which conditionally positive definite functions of order one can be used for the scattered data interpolation problem. Furthermore, all of the results dealt with so far involve radial basis functions that are centered at the given data sites. There are only limited results addressing the situation in which the centers for the basis functions and the data sites may differ.

fasshauer@iit.edu MATH 590 – Chapter 5 33

Appendix References

References I

Buhmann, M. D. (2003). Radial Basis Functions: Theory and Implementations. Cambridge University Press. Cheney, E. W. and Light, W. A. (1999). A Course in Approximation Theory. Brooks/Cole (Pacific Grove, CA). Fasshauer, G. E. (2007). Meshfree Approximation Methods with MATLAB. World Scientific Publishers. Feller, W. (1966). An Introduction to Probability Theory and Its Application, Vol. 2. Wiley & Sons, New York. Iske, A. (2004). Multiresolution Methods in Scattered Data Modelling. Lecture Notes in Computational Science and Engineering 37, Springer Verlag (Berlin).

fasshauer@iit.edu MATH 590 – Chapter 5 34

Appendix References

References II

Wendland, H. (2005a). Scattered Data Approximation. Cambridge University Press (Cambridge). Widder, D. V. (1941). The Laplace Transform. Princeton University Press (Princeton). Askey, R. (1973). Radial characteristic functions. TSR #1262, University of Wisconsin-Madison. Buhmann, M. D. (1993a). New developments in the theory of radial basis function interpolation. in Multivariate Approximation: From CAGD to Wavelets, Kurt Jetter and Florencio Utreras (eds.), World Scientific Publishing (Singapore), pp. 35–75.

fasshauer@iit.edu MATH 590 – Chapter 5 35

Appendix References

References III

Micchelli, C. A. (1986). Interpolation of scattered data: distance matrices and conditionally positive definite functions.

• Constr. Approx. 2, pp. 11–22.

Schoenberg, I. J. (1938a). Metric spaces and completely monotone functions.

• Ann. of Math. 39, pp. 811–841.

Williamson, R. E. (1956). Multiply monotone functions and their Laplace transform. Duke Math. J. 23, pp. 189–207.

fasshauer@iit.edu MATH 590 – Chapter 5 36