Limits to Nonlinear Inversion Klaus Mosegaard Univ. of Copenhagen - PowerPoint PPT Presentation

Limits to Nonlinear Inversion Klaus Mosegaard Univ. of Copenhagen September 2008 Klaus Mosegaard (Univ. of Copenhagen) Limits to Nonlinear Inversion September 2008 1 / 37

Outline (and basic theses to be substantiated) 1 The most di ffi cult task: To find a solution! . 2 Once the solutions are found, evaluation of uncertainties is usually relatively easy. 3 If the inversion algorithm has not converged properly to the solution(s), this is the most significant source of uncertainty ! 4 The futility of blind inversion - the use of general purpose algorithms. 5 Inversion algorithms built for the specific problem perform better! Klaus Mosegaard (Univ. of Copenhagen) Limits to Nonlinear Inversion September 2008 2 / 37

The most di ffi cult problem: To find a solution! Klaus Mosegaard (Univ. of Copenhagen) Limits to Nonlinear Inversion September 2008 3 / 37

The logic of Data Analysis M Klaus Mosegaard (Univ. of Copenhagen) Limits to Nonlinear Inversion September 2008 4 / 37

The logic of Data Analysis M M(p) Klaus Mosegaard (Univ. of Copenhagen) Limits to Nonlinear Inversion September 2008 5 / 37

The logic of Data Analysis M M(d ) 1 M(p) Klaus Mosegaard (Univ. of Copenhagen) Limits to Nonlinear Inversion September 2008 6 / 37

The logic of Data Analysis M M(d ) 2 M(d ) 1 M(p) Klaus Mosegaard (Univ. of Copenhagen) Limits to Nonlinear Inversion September 2008 7 / 37

The logic of Data Analysis M M(d ) 2 M(d ) 1 M(p) M(s) = M(d ) « M(d ) « M(p) 1 2 Klaus Mosegaard (Univ. of Copenhagen) Limits to Nonlinear Inversion September 2008 8 / 37

The Bayesian view Define indicator functions: � 1 if m ∈ M ( d j ) L j ( m ) = 0 otherwise � 1 if m ∈ M ( p ) ρ ( m ) = 0 otherwise � 1 if m is a solution σ ( m ) = 0 otherwise then σ ( m ) = L 1 ( m ) . . . L N ( m ) ρ ( m ) � �� � L ( m | d ) “Softening” the indicator functions to probability densities leaves us with Bayes’ Rule. Klaus Mosegaard (Univ. of Copenhagen) Limits to Nonlinear Inversion September 2008 9 / 37

The Deterministic view Models consistent with one datum usually reside in a “narrow neighbourhood” of a manifold with dimension Dim( M ) − 1 M M(d ) 1 Klaus Mosegaard (Univ. of Copenhagen) Limits to Nonlinear Inversion September 2008 10 / 37

The Deterministic view Models consistent with N independent data usually reside in a “narrow neighbourhood” of a manifold with dimension Dim( M ) − N M M(d ) 2 M(d « d ) 1 2 M(d ) 1 Klaus Mosegaard (Univ. of Copenhagen) Limits to Nonlinear Inversion September 2008 11 / 37

The “Curse of Dimensionality” The volume of the solution space decreases at least exponentially with the number of independent data ! n 2 n+1 ! n 4 3 ! R 3 2R ! R 2 R 2n (2n+1)!! R 2n+1 (...) n! (2R) 2 (2R) 3 (2R) 2n (2R) 2n+1 2R (...) Volume hypersphere / Volume hypercube 1.0 1 0.8 0.8 0.6 0.6 0.4 0.4 0.2 0.2 0.0 0 2 4 6 8 10 1 2 3 4 5 6 7 8 9 10 11 Dimension Klaus Mosegaard (Univ. of Copenhagen) Limits to Nonlinear Inversion September 2008 12 / 37

Preliminary observations Let Dim ( M ) : Dimension of model parameter space Dim ( D ) : Dimension of data space Dim ( P ) : Number of independent a priori constraints Observation 1 Given the path to a point in the solution space, the search time along the path is only weakly dependent on Dim ( M ) , Dim ( D ) and Dim ( P ) . Observation 2 Given no information about the solution space, the random search time increases at least exponentially with Dim ( M ) + Dim ( D ) + Dim ( P ) (1) when Dim ( M ) ≥ Dim ( D ) Klaus Mosegaard (Univ. of Copenhagen) Limits to Nonlinear Inversion September 2008 13 / 37

Once the solutions are found, evaluation of uncertainties, is usually relatively easy! Klaus Mosegaard (Univ. of Copenhagen) Limits to Nonlinear Inversion September 2008 14 / 37

Search and sampling Klaus Mosegaard (Univ. of Copenhagen) Limits to Nonlinear Inversion September 2008 15 / 37

Search and sampling Klaus Mosegaard (Univ. of Copenhagen) Limits to Nonlinear Inversion September 2008 16 / 37

Search and sampling Klaus Mosegaard (Univ. of Copenhagen) Limits to Nonlinear Inversion September 2008 17 / 37

Search and sampling Klaus Mosegaard (Univ. of Copenhagen) Limits to Nonlinear Inversion September 2008 18 / 37

If the inversion algorithm has not converged properly to the solution(s), this is the most significant source of uncertainty! Klaus Mosegaard (Univ. of Copenhagen) Limits to Nonlinear Inversion September 2008 19 / 37

Incomplete convergence Klaus Mosegaard (Univ. of Copenhagen) Limits to Nonlinear Inversion September 2008 20 / 37

The futility of blind inversion Klaus Mosegaard (Univ. of Copenhagen) Limits to Nonlinear Inversion September 2008 21 / 37

The question Which one of the following general purpose algorithms is the most e ffi cient? Simulated Annealing, Metropolis Algorithm, Random Search, Rejection Sampling, Genetic Algorithm, Taboo Search, Neighbourhood Algorithm, . . . ? Klaus Mosegaard (Univ. of Copenhagen) Limits to Nonlinear Inversion September 2008 22 / 37

A di ff erent viewpoint: Double-discrete Analysis of Inverse Problems Klaus Mosegaard (Univ. of Copenhagen) Limits to Nonlinear Inversion September 2008 23 / 37

Double-discrete data analysis Here, we shall assume that model parameters are doubly discrete : There is a finite number of model parameters (this is the usual assumption in parameter estimation) Model parameters can only take a finite number of parameter values ! Figure: Original image, image with few pixels, and image with few color levels Klaus Mosegaard (Univ. of Copenhagen) Limits to Nonlinear Inversion September 2008 24 / 37

How fine a discretization is needed for an inverse problem? The misfit function f ( m ) usually inherits continuity from d = g ( m ) , e.g., f ( m ) = � d − g ( m ) � 2 2 σ 2 Now we can define a grid of points representing small regions ∆ m 1 ∆ m 2 . . . of almost constant f ( m ) . Klaus Mosegaard (Univ. of Copenhagen) Limits to Nonlinear Inversion September 2008 25 / 37

How fine a discretization of parameters values is needed? Figure: The Victoria Crater in 256 colors, 16 colors, and 4 colors. Klaus Mosegaard (Univ. of Copenhagen) Limits to Nonlinear Inversion September 2008 26 / 37

Example: Seismic reflection data ∆ m i < 2 σ 2 ǫ / � w � 2 , where σ is the standard deviation of the noise, ǫ is the desired fractional change in misfit over ∆ m i , and w is the seismic wavelet. Klaus Mosegaard (Univ. of Copenhagen) Limits to Nonlinear Inversion September 2008 27 / 37

The discrete counterpart to ”The Curse of Dimensionality” The inverse problem: d 1 = g 1 ( m 1 , m 2 . . . , m M ) d 2 = g 2 ( m 1 , m 2 . . . , m M ) . . . d K = g K ( m 1 , m 2 . . . , m M ) Here, we can freely chose one out of N values for M − K model parameters. This can be done in N M − K ways. After this we have K equation with K unknowns left, and they may have a solution in one, several or all of the above N M − K cases. Proposition The curse of combinatorics. K data reduce the solution space by a factor ≤ N − K Klaus Mosegaard (Univ. of Copenhagen) Limits to Nonlinear Inversion September 2008 28 / 37

A Double-discrete Analysis of the Performance of Inversion Algorithms Klaus Mosegaard (Univ. of Copenhagen) Limits to Nonlinear Inversion September 2008 29 / 37

The typical scenario for nonlinear inversion In the relations d i = g i ( m ) . we have no closed-form mathematical expression for g i ( m ) . We only have a programme that is able to evaluate g i ( m ) for given values of the parameters in m . In short: We are performing a blind search for the solution. Klaus Mosegaard (Univ. of Copenhagen) Limits to Nonlinear Inversion September 2008 30 / 37

Notation 1 Two finite sets X and Y , The set F X of all fit functions/probability distributions f : X → Y . Klaus Mosegaard (Univ. of Copenhagen) Limits to Nonlinear Inversion September 2008 31 / 37

Notation 2 A sample of size m < | X | : { ( x 1 , y 1 ) , . . . , ( x m , y m ) } . The set F X | C of all fit functions/probability distributions defined on X , but with fixed values in C . Klaus Mosegaard (Univ. of Copenhagen) Limits to Nonlinear Inversion September 2008 32 / 37

Proposition The total number of functions intersecting the m samples is |F X | C | = | Y | | X | − m . (2) This number is independent of the location of the sample points. The probability that an algorithm a sees the values y 1 , . . . , y m in the first m steps is then P ( y 1 , . . . , y m | f, m, a ) = | Y | | X | − m = | Y | − m (3) | Y | | X | This number is independent of the algorithm. Klaus Mosegaard (Univ. of Copenhagen) Limits to Nonlinear Inversion September 2008 33 / 37