Limitations of Realistic A Faster Method: . . . Monte-Carlo - PowerPoint PPT Presentation

Need for Data Processing Need to Take . . . Case of Interval . . . How to Compute the . . . Limitations of Realistic A Faster Method: . . . Monte-Carlo Techniques Monte-Carlo: . . . Proof : Case of . . . in Estimating Interval General Case Why Cauchy . . . Uncertainty Home Page Title Page Andrzej Pownuk, Olga Kosheleva, and Vladik Kreinovich ◭◭ ◮◮ Computational Science Program ◭ ◮ University of Texas at El Paso El Paso, TX 79968, USA Page 1 of 21 ampownuk@utep.edu, olgak@utep.edu, vladik@utep.edu Go Back Full Screen Close Quit

Need for Data Processing Need to Take . . . 1. Need for Data Processing Case of Interval . . . • We want to predict the future state of the world, i.e., How to Compute the . . . the future values y of different quantities. A Faster Method: . . . Monte-Carlo: . . . • For this, we need to know how y depends on the current Proof : Case of . . . values x 1 , . . . , x n of the related quantities: General Case y = f ( x 1 , . . . , x n ) . Why Cauchy . . . • Then, we measure x i and make a prediction Home Page y = f ( � x 1 , . . . , � x n ) . � Title Page • Weather prediction shows that the data processing al- ◭◭ ◮◮ gorithm f can be very complex. ◭ ◮ • Data processing is also needed if we are interested in a Page 2 of 21 difficult-to-measure quantity y . Go Back • To estimate y , we measure easier-to-measure quantities Full Screen x 1 , . . . , x n related to y by a known dependence Close y = f ( x 1 , . . . , x n ) . Quit

Need for Data Processing Need to Take . . . 2. Need to Take Uncertainty Into Account When Case of Interval . . . Processing Data How to Compute the . . . • Measurement are never absolutely accurate: in general, A Faster Method: . . . Monte-Carlo: . . . def ∆ x i = � x i − x i � = 0 . Proof : Case of . . . • As a result, the estimate � y = f ( � x 1 , . . . , � x n ) is, in gen- General Case eral, different from the ideal value y = f ( x 1 , . . . , x n ). Why Cauchy . . . Home Page def • To estimate the accuracy ∆ y = � y − y , we need to have Title Page some information about the measurement errors ∆ x i . ◭◭ ◮◮ • Traditional engineering approach assumes that we ◭ ◮ know the probability distribution of each ∆ x i . Page 3 of 21 • Often, ∆ x i ∼ N (0 , σ i ), and different ∆ x i are assumed to be independent. Go Back • In such situations, our goal is to find the probability Full Screen distribution for ∆ y . Close Quit

Need for Data Processing Need to Take . . . 3. Case of Interval Uncertainty Case of Interval . . . • Often, we only know the upper bound ∆ i : | ∆ x i | ≤ ∆ i . How to Compute the . . . A Faster Method: . . . • Then, the only information about the x i is that Monte-Carlo: . . . def x i ∈ x i = [ � x i − ∆ i , � x i + ∆ i ] . Proof : Case of . . . • Different x i ∈ x i lead, in general, to different General Case y = f ( x 1 , . . . , x n ) . Why Cauchy . . . Home Page • We want to find the range y of possible values of y : Title Page y = { f ( x 1 , . . . , x n ) : x 1 ∈ x 1 , . . . , x n ∈ x n } . ◭◭ ◮◮ • Often, measurement errors are relatively small. ◭ ◮ • We can then only keep terms linear in ∆ x i : Page 4 of 21 � n = ∂f def ∆ y = c i · ∆ x i , where c i . Go Back ∂x i i =1 � n Full Screen • In this case, y = [ � y − ∆ , � y + ∆], where ∆ = | c i | · ∆ i . Close i =1 Quit

Need for Data Processing Need to Take . . . 4. How to Compute the Interval Range: Case of Interval . . . Linearized Case How to Compute the . . . • Sometimes, we have explicit expressions or efficient al- A Faster Method: . . . gorithms for the partial derivatives c i . Monte-Carlo: . . . Proof : Case of . . . • Often, however, we proprietary software in our compu- General Case tations. Why Cauchy . . . • Then, we cannot use differentiation formulas or auto- Home Page matic differentiation (AD) tools. Title Page • We can use numerical differentiation: ◭◭ ◮◮ c i ≈ f ( � x 1 , . . . , � x i − 1 , � x i + h i , � x i +1 , . . . , � x n ) − � y . ◭ ◮ h i Page 5 of 21 • Problem: We need n + 1 calls to f , to compute � y and n values c i . Go Back • When f is time-consuming and n is large, this takes Full Screen too long. Close Quit

Need for Data Processing Need to Take . . . 5. A Faster Method: Cauchy-Based Monte-Carlo Case of Interval . . . • Idea: use Cauchy distribution ρ ∆ ( x ) = ∆ 1 How to Compute the . . . π · 1 + x 2 / ∆ 2 . A Faster Method: . . . • Why: when ∆ x i ∼ ρ ∆ i ( x ) are indep., then Monte-Carlo: . . . � � n n Proof : Case of . . . ∆ y = c i · ∆ x i ∼ ρ ∆ ( x ), with ∆ = | c i | · ∆ i . i =1 i =1 General Case • Thus, we simulate ∆ x ( k ) Why Cauchy . . . ∼ ρ ∆ i ( x ); then, i Home Page ∆ y ( k ) def x 1 − ∆ x ( k ) = � y − f ( � 1 , . . . ) ∼ ρ ∆ ( x ). Title Page • Maximum Likelihood method can estimate ∆: � � ◭◭ ◮◮ N N 1 + (∆ y ( k ) ) 2 / ∆ 2 = N 1 ρ ∆ (∆ y ( k ) ) → max, so 2 . ◭ ◮ k =1 k =1 • To find ∆ from this equation, we can use, e.g., the Page 6 of 21 1 ≤ k ≤ N | ∆ y ( k ) | . bisection method for ∆ = 0 and ∆ = max Go Back Full Screen Close Quit

Need for Data Processing Need to Take . . . 6. Monte-Carlo: Successes and Limitations Case of Interval . . . √ • Fact: for Monte-Carlo, accuracy is ε ∼ 1 / N . How to Compute the . . . A Faster Method: . . . • Good news: the number N of calls to f depends only Monte-Carlo: . . . the desired accuracy ε . Proof : Case of . . . • Example: to find ∆ with accuracy 20% and certainty General Case 95%, we need N = 200 iterations. Why Cauchy . . . Home Page • Limitation: this method is not realistic ; indeed: Title Page – we know that ∆ x i is inside [ − ∆ i , ∆ i ], but ◭◭ ◮◮ – Cauchy-distributed variable has a high probability to be outside this interval. ◭ ◮ • Natural question: is it a limitation of our method, or Page 7 of 21 of a problem itself? Go Back • Our answer: for interval uncertainty, a realistic Monte- Full Screen Carlo method is not possible. Close Quit

Need for Data Processing Need to Take . . . 7. Proof : Case of Independent Variables Case of Interval . . . • It is sufficient to prove that we cannot get the correct How to Compute the . . . estimate for one specific function A Faster Method: . . . Monte-Carlo: . . . f ( x 1 , . . . , x n ) = x 1 + . . . + x n , when ∆ y = ∆ x 1 + . . . +∆ x n . Proof : Case of . . . • When each variables ∆ x i is in the interval [ − δ, δ ], then General Case the range of ∆ y is [ − ∆ , ∆], where ∆ = n · δ . Why Cauchy . . . Home Page • In Monte-Carlo, ∆ y ( k ) = ∆ x ( k ) + . . . + ∆ x ( k ) n . 1 Title Page • ∆ ( k ) are i.i.d. Due to the Central Limit Theorem, when i ◭◭ ◮◮ n → ∞ , the distribution of the sum tends to Gaussian. ◭ ◮ • For a normal distribution, with very high confidence, Page 8 of 21 ∆ y ∈ [ µ − k · σ, µ + k · σ ]. • Here, σ ∼ √ n , so this interval has width w ∼ √ n . Go Back Full Screen • However, the actual range of ∆ y is ∼ n ≫ w . Q.E.D. Close Quit

Need for Data Processing Need to Take . . . 8. General Case Case of Interval . . . • Let’s take f ( x 1 , . . . , x n ) = s 1 · x 1 + . . . + s n · x n , where How to Compute the . . . s i ∈ {− 1 , 1 } . A Faster Method: . . . � n Monte-Carlo: . . . • Then, ∆ = | c i | · ∆ i = n · δ. Proof : Case of . . . i =1 General Case • Let ε > 0, δ > 0, and p ∈ (0 , 1). We consider proba- Why Cauchy . . . bility distributions P on the set of all vectors Home Page (∆ x 1 . . . , ∆ x n ) ∈ [ − δ, δ ] × . . . × [ − δ, δ ] . Title Page • We say that P is a ( p, ε ) -realistic Monte-Carlo estima- ◭◭ ◮◮ tion (MCE) if for all s i ∈ {− 1 , 1 } , we have ◭ ◮ Prob( s 1 · ∆ x 1 + . . . + s n · ∆ x n ≥ n · δ · (1 − ε )) ≥ p. Page 9 of 21 • Result. If for every n , we have a ( p n , ε ) -realistic MCE, then p n ≤ β · n · c n for some β > 0 and c < 1 . Go Back • For probability p n , we need 1 /p n ∼ c − n simulations – Full Screen more than n + 1 for numerical differentiation. Close Quit

Need for Data Processing Need to Take . . . 9. Why Cauchy Distribution: Formulation of the Case of Interval . . . Problem How to Compute the . . . • We want to find a family of probability distributions A Faster Method: . . . with the following property: Monte-Carlo: . . . Proof : Case of . . . – when independent X 1 , . . . , X n have distributions General Case from this family with parameters ∆ 1 , . . . , ∆ n , Why Cauchy . . . – then each Y = c 1 · X 1 + . . . + c n · X n ∼ ∆ · X , where Home Page � n X corr. to parameter 1, and ∆ = | c i | · ∆ i . Title Page i =1 • In particular, for ∆ 1 = . . . = ∆ n = 1, the desired ◭◭ ◮◮ property of this probability distribution is as follows: ◭ ◮ – if we have n independent identically distributed Page 10 of 21 random variables X 1 , . . . , X n , Go Back – then each Y = c 1 · X 1 + . . . + c n · X n has the same � n Full Screen distribution as ∆ · X i , where ∆ = | c i | . i =1 Close Quit