Expert Knowledge Makes Towards an . . . Towards an . . . - - PowerPoint PPT Presentation

Empirical Observation . . . Towards an Explanation Towards an . . . Towards an . . . Reference Home Page Title Page ◭◭ ◮◮ ◭ ◮ Page 1 of 6 Go Back Full Screen Close Quit

Expert Knowledge Makes Predictions More Accurate: Theoretical Explanation of an Empirical Observation

Julio Urenda1,2, Marco Cardiel2, Laura Hinojos2, Oliver Martinez2, and Vladik Kreinovich2

1Department of Mathematical Sciences 2Department of Computer Science

University of Texas at El Paso, El Paso, TX 79968, USA jcurenda@utep.edu, macardiel@miners.utep.edu, ljhinojos@miners.utep.edu, omartinez14@miners.utep.edu, vladik@utep.edu

Empirical Observation . . . Towards an Explanation Towards an . . . Towards an . . . Reference Home Page Title Page ◭◭ ◮◮ ◭ ◮ Page 2 of 6 Go Back Full Screen Close Quit

1. Empirical Observation That Needs Explaining

• It is known that the use of expert knowledge makes

predictions more accurate.

• For example, computer-based meteorological forecasts

are regularly corrected by experts.

• A typical improvement is that the accuracy consis-

tently improves by 10%.

• How can we explain this?

Empirical Observation . . . Towards an Explanation Towards an . . . Towards an . . . Reference Home Page Title Page ◭◭ ◮◮ ◭ ◮ Page 3 of 6 Go Back Full Screen Close Quit

2. Towards an Explanation

• Use of expert knowledge means, in effect, that we com-

bine: – an estimate produced by a computer model and – an expert estimate.

• Let σm and σe denote the standard deviations, corre-

spondingly, of the model and of the expert estimate.

• In effect, the only information that we have about com-

paring the two accuracies is that – expert estimates are usually less accurate – than model results: σm < σe.

• So, if we fix σe, then the only thing we know about σm

is that σm is somewhere between 0 and σe.

Empirical Observation . . . Towards an Explanation Towards an . . . Towards an . . . Reference Home Page Title Page ◭◭ ◮◮ ◭ ◮ Page 4 of 6 Go Back Full Screen Close Quit

3. Towards an Explanation (cont-d)

• We have no reason to assume that some values from

the interval [0, σe] are more probable than others.

• Thus, it makes sense to assume that all these values

are equally probable.

• So, we have a uniform distribution on this interval.
• For this uniform distribution, the average value of σm

is equal to 0.5 · σe.

• Thus, we have σe = 2 · σm.
• In general:

– if we combine two estimates xm and xe with accu- racies σm and σe, – then the combined estimate xc is obtained by min- imizing the sum (xm − xc)2 σ2

m

+ (xe − xc)2 σ2

e

.

Empirical Observation . . . Towards an Explanation Towards an . . . Towards an . . . Reference Home Page Title Page ◭◭ ◮◮ ◭ ◮ Page 5 of 6 Go Back Full Screen Close Quit

4. Towards an Explanation (cont-d)

• The resulting estimate is xc = xm · σ−2

m + xe · σ−2 e

σ−2

m + σ−2 e

, with accuracy σ2

c =

1 σ−2

m + σ−2 e

.

• For σe = 2σm, we have σ−2

e

= 0.25 · σ−2

m , thus σ2 c =

σ2

m ·

1 1 + 0.25 = σ2

m ·

1 1.25 = 0.8 · σ2

m, thus σc ≈ 0.9 · σm.

• So we indeed get a 10% increase in the resulting pre-

diction.

Empirical Observation . . . Towards an Explanation Towards an . . . Towards an . . . Reference Home Page Title Page ◭◭ ◮◮ ◭ ◮ Page 6 of 6 Go Back Full Screen Close Quit

5. Reference

• N. Silver, The Signal and the Noise: Why So Many

Decisions Fail – but Some Don’t, Penguin Press, New York, 2012.