Reliability of Expert Judgments and Uncertainty Judgments and - - PDF document

▶
reliability of expert judgments and uncertainty judgments
SMART_READER_LITE
LIVE PREVIEW

Reliability of Expert Judgments and Uncertainty Judgments and - - PDF document

Mar 31, 2023 328 likes •637 views

Expert Reliability & Uncertainty Steve Begg, ASP, University Adelaide SPE Distinguished Lecturer Program Primary funding is provided by The SPE Foundation through member donations S and a contribution from Offshore Europe The Society is

slide-1
SLIDE 1

Steve Begg, ASP, University Adelaide Expert Reliability & Uncertainty 1 SPE DL 2010

SPE Distinguished Lecturer Program

Primary funding is provided by

S The SPE Foundation through member donations and a contribution from Offshore Europe

The Society is grateful to those companies that allow their professionals to serve as lecturers

2010-2011 SPE Distinguished Lecture Experts and Uncertainty Steve Begg 1

professionals to serve as lecturers Additional support provided by AIME

Society of Petroleum Engineers Distinguished Lecturer Program

www.spe.org/dl

Reliability of Expert Judgments and Uncertainty

SPE Distinguished Lecture, 2010-2011

Judgments and Uncertainty Assessments

Steve Begg

Australian School of Petroleum, University of Adelaide Centre for Improved Business Performance

slide-2
SLIDE 2

Steve Begg, ASP, University Adelaide Expert Reliability & Uncertainty 2 SPE DL 2010

Reliability of Expert Judgments and Uncertainty

SPE Distinguished Lecture, 2010-2011

Judgments and Uncertainty Assessments

Steve Begg

Australian School of Petroleum, University of Adelaide

“All business proceeds on beliefs, or j d t f b biliti d t

Centre for Improved Business Performance

judgments of probabilities, and not on certainties".

Charles W. Eliot

Outline

  • The Nature of Uncertainty

P l P b bilit d

  • People, Probability and

Judgment

  • Performance of Industry

Experts

  • Conclusions

2010-2011 SPE Distinguished Lecture Experts and Uncertainty Steve Begg 4

Conclusions

slide-3
SLIDE 3

Steve Begg, ASP, University Adelaide Expert Reliability & Uncertainty 3 SPE DL 2010

Industry Performance

  • Comments & Observations

– “Every one of our 10 most important projects failed to generate the desired return ” (Super Major) generate the desired return. (Super Major) – “The actual performance of our key assets wasn’t even within the P1 to P99 range.” (Large Independent) – “I want your guarantee that we will not spend more than the P50 on this project!” (CEO to Manager) – “a decade of unprofitable growth”; vast majority of

2010-2011 SPE Distinguished Lecture Experts and Uncertainty Steve Begg 5

– a decade of unprofitable growth ; vast majority of projects take longer, cost more and produce less than predicted; 1-in-8 of major offshore are “disasters” (IPA)

The fundamental problem: Industry performance not living up to expectations, or possibilities

Uncertainty

2010-2011 SPE Distinguished Lecture Experts and Uncertainty Steve Begg 8

slide-4
SLIDE 4

Steve Begg, ASP, University Adelaide Expert Reliability & Uncertainty 4 SPE DL 2010

The fundamental problem: Industry performance not living up to expectations, or possibilities

  • People tend to grossly under-estimate uncertainty

– number of uncertain factors and the magnitude of uncertainty uncertainty – complexity of the relationships between them and therefore un-anticipated non-intuitive outcomes)

  • Better decision-making, at asset and portfolio levels,

first requires accurate (= unbiased & appropriate range) uncertainty assessment

2010-2011 SPE Distinguished Lecture Experts and Uncertainty Steve Begg 9

  • Reduce uncertainty only if

– it can change a decision (eg Mitigate downside risk and/or capture upside opportunities) AND – expected benefit of reduction is less than its cost

The fundamental problem: Industry performance not living up to expectations, or possibilities

Better performance Better Decisions

2010-2011 SPE Distinguished Lecture Experts and Uncertainty Steve Begg 10

Better uncertainty assessment

slide-5
SLIDE 5

Steve Begg, ASP, University Adelaide Expert Reliability & Uncertainty 5 SPE DL 2010

Probability: The Language of Uncertainty

  • Classical (Theoretical)

Number of outcomes representing the occurrence of an event Total number of possible outcomes 30 d b ll d 70 b ll i b P(R d) 30% – e.g. 30 red balls and 70 green balls in a bag. P(Red) = 30%

  • Relative Frequency

– proportion of times an event occurs in the long run – can be ESTIMATED from sample data ASSUMING identical events e.g. 15 out of 20 wells drilled were dry holes. P(Dry) = 75% – More accurate with greater sample size. May not apply to future.

2010-2011 SPE Distinguished Lecture Experts and Uncertainty Steve Begg 11

  • Subjective

– Personal degree of belief of the likelihood of a future event occurring (or of the unknown outcome of a past event) – May be based on some past similar / analogous occurrences

Uncertainty: not knowing if a statement is true or not

Throw a die and hide top face. What is the probability of a 3? 1/6 Now you get information. Has has the top face changed? No

2010-2011 SPE Distinguished Lecture Experts and Uncertainty Steve Begg 12

What is the probability of a 3 now?

Uncertainty is a function of what you know. There is no “right” uncertainty (or PDF)!

p g

  • Has the probability of a 3 changed? Yes!

1/3

slide-6
SLIDE 6

Steve Begg, ASP, University Adelaide Expert Reliability & Uncertainty 6 SPE DL 2010

Uncertainty is in OUR heads – it’s a function of

  • ur state of knowledge

Different people can, legitimately, hold

Its not a feature of the “system”. A consequence:

0.333 0.333

p p g y different views about the uncertainty of an unknown quantity

Person A Person B

2010-2011 SPE Distinguished Lecture Experts and Uncertainty Steve Begg 13

1 2 3 5 6 4 Outcome Prob. 1 2 3 5 6 4 Outcome Prob.

Uncertainty is in YOUR head: What’s it worth to know more?

Another consequence:

Information might have value by virtue of its ability to change probabilities its ability to change probabilities

  • Betting Game
  • Win $100 if it is a 3.
  • Lose $10 if it is not
  • How much is it worth to look at

?

2010-2011 SPE Distinguished Lecture Experts and Uncertainty Steve Begg 14

How much is it worth to look at (get information about) the centre?

Corollary: you might change your mind on whether to bet or not as a result of getting new information

slide-7
SLIDE 7

Steve Begg, ASP, University Adelaide Expert Reliability & Uncertainty 7 SPE DL 2010

Probability is subjective (personal) and depends upon your information

Person A Info Person B Info Shared Info

2010-2011 SPE Distinguished Lecture Experts and Uncertainty Steve Begg 15

Person A PDF Person B PDF

Probability is subjective: games of chance – eg coin tossing

Person A Info (A thinks there is a Person B Info (A thinks there is a small chance the coin is biased towards heads) Info Shared Info

2010-2011 SPE Distinguished Lecture Experts and Uncertainty Steve Begg 16

Person A PDF Person B PDF Heads Tails Heads Tails

slide-8
SLIDE 8

Steve Begg, ASP, University Adelaide Expert Reliability & Uncertainty 8 SPE DL 2010

Probability is subjective: implications for companies in joint ventures

Company A Info Company B Info Info Shared Info

2010-2011 SPE Distinguished Lecture Experts and Uncertainty Steve Begg 17

Company A PDF Company B PDF

Uncertainty vs Risk

Uncertainty Risk

2010-2011 SPE Distinguished Lecture Experts and Uncertainty Steve Begg 18

  • A Risk (noun!) is one possible consequence of uncertainty. It

has a negative connotation, which is “personal” to the D-M

  • an event that has a negative impact on DM’s objectives
  • It is specified by defining the event and assessing its probability,
slide-9
SLIDE 9

Steve Begg, ASP, University Adelaide Expert Reliability & Uncertainty 9 SPE DL 2010

Don’t take a biassed approach to managing consequences of uncertainty

  • Risk is only one outcome of uncertainty - so is Opportunity!

– often over-looked – it is a source of value creation

Consequences of Uncertainty

Risk

Opportunity

2010-2011 SPE Distinguished Lecture Experts and Uncertainty Steve Begg 19

  • Possibility of loss or injury
  • A dangerous element or factor
  • The degree of probability of

loss

  • Possibility of exceeding

expectations

  • Upside potential
  • A wonderful element or factor

Uncertainty vs Variability

Variability of all sand- body widths Width Uncertainty in individual sand-body width ? ?

2010-2011 SPE Distinguished Lecture Experts and Uncertainty Steve Begg 20

y ? ? Width

Sand 1

? ?

Sand 2

slide-10
SLIDE 10

Steve Begg, ASP, University Adelaide Expert Reliability & Uncertainty 10 SPE DL 2010

Uncertainty vs Variability

Variability of all sand- body widths

A di t ib ti th t

Width Uncertainty in individual sand-body width ? ?

A distribution that describes the variability

  • f a natural phenomenon is

not usually appropriate to describe the uncertainty

2010-2011 SPE Distinguished Lecture Experts and Uncertainty Steve Begg 21

y ? ? Width

Sand 1

? ?

Sand 2

describe the uncertainty

  • f a single occurrence

Gambling (probability = repeated outcomes) vs. “Real World” (probability = degree of belief)

Uncertainty Quantification All Identified Some missed

  • r unknowable

Known Distribution Type Unknown Distribution Type

  • 1. Identify Possible Outcomes
  • 2. Assign Probabilities to Outcomes

2010-2011 SPE Distinguished Lecture Experts and Uncertainty Steve Begg 22

Known Parameters Unknown Parameters Games of Chance: Classical Prob & Stats Oil & Gas: Subjective

slide-11
SLIDE 11

Steve Begg, ASP, University Adelaide Expert Reliability & Uncertainty 11 SPE DL 2010

Outline

  • The Nature of Uncertainty

P l P b bilit d

  • People, Probability and

Judgment

  • Performance of Industry

Experts

  • Conclusions

2010-2011 SPE Distinguished Lecture Experts and Uncertainty Steve Begg 23

Conclusions

Estimate the gray %

Using a scale of 0% (black) to 100% (white) estimate the % gray of squares A and B

2010-2011 SPE Distinguished Lecture Experts and Uncertainty Steve Begg 24

slide-12
SLIDE 12

Steve Begg, ASP, University Adelaide Expert Reliability & Uncertainty 12 SPE DL 2010

*Estimate the aspect ratio

2010-2011 SPE Distinguished Lecture Experts and Uncertainty Steve Begg 25

Perceptual Limitations as a Metaphor Cognitive Limitations

  • Awareness of illusions by itself does

Awareness of illusions, by itself, does not produce a more accurate perception.

  • Illusions & cognitive errors therefore,

can be extremely difficult to

2010-2011 SPE Distinguished Lecture Experts and Uncertainty Steve Begg 27

can be extremely difficult to

  • vercome.
slide-13
SLIDE 13

Steve Begg, ASP, University Adelaide Expert Reliability & Uncertainty 13 SPE DL 2010

Judging likelihoods of events

  • Linda is a 31 years old, single, outspoken and very bright.

She majored in philosophy. As a student she was deeply concerned with issues of discrimination and social justice, and also participated in anti-nuclear demonstrations. p p Which is the more likely alternative? a) Linda is a bank teller b) Linda is a bank teller and active in the feminist movement.

2010-2011 SPE Distinguished Lecture Experts and Uncertainty Steve Begg 29

Answer:______

*Discussion of Linda Question

  • Nearly 90% of respondents choose the second alternative

(bank teller and active in the feminist movement), even though this is provably logically incorrect bank tellers feminists

2010-2011 SPE Distinguished Lecture Experts and Uncertainty Steve Begg 31

P(Bank Teller) > P(Bank Teller AND Feminist)

feminist bank tellers

Junctions (“ands”) are always less likely than stand-alone statements.

slide-14
SLIDE 14

Steve Begg, ASP, University Adelaide Expert Reliability & Uncertainty 14 SPE DL 2010

Cognitive Illusions

  • The description of Linda is more representative of a

feminist bank teller so people, wrongly, conclude it is more likely that she is a feminist and a bank teller

  • Kahneman & Tversky (1982)

– “As the amount of detail in a scenario increases, its probability can only decrease steadily, but its representativeness and hence its apparent likelihood may increase.” “Th li t ti b li i i

2010-2011 SPE Distinguished Lecture Experts and Uncertainty Steve Begg 33

– “The reliance on representativeness, we believe, is a primary reason for the unwarranted appeal of detailed scenarios and the illusory sense of insight that such constructions often provide.”

  • Implications: consider a “rich” description of a

reservoir depositional environment

Count the Passes

2010-2011 SPE Distinguished Lecture Experts and Uncertainty Steve Begg 34

slide-15
SLIDE 15

Steve Begg, ASP, University Adelaide Expert Reliability & Uncertainty 15 SPE DL 2010

Heuristics, Biases & Uncertainty

  • Heuristics

– simple rules of thumb and mental shortcuts

Human beings are not endowed with

simple rules of thumb and mental shortcuts

  • Biases

– systematic errors that are a result of the use of heuristics

  • Our “mental wiring” is just not good when it

Human beings are not endowed with rational probabilistic thinking and optimal behaviour under uncertainty.

2010-2011 SPE Distinguished Lecture Experts and Uncertainty Steve Begg 36

comes to uncertainty

– Intuition and “gut feel” often significantly wrong

Bias & error => poor decisions => undesirable outcomes Heuristics, Biases & Uncertainty

  • Heuristics

– simple rules of thumb and mental shortcuts simple rules of thumb and mental shortcuts

  • Biases

– systematic errors that are a result of the use of heuristics

  • Our “mental wiring” is just not good when it

2010-2011 SPE Distinguished Lecture Experts and Uncertainty Steve Begg 37

comes to uncertainty

– Intuition and “gut feel” often significantly wrong

slide-16
SLIDE 16

Steve Begg, ASP, University Adelaide Expert Reliability & Uncertainty 16 SPE DL 2010

Evidence of Bias: Data from IPA

All 1000+ projects If the Forecasted production is the “Base Case”, we h ld h i t l projects in the study No projects should have approximately as many projects producing more than expected as less than expected !!

2010-2011 SPE Distinguished Lecture Experts and Uncertainty Steve Begg 39

50 100 150 200 250 300

Basis for development sanction

0% 50% 100% 150% 200% 250% 300%

Outline

  • Introduction

Th N t f U t i t

  • The Nature of Uncertainty
  • People, Probability and

Judgment

  • Performance of Industry

Experts

2010-2011 SPE Distinguished Lecture Experts and Uncertainty Steve Begg 40

Experts

  • Conclusions
slide-17
SLIDE 17

Steve Begg, ASP, University Adelaide Expert Reliability & Uncertainty 17 SPE DL 2010

Assing the abilty of experts to assign P10-P90 ranges

Lower Limit (P10) Upper Limit (P90) 80% Chance 10% 10% Question 1 Question 2 Question 3 etc

E.g. What was daily average oil production in the USA in 2003?

2010-2011 SPE Distinguished Lecture Experts and Uncertainty Steve Begg 41

Answer: 7,454,000 g y g p

Overconfidence Results: Large Industry Sample – using “industry- related” questions

0.30 0.35 pants

Expected Observed

0.10 0.15 0.20 0.25 portion of Particip

2010-2011 SPE Distinguished Lecture Experts and Uncertainty Steve Begg 42

0.00 0.05 1 2 3 4 5 6 7 8 9 10 Prop Questions Correct /10

slide-18
SLIDE 18

Steve Begg, ASP, University Adelaide Expert Reliability & Uncertainty 18 SPE DL 2010

Probability Intuition: Assessing co-variation

Present Absent Present 16 4 Seismic Hydro-carbons Present 16 4 Absent 4 1

  • Which cells of the table are needed to determine whether

seismic anomalies are associated with hydrocarbons

Upper Upper Lower Lower

Anomaly

2010-2011 SPE Distinguished Lecture Experts and Uncertainty Steve Begg 43

Upper Upper Lower Lower Left Right Left Right

  • According to the data in the table, do seismic anomalies

increase the probability of hydrocarbon presence? Yes/No

Probability Intuition: Assessing co-variation

Present Absent Total Present

16 4 20

Hydrocarbons Seismic

  • All 4 pieces on information (cells) are required
  • Conditional probabilities of HC presence given the

presence/absence of seismic anomaly

Absent

4 1 5

Total 20 5 25 Seismic Anomaly

2010-2011 SPE Distinguished Lecture Experts and Uncertainty Steve Begg 44

P(HC Present | Anomaly present) = 16/20 = 80% P(HC Present | Anomaly absent) = 4/5 = 80%

  • Probability of HC being present is the same, whether or

not a seismic anomaly is present – so no information

slide-19
SLIDE 19

Steve Begg, ASP, University Adelaide Expert Reliability & Uncertainty 19 SPE DL 2010

Assessing Co-variation (presumed association): Results

80 90 100

Correct Incorrect

30 40 50 60 70 80

  • f Participants

2010-2011 SPE Distinguished Lecture Experts and Uncertainty Steve Begg 45

10 20 30 First Both %

Correct Answer Correct for right reason

Probability Intuition: Assessing Reliability of Predictors

  • Historical estimates suggest one in every 1000

blow-out preventers has serious cracks.

  • Suppose x-ray analysis is a very good but not
  • Suppose x-ray analysis is a very good, but not

perfect, detector of these cracks.

– If a blow-out preventer has cracks, x-rays will correctly say it has them 99% of the time – If a blow-out preventer does not have cracks, x-rays will wrongly say that it has them 2% of the time

2010-2011 SPE Distinguished Lecture Experts and Uncertainty Steve Begg 46

  • A blow-out preventer has been x-rayed at random

and the result was positive!

– What is your intuitive assessment of the chances that is cracked?

slide-20
SLIDE 20

Steve Begg, ASP, University Adelaide Expert Reliability & Uncertainty 20 SPE DL 2010

Probability Intuition: Assessing Reliability of Predictors

  • Historical estimates suggest one in every 1000

blow-out preventers has serious cracks.

  • Suppose x-ray analysis is a very good but not
  • Suppose x-ray analysis is a very good, but not

perfect, detector of these cracks.

– If a blow-out preventer has cracks, x-rays will correctly say it has them 99% of the time – If a blow-out preventer does not have cracks, x-rays will wrongly say that it has them 2% of the time

4.7%!

P( test positive given crack)=99% P( crack given test positive)=4.7%

2010-2011 SPE Distinguished Lecture Experts and Uncertainty Steve Begg 47

  • A blow-out preventer has been x-rayed at random

and the result was positive!

– What is your intuitive assessment of the chances that is cracked?

Reliability of predictors: Results

70 80 90 30 40 50 60

No of Participants

2010-2011 SPE Distinguished Lecture Experts and Uncertainty Steve Begg 48

10 20 20 40 60 80 100

Estimated Probability

slide-21
SLIDE 21

Steve Begg, ASP, University Adelaide Expert Reliability & Uncertainty 21 SPE DL 2010

Anchoring - Subtle changes in wording of a question can significantly impact responses

Question Outcome

How long was the movie? 130 min

  • g

as e

  • e

How short was the movie? 30 100 min

Question Outcome

2010-2011 SPE Distinguished Lecture Experts and Uncertainty Steve Begg 49

How wide are the channels? How narrow are the channels?

Anchoring Question: Large Industry Sample

  • Alternate versions of question with high and low

anchors given to two groups

Hi h A h G “W ld d il i – High Anchor Group: “Were world proved oil reserves in 2003 greater or less than 1721.6 Billion Barrels?” – Low Anchor Group: “Were world proved oil reserves in 2003 greater or less than 573.9 Billion Barrels?”

B th th k d

2010-2011 SPE Distinguished Lecture Experts and Uncertainty Steve Begg 50

  • Both groups then asked

– “What is your best estimate of the world proved oil reserves in 2003?”

slide-22
SLIDE 22

Steve Begg, ASP, University Adelaide Expert Reliability & Uncertainty 22 SPE DL 2010

Anchoring Results: Large Industry Sample

3000 3500 Anchor 1932 1722 1000 1500 2000 2500 mated World Proved ves 2003 +/- 1sd Estimate

2010-2011 SPE Distinguished Lecture Experts and Uncertainty Steve Begg 51

574 682 500 Low High Anchor Group Mean Estim Reserv

Anchoring Results: Large Industry Sample

3000 3500 Anchor 1932 1722 1000 1500 2000 2500 mated World Proved ves 2003 +/- 1sd Estimate

Common approach in E&P project evaluation: “Let’s start with a base case and then build some scenarios around it ”

2010-2011 SPE Distinguished Lecture Experts and Uncertainty Steve Begg 52

574 682 500 Low High Anchor Group Mean Estim Reserv

then build some scenarios around it.

slide-23
SLIDE 23

Steve Begg, ASP, University Adelaide Expert Reliability & Uncertainty 23 SPE DL 2010

  • Unpacking Question (RoW Packed)

– “What % of world oil consumption is accounted for by each of the following regions: North

Unpacking

for by each of the following regions: North America, Europe/Eurasia and the Rest of the World?”

  • Unpacking Question (RoW Unpacked)

“What % of world oil consumption is accounted

2010-2011 SPE Distinguished Lecture Experts and Uncertainty Steve Begg 53

– What % of world oil consumption is accounted for by each of the following regions: North America, Europe/Eurasia, South and Central America, Middle East, Africa, Asia Pacific and the Rest of the World?”.

Unpacking Question Results

50 60 Oil 1SD

True: Packed Unpacked

20 30 40 mated % of World umption (2003) +/-

Unpacked

2010-2011 SPE Distinguished Lecture Experts and Uncertainty Steve Begg 54

10 N America Europe/Eur Rest of World Esti Consu Region

slide-24
SLIDE 24

Steve Begg, ASP, University Adelaide Expert Reliability & Uncertainty 24 SPE DL 2010

Availability and Vividness Bias – Seeing What We Believe

  • The tendency people have

to base estimates of frequencies (probabilities) frequencies (probabilities)

  • n the most readily

available, recent and vivid information they can remember

  • how many events of a

particular type are available to memory

2010-2011 SPE Distinguished Lecture Experts and Uncertainty Steve Begg 56

to memory

  • more available events are

judged more likely

  • Memory is limited to 7

“chunks”

Bias from the Availability Heuristic & Saliency

  • Tversky and Kahneman (1974)

– “decision makers assess the frequency of a class or the probability of an event by the ease with which the probability of an event by the ease with which instances or occurrences can be brought to mind.”

  • Managers conducting performance appraisals:

– Working from memory, vivid instances of an employee’s behaviour (either positive or negative) will be most easily recalled from memory and will appear

2010-2011 SPE Distinguished Lecture Experts and Uncertainty Steve Begg 57

more numerous than more commonplace instances. – Managers give more weight to performance during the three months prior to the evaluation than to the previous nine months of the evaluation period.

slide-25
SLIDE 25

Steve Begg, ASP, University Adelaide Expert Reliability & Uncertainty 25 SPE DL 2010

Does it Matter? Overconfidence Model

Biassed True

2010-2011 SPE Distinguished Lecture Experts and Uncertainty Steve Begg 59

12500 15000 17500 20000 22500 Area OC20 True 10th/20th percentile

400

  • n

Economic Impact of Overconfidence

Welsh, Begg & Bratvold (2007) SPE 110765

100 200 300 Real E(NPV), $Millio

NPV EV

Assessed (Overconfident) Value

2010-2011 SPE Distinguished Lecture Experts and Uncertainty Steve Begg 61

  • 100

0% 5% 10% 15% 20% 25% 30% Overconfidence

slide-26
SLIDE 26

Steve Begg, ASP, University Adelaide Expert Reliability & Uncertainty 26 SPE DL 2010

Risk Aversion and Incentives

  • Your track record hasn’t been too good recently. You can

recommend one of two investments. Which one? P Pf EV $MM Ps Pf EV, $MM “Safe” 80% 20% 10 “Risky” 10% 90% 20

  • You should take a corporate (organizational) attitude to risk,

not a personal one, and recommend “Risky” based on EV

2010-2011 SPE Distinguished Lecture Experts and Uncertainty Steve Begg 63

  • a pe so a o e, a d eco

e d s y based o – any other choice (being risk-averse or risk-seeking) is value destroying (money–losing)

  • Most incentive policies, focused on reward by outcome,

encourage inappropriate risk-aversion, therefore value loss!

Risk attitudes. Will you play the game?

Tails 50% (1) (10) (100) (1,000) (10,000) (100,000) (1,000,000)

Play game 1

Yes Outcome Tails Heads 50% 3

E=1

30

E=10

300

E=100

3,000

E=1,000

30,000

E=10,000

300,000

E=100,000

3,000,000

E=1,000,000

2010-2011 SPE Distinguished Lecture Experts and Uncertainty Steve Begg 64

game 1

No

slide-27
SLIDE 27

Steve Begg, ASP, University Adelaide Expert Reliability & Uncertainty 27 SPE DL 2010

Risk attitudes

Tails 50%

Play game 2

Outcome Tails Heads Yes 50% 3

E=1.5

30

E=15

300

E=150

3,000

E=1,500

30,000

E=15,000

300,000

E=150,000

3,000,000

E=1,500,000

2010-2011 SPE Distinguished Lecture Experts and Uncertainty Steve Begg 65

game 2

No 1 10 100 1,000 10,000 100,000 1,000,000

Risk Aversion: Consequences

  • Expected Value rule is sometimes not followed because its

focus on an average outcome ignores

– the range of outcomes and their probabilities (risks) the consequences of the outcome – the consequences of the outcome

  • Risk aversion demonstrably leads to lower returns in the long

run (=over many decisions)

– but can be optimal in some circumstances – when a large part of

  • ur wealth is invested in few items

– Most of us buy insurance

Re ard/penalt mechanisms can create a mis alignment

2010-2011 SPE Distinguished Lecture Experts and Uncertainty Steve Begg 66

  • Reward/penalty mechanisms can create a mis-alignment

between organizational and individual attitudes to (tolerance for) risk

– Leading to inappropriate risk-aversion and therefore lower returns – Eg “risky” projects not brought forward

slide-28
SLIDE 28

Steve Begg, ASP, University Adelaide Expert Reliability & Uncertainty 28 SPE DL 2010

Reasoning under uncertainty = using the rules of probability

  • In terms of practical applicability, probability theory

is comparable with geometry;

  • both are branches of applied mathematics that are
  • both are branches of applied mathematics that are

directly linked with the problems of daily life.

  • While most people have a natural feel for geometry

(at least to some extent), many people clearly have trouble developing a good intuition for probability.

  • In no other branch of mathematics is it so easy to

2010-2011 SPE Distinguished Lecture Experts and Uncertainty Steve Begg 67

  • ot e b a c
  • at e

at cs s t so easy to make mistakes as in probability theory.

  • Conditional probabilities, and Bayes theorem in

particular, are especially difficult

Reasoning under uncertainty = using the rules of probability

  • In terms of practical applicability, probability theory

is comparable with geometry;

  • both are branches of applied mathematics that are

“The theory of probabilities is at bottom nothing but common sense reduced to calculus; …

  • both are branches of applied mathematics that are

directly linked with the problems of daily life.

  • While most people have a natural feel for geometry

(at least to some extent), many people clearly have trouble developing a good intuition for probability.

  • In no other branch of mathematics is it so easy to

; It teaches us to avoid the illusions which often mislead us; … there is no science more worthy of our contemplations nor a more useful one for admission to our system of public education ”

2010-2011 SPE Distinguished Lecture Experts and Uncertainty Steve Begg 68

  • ot e b a c
  • at e

at cs s t so easy to make mistakes as in probability theory.

  • Conditional probabilities, and Bayes theorem in

particular, are especially difficult

to our system of public education.

Laplace – Theorie Analytique des Probabilites

slide-29
SLIDE 29

Steve Begg, ASP, University Adelaide Expert Reliability & Uncertainty 29 SPE DL 2010

Conclusions

  • In our context, uncertainty is a function of what we

know about a situation – its in our heads, not a part

  • f the “system”
  • there is no single, “right” probability for an uncertain event
  • variability is not the same thing as uncertainty
  • Evolution has not “wired” our brains for a good

natural ability to assess uncertainty

  • training helps, but even industry experts, including those

2010-2011 SPE Distinguished Lecture Experts and Uncertainty Steve Begg 69

g y g whose job it is to deal with uncertainty, are not great

  • Do not use intuition to propagate (amalgmate)

assessed uncertainties

  • use the rules of probability, or Monte Carlo simulation

Acknowledgements

  • Matthew Welsh, University of Adelaide

Reidar Bratvold University of Stavanger

  • Reidar Bratvold, University of Stavanger
  • Michael Lee, University of California, Irvine
  • and, again, SPE staff for organization of the tour

and SPE Foundation for financial support

2010-2011 SPE Distinguished Lecture Experts and Uncertainty Steve Begg 70

and SPE Foundation for financial support

slide-30
SLIDE 30

Steve Begg, ASP, University Adelaide Expert Reliability & Uncertainty 30 SPE DL 2010

2010-2011 SPE Distinguished Lecture Experts and Uncertainty Steve Begg 71

“I make decisions as much with my gut as my brain. Let’s eat.