Technology, Oct. 24, 2011 Websites & Links Radiation - PowerPoint PPT Presentation

UNCERTAINTY AMBIGUITY Roger Cooke Resources for the Future Dept. Math, Delft Univ. of INDECISION Technology, Oct. 24, 2011

Websites & Links • Radiation Protection Dosimetry 90: (2000) http://rpd.oxfordjournals.org/cgi/content/short/90/3/295 • NUREG EU Probabilistic accident consequence uncertainty analysis http://www.osti.gov/bridge/basicsearch.jsp http://www.osti.gov/energycitations/basicsearch.jsp • EU Probabilistic accident consequence uncertainty assessment using COSYMA http://cordis.europa.eu/fp5-euratom/src/lib_docs.htm • RFF workshop expert judgment http://www.rff.org/rff/Events/Expert-Judgment-Workshop.cfm • TU Delft Website http://dutiosc.twi.tudelft.nl/~risk/ UNCERTAINTY 1 AMBIGUITY INDECISION

History Structured Expert Judgment in Risk Analysis • WASH 1400 (Rasmussen Report, 1975) Very Different Guidelines: • IEEE Std 500 (1977) • Canvey Island (1978) The story you hear today is NOT the only story • NUREG 1150 (1989) • T-book (Swedish Reliability Data Base 1994) • USNRC-EU (1995-1997) • Guidance on Uncertainty and Use of Experts. NUREG/CR-6372, 1997 • Procedures Guide for Structured Expert Judgment, EUR 18820EN, 2000 UNCERTAINTY • Morgan, et al “Best Practice Approaches for 2 AMBIGUITY Characterizing, Communicating, and Incorporating INDECISION Scientific Uncertainty in Climate Decision Making 2009

Overview • Foundations 101 • Rational Consensus / Classical Model • DATA / Validation • Take Home NOT • Stakeholder preference (values) • Dependence • Fitting models to EJ (probabilistic inversion) UNCERTAINTY 1 AMBIGUITY INDECISION

UNCERTAINTY How harmful is 100Gy gamma radiation In 1 hr? AMBIGUITY AMBIGUITY Is John (1.87m) tall? Is John (1.87m) tall? INDECISION Evacuate?

UNCERTAINTY What Is? AMBIGUITY What means? INDECISION What’s best?

UNCERTAINTY Experts’ job AMBIGUITY Analysts’ job INDECISION Problem owners’ job

Christine Todd Whitman Administrator EPA, 2001-2003 “A big part of my frustration was that scientists would give me a range. And I would ask, „Please just tell me at which point you are safe, and we can do that.‟ But they would give a range, say, from 5 to 25 parts per billion” Christine Todd Whitman, quoted in Environmental Science & Technology YOU are paid to decide Online, April 20, 2005 under uncertainty

Operational Definitions • The philosophy of science: semantic analysis: Mach, Hertz, Einstein, Bohr • A Modern rendering: IF BOB says “The Loch Ness monster exists with degree of possibility 0.0731” to which sentences in the natural language not containing “degree of possibility” is BOB committed? UNCERTAINTY 1 AMBIGUITY INDECISION

Operational definition: Subjective probability Consider two events: F: France wins next World Cup Soccer tournament US: USA wins next World Cup Soccer tournament. Two lottery tickets: L(F): worth $10,000 if F, worth $1000 otherwise L(US): worth $10,000 if US, worth $1000 otherwise. John may choose ONE . John's degree belief (F) John’s degree belief (US) is operationalized as UNCERTAINTY John chooses L(F) in the above choice situation 1 AMBIGUITY INDECISION

Fundamental Theorem of Decision Theory UNCERTAINTY AMBIGUITY If, eg : INDECISION B: Belgium wins next World Cup Soccer tournament. L(F) > L(US); L(US) > L(B); L(F) > L(B) ?? L(F or B) > L(US or B) ?? L(F) > L(US) (plus some technical axioms) Then There is a UNIQUE probability P which represents degree of belief: DegBel(F) > DegBel(US) P(F) > P(US) AND a Utility function, unique op to 0 and 1, that represents values: Exp’d Utility (L(F)) > Exp’d Utility (L(US)) L(F) > L(US) PROOF (4 hrs) EJCoursenotes-Theory-Rational-Decision.doc

RATIONAL CONSENSUS

Goals of an EJ study • Census • Political consensus • Rational consensus EJCoursenotes_review-EJ-literature.doc UNCERTAINTY AMBIGUITY INDECISION

EJ for RATIONAL CONSENSUS : RESS-TUDdatabase.pdf Parties pre-commit to a method which satisfies necessary conditions for scientific method: Traceability/accountability Neutrality (don’t encourage untruthfulness) Fairness (ab initio, all experts equal) Empirical control (performance meas’t) Withdrawal post hoc incurs burden of proof. Goal: comply with principals and combine experts‟ judgments to get a Good Probability Assessor “Classical Model for EJ” UNCERTAINTY AMBIGUITY INDECISION

CLASSICAL MODEL What is a GOOD subjective probability assessor? • Calibration, statistical likelihood – Are the expert‟s probability statements statistically accurate? P-value of statistical test • Informativeness – Probability mass concentrated in a small region, relative to background measure • Nominal values near truth UNCERTAINTY • ? AMBIGUITY INDECISION

Performance based score (weight): Calibration information cutoff Requires that experts assess uncertainty for variables for which we (will) know the true values: Calibration / performance / seed variables any expert, or combination of experts (Decision Maker, dm), can be regarded as a statistical hypothesis UNCERTAINTY AMBIGUITY INDECISION

Performance score is a strictly proper scoring rule Expert maximizes long run expected score by, and only by, stating percentiles which (s)he believes EJCoursenotes-ScoringRules.doc (4 hrs) Uncertainty Ambiguity Indecision

Combining Experts Uncertainty Ambiguity Equal weight decision maker Indecision • Easy • Sometimes OK • Sometimes NOT Performance Based Combinations • Cut-off chosen to optimize DM performance, linear pool of weighted experts

Does weighting Matter? Uncertainty Ambiguity Indecision

Harvard-Kuwait SEJ Health Effects of Oil Fires: UN Claims commission All cause mortality, percent increase per 1 μ g/m 3 increase in PM 2.5 (RESS-PM25.pdf) Amer Cancer Soc. Six Cities Study Harvard Harvard (reanal.) (reanal.) Kuwait, Kuwait, Equal weights Performance (US) weights (US) Median/best 0.7 1.4 0.9657 0.6046 estimate Ratio 95%/5% 2.5 4.8 257 63 UNCERTAINTY 2 AMBIGUITY INDECISION

68% of 84 NIS established since 1959 associated with transoceanic shipping (Ricciardi 2006) 22

Robert Wood Johnson Foundation

Campylobacter: Chicken Processing Model Transport from skin Environment Chicken aextA N ext N env a int w int Feces C int b c env c a (1-a ) w int int

Campylobacter Infection

Expert 10 Expert 7 • Expert no. : 7 Expert name: GM • Items • 1(L) [-----*-----] • Real ::::::::::::::::::::::::::::::::::::::::::::::#::::::::::::::::::::::::: • Expert no. : 10 Expert name: PE • Items • 2(L) [-----*-----] • 1(L) [-*---] • Real :::::::::::::::::::::::::::::::::::::::#:::::::::::::::::::::::::::::::: • Real ::::::::::::::::::::::::::::::::::::::::::::::#::::::::::::::::::::::::: • 3(L) [-----*-----] • 2(L) [-----*-----] • Real :::::::::::::::::::::::::::::::::::::::::::::::#:::::::::::::::::::::::: • Real :::::::::::::::::::::::::::::::::::::::#:::::::::::::::::::::::::::::::: • 4(L) [-----*-----] • 3(L) [-*-] • Real ::::::::::::::::::::::::::::::::::::#::::::::::::::::::::::::::::::::::: • Real :::::::::::::::::::::::::::::::::::::::::::::::#:::::::::::::::::::::::: • 5(L) [-----*-----] • 4(L) [---*-] • Real :::::::::::::::::::::::::::::::::::::#:::::::::::::::::::::::::::::::::: • Real ::::::::::::::::::::::::::::::::::::#::::::::::::::::::::::::::::::::::: • 6(U) [----------*--] • 5(L) [] • Real :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::#:::::::: • Real :::::::::::::::::::::::::::::::::::::#:::::::::::::::::::::::::::::::::: • 7(U) [*] • 6(U) [--*--] • Real :::::::::::::::::::::::::::::#:::::::::::::::::::::::::::::::::::::::::: • Real :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::#:::::::: • 8(U) [-------*--------------] • 7(U) [----*-----] • Real ::::::::::::::::::::::::::::::::::::::::::::::::::::::::#::::::::::::::: • Real :::::::::::::::::::::::::::::#:::::::::::::::::::::::::::::::::::::::::: • 9(L) [-----------*-----------] • 8(U) [-------*-------] • Real ::::::::::::::::#::::::::::::::::::::::::::::::::::::::::::::::::::::::: • Real ::::::::::::::::::::::::::::::::::::::::::::::::::::::::#::::::::::::::: • 10(L) [-----*----------] • 9(L) [---*-] • Real ::::::::::::::::::#::::::::::::::::::::::::::::::::::::::::::::::::::::: • Real ::::::::::::::::#::::::::::::::::::::::::::::::::::::::::::::::::::::::: • 10(L) [-*-] • Real ::::::::::::::::::#:::::::::::::::::::::::::::::::::::::::::::::::::::::