Confidence in Belief, Weight of Evidence and Uncertainty Reporting - - PowerPoint PPT Presentation

▶
confidence in belief weight of evidence and uncertainty
SMART_READER_LITE
LIVE PREVIEW

Confidence in Belief, Weight of Evidence and Uncertainty Reporting - - PowerPoint PPT Presentation

May 05, 2023 167 likes •439 views

Confidence in Belief, Weight of Evidence and Uncertainty Reporting Brian Hill hill@hec.fr www.hec.fr/hill GREGHEC, CNRS & HEC Paris July 3, 2019 1 / 9 Weight of Evidence & Confidence in beliefs Unknown urn: 100

slide-1
SLIDE 1

Confidence in Belief, Weight of Evidence and Uncertainty Reporting

Brian Hill

hill@hec.fr www.hec.fr/hill GREGHEC, CNRS & HEC Paris

July 3, 2019

1 / 9

slide-2
SLIDE 2

Weight of Evidence & Confidence in beliefs

➓ Unknown urn: 100 balls, each red or black. ➓ Known urn: 100 balls, 50 red, 50 black. ➓ ➓ ➓

2 / 9

slide-3
SLIDE 3

Weight of Evidence & Confidence in beliefs

➓ Unknown urn: 100 balls, each red or black. ➓ Known urn: 100 balls, 50 red, 50 black. Keynes Your beliefs about the colour of the next ball drawn? ➓ Balance of evidence: same ➓ Weight of evidence: different Bayesian belief: same (1

2).

➓

2 / 9

slide-4
SLIDE 4

Weight of Evidence & Confidence in beliefs

➓ Unknown urn: 100 balls, each red or black. ➓ Known urn: 100 balls, 50 red, 50 black. Keynes Your beliefs about the colour of the next ball drawn? ➓ Balance of evidence: same ➓ Weight of evidence: different Bayesian belief: same (1

2).

Ellsberg Which urn would you rather bet on? ➓ Known urn Bayesian decision: indifferent.

2 / 9

slide-5
SLIDE 5

Weight of Evidence & Confidence in beliefs

Ellsberg preferences justified by: ➓ higher weight of evidence for known urn ➓ more confidence in probability 1

2 judgement for that urn

Moral

Bayesianism denies any role for confidence in beliefs or weight

  • f evidence in choice

3 / 9

slide-6
SLIDE 6

Weight of Evidence & Confidence in beliefs

Ellsberg preferences justified by: ➓ higher weight of evidence for known urn ➓ more confidence in probability 1

2 judgement for that urn

Moral

Bayesianism denies any role for confidence in beliefs or weight

  • f evidence in choice

However confidence in probability judgements reported by the IPCC, US DIA etc.

3 / 9

slide-7
SLIDE 7

Confidence in Beliefs

Belief state: ➓ Beliefs or Credal judgements

➓ probability judgements reflecting direction evidence is pointing

➓ Confidence in beliefs

➓ subjective appraisal of the support for them

4 / 9

slide-8
SLIDE 8

Confidence in Beliefs

Belief state: ➓ Beliefs or Credal judgements

➓ probability judgements reflecting direction evidence is pointing ú balance

➓ Confidence in beliefs

➓ subjective appraisal of the support for them ú weight

4 / 9

slide-9
SLIDE 9

Confidence in Beliefs

Belief state: ➓ Beliefs or Credal judgements

➓ probability judgements reflecting direction evidence is pointing ú balance

➓ Confidence in beliefs

➓ subjective appraisal of the support for them ú weight

This paper: ➓ Formal model of weight of evidence (via confidence) ➓ Support effective uncertainty reporting

4 / 9

slide-10
SLIDE 10

Confidence in beliefs / Weight of Evidence

Model

➓ A nested family of sets of probability measures ♣ q ✏ ♣ q ✏

♣ q ✏ ♣ q ✏ ♣ q P r s

Confidence Level: Low High

∆♣Sq

5 / 9

slide-11
SLIDE 11

Confidence in beliefs / Weight of Evidence

Model

➓ A nested family of sets of probability measures

➓ generalisation of credal sets

♣ q ✏ ♣ q ✏

♣ q ✏ ♣ q ✏ ♣ q P r s

Confidence Level: Low High

∆♣Sq

5 / 9

slide-12
SLIDE 12

Confidence in beliefs / Weight of Evidence

Model

➓ A nested family of sets of probability measures

➓ portrays precision / weight trade-off ➓ without requiring the agent to settle on a single set.

♣ q ✏ ♣ q ✏

probability measures with p♣RKnownq ✏ 1

2

probability measures with p♣RUnknownq ✏ 1

2

♣ q P r s

Confidence Level: Low High

∆♣Sq

5 / 9

slide-13
SLIDE 13

Confidence in beliefs / Weight of Evidence

Model

➓ A nested family of sets of probability measures

➓ portrays precision / weight trade-off ➓ without requiring the agent to settle on a single set.

♣ q ✏ ♣ q ✏

probability measures with p♣RKnownq ✏ 1

2

♣ q ✏ p♣RUnknownq P r0.3, 0.7s

Confidence Level: Low High

∆♣Sq

5 / 9

slide-14
SLIDE 14

Confidence in beliefs / Weight of Evidence

Model

➓ A nested family of sets of probability measures

➓ has solid connections to decision, which carry over to weight of evidence

♣ q ✏ ♣ q ✏

♣ q ✏ ♣ q ✏ ♣ q P r s

Confidence Level: Low High

∆♣Sq

5 / 9

slide-15
SLIDE 15

Uncertainty Reporting

Desiderata

  • 1. Clean belief / value separation
  • 2. Unambiguous uncertainty language

Gilboa, Marinacci, “Ambiguity and the Bayesian Paradigm”, 2013; “Confidence in Beliefs and Rational Decision Making” Economics & Philosophy, 2019 6 / 9

slide-16
SLIDE 16

Uncertainty Reporting

Desiderata

  • 1. Clean belief / value separation
  • 2. Unambiguous uncertainty language

Bayesian Clean Separation: ➓ probability (beliefs) vs. utility (desires / values)

Gilboa, Marinacci, “Ambiguity and the Bayesian Paradigm”, 2013; “Confidence in Beliefs and Rational Decision Making” Economics & Philosophy, 2019 6 / 9

slide-17
SLIDE 17

Uncertainty Reporting

Desiderata

  • 1. Clean belief / value separation
  • 2. Unambiguous uncertainty language

Bayesian Clean Separation: ➓ probability (beliefs) vs. utility (desires / values) Credal sets / multiple priors No Clean Separation: ➓ Set of priors can reflect both beliefs and attitudes to / taste for uncertainty

Gilboa, Marinacci, “Ambiguity and the Bayesian Paradigm”, 2013; “Confidence in Beliefs and Rational Decision Making” Economics & Philosophy, 2019 6 / 9

slide-18
SLIDE 18

Uncertainty Reporting

Desiderata

  • 1. Clean belief / value separation
  • 2. Unambiguous uncertainty language

Bayesian Clean Separation: ➓ probability (beliefs) vs. utility (desires / values) Credal sets / multiple priors No Clean Separation: ➓ Set of priors can reflect both beliefs and attitudes to / taste for uncertainty Confidence approach Clean Separation: ➓ Nested family: beliefs & confidence in beliefs ➓ Uncertainty attitudes: another parameter

Gilboa, Marinacci, “Ambiguity and the Bayesian Paradigm”, 2013; “Confidence in Beliefs and Rational Decision Making” Economics & Philosophy, 2019 6 / 9

slide-19
SLIDE 19

Uncertainty Reporting

Desiderata

  • 1. Clean belief / value separation
  • 2. Unambiguous uncertainty language

Challenge: calibrate confidence levels across agents.

7 / 9

slide-20
SLIDE 20

Uncertainty Reporting

Desiderata

  • 1. Clean belief / value separation
  • 2. Unambiguous uncertainty language

Challenge: calibrate confidence levels across agents. How are probabilities calibrated? ➓ on “objectively uncertain / chance” events.

7 / 9

slide-21
SLIDE 21

Uncertainty Reporting

Desiderata

  • 1. Clean belief / value separation
  • 2. Unambiguous uncertainty language

Challenge: calibrate confidence levels across agents. How are probabilities calibrated? ➓ on “objectively uncertain / chance” events. In fact: Principal Principle (ordinal version) ù ñ “Objective uncertainty” set of events calibrate probability levels across (rational) agents.

7 / 9

slide-22
SLIDE 22

Uncertainty Reporting

Desiderata

  • 1. Clean belief / value separation
  • 2. Unambiguous uncertainty language

Challenge: calibrate confidence levels across agents. Idea: use “objective” comparisons of weight of evidence.

7 / 9

slide-23
SLIDE 23

Uncertainty Reporting

Desiderata

  • 1. Clean belief / value separation
  • 2. Unambiguous uncertainty language

Challenge: calibrate confidence levels across agents. Idea: use “objective” comparisons of weight of evidence. In fact: Weight-of-Evidence Principal Principle ù ñ “Objective weight-of-evidence” set of probability judgements calibrate confidence levels across (rational) agents.

7 / 9

slide-24
SLIDE 24

Uncertainty Reporting

Desiderata

  • 1. Clean belief / value separation
  • 2. Unambiguous uncertainty language

Challenge: calibrate confidence levels across agents. Idea: use “objective” comparisons of weight of evidence. In fact: Weight-of-Evidence Principal Principle ù ñ “Objective weight-of-evidence” set of probability judgements calibrate confidence levels across (rational) agents. Confidence Elicitation Web Tool http://confidence.hec.fr/app/

7 / 9

slide-25
SLIDE 25

Confidence in Beliefs

This paper: ➓ Use to model weight of evidence ➓ Support effective uncertainty reporting General Project ➓ Model of confidence in beliefs ➓ Role in decision making ➓ Solid normative credentials ➓ Application to IPCC uncertainty language ➓ Belief updating ➓ Elicitation . . .

8 / 9

slide-26
SLIDE 26

Thank you.

hill@hec.fr www.hec.fr/hill

Further details:

➓ Confidence and Decision, Games and Economic Behavior, 82: 675–692, 2013. ➓ Incomplete Preferences and Confidence, Journal of Mathematical Economics, 65: 83-103, 2016. ➓ Confidence in Beliefs and Rational Decision Making, Economics and Philosophy, 32: 223-258, 2019. ➓ Climate Change Assessments: Confidence, Probability and Decision, Philosophy of Science, 84: 500-522, 2017 (with R. Bradley, C. Helgeson). ➓ Combining probability with qualitative degree-of-certainty metrics in assessment, Climatic Change 149: 517-525, 2018 (with R. Bradley, C. Helgeson). Web tool: ➓ http://confidence.hec.fr/app/.

9 / 9