A consistent approach to inconsistencies Fabian Khlinger (Kavli - - PowerPoint PPT Presentation

▶
a consistent approach to inconsistencies
SMART_READER_LITE
LIVE PREVIEW

A consistent approach to inconsistencies Fabian Khlinger (Kavli - - PowerPoint PPT Presentation

Mar 30, 2023 479 likes •740 views

A consistent approach to inconsistencies Fabian Khlinger (Kavli IPMU) in collaboration with Benjamin Joachimi (UCL) SCLSS workshop Oxford, 19 th April 2018 I. Motivation Typical questions arising in a (LSS) data analysis: 1. Is model 0

slide-1
SLIDE 1

A consistent approach to inconsistencies

Fabian Köhlinger (Kavli IPMU) in collaboration with Benjamin Joachimi (UCL) SCLSS workshop Oxford, 19th April 2018

slide-2
SLIDE 2
  • I. Motivation
slide-3
SLIDE 3

Typical questions arising in a (LSS) data analysis:

  • 1. Is model 0 (e.g. wCDM) more likely than my fiducial model 1 (e.g.

𝚳CDM)?

  • 2. Is data set 1 (e.g. Planck) consistent with data set 0 (e.g. cosmic

shear)?

  • 3. Is split 1 of my data set (e.g. z-bin X) consistent with another split 0
  • f the same data set (e.g. all other z-bins)?
slide-4
SLIDE 4

Typical questions arising in a (LSS) data analysis:

  • 1. Is model 0 (e.g. wCDM) more likely than my fiducial model 1 (e.g.

𝚳CDM)?

  • 2. Is data set 1 (e.g. Planck) consistent with data set 0 (e.g. cosmic

shear)?

  • 3. Is split 1 of my data set (e.g. z-bin X) consistent with another split 0
  • f the same data set (e.g. all other z-bins)?
slide-5
SLIDE 5
  • II. Bayesian approach to

(in)consistency

slide-6
SLIDE 6

6

  • 1. Bayesian evidence:

evidence likelihood prior

The evidence is the average of the likelihood over the prior, so it automatically implements Occam’s razor.

data hypothesis, model parameters

slide-7
SLIDE 7

7

Calculate the ratio of probabilities that each model is correct (given the data):

  • 2. Bayes factor:

Bayes factor

typically set to 1 a priori

H0: ‘hypothesis for model 1’ H1: ‘hypothesis for model 0’ d: data

Bayes’ theorem

slide-8
SLIDE 8

8

  • 2. Bayes factor:

> 1

H0 is more likely to be true than H1

(Nested) model comparison: H0: ‘wCDM' vs. H1: ‘𝚳CDM' Data set comparison: H0: ‘there is one common set of parameters describing e.g. Planck and cosmic shear’ vs. H1: ‘each data set requires its own set of parameters’

e.g. Marshall, Rajguru & Slosar (2006)

slide-9
SLIDE 9

9

  • 2. Bayes factor:

> 1

H0 is more likely to be true than H1

Data set comparison: H0: ‘one common set of parameters is sufficient for describing the fiducial (= split 1 + … + split N) data set’ vs. H1: ‘each split i of the data set requires its own set of parameters’

This does NOT hold for correlated data sets!

slide-10
SLIDE 10

10

  • 2. Bayes factor:

> 1

H0 is more likely to be true than H1

Data set comparison: H0: ‘one common set of parameters is sufficient for describing the fiducial (= split 1 + … + split N) data set’ vs. H1: ‘each split i of the data set requires its own set of parameters’

This does NOT hold for correlated data sets!

slide-11
SLIDE 11

11

  • 3. Posterior predictive distribution (PPD):

posterior sample PPD likelihood of new data

: original data : PPD split samples
 : PPD joint sample Can the model(s) describe the data? Are ‘split’ models consistent? quantify this by:

  • comparing the difference between joint and split PPDs to zero
  • comparing the (Gaussian) data distribution to the corresponding PPDs

ˆ ds

ˆ dj

d

The PPD is the average of the likelihood of the new data over the posterior of the parameters of a given model.

slide-12
SLIDE 12

12

Quantify tension between Gaussian data distribution and PPDs by calculating overlap with m𝜏-region.

  • 3. Posterior predictive distribution (PPD):

FK+ in prep.

slide-13
SLIDE 13
  • III. Test case:

cosmic shear correlation functions from KiDS-450

slide-14
SLIDE 14

a) Systematics in z-bin 3?

slide-15
SLIDE 15

15

  • 1. Data and PPDs:

z-bin 3 (incl. cross-correlations) vs. all other correlations

+

black: data from KiDS-450 (Hildebrandt+ 2017) red: mode of joint PPD blue: modes of split PPDs

FK+ in prep.

slide-16
SLIDE 16

16

  • 2. Comparison of key parameters:

amplitude of intrinsic alignment model

z-bin 3 (incl. cross-correlations) vs. all other correlations

FK+ in prep.

slide-17
SLIDE 17

17

  • 3. Comparison in data space:

z-bin 3 (incl. cross-correlations) vs. all other correlations

+

FK+ in prep.

red: mode of joint PPD blue: modes of split PPDs

slide-18
SLIDE 18

18

  • 4. Comparing difference of PPDs:

z-bin 3 (incl. cross-correlations) vs. all other correlations

+

red: mode of joint PPD blue: modes of split PPDs

FK+ in prep. FK+ in prep.

slide-19
SLIDE 19

b) Scale-dependent systematics?

slide-20
SLIDE 20

20

  • 1. Data and PPDs:

Large scales vs. small scales

+

black: data from KiDS-450 (Hildebrandt+ 2017) red: mode of joint PPD blue: modes of split PPDs

FK+ in prep.

slide-21
SLIDE 21

21

  • 2. Comparison of key parameters:

Large scales vs. small scales

amplitude of intrinsic alignment model

FK+ in prep.

slide-22
SLIDE 22

22

  • 3. Comparison in data space:

+

FK+ in prep.

red: mode of joint PPD blue: modes of split PPDs

Large scales vs. small scales

slide-23
SLIDE 23

23

  • 4. Comparing difference of PPDs:

Large scales vs. small scales

+

red: mode of joint PPD blue: modes of split PPDs

FK+ in prep. FK+ in prep.

slide-24
SLIDE 24
  • IV. Summary
slide-25
SLIDE 25

25

  • 1. Bayesian evidence and the Bayes factor are powerful

concepts for model comparison

  • can be expanded to consistency checks of

(correlated) datasets

  • 2. Quantification of consistency with Bayes factor is not
  • ptimal:
  • all information compressed into one number
  • no hints to from where systematics arise
  • mind the priors…
  • 3. Complementary tool: PPDs
  • systematics apparent in data space
  • can be compressed into various numbers (𝛕-levels)

Summary: