Statistical significance in CP violation Mattias Blennow emb@kth.se - - PowerPoint PPT Presentation

Statistical significance in CP violation

Mattias Blennow emb@kth.se

KTH Theoretical Physics

June 22, 2015, Invisibles 15, Madrid, Spain

Mattias Blennow KTH Theoretical Physics Statistical significance in CP violation

Parameter estimation sensitivity

Define the test statistic “∆χ2” ∆χ2(θ) = −2 log

• L(θ|d)

supθ′ L(θ′|d)

• Assume it is χ2 distributed with n degrees of freedom

Use the data set without statistical fluctuations (Asimov data) Quote result

Mattias Blennow KTH Theoretical Physics Statistical significance in CP violation

The interpretation

∆χ2 is asymptotically χ2 (Wilks’ theorem) The Asimov data (expected data without fluctuations) is representative Several requirements, not always fulfilled

3 .10

• 3

10

• 2

7 .10

• 2

true sin

22θ13

π/2 π 3π/2 2π true δCP Probability to observe a non-zero θ13 at 99.73% CL in T2K < 20% 20%-50% 50%-90% 90%-99% > 99%

Schwetz, Phys.Lett. B648 (2007) 54 Mattias Blennow KTH Theoretical Physics Statistical significance in CP violation

For the mass ordering

Mass ordering is not nested Wilks’ theorem not applicable Test statistic T = χ2

IO − χ2 NO

2

χ ∆

• 40
• 20

20 40 )

2

χ ∆ PDF( 0.02 0.04 0.06 0.08 0.1

NH MC IH MC NH Norm. Approx. IH Norm. Approx.

Qian, et al., Phys.Rev. D86 (2012) 113011

T is approximately Gaussian for many situations T ≃ N(T0, 2

• T0)

T0 = value for Asimov data

Mattias Blennow KTH Theoretical Physics Statistical significance in CP violation

What is sensitivity?

Sensitivity (median) What is the expected rejection

• f a false ordering?

(Given a parameter set) Interpretation: It is representative for how well the experiment will do 50 % probability of not reaching it 50 % probability of doing better Not 50 % probability of “being wrong” Not the only relevant quantity, distribution matters (do Brazilian bands!)

Mattias Blennow KTH Theoretical Physics Statistical significance in CP violation

Mass ordering results

10 20 30 T0 1 2 3 4 5 6 sensitivity (σ) 10 20 30 1 2 3 4 5 6 median (2 sided) median (1 sided) crossing (2 sided) crossing (1 sided) MB, Coloma, Huber, Schwetz, JHEP 03(2014)028

MC Β0.5 Gaussian 1sided Gaussian 2sided ΑΒ

NOvA

150 100 50 50 100 150 2 4 6 ∆° Sensitivity Σ MC Β0.5 Gaussian 1sided Gaussian 2sided ΑΒ

LBNE10kt

150 100 50 50 100 150 2 4 6 ∆° Sensitivity Σ

Mattias Blennow KTH Theoretical Physics Statistical significance in CP violation

Other measurements

CP violation Nested hypothesis Does not mean that Wilk’s theorem holds, cyclic parameters Rest of this talk θ23 octant Degeneracies closer Wilk’s theorem still violated A priori, a dedicated study is needed

Mattias Blennow KTH Theoretical Physics Statistical significance in CP violation

Setup for CP violation

0.02 0.04 0.06 0.08 0.1 0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.08 0.09 0.1

Peµ Peµ

Blennow, Smirnov, Adv.High Energy Phys. 2013 (2013) 972485

295 km, 0.65 GeV

Test statistic S = min

δ=0,π χ2 − min global χ2

Why not necessarily gaussian?

Cyclic parameter Several points in null hypothesis (δ = 0, π) Degeneracies

Distribution should always be checked or argued for

Mattias Blennow KTH Theoretical Physics Statistical significance in CP violation

Critical values

Expectation from null hypothesis Red line shows the χ2 distribution NOvA to lower cutoff values More sensitive experiments to higher cutoff values

1 Σ 2 Σ 3 Σ Χ2 ESS LBNE NOΝA T2HK T2HK20 2 4 6 8 10 0.001 0.01 0.1 1 S 1CDF

MB, Coloma, Fernandez-Martinez, JHEP 1503 (2015) 005 Mattias Blennow KTH Theoretical Physics Statistical significance in CP violation

Median deviations

Need to consider the expected outcome

Median Asimov ESS LBNE NOΝA T2HK 45 90 135 180 225 270 315 360 10 20 30 40 50 60 70 ∆ S

MB, Coloma, Fernandez-Martinez, JHEP 1503 (2015) 005

Agrees quite well for most experiments Lower than Asimov data for NOvA Depending on δ for

• ther experiments

Mattias Blennow KTH Theoretical Physics Statistical significance in CP violation

Sensitivity results

Combining the distributions with the cutoffs

NOΝA Median Asimov 45 90 135 180 225 270 315 360 0.0 0.5 1.0 1.5 2.0 2.5 3.0 ∆ Σ T2HK Median Asimov 45 90 135 180 225 270 315 360 0.0 0.5 1.0 1.5 2.0 2.5 3.0 ∆ Σ LBNE Median Asimov 45 90 135 180 225 270 315 360 0.0 0.5 1.0 1.5 2.0 2.5 3.0 ∆ Σ ESS Median Asimov 45 90 135 180 225 270 315 360 0.0 0.5 1.0 1.5 2.0 2.5 3.0 ∆ Σ

MB, Coloma, Fernandez-Martinez, JHEP 1503 (2015) 005 Mattias Blennow KTH Theoretical Physics Statistical significance in CP violation

Handling of nuisance parameters

Systematics and previous measurements: Addition to the χ2 function χ2(θ, ξ) = χ2

0(θ, ξ) + (ξ − ¯

ξ)2 σ2

ξ

ξ is the fit value of the nuisance parameter ¯ ξ is the experimental measurement or theoretical prediction A priori: Should calibrate χ2 for all true values of ξtrue In reality: Little dependence on the true value, calibrate for ξtrue = ¯ ξ for existing experiments

Mattias Blennow KTH Theoretical Physics Statistical significance in CP violation

Current hints

90 180 270 δCP 1 2 3 4 5 6 7 8 ∆χ

2

sin

2θ23 = 0.4

68% 90% 95% 99%

90 180 270 δCP sin

2θ23 = 0.5

68% 90% 95% 99%

90 180 270 360 δCP sin

2θ23 = 0.6

68% 90% 95% 99%

• bserved data

∆χ

2 levels

NuFit 2.0 (2014)

Gonzalez-Garcia, Maltoni, Schwetz, JHEP 1411 (2014) 052 www.nu-fit.org, 2014 Mattias Blennow KTH Theoretical Physics Statistical significance in CP violation

Heuristic interpretation

NΝ NΝ

Large errors on rates: Difference between best fit and null hypothesis small Medium errors: Curvature plays a role Small errors: Possible outcomes essentially linearly related to δ

Parameter space is curved Do not expect χ2 Can we understand the deviations?

1 Σ 2 Σ 3 Σ s0 s1 s0.3 2 4 6 8 10 0.001 0.01 0.1 1 S 1CDF

Mattias Blennow KTH Theoretical Physics Statistical significance in CP violation

Summary and conclusions

Wilks’ theorem is not a priori applicable to the neutrino CP violation, the test statistic is not χ2 distributed More precise experiments → χ2 Critical values will depend on the experiments Generally: Lower critical values for low precision experiments Also expect lower χ2 than Asimov for those The usual Asimov + χ2 approximation is a relatively good estimator

Mattias Blennow KTH Theoretical Physics Statistical significance in CP violation