Statistical modeling of summary values leads to accurate Approximate - PowerPoint PPT Presentation

Centre for Outbreak Analysis and M odelling Statistical modeling of summary values leads to accurate Approximate Bayesian Computations Oliver Ratmann (Imperial College London, UK) Anton Camacho (London School of Hygiene & Tropical Medicine, UK) Adam Meijer (National Institute of the Environment & Public Health, NL) Gé Donker (Netherlands Institute for Health Services Research, NL) Tuesday, 7 January 14

Standard ABC (Beaumont 2002) summary stat tolerance ABC approximation to likelihood is exact if 1) summary statistics are sufficient 2) upper and lower tolerances coincide Tuesday, 7 January 14

Standard ABC (Beaumont 2002) summary stat tolerance ABC approximation to likelihood in practice not feasible, is exact if ‘asymptotic’ argument 1) summary statistics are sufficient 2) upper and lower tolerances coincide Tuesday, 7 January 14

Standard ABC is noisy even with sufficient summary statistics (Fernhead & Prangle 2012) 3.0 n=60 2.5 naive tolerances τ - =0.35 τ + =1.65 2.0 n-ABC estimate of π τ ( σ 2 | x ) π ( σ 2 | x ) a r g m a x σ 2 1.5 π ( σ 2 | x ) 1.0 0.5 0.0 0.0 0.5 1.0 1.5 2.0 2.5 σ 2 Tuesday, 7 January 14

ABC* (Ratmann, Camacho, Meijer, Donker; arXiv 2013) Can we construct ABC such that 3.0 n=60 inference is accurate calibrated • wrt point estimate, eg MAP 2.5 tolerances • wrt overall similarity in distribution, eg KL τ − =0.572 n − ABC estimate of π τ ( σ 2 |x) τ + =1.808 divergence 2.0 m=97 • and maintain computational feasibility 1.5 If yes, under which conditions? 1.0 How general are these? π ( σ 2 |x) 0.5 argmax σ 2 π ( σ 2 |x) 0.0 0.0 0.5 1.0 1.5 2.0 2.5 3.0 σ 2 Tuesday, 7 January 14

ABC* step 1 To avoid asymptotics, c − ≤ T interpret ABC accept/reject step � � s 1: n ( x ) , s 1: m ( y ) � ≤ c + R = as the outcome of a decision test T-test ABC objective: declare , unequal objective: declare , equal µ ( θ ) µ x µ ( θ ) µ x H 0 : , equal H 0 : , unequal µ ( θ ) µ x µ ( θ ) µ x H 1 : , unequal H 1 : , equal µ ( θ ) µ x µ ( θ ) µ x rejection region: ( −∞ , c − ] ∪ [ c + , ∞ ) rejection region: [ c − , c + ] Tuesday, 7 January 14

ABC* step 1 To avoid asymptotics, c − ≤ T interpret ABC accept/reject step � � s 1: n ( x ) , s 1: m ( y ) � ≤ c + R = as the outcome of a decision test T-test ABC objective: declare , unequal objective: declare , equal µ ( θ ) µ x µ ( θ ) µ x H 0 : , equal H 0 : , unequal µ ( θ ) µ x µ ( θ ) µ x H 1 : , unequal H 1 : , equal µ ( θ ) µ x µ ( θ ) µ x rejection region: ( −∞ , c − ] ∪ [ c + , ∞ ) rejection region: [ c − , c + ] c − c + , are fully determined sth P ( R | H 0 ) ≤ α Tuesday, 7 January 14

ABC* step 1 To avoid asymptotics, c − ≤ T interpret ABC accept/reject step � � s 1: n ( x ) , s 1: m ( y ) � ≤ c + R = as the outcome of a decision test T-test ABC objective: declare , unequal objective: declare , equal µ ( θ ) µ x µ ( θ ) µ x H 0 : , equal H 0 : , unequal µ ( θ ) µ x µ ( θ ) µ x H 1 : , unequal H 1 : , equal µ ( θ ) µ x µ ( θ ) µ x rejection region: ( −∞ , c − ] ∪ [ c + , ∞ ) rejection region: [ c − , c + ] c − c + , are fully determined sth P ( R | H 0 ) ≤ α Let ρ = µ − µ x then ABC approximation to likelihood is the power function of the test ρ → P ( R | ρ ) Tuesday, 7 January 14

ABC* step 1 To avoid asymptotics, c − ≤ T interpret ABC accept/reject step � � s 1: n ( x ) , s 1: m ( y ) � ≤ c + R = as the outcome of a decision test T-test ABC objective: declare , unequal objective: declare , equal µ ( θ ) µ x µ ( θ ) µ x H 0 : , equal H 0 : , unequal µ ( θ ) µ x µ ( θ ) µ x H 1 : , unequal H 1 : , equal µ ( θ ) µ x µ ( θ ) µ x rejection region: ( −∞ , c − ] ∪ [ c + , ∞ ) rejection region: [ c − , c + ] c − c + , are fully determined sth P ( R | H 0 ) ≤ α holds for specific test: Let ρ = µ − µ x then ABC approximation to two sided, one sample likelihood is the power function equivalence hypothesis test of the test ρ → P ( R | ρ ) Tuesday, 7 January 14

Example: test variance suppose x 1: n ∼ N (0 , σ 2 y 1: m ∼ N (0 , σ 2 ) x ) then Tuesday, 7 January 14

Example: test variance for simplicity, suppose summary values equal data x 1: n ∼ N (0 , σ 2 y 1: m ∼ N (0 , σ 2 ) x ) then ρ = σ 2 / ˆ σ 2 x ρ ? = 1 ∈ [ τ − , τ + ] H 0 : ρ / H 1 : ρ ∈ [ τ − , τ + ] m y ) 2 1 ( y i − ¯ X T = S 2 ( y 1: m ) /S 2 ( x 1: n ) = ρ n − 1 σ 2 i =1 ρ n − 1 χ 2 ∼ m − 1 Tuesday, 7 January 14

Example: test variance for simplicity, suppose summary values equal data x 1: n ∼ N (0 , σ 2 y 1: m ∼ N (0 , σ 2 ) x ) point of equality then ρ = σ 2 / ˆ σ 2 x ρ ? = 1 ∈ [ τ − , τ + ] H 0 : ρ / H 1 : ρ ∈ [ τ − , τ + ] m y ) 2 1 ( y i − ¯ X T = S 2 ( y 1: m ) /S 2 ( x 1: n ) = ρ n − 1 σ 2 i =1 ρ n − 1 χ 2 ∼ m − 1 Tuesday, 7 January 14

Example: test variance for simplicity, suppose summary values equal data x 1: n ∼ N (0 , σ 2 y 1: m ∼ N (0 , σ 2 ) x ) point of equality then tolerances on population level ρ = σ 2 / ˆ σ 2 x ρ ? = 1 ∈ [ τ − , τ + ] H 0 : ρ / H 1 : ρ ∈ [ τ − , τ + ] m y ) 2 1 ( y i − ¯ X T = S 2 ( y 1: m ) /S 2 ( x 1: n ) = ρ n − 1 σ 2 i =1 ρ n − 1 χ 2 ∼ m − 1 Tuesday, 7 January 14

Example: test variance suppose know distribution of T, c − c + can work out , x 1: n ∼ N (0 , σ 2 y 1: m ∼ N (0 , σ 2 ) x ) then ρ = σ 2 / ˆ σ 2 x ρ ? = 1 ∈ [ τ − , τ + ] H 0 : ρ / H 1 : ρ ∈ [ τ − , τ + ] m y ) 2 1 ( y i − ¯ X T = S 2 ( y 1: m ) /S 2 ( x 1: n ) = ρ n − 1 σ 2 i =1 ρ n − 1 χ 2 ∼ m − 1 Tuesday, 7 January 14

Example: test variance suppose know distribution of T, c − c + can work out , x 1: n ∼ N (0 , σ 2 y 1: m ∼ N (0 , σ 2 ) x ) then ρ = σ 2 / ˆ σ 2 x ρ ? = 1 ∈ [ τ − , τ + ] H 0 : ρ / H 1 : ρ ∈ [ τ − , τ + ] m y ) 2 1 ( y i − ¯ X T = S 2 ( y 1: m ) /S 2 ( x 1: n ) = ρ n − 1 σ 2 i =1 ρ n − 1 χ 2 ∼ m − 1 Tuesday, 7 January 14

Example: test variance suppose know distribution of T, c − c + can work out , x 1: n ∼ N (0 , σ 2 y 1: m ∼ N (0 , σ 2 ) x ) and power function then 0.8 ρ = σ 2 / ˆ σ 2 x ρ ? = 1 0.6 ∈ [ τ − , τ + ] H 0 : ρ / power H 1 : ρ ∈ [ τ − , τ + ] 0.4 m y ) 2 1 ( y i − ¯ X T = S 2 ( y 1: m ) /S 2 ( x 1: n ) = ρ 0.2 n − 1 σ 2 i =1 ρ n − 1 χ 2 ∼ m − 1 0.0 0.5 1.0 1.5 2.0 ρ Tuesday, 7 January 14

Example: test variance suppose know distribution of T, c − c + can work out , x 1: n ∼ N (0 , σ 2 y 1: m ∼ N (0 , σ 2 ) x ) and power function and calibrate then move mode ρ = σ 2 / ˆ σ 2 0.8 x ρ ? = 1 ∈ [ τ − , τ + ] H 0 : ρ / 0.6 power H 1 : ρ ∈ [ τ − , τ + ] 0.4 m y ) 2 1 ( y i − ¯ X T = S 2 ( y 1: m ) /S 2 ( x 1: n ) = ρ n − 1 σ 2 0.2 i =1 ρ n − 1 χ 2 ∼ m − 1 0.0 increase 0.5 1.0 1.5 2.0 ρ Tuesday, 7 January 14

Example: test variance increase suppose know distribution of T, c − c + can work out , x 1: n ∼ N (0 , σ 2 y 1: m ∼ N (0 , σ 2 ) x ) and power function and calibrate then move mode ρ = σ 2 / ˆ σ 2 0.8 x ρ ? = 1 ∈ [ τ − , τ + ] H 0 : ρ / 0.6 power H 1 : ρ ∈ [ τ − , τ + ] 0.4 m y ) 2 1 ( y i − ¯ X T = S 2 ( y 1: m ) /S 2 ( x 1: n ) = ρ n − 1 σ 2 0.2 i =1 ρ n − 1 χ 2 tighten ∼ m − 1 0.0 increase 0.5 1.0 1.5 2.0 ρ Tuesday, 7 January 14

Example: test variance suppose x 1: n ∼ N (0 , σ 2 y 1: m ∼ N (0 , σ 2 ) x ) exact then 2.0 posterior n=60 ρ = σ 2 / ˆ σ 2 calibrated x ρ ? = 1 tolerances τ − =0.477 1.5 n − ABC estimate of π τ ( σ 2 |x) τ + =2.2 ∈ [ τ − , τ + ] H 0 : ρ / naive H 1 : ρ ∈ [ τ − , τ + ] tolerances τ − =0.35 1.0 τ + =1.65 m y ) 2 1 ( y i − ¯ X T = S 2 ( y 1: m ) /S 2 ( x 1: n ) = ρ π ( σ 2 |x) n − 1 σ 2 i =1 argmax σ 2 0.5 ρ n − 1 χ 2 ∼ π ( σ 2 |x) m − 1 calibrated tolerances 0.0 0.0 0.5 1.0 1.5 2.0 2.5 3.0 Tuesday, 7 January 14

Example: test variance suppose 3.0 x 1: n ∼ N (0 , σ 2 y 1: m ∼ N (0 , σ 2 ) x ) n=60 exact calibrated 2.5 posterior then tolerances τ − =0.572 n − ABC estimate of π τ ( σ 2 |x) τ + =1.808 ρ = σ 2 / ˆ σ 2 2.0 x m=97 ρ ? = 1 calibrated ∈ [ τ − , τ + ] tolerances H 0 : ρ / 1.5 τ − =0.726 H 1 : ρ ∈ [ τ − , τ + ] τ + =1.392 m=300 1.0 m y ) 2 1 ( y i − ¯ π ( σ 2 |x) X T = S 2 ( y 1: m ) /S 2 ( x 1: n ) = ρ n − 1 σ 2 i =1 0.5 argmax σ 2 ρ n − 1 χ 2 π ( σ 2 |x) ∼ m − 1 tighten 0.0 calibrated tolerances calibrated m 0.0 0.5 1.0 1.5 2.0 2.5 3.0 σ 2 Tuesday, 7 January 14

Example: test variance suppose x 1: n ∼ N (0 , σ 2 y 1: m ∼ N (0 , σ 2 ) x ) then ρ = σ 2 / ˆ σ 2 x ρ ? = 1 ∈ [ τ − , τ + ] H 0 : ρ / H 1 : ρ ∈ [ τ − , τ + ] m y ) 2 1 ( y i − ¯ X T = S 2 ( y 1: m ) /S 2 ( x 1: n ) = ρ n − 1 σ 2 i =1 ρ n − 1 χ 2 ∼ m − 1 calibrated tolerances calibrated m Tuesday, 7 January 14