New Zealand as a Social Laboratory [A James Cook Fellowship - - PowerPoint PPT Presentation

New Zealand as a “Social Laboratory”

[A James Cook Fellowship Proposal]

COMPASS Seminar Series Monday, 3 August 2015, Fale Pacifika

Professor Peter Davis Department of Sociology, COMPASS Research Centre

New Zealand as a “Social Laboratory”

[A James Cook Fellowship Proposal] Preamble - Making Knowledge Claims

• The Year of Evaluation – RCTs
• Impact of societal inequality
• Improving inference with better design
• The simulation approach

The University of Auckland New Zealand

Pickett and Wilkinson, Soc Sci Med 2015

5

The University of Auckland New Zealand

Pickett and Wilkinson, Soc Sci Med 2015

6

The University of Auckland New Zealand

Avendano article, Soc Sci Med 2012

7

The University of Auckland New Zealand

Pickett and Wilkinson, Soc Sci Med 2015

8

Avendano 2012

9

New Zealand

Correlation = -0.92 Source: Avendano data

France

Source: Avendano data Correlation = +0.96

11

New Zealand as a “Social Laboratory”

[A James Cook Fellowship Proposal] Preamble - Making Knowledge Claims

• The Year of Evaluation – RCTs
• Impact of societal inequality
• Improving inference with better design
• The simulation approach

The University of Auckland New Zealand

Causal Inference in Observational Settings

7th Wellington Colloquium Statistics NZ 30 August 2013 Professor Peter Davis University of Auckland, New Zealand and COMPASS Research Centre www.compass.auckland.ac.nz

New Zealand as a “Social Laboratory”

[A James Cook Fellowship Proposal]

The University of Auckland New Zealand

Assessing policy counterfactuals with a simulation-based inquiry system.

Peter Davis and Colleagues COMPASS Research Centre University of Auckland New Zealand www.compass.auckland.ac.nz

DISCLAIMER: Access to the data used in this study was provided by Statistics New Zealand under conditions designed to give effect to the security and confidentiality provisions of the Statistics Act 1975. The results presented in this study are the work of the author, not Statistics New Zealand.

17

The University of Auckland New Zealand

Assessing counterfactuals

Counterfactual paradigm of causal reasoning

If the putative causal factor had not been present, we would not have observed the recorded outcome.

• Randomised Controlled Trials (RCTs)
• Experimental and quasi-experimental methods
• Observational designs and statistical analysis

Simulation techniques

18

The University of Auckland New Zealand

Simulation techniques

Simulation – in our case, social simulation

Use of computer models (computational techniques) to “mimic” social phenomena (e.g. social processes).

• Understand phenomena better in constructing the models
• Once understood and validated, one can alter features
• Particularly useful for sub-groups and future projections
• Ability to combine different data sources in a single model
• Overcome privacy and confidentiality issues

19

The University of Auckland New Zealand

Model Year Locality Type Life stage Domain Software Data Funder

Collaborators

End-users

MOSC 2005-8 NZ ABM/MSM Adults Marriage market, residential segregation NetLogo Repast Java Census Marsden UOA PCASO 2005-8 NZ Static discrete- time MSM Older people Health care SAS NATMEDCA NZHS ANHS HRC UOA NatSem BCASO 2009-12 NZ Dynamic discrete- time MSM Older people Health & social care R NZHS NZDS Census HRC UOA NatSem MEL-C 2009-13 NZ Children Health, education, conduct Java R CHDS DMDHS PIFS THNR Census2006 MBIE UOA NatSem StatCan MOE MOH MOJ MSD Te Puni Kokiri

SUPERU

Children’s Commissioner KNOW- LAB 2013-16 World Children & young people Health, education, conduct, etc. Published literature MBIE UOA StatCan SOCLAB 2015-17 NZ Whole society Social Modgen/ OpenM++ NZLC Royal Society UOA Open-source modelling community TPM 2015-20 NZ Whole society Social Census TEC UOA MOTU

LEGEND

Model Data Funder MOSC: Modelling social change PCASO: Primary care in an ageing society BCASO: Balance of care in an ageing society MEL-C: Modelling the early life course KNOW-LAB: Knowledge laboratory SOC-LAB: NZ social laboratory TPM CORE: Te Punaha Matatini NATMEDCA: National Primary Medical Care NZHS: NZ Health Survey NZDS: NZ Disability Survey ANHS: Australian National Health Survey CHDS: Christchurch Health & Development Study DMHDS: Dunedin Multidisciplinary Health & Development Study PIFS: Pacific Island Families Study THNR: Te Hoe Nuku Roa NZLC: NZ Longitudinal Census HRC: Health Research Council MBIE: Ministry of Business, Innovation & Employment Royal Society: James Cook Fellowship TEC: Tertiary Education Commission Collaborators UOA: University of Auckland NatSem: National Centre for Social & Economic Modelling, University of Canberra StatCan: Statistics Canada MOTU: Motu Economic and Public Policy Research Type End-users ABM: Agent based model MSM: Micro-simulation model MOE: Ministry of Education, MOH: Ministry of Health MOJ: Ministry of Justice, MSD: Ministry of Social Development

Simulation at COMPASS

20

The University of Auckland New Zealand

Micro-simulation approach.

We start with a sample of individuals

Real (studies) / synthetic (derived from Census)

We derive statistical rules to create a “virtual cohort” that mimics the “real” one

Derive rules best able to reproduce real data Apply these rules to the base file to create a synthetic sample of typical biographies through life course

We then simulate what might happen if policy were to change, by altering parameters

Using software application to test counterfactuals

21

Virtual versus real cohort: family doctor visits, reading ability, and conduct problems, by year of age

22

Year Real cohort (CHDS) n=1017 Virtual cohort (simulated) n=1017 Absolute error Absolute error / CHDS mean Family doctor visits (mean (95% CI)) 1 5.82 5.82

• 2

5.34 5.28 0.06

• 3

3.31 3.18 0.13

• 4

3.13 3.15 0.02

• 5

3.22 3.12 0.10

• 6

3.35 3.32 0.03

• 7

2.43 2.41 0.02

• 8

2.14 2.15 0.01

• 9

1.96 1.90 0.06

• 10

1.65 1.68 0.03

• All years

3.24 3.20 (3.15-3.25) 0.04 1.2% Reading ability: BURT score (mean (95% CI)) 8 45.3 45.3

• 9

54.4 54.7 0.3

• 10

64.1 63.7 0.4

• 11

72.8 71.9 0.9

• 12

79.5 78.9 0.6

• 13

85.2 84.6 0.6

• All years

66.9 66.5 (65.7-67.4) 0.4 0.6% Conduct problems (mean (95% CI)) 6 10.6 10.6

• 7

24.6 24.8 0.2

• 8

24.4 25.0 0.6

• 9

24.7 25.3 0.6

• 10

24.9 25.6 0.7

• All years

21.8 22.3 (22.1-22.4) 0.5 2.3%

The University of Auckland New Zealand

Inquiry Tool

23

New Zealand as a “Social Laboratory”

[A James Cook Fellowship Proposal] ANY QUESTIONS AT THIS POINT!?

New Zealand as a “Social Laboratory”

[A James Cook Fellowship Proposal] Assessing Counterfactuals about Society

• Background and concept
• Central ingredients of project
• COMPASS team contribution
• Building blocks
• Book proposal

Outline

• Background
• Central ingredients of the James Cook
• COMPASS team contribution
• Building blocks

– New Zealand Longitudinal Census (1981-2013) – Synthetic base file – Estimating equations – Open-source micro-simulation platform

• Book proposal

Background – assessing counterfactuals

• New Zealand, an early policy pioneer

– 1890-1920 seen by observers as “social laboratory” – Social policies tried out by reforming governments – A “natural experiment” in a new, fluid society

• How to draw credible inferences about policy?

– RCTs, experimental and quasi-experimental designs – Non-experimental work (e.g. case studies) – Virtual “experiments”, using simulation techniques – Any precedents? Think of climate change scenarios

Background – assessing counterfactuals

• New Zealand, an early policy pioneer

– 1890-1920 seen by observers as “social laboratory” – Social policies tried out by reforming governments – A “natural experiment” in a new, fluid society

• How to draw credible inferences about policy?

– RCTs, experimental and quasi-experimental designs – Non-experimental work (e.g. case studies) – Virtual “experiments”, using simulation techniques – Any precedents? Think of climate change scenarios

Central Ingredients of James Cook

• Three aims

– Create model of NZ pop via synthetic cohort – Statistical model from NZLC to generate cohorts – Conduct experiments, “virtual counterfactuals”

• 1. Constructing smaller, synthetic cohorts

Need synthetic starting file for each cohort, 1981

• 2. Estimating statistical model driving cohorts

Method for reproducing biographical trajectories

• 3. Testing “virtual counterfactuals”

Particular interest in impact of social assets

Three “Virtual Counterfactuals”

• Health impact of 1980s/1990s restructuring

– Blakely et al. use a cross-comparative counterfactual (Norway) – We can try returning key exogenous parameters to the long- term pre-disruption trend line – Assess impact on health inequalities

• Long-term impact of “Working For Families”

– What would have happened had “normal” settings applied? – Did the in-work tax credit work against the poor?

• Impact of social assets on valued goals and outcomes

– Determine relationship of assets (e.g. social and cultural capital) to achievement of valued goals and outcomes – Alter distribution of these non-monetary assets to assess impact

The University of Auckland New Zealand

Senior RF - Barry Milne

Research-Policy Collaboration – Published 2014

34

The University of Auckland New Zealand

Senior RF – Roy Lay-Yee

Determinants and Disparities – Published 2015

35

The University of Auckland New Zealand

Statistician – Jessica McLay

Regression Estimation for Dynamic Microsimulation (McLay et al.)

• ACCEPTED WITH REVISIONS (International Journal of

Microsimulation)

Abstract: Microsimulation models seek to represent real-world processes and can generate extensive amounts of synthetic data. Most often, the parameters that drive the data generation process are estimated by statistical modelling techniques, such as regression. But which techniques are best suited to this purpose? We assess the performance of five regression-style estimation techniques: ordinary least squares regression with a lagged dependent variable, random effects with and without an autoregressive order 1within-unit error structure, a hybrid model combining features from both econometric fixed effects and random effects models, and a dynamic panel model estimated with system generalised method of moments. The criterion for good performance was the proximity of fit of simulated data to empirical data on various characteristics. It was found that ordinary least squares regression with a lagged dependent variable out-performed the other techniques. Random effects with autoregressive errors of the first order was the next best, followed by standard random effects. The dynamic panel model came fourth followed by the hybrid model. This empirical assessment provides practical guidance to those contemplating dynamic microsimulation and other applications using regression-style techniques of synthetic data generation.

36

Outline

• Background
• Central ingredients of the James Cook
• COMPASS team contribution
• Building blocks

– New Zealand Longitudinal Census (1981-2013) – Synthetic base file – Estimating equations – Open-source micro-simulation platform

• Book proposal

Cohort Estimating Equations

Open-Source Simulation Software

Book Proposal

Central Ingredients of James Cook

• Three aims

– Create model of NZ pop via synthetic cohorts – Statistical model from NZLC to generate cohorts – Conduct experiments, “virtual counterfactuals”

• 1. Constructing smaller, synthetic cohorts

Need synthetic starting file for each cohort, 1981

• 2. Estimating statistical model driving cohorts

Method for reproducing biographical trajectories

• 3. Testing “virtual counterfactuals”

Particular interest in impact of social assets

New Zealand as a “Social Laboratory”

[A James Cook Fellowship Proposal] QUESTIONS, COMMENTS!