Learning more with every year : School year productivity and - - PowerPoint PPT Presentation

Learning more with every year: School year productivity and international learning divergence

Abhijeet Singh

University of Oxford Presentation at CSAE Conference 2015 March 2015

Introduction

What we know about learning outcomes in developing countries

◮ Learning outcomes very poor in many developing countries,

especially in South Asia and SSA

◮ Less known in comparative settings (under-represented in

PISA, TIMSS)

◮ But available results suggest differences in performance within

developing countries is big

◮ Vietnam-Peru gap in math is 1.4 SD in PISA 2012 ◮ US-Finland gap - 0.38 SD

Introduction

What we know about learning outcomes in developing countries

◮ Learning outcomes very poor in many developing countries,

especially in South Asia and SSA

◮ Less known in comparative settings (under-represented in

PISA, TIMSS)

◮ But available results suggest differences in performance within

developing countries is big

◮ Vietnam-Peru gap in math is 1.4 SD in PISA 2012 ◮ US-Finland gap - 0.38 SD

Introduction

What we know about learning outcomes in developing countries

◮ Learning outcomes very poor in many developing countries,

especially in South Asia and SSA

◮ Less known in comparative settings (under-represented in

PISA, TIMSS)

◮ But available results suggest differences in performance within

developing countries is big

◮ Vietnam-Peru gap in math is 1.4 SD in PISA 2012 ◮ US-Finland gap - 0.38 SD

Introduction

What we know about learning outcomes in developing countries

◮ Learning outcomes very poor in many developing countries,

especially in South Asia and SSA

◮ Less known in comparative settings (under-represented in

PISA, TIMSS)

◮ But available results suggest differences in performance within

developing countries is big

◮ Vietnam-Peru gap in math is 1.4 SD in PISA 2012 ◮ US-Finland gap - 0.38 SD

Introduction

What we know about learning outcomes in developing countries

◮ Learning outcomes very poor in many developing countries,

especially in South Asia and SSA

◮ Less known in comparative settings (under-represented in

PISA, TIMSS)

◮ But available results suggest differences in performance within

developing countries is big

◮ Vietnam-Peru gap in math is 1.4 SD in PISA 2012 ◮ US-Finland gap - 0.38 SD

What is this paper about?

Three main questions:

◮ When do learning gaps emerge? ◮ How do they evolve over the educational/age trajectory of

children?

◮ What are the causes of divergence?

◮ can we say anything about the relative effectiveness of

schooling systems?

◮ PISA at 15 years pretty uninformative about these questions.

What is this paper about?

Three main questions:

◮ When do learning gaps emerge? ◮ How do they evolve over the educational/age trajectory of

children?

◮ What are the causes of divergence?

◮ can we say anything about the relative effectiveness of

schooling systems?

◮ PISA at 15 years pretty uninformative about these questions.

What is this paper about?

Three main questions:

◮ When do learning gaps emerge? ◮ How do they evolve over the educational/age trajectory of

children?

◮ What are the causes of divergence?

◮ can we say anything about the relative effectiveness of

schooling systems?

◮ PISA at 15 years pretty uninformative about these questions.

What is this paper about?

Three main questions:

◮ When do learning gaps emerge? ◮ How do they evolve over the educational/age trajectory of

children?

◮ What are the causes of divergence?

◮ can we say anything about the relative effectiveness of

schooling systems?

◮ PISA at 15 years pretty uninformative about these questions.

What is this paper about?

Three main questions:

◮ When do learning gaps emerge? ◮ How do they evolve over the educational/age trajectory of

children?

◮ What are the causes of divergence?

◮ can we say anything about the relative effectiveness of

schooling systems?

◮ PISA at 15 years pretty uninformative about these questions.

Why this matters

◮ Knowing when and how learning gaps evolve is informative for

understanding when policy interventions might work:

◮ Effectiveness of interventions varies importantly across the age

• f children

◮ Understanding sources of divergence useful for identifying

domains in which intervention necessary

◮ we don’t just want a league table.

◮ Important differences between educational systems may have

important information for policy

◮ But most economics of education in developing countries is

focused on specific interventions within a given institutional setting

◮ no work on ‘business-as-usual’ productivity of time spent in

school

Why this matters

◮ Knowing when and how learning gaps evolve is informative for

understanding when policy interventions might work:

◮ Effectiveness of interventions varies importantly across the age

• f children

◮ Understanding sources of divergence useful for identifying

domains in which intervention necessary

◮ we don’t just want a league table.

◮ Important differences between educational systems may have

important information for policy

◮ But most economics of education in developing countries is

focused on specific interventions within a given institutional setting

◮ no work on ‘business-as-usual’ productivity of time spent in

school

Why this matters

◮ Knowing when and how learning gaps evolve is informative for

understanding when policy interventions might work:

◮ Effectiveness of interventions varies importantly across the age

• f children

◮ Understanding sources of divergence useful for identifying

domains in which intervention necessary

◮ we don’t just want a league table.

◮ Important differences between educational systems may have

important information for policy

◮ But most economics of education in developing countries is

focused on specific interventions within a given institutional setting

◮ no work on ‘business-as-usual’ productivity of time spent in

school

What I do

Use child level panel data for Ethiopia, India, Peru and Vietnam to:

◮ Compare distributions of achievement of children at 5 and 8

years across four Young Lives countries

◮ Examine how the gap evolves over the age group of the

children

◮ Is there growth between 5-8 years? ◮ Do rankings change across ages?

◮ Estimate value-added (VA) models examining sources of the

gap

◮ Causally identify differential productivity of schooling with VA

and IV estimates using enrollment discontinuities

What I do

Use child level panel data for Ethiopia, India, Peru and Vietnam to:

◮ Compare distributions of achievement of children at 5 and 8

years across four Young Lives countries

◮ Examine how the gap evolves over the age group of the

children

◮ Is there growth between 5-8 years? ◮ Do rankings change across ages?

◮ Estimate value-added (VA) models examining sources of the

gap

◮ Causally identify differential productivity of schooling with VA

and IV estimates using enrollment discontinuities

What I do

Use child level panel data for Ethiopia, India, Peru and Vietnam to:

◮ Compare distributions of achievement of children at 5 and 8

years across four Young Lives countries

◮ Examine how the gap evolves over the age group of the

children

◮ Is there growth between 5-8 years? ◮ Do rankings change across ages?

◮ Estimate value-added (VA) models examining sources of the

gap

◮ Causally identify differential productivity of schooling with VA

and IV estimates using enrollment discontinuities

What I do

Use child level panel data for Ethiopia, India, Peru and Vietnam to:

◮ Compare distributions of achievement of children at 5 and 8

years across four Young Lives countries

◮ Examine how the gap evolves over the age group of the

children

◮ Is there growth between 5-8 years? ◮ Do rankings change across ages?

◮ Estimate value-added (VA) models examining sources of the

gap

◮ Causally identify differential productivity of schooling with VA

and IV estimates using enrollment discontinuities

What I do

Use child level panel data for Ethiopia, India, Peru and Vietnam to:

◮ Compare distributions of achievement of children at 5 and 8

years across four Young Lives countries

◮ Examine how the gap evolves over the age group of the

children

◮ Is there growth between 5-8 years? ◮ Do rankings change across ages?

◮ Estimate value-added (VA) models examining sources of the

gap

◮ Causally identify differential productivity of schooling with VA

and IV estimates using enrollment discontinuities

Contribution

◮ First analysis of the emergence and evolution of gaps in

cognitive achievement across countries, using internationally comparable child-level panel data, at a critical age for skill formation

◮ similar work on racial gaps in US, socio-economic gaps in the

UK etc. but nothing across countries

◮ no studies of comparable age range in developing countries

◮ Causal identification of learning-gains-per-year in different

countries using micro panel and RD-based identification

Contribution

◮ First analysis of the emergence and evolution of gaps in

cognitive achievement across countries, using internationally comparable child-level panel data, at a critical age for skill formation

◮ similar work on racial gaps in US, socio-economic gaps in the

UK etc. but nothing across countries

◮ no studies of comparable age range in developing countries

◮ Causal identification of learning-gains-per-year in different

countries using micro panel and RD-based identification

Contribution

◮ First analysis of the emergence and evolution of gaps in

cognitive achievement across countries, using internationally comparable child-level panel data, at a critical age for skill formation

◮ similar work on racial gaps in US, socio-economic gaps in the

UK etc. but nothing across countries

◮ no studies of comparable age range in developing countries

◮ Causal identification of learning-gains-per-year in different

countries using micro panel and RD-based identification

Data

Young Lives survey structure

Round 1 Round 2 Round 3

5 10 15 Age in years

Oct 2002 Dec 2006 Nov 2009

Time Younger cohort Older cohort

Graph shows median age of children and time of interview across countries

By age of children

Timing of survey rounds

Data

Young Lives survey test data

◮ Use data from the 2006/7 and 2009 rounds on quantitative

proficiency

◮ Cognitive Development Assessment Quant. sub-scale for 5

year old sample

◮ Mathematics tests for 8 year old children

◮ Identical tests administered across all four countries in each

round

◮ can be linked within round across the four countries using Item

Response Theory

Data

Young Lives survey test data

◮ Use data from the 2006/7 and 2009 rounds on quantitative

proficiency

◮ Cognitive Development Assessment Quant. sub-scale for 5

year old sample

◮ Mathematics tests for 8 year old children

◮ Identical tests administered across all four countries in each

round

◮ can be linked within round across the four countries using Item

Response Theory

Data

Table : Descriptives on age and school progression

Cohort Variable Statistics Ethiopia India Peru Vietnam Older cohort Age of entry Mean 7.19 5.04 5.88 6.07 SD 1.52 0.71 0.57 0.48 YC 2006 (5-years) Enrolment Mean 0.04 0.45 0.01 0.01 YC 2009 (8-years) Enrolment Mean 0.77 0.99 0.98 0.98 OC 2006 (12-years) Enrolment Mean 0.95 0.89 0.99 0.97 OC 2009 (15-years) Enrolment Mean 0.89 0.77 0.92 0.77 YC 2009 (8-years) Grade Mean 0.64 1.63 1.31 1.71 SD 0.77 1 0.58 0.57 OC 2006 (12-years) Grade Mean 3.17 5.61 4.91 5.57 SD 1.68 1.25 1.11 0.94 OC 2009 (15-years) Grade Mean 5.55 8.15 7.72 8.29 SD 2.05 1.73 1.31 1.25 Grade refers to highest grade completed

Learning differences at 5 and 8

Table : Linked test scores at 5,and 8 years

Age group Statistics Countries Ethiopia India Peru Vietnam 5 years Mean 454 498.3 520.4 524.7 SD 102.1 94.8 97.6 89.1 N 1846 1904 1893 1935 8 years Mean 419.1 495.9 518.2 563.6 SD 100.7 84.6 68.3 85.3 N 1885 1930 1943 1964

Scores are IRT test scores generated within an age sample,pooling data from all countries, and normalized to have a mean of 500 and an SD of 100 in the pooled sample. Scores are comparable across countries but not across age groups.

Do rankings change across age groups?

.2 .4 .6 .8 1 200 400 600 800 1000

CDA Scores, 2006

5 years

.2 .4 .6 .8 1 200 400 600 800 1000

Math Scores, 2009

8 years Empirical CDFs

Distribution of achievement

Ethiopia India Peru Vietnam

Rankings are unchanged but are the gaps growing?

Between 5 and 8 years of age

Rankings are unchanged but are the gaps growing?

Between 5 and 8 years of age

p10 p90 400 450 500 550 600 Math scores (2009) 300 400 500 600 700 CDA scores (2006) Ethiopia India Peru Vietnam

Rankings are unchanged but are the gaps growing?

Between 12 and 15 years of age

p10 p75 350 400 450 500 550 600 Math scores (2009) 350 400 450 500 550 Math scores (2006) Ethiopia India Peru Vietnam

Where are the gaps coming from?

◮ Knowing differences in levels and trends between countries

informative but not enough.

◮ Even trend differences need not imply differential effectiveness

• f schools across countries

◮ endowments differ - e.g. parental education, home inputs,

nutrition, other environmental differences

◮ but differential effectiveness, and malleable environmental

sources of learning divergence, are where policy might make a difference

Where are the gaps coming from?

◮ Knowing differences in levels and trends between countries

informative but not enough.

◮ Even trend differences need not imply differential effectiveness

• f schools across countries

◮ endowments differ - e.g. parental education, home inputs,

nutrition, other environmental differences

◮ but differential effectiveness, and malleable environmental

sources of learning divergence, are where policy might make a difference

Where are the gaps coming from?

◮ Knowing differences in levels and trends between countries

informative but not enough.

◮ Even trend differences need not imply differential effectiveness

• f schools across countries

◮ endowments differ - e.g. parental education, home inputs,

nutrition, other environmental differences

◮ but differential effectiveness, and malleable environmental

sources of learning divergence, are where policy might make a difference

Where are the gaps coming from?

◮ Knowing differences in levels and trends between countries

informative but not enough.

◮ Even trend differences need not imply differential effectiveness

• f schools across countries

◮ endowments differ - e.g. parental education, home inputs,

nutrition, other environmental differences

◮ but differential effectiveness, and malleable environmental

sources of learning divergence, are where policy might make a difference

Do child-specific endowments explain divergence?

Value-added models with common coefficients: Specifications

Yica = φc (1) +β1.Yic,a−1 (2) +β2.Xi (3) +β3.TUica + ica (4)

◮ Xi (Background) - male, eldest child, wealth index, age,

caregiver’s education, height-for-age in 2009

◮ TUica (time use) - time use on different activities ◮ Yi,2006 (lagged achievement) - 2006 quantitative achievement

measures

Do child-specific endowments explain divergence?

Value-added models with common coefficients: Results

(1) (2) (3) (4) Dep var: Mathematics score (2009) VARIABLES 8-years old Country dummies India 76.3*** 64.5*** 61.6*** 16.3*** (3.01) (2.92) (2.97) (3.58) Peru 96.7*** 79.1*** 65.2*** 48.2*** (2.75) (2.71) (2.69) (2.85) Vietnam 146*** 127*** 108*** 92.2*** (3.04) (3.06) (2.97) (3.45) Lagged test scores Y Y Y Background vars (Xic) Y Y Time use (TUic,a) Y

Huber-White standard errors in parentheses *** p<0.01, ** p<0.05, * p<0.1.

Does differential productivity of home inputs explain divergence?

Country-specific production function estimates

◮ Previous specification had a very strong implicit assumption:

the effect of inputs on achievement is the same across countries

◮ So I run the same specifications separately for each country

sample

◮ allows for each input parameter to be different across countries ◮ but makes interpretation difficult since four sets of input

coefficients

◮ Key result: Between 5-8 years, divergence with Vietnam not

explained by levels of inputs

Does differential productivity of home inputs explain divergence?

Country-specific production function estimates

◮ Previous specification had a very strong implicit assumption:

the effect of inputs on achievement is the same across countries

◮ So I run the same specifications separately for each country

sample

◮ allows for each input parameter to be different across countries ◮ but makes interpretation difficult since four sets of input

coefficients

◮ Key result: Between 5-8 years, divergence with Vietnam not

explained by levels of inputs

Does differential productivity of home inputs explain divergence?

Country-specific production function estimates

◮ Previous specification had a very strong implicit assumption:

the effect of inputs on achievement is the same across countries

◮ So I run the same specifications separately for each country

sample

◮ allows for each input parameter to be different across countries ◮ but makes interpretation difficult since four sets of input

coefficients

◮ Key result: Between 5-8 years, divergence with Vietnam not

explained by levels of inputs

Does differential productivity of home inputs explain divergence?

Country-specific production function estimates

◮ Previous specification had a very strong implicit assumption:

the effect of inputs on achievement is the same across countries

◮ So I run the same specifications separately for each country

sample

◮ allows for each input parameter to be different across countries ◮ but makes interpretation difficult since four sets of input

coefficients

◮ Key result: Between 5-8 years, divergence with Vietnam not

explained by levels of inputs

Does differential productivity of home inputs explain divergence?

Country-specific production function estimates

◮ Previous specification had a very strong implicit assumption:

the effect of inputs on achievement is the same across countries

◮ So I run the same specifications separately for each country

sample

◮ allows for each input parameter to be different across countries ◮ but makes interpretation difficult since four sets of input

coefficients

◮ Key result: Between 5-8 years, divergence with Vietnam not

explained by levels of inputs

Predicted mean scores under counterfactual scenarios

8-year olds

Coefficients (βc) Without time use With time use Ethiopia India Peru Vietnam Ethiopia India Peru Vietnam Ethiopia 420.79 485.28 495.47 523.15 420.75 390.94 486.66 488.38 (9.87) (10.64) (5.49) (13.48) (10.85) (16.72) (9.62) (19.19) Inputs India 450.36 497.32 503.74 539.9 487.38 497.32 516.86 563.24 (Xic; TUica) (11.54) (9.59) (4.97) (11.02) (10.39) (9.87) (7.99) (14.79) Yic,a−1 Peru 470.66 514.64 517.73 559.32 479.48 468.87 517.74 557.66 (11.35) (10.7) (4.65) (10.53) (10.93) (10.96) (5.65) (11.68) Vietnam 478.69 518.05 522.35 567.03 492.1 476.78 520.84 568.22 (11.08) (9.76) (4.51) (9.16) (12.06) (13.14) (7.09) (11.43) Cells contain linear predictions of test scores using combinations of country-specific production function parameters (βc) with country-specific input levels (Xic and TUic). Standard errors of predictions in parentheses.

Estimating the quality of schooling

◮ Specifications above include no schooling measures

◮ But we know exposure of schooling differs, esp. in Ethiopia ◮ Suspect that quality of schooling differs too

◮ What I do: include highest grade completed in the

specifications and re-estimate

◮ Identification reliant on relevant unobserved heterogeneity

being absorbed by controls and lag

◮ Will show RD-type IV estimates

◮ These are the most ‘complete’ VA specifications in the paper

Estimating the quality of schooling

◮ Specifications above include no schooling measures

◮ But we know exposure of schooling differs, esp. in Ethiopia ◮ Suspect that quality of schooling differs too

◮ What I do: include highest grade completed in the

specifications and re-estimate

◮ Identification reliant on relevant unobserved heterogeneity

being absorbed by controls and lag

◮ Will show RD-type IV estimates

◮ These are the most ‘complete’ VA specifications in the paper

Estimating the quality of schooling

◮ Specifications above include no schooling measures

◮ But we know exposure of schooling differs, esp. in Ethiopia ◮ Suspect that quality of schooling differs too

◮ What I do: include highest grade completed in the

specifications and re-estimate

◮ Identification reliant on relevant unobserved heterogeneity

being absorbed by controls and lag

◮ Will show RD-type IV estimates

◮ These are the most ‘complete’ VA specifications in the paper

VAMs with grade effectiveness

8-year olds

(1) (2) (3) (4) (5) (6) (7) (8) VARIABLES Dep var: Mathematics score (2009) Without time use With time use Ethiopia India Peru Vietnam Ethiopia India Peru Vietnam Highest grade completed 40.9*** 27.4*** 33.6*** 60.9*** 28.4*** 25.4*** 32.6*** 55.2*** (4.67) (2.03) (3.60) (14.6) (4.48) (1.62) (3.55) (10.9) Male 3.26 12.7*** 8.73*** 1.65 4.44 11.6*** 8.92*** 1.62 (5.61) (3.05) (2.22) (2.39) (4.82) (3.13) (2.47) (2.66) Caregiver’s education level 3.76*** 2.40*** 2.23*** 3.16*** 2.74*** 1.86*** 2.10*** 2.18*** (0.66) (0.70) (0.49) (0.80) (0.52) (0.49) (0.48) (0.72) Age in months 1.26** 0.51

• 0.067

0.18 1.30** 0.60 0.0079 0.69 (0.53) (0.45) (0.30) (1.10) (0.56) (0.41) (0.30) (0.87) Height-for-age (2009) 9.31*** 5.38** 5.22** 7.14*** 5.30** 4.79** 4.82** 4.81*** (2.64) (2.21) (1.92) (1.78) (2.33) (1.85) (1.73) (1.56) Wealth index (2006) 151*** 53.6** 17.6* 78.3*** 105*** 31.0* 18.1* 59.0*** (25.9) (23.8) (8.80) (20.9) (18.8) (17.8) (8.91) (19.0) Lagged CDA scores (2006) 0.067*** 0.13*** 0.100*** 0.065* 0.045* 0.12*** 0.100*** 0.049 (0.023) (0.027) (0.021) (0.032) (0.022) (0.027) (0.020) (0.030) Constant 196*** 306*** 401*** 354*** 129* 97.6* 313*** 333*** (49.2) (45.5) (29.5) (74.1) (72.0) (53.8) (38.8) (65.5) Observations 1,835 1,892 1,888 1,907 1,834 1,892 1,881 1,858 R-squared 0.340 0.276 0.343 0.437 0.410 0.365 0.370 0.458 Robust standard errors in parentheses. Standard errors are clustered at site level. *** p<0.01, ** p<0.05, * p<0.1

Can you trust VA estimates

Comparing with IV results

◮ What if you don’t believe that grades completed are

conditionally exogenous?

◮ Identification reliant on relevant unobserved heterogeneity

being absorbed by controls and lag

◮ Way out: try looking for an IV which affects the highest grade

completed at a particular age

◮ but does not directly determine learning, conditional on

controls

◮ Solution: Plausibly exogenous variation coming from

enrolment thresholds

◮ Creates discontinuity in the number of grades completed at

particular calendar months

◮ Conditional on age and previous learning, should be excludable

Can you trust VA estimates

Comparing with IV results

◮ What if you don’t believe that grades completed are

conditionally exogenous?

◮ Identification reliant on relevant unobserved heterogeneity

being absorbed by controls and lag

◮ Way out: try looking for an IV which affects the highest grade

completed at a particular age

◮ but does not directly determine learning, conditional on

controls

◮ Solution: Plausibly exogenous variation coming from

enrolment thresholds

◮ Creates discontinuity in the number of grades completed at

particular calendar months

◮ Conditional on age and previous learning, should be excludable

Can you trust VA estimates

Comparing with IV results

◮ What if you don’t believe that grades completed are

conditionally exogenous?

◮ Identification reliant on relevant unobserved heterogeneity

being absorbed by controls and lag

◮ Way out: try looking for an IV which affects the highest grade

completed at a particular age

◮ but does not directly determine learning, conditional on

controls

◮ Solution: Plausibly exogenous variation coming from

enrolment thresholds

◮ Creates discontinuity in the number of grades completed at

particular calendar months

◮ Conditional on age and previous learning, should be excludable

Can you trust VA estimates

Comparing with IV results

◮ What if you don’t believe that grades completed are

conditionally exogenous?

◮ Identification reliant on relevant unobserved heterogeneity

being absorbed by controls and lag

◮ Way out: try looking for an IV which affects the highest grade

completed at a particular age

◮ but does not directly determine learning, conditional on

controls

◮ Solution: Plausibly exogenous variation coming from

enrolment thresholds

◮ Creates discontinuity in the number of grades completed at

particular calendar months

◮ Conditional on age and previous learning, should be excludable

Enrolment threshold based discontinuities in grade completion

.5 1 1.5 2 .5 1 1.5 2

J a n 1 F e b 1 M a r 1 A p r 1 M a y 1 J u n 1 J u l 1 A u g 1 S e p t 1 O c t 1 N

• v

1 D e c 1 J a n 2 F e b 2 M a r 2 A p r 2 M a y 2 J u n 2 J u l 2 A u g 2 S e p 2 O c t 2 O c t 2 J a n 1 F e b 1 M a r 1 A p r 1 M a y 1 J u n 1 J u l 1 A u g 1 S e p t 1 O c t 1 N

• v

1 D e c 1 J a n 2 F e b 2 M a r 2 A p r 2 M a y 2 J u n 2 J u l 2 A u g 2 S e p 2 O c t 2 O c t 2 J a n 1 F e b 1 M a r 1 A p r 1 M a y 1 J u n 1 J u l 1 A u g 1 S e p t 1 O c t 1 N

• v

1 D e c 1 J a n 2 F e b 2 M a r 2 A p r 2 M a y 2 J u n 2 J u l 2 A u g 2 S e p 2 O c t 2 O c t 2 J a n 1 F e b 1 M a r 1 A p r 1 M a y 1 J u n 1 J u l 1 A u g 1 S e p t 1 O c t 1 N

• v

1 D e c 1 J a n 2 F e b 2 M a r 2 A p r 2 M a y 2 J u n 2 J u l 2 A u g 2 S e p 2 O c t 2 O c t 2

Ethiopia India Peru Vietnam Average grade attained

Graphs by country

By month of birth

Mean grade completed by 2009

IV specifications

First stage: gradesi,2009 = µ + γ1.Thresholdi + γ2.Xi + γ3.sitei + ica (5) Second stage: Yic,a = αc + β1.Yic,a−1 + β2.Xic + β3.gradeica + γ.sitei + ica +β4.TUic,a

◮ Same as old VAM but for inclusion of site fixed effects

◮ OK here because not comparing constant terms

IV specifications

First stage: gradesi,2009 = µ + γ1.Thresholdi + γ2.Xi + γ3.sitei + ica (5) Second stage: Yic,a = αc + β1.Yic,a−1 + β2.Xic + β3.gradeica + γ.sitei + ica +β4.TUic,a

◮ Same as old VAM but for inclusion of site fixed effects

◮ OK here because not comparing constant terms

Discontinuity based results on grade effectiveness

Peru and Vietnam

(1) (2) (3) (4) VARIABLES Dep var: Math scores (2009) Peru Vietnam Highest grade completed 20.1*** 20.9*** 47.3*** 46.3*** (7.61) (7.96) (7.49) (7.16) Male 9.43*** 9.96*** 1.34 1.56 (2.39) (2.63) (2.36) (2.46) Caregiver’s education level 2.31*** 2.14*** 3.05*** 2.41*** (0.40) (0.37) (0.61) (0.55) Age in months 0.94 0.87 0.41 0.64 (0.66) (0.71) (0.57) (0.53) Height-for-age (2009) 6.15*** 5.59*** 6.00*** 4.18*** (2.20) (2.00) (1.96) (1.44) Wealth index (2006) 29.7*** 29.0*** 40.2** 28.6** (7.67) (7.84) (16.2) (13.4) Lagged CDA scores (2006) 0.13*** 0.12*** 0.11*** 0.088*** (0.020) (0.020) (0.031) (0.027) Constant 290*** 227*** 375*** 316*** (58.2) (69.2) (55.5) (60.2) Observations 1,888 1,881 1,907 1,858 R-squared 0.366 0.393 0.481 0.504 Kleibergen-Paap F-statistic 108 110 113 152 Robust standard errors in parentheses. Standard errors are clustered at site level. *** p<0.01, ** p<0.05, * p<0.1 Test scores are IRT scores normalized to have a mean of 500 and SD of 100 in the pooled four-country sample at each age. Estimation includes a vector of site fixed effects and other covariates, coefficients for which are not reported.

Robustness checks

◮ Flexible lags: Possibility (even suggestion) of non-linearity in

the effect of lag on current achievement

◮ estimate everything with third-order polynomial of lag / bins of

achievement

◮ Measurement error in the lag

◮ instrument lag with vocabulary test in the 8-year old cohort ◮ assumes independent measurement error across tests

◮ Overall: Persistence parameter might be off but basic story

stays.

Robustness checks

◮ Flexible lags: Possibility (even suggestion) of non-linearity in

the effect of lag on current achievement

◮ estimate everything with third-order polynomial of lag / bins of

achievement

◮ Measurement error in the lag

◮ instrument lag with vocabulary test in the 8-year old cohort ◮ assumes independent measurement error across tests

◮ Overall: Persistence parameter might be off but basic story

stays.

Robustness checks

◮ Flexible lags: Possibility (even suggestion) of non-linearity in

the effect of lag on current achievement

◮ estimate everything with third-order polynomial of lag / bins of

achievement

◮ Measurement error in the lag

◮ instrument lag with vocabulary test in the 8-year old cohort ◮ assumes independent measurement error across tests

◮ Overall: Persistence parameter might be off but basic story

stays.

Robustness checks

◮ Flexible lags: Possibility (even suggestion) of non-linearity in

the effect of lag on current achievement

◮ estimate everything with third-order polynomial of lag / bins of

achievement

◮ Measurement error in the lag

◮ instrument lag with vocabulary test in the 8-year old cohort ◮ assumes independent measurement error across tests

◮ Overall: Persistence parameter might be off but basic story

stays.

Robustness checks

◮ Flexible lags: Possibility (even suggestion) of non-linearity in

the effect of lag on current achievement

◮ estimate everything with third-order polynomial of lag / bins of

achievement

◮ Measurement error in the lag

◮ instrument lag with vocabulary test in the 8-year old cohort ◮ assumes independent measurement error across tests

◮ Overall: Persistence parameter might be off but basic story

stays.

Robustness checks

◮ Flexible lags: Possibility (even suggestion) of non-linearity in

the effect of lag on current achievement

◮ estimate everything with third-order polynomial of lag / bins of

achievement

◮ Measurement error in the lag

◮ instrument lag with vocabulary test in the 8-year old cohort ◮ assumes independent measurement error across tests

◮ Overall: Persistence parameter might be off but basic story

stays.

Pulling it all together

◮ Levels of learning are low except for Vietnam ◮ Differences start early by 5 and grow further later ◮ Between 5-8, divergence with Vietnam reflects differential

effectiveness of schooling

◮ School productivity differences are huge!

◮ Between 12-15, vars predetermined by 12 (including stock of

learning) more important than any schooling differences

Pulling it all together

◮ Levels of learning are low except for Vietnam ◮ Differences start early by 5 and grow further later ◮ Between 5-8, divergence with Vietnam reflects differential

effectiveness of schooling

◮ School productivity differences are huge!

◮ Between 12-15, vars predetermined by 12 (including stock of

learning) more important than any schooling differences

Pulling it all together

◮ Levels of learning are low except for Vietnam ◮ Differences start early by 5 and grow further later ◮ Between 5-8, divergence with Vietnam reflects differential

effectiveness of schooling

◮ School productivity differences are huge!

◮ Between 12-15, vars predetermined by 12 (including stock of

learning) more important than any schooling differences

Pulling it all together

◮ Levels of learning are low except for Vietnam ◮ Differences start early by 5 and grow further later ◮ Between 5-8, divergence with Vietnam reflects differential

effectiveness of schooling

◮ School productivity differences are huge!

◮ Between 12-15, vars predetermined by 12 (including stock of

learning) more important than any schooling differences

Pulling it all together

◮ Levels of learning are low except for Vietnam ◮ Differences start early by 5 and grow further later ◮ Between 5-8, divergence with Vietnam reflects differential

effectiveness of schooling

◮ School productivity differences are huge!

◮ Between 12-15, vars predetermined by 12 (including stock of

learning) more important than any schooling differences

What these results imply

◮ Early divergence provides suggestive support for preschool

interventions

◮ Evidence (except on nutrition) usually based on OECD or LAC

◮ But major divergence after 5 is due to differences in school

productivity at primary school level

◮ It isn’t all over by 5. School productivity is a variable policy

can affect!

◮ Differences in school productivity across countries raise an

important question:

◮ why is productivity so much higher in some countries? ◮ This is not the focus of most of the work in education in dev

econ (but still important)

What these results imply

◮ Early divergence provides suggestive support for preschool

interventions

◮ Evidence (except on nutrition) usually based on OECD or LAC

◮ But major divergence after 5 is due to differences in school

productivity at primary school level

◮ It isn’t all over by 5. School productivity is a variable policy

can affect!

◮ Differences in school productivity across countries raise an

important question:

◮ why is productivity so much higher in some countries? ◮ This is not the focus of most of the work in education in dev

econ (but still important)

What these results imply

◮ Early divergence provides suggestive support for preschool

interventions

◮ Evidence (except on nutrition) usually based on OECD or LAC

◮ But major divergence after 5 is due to differences in school

productivity at primary school level

◮ It isn’t all over by 5. School productivity is a variable policy

can affect!

◮ Differences in school productivity across countries raise an

important question:

◮ why is productivity so much higher in some countries? ◮ This is not the focus of most of the work in education in dev

econ (but still important)

What these results imply

◮ Early divergence provides suggestive support for preschool

interventions

◮ Evidence (except on nutrition) usually based on OECD or LAC

◮ But major divergence after 5 is due to differences in school

productivity at primary school level

◮ It isn’t all over by 5. School productivity is a variable policy

can affect!

◮ Differences in school productivity across countries raise an

important question:

◮ why is productivity so much higher in some countries? ◮ This is not the focus of most of the work in education in dev

econ (but still important)

Comments/Questions/Feedback

How does the YL sample compare internationally?

Proportion correct on identical link items: 12-y olds compared with TIMSS Grade 4 TIMSS 2003 (G4) Q.1 Q.2 Q.3 Q.4 Q.5 Q.6 Canada - Quebec 0.92 0.93 0.85 0.89 0.69 0.64 England 0.93 0.96 0.88 0.86 0.79 0.82 Hong Kong 0.98 0.98 0.92 0.85 0.95 0.75 Italy 0.92 0.97 0.83 0.85 0.73 0.79 Japan 0.97 0.99 0.93 0.90 0.89 0.90 Singapore 0.97 0.97 0.94 0.90 0.94 0.88 USA 0.93 0.94 0.88 0.89 0.67 0.67 Young Lives Ethiopia 0.61 0.72 0.51 0.50 0.40 0.57 India 0.74 0.82 0.60 0.68 0.39 0.71 Peru 0.70 0.91 0.68 0.77 0.51 0.65 Vietnam 0.84 0.94 0.76 0.69 0.75 0.85

Grade 4 students in TIMSS aged 10 years on average

A lot differs across samples

At 8 years of age

Ethiopia India Peru Vietnam Mean SD N Mean SD N Mean SD N Mean SD N Child and background characteristics (Xic) Male 0.53 0.5 1881 0.53 0.5 1903 0.5 0.5 1892 0.51 0.5 1916 First born 0.23 0.42 1881 0.39 0.49 1903 0.37 0.48 1892 0.46 0.5 1916 Caregiver’s Education 2.95 3.73 1874 3.7 4.44 1900 7.75 4.64 1892 6.88 3.83 1908 Age in months 97.48 4.05 1879 96.03 3.92 1903 95.35 3.63 1890 97.09 3.75 1915 Height-for-age z-score

• 1.21

1.05 1877

• 1.44

1.03 1898

• 1.14

1.03 1890

• 1.07

1.05 1900 Wealth index (2006) 0.28 0.18 1881 0.46 0.2 1902 0.47 0.23 1892 0.51 0.2 1914 Time use (hours spent on a typical day; TUic,a) — Doing domestic tasks 1.66 1.37 1881 0.33 0.58 1903 0.87 0.7 1887 0.54 0.66 1899 — Tasks on family farm/business etc. 1.5 2.22 1880 0.01 0.1 1903 0.25 0.66 1886 0.09 0.48 1897 — Paid work outside household 0.01 0.28 1880 0.01 0.2 1903 0.08 1887 0.07 1897 — At school 4.91 2.54 1881 7.72 0.95 1903 6.02 0.9 1887 5.04 1.31 1898 — Studying outside school time 0.99 0.89 1881 1.86 1.09 1903 1.87 0.83 1886 2.82 1.49 1897 — General leisure etc. 4.44 2.39 1881 4.71 1.54 1903 4.13 1.65 1887 5.55 1.65 1898 — Caring for others 0.83 1.21 1881 0.21 0.5 1903 0.48 0.88 1886 0.24 0.66 1878

A lot differs across samples

At 15 years of age

Ethiopia India Peru Vietnam Mean SD N Mean SD N Mean SD N Mean SD N Child and background characteristics (Xic) Male 0.51 0.5 971 0.49 0.5 976 0.53 0.5 664 0.49 0.5 972 First born 0.2 0.4 971 0.31 0.46 976 0.31 0.46 664 0.37 0.48 972 Caregiver’s Education 2.93 3.49 967 2.86 4.05 976 7.27 4.57 663 6.77 3.85 971 Age in months 180.34 3.58 971 179.76 4.24 975 179.1 4.1 661 181.12 3.83 972 Height-for-age z-score

• 1.37

1.28 968

• 1.64

1 970

• 1.48

0.9 657

• 1.43

0.91 967 Wealth index (2006) 0.3 0.17 971 0.47 0.2 976 0.52 0.23 664 0.52 0.19 970 Time use (hours spent on a typical day; TUic,a) — Doing domestic tasks 2.55 1.65 970 1.45 1.35 975 1.42 1.07 662 1.44 0.96 958 — Tasks on family farm/business etc. 1.34 2.09 970 0.49 1.72 975 0.68 1.49 662 1.05 2.13 958 — Paid work outside household 0.4 1.63 970 1.04 2.77 975 0.41 1.72 662 0.47 2 958 — At school 5.55 2.17 970 6.39 3.59 975 5.91 2.01 662 4.23 2.34 946 — Studying outside school time 1.84 1.23 970 2.01 1.54 975 2.09 1.12 662 3.06 2.13 941 — General leisure etc. 2.98 1.71 970 4.1 2.32 975 3.24 1.48 662 4.97 2.23 955 — Caring for others 0.67 0.93 970 0.28 0.75 975 0.73 1.18 662 0.16 0.64 951

Predicted mean scores under counterfactual scenarios

8-year olds

Coefficients (βc) Without time use With time use Ethiopia India Peru Vietnam Ethiopia India Peru Vietnam Ethiopia 420.79 485.28 495.47 523.15 420.75 390.94 486.66 488.38 (9.87) (10.64) (5.49) (13.48) (10.85) (16.72) (9.62) (19.19) Inputs India 450.36 497.32 503.74 539.9 487.38 497.32 516.86 563.24 (Xic; TUica) (11.54) (9.59) (4.97) (11.02) (10.39) (9.87) (7.99) (14.79) Yic,a−1 Peru 470.66 514.64 517.73 559.32 479.48 468.87 517.74 557.66 (11.35) (10.7) (4.65) (10.53) (10.93) (10.96) (5.65) (11.68) Vietnam 478.69 518.05 522.35 567.03 492.1 476.78 520.84 568.22 (11.08) (9.76) (4.51) (9.16) (12.06) (13.14) (7.09) (11.43) Cells contain linear predictions of test scores using combinations of country-specific production function parameters (βc) with country-specific input levels (Xic and TUic). Standard errors of predictions in parentheses.

Predicted mean scores under counterfactual scenarios

15-year olds

Coefficients (βc) Without time use With time use Ethiopia India Peru Vietnam Ethiopia India Peru Vietnam Ethiopia 443.17 448.98 507.7 502.26 443.15 453.52 512.33 524.55 (10.54) (10.14) (7.12) (9.21) (12.13) (11.38) (8.5) (12.03) Inputs India 495.01 482.61 524.44 529.91 496.15 482.86 531.03 549.28 ( ¯ Xic; TUica;) 13.12) (9.84) (6.54) (8.67) (14.96) (10.38) (9.05) (12.55) Yic,a−1 Peru 493.1 493.53 529.74 546.1 483.25 481.28 529.74 557.88 (12.65) (9.86) (6.04) 8.74) (14.14) (10.68) (7.25) (11.58) Vietnam 525.34 515.18 542.1 557.05 521.01 504.53 535.76 558.18 (12.65) (10.25) (6.56) 8.54) (14.1) (11.36) (9.14) (10.56) Cells contain linear predictions of test scores using combinations of country-specific production function parameters (βc) with country-specific input levels (Xic and TUic). Standard errors of predictions in parentheses.

Appendix: Item Response Theory

How I link scores

◮ Decades long history in education and psychometrics – GRE,

GMAT, SAT, NAEP, TIMSS

◮ The basic idea:The focus of IRT is at the item level.

◮ Models the probability that an individual with given ability will

get an item right

◮ The overall ability estimate (test score) generated by analyzing

an individual’s response to different items each defined by their

• wn characteristics

◮ Many advantages (see e.g. Das and Zajonc, 2010):

◮ Most importantly (for me) the ability to link ◮ But also much better diagnostics for cross-cultural comparisons ◮ Less arbitrary than summing up correct responses

◮ Caveat: Linking requires common items across samples

◮ can’t directly compare across age groups

Appendix: Item Response Theory

How I link scores

◮ Decades long history in education and psychometrics – GRE,

GMAT, SAT, NAEP, TIMSS

◮ The basic idea:The focus of IRT is at the item level.

◮ Models the probability that an individual with given ability will

get an item right

◮ The overall ability estimate (test score) generated by analyzing

an individual’s response to different items each defined by their

• wn characteristics

◮ Many advantages (see e.g. Das and Zajonc, 2010):

◮ Most importantly (for me) the ability to link ◮ But also much better diagnostics for cross-cultural comparisons ◮ Less arbitrary than summing up correct responses

◮ Caveat: Linking requires common items across samples

◮ can’t directly compare across age groups

Appendix: Item Response Theory

How I link scores

◮ Decades long history in education and psychometrics – GRE,

GMAT, SAT, NAEP, TIMSS

◮ The basic idea:The focus of IRT is at the item level.

◮ Models the probability that an individual with given ability will

get an item right

◮ The overall ability estimate (test score) generated by analyzing

an individual’s response to different items each defined by their

• wn characteristics

◮ Many advantages (see e.g. Das and Zajonc, 2010):

◮ Most importantly (for me) the ability to link ◮ But also much better diagnostics for cross-cultural comparisons ◮ Less arbitrary than summing up correct responses

◮ Caveat: Linking requires common items across samples

◮ can’t directly compare across age groups

Appendix: Item Response Theory

How I link scores

◮ Decades long history in education and psychometrics – GRE,

GMAT, SAT, NAEP, TIMSS

◮ The basic idea:The focus of IRT is at the item level.

◮ Models the probability that an individual with given ability will

get an item right

◮ The overall ability estimate (test score) generated by analyzing

an individual’s response to different items each defined by their

• wn characteristics

◮ Many advantages (see e.g. Das and Zajonc, 2010):

◮ Most importantly (for me) the ability to link ◮ But also much better diagnostics for cross-cultural comparisons ◮ Less arbitrary than summing up correct responses

◮ Caveat: Linking requires common items across samples

◮ can’t directly compare across age groups

Appendix: Item Response Theory

Item Characteristic Curve

Appendix: Item Response Theory

3 Parameter Logistic (3PL) Model

Item Response Function: Pg(θi) = cg + 1 − cg 1 + exp(−1.7.ag.(θi − bg)) (6)

◮ cg is the pseudo-guessing parameter - with multiple choice

questions, even the lowest ability can get some answers right. Set to zero for non-MCQ to get 2PL model

◮ bg is the difficulty parameter - the level at which the

probability of getting item right is 0.5 in 2 PL

◮ ag is the discrimination parameter - slope of the ICC at b –

how quickly the likelihood of success changes with respect to ability.

Appendix: Item Response Theory

3 Parameter Logistic (3PL) Model

Item Response Function: Pg(θi) = cg + 1 − cg 1 + exp(−1.7.ag.(θi − bg)) (6)

◮ cg is the pseudo-guessing parameter - with multiple choice

questions, even the lowest ability can get some answers right. Set to zero for non-MCQ to get 2PL model

◮ bg is the difficulty parameter - the level at which the

probability of getting item right is 0.5 in 2 PL

◮ ag is the discrimination parameter - slope of the ICC at b –

how quickly the likelihood of success changes with respect to ability.

Appendix: Item Response Theory

3 Parameter Logistic (3PL) Model

Item Response Function: Pg(θi) = cg + 1 − cg 1 + exp(−1.7.ag.(θi − bg)) (6)

◮ cg is the pseudo-guessing parameter - with multiple choice

questions, even the lowest ability can get some answers right. Set to zero for non-MCQ to get 2PL model

◮ bg is the difficulty parameter - the level at which the

probability of getting item right is 0.5 in 2 PL

◮ ag is the discrimination parameter - slope of the ICC at b –

how quickly the likelihood of success changes with respect to ability.

Appendix: Item Response Theory

Core Assumptions

• 1. Unidimensionality - A single latent individual-specific trait

determines performance on the test

• 2. No Differential Item Functioning: Implicit in ICC, item

characteristics are person-invariant

2.1 particularly important in cross-cultural settings

• 3. (Conditional) local independence:

3.1 Item responses are independent across individuals (no cheating!) 3.2 Conditional on ability, item responses are locally independent across questions for the same individual

Under these assumptions, can recover estimates of ability and item characteristics given matrix of responses by individuals

Appendix: Item Response Theory

Core Assumptions

• 1. Unidimensionality - A single latent individual-specific trait

determines performance on the test

• 2. No Differential Item Functioning: Implicit in ICC, item

characteristics are person-invariant

2.1 particularly important in cross-cultural settings

• 3. (Conditional) local independence:

3.1 Item responses are independent across individuals (no cheating!) 3.2 Conditional on ability, item responses are locally independent across questions for the same individual

Under these assumptions, can recover estimates of ability and item characteristics given matrix of responses by individuals

Appendix: Item Response Theory

Core Assumptions

• 1. Unidimensionality - A single latent individual-specific trait

determines performance on the test

• 2. No Differential Item Functioning: Implicit in ICC, item

characteristics are person-invariant

2.1 particularly important in cross-cultural settings

• 3. (Conditional) local independence:

3.1 Item responses are independent across individuals (no cheating!) 3.2 Conditional on ability, item responses are locally independent across questions for the same individual

Under these assumptions, can recover estimates of ability and item characteristics given matrix of responses by individuals

Appendix: Item Response Theory

Core Assumptions

• 1. Unidimensionality - A single latent individual-specific trait

determines performance on the test

• 2. No Differential Item Functioning: Implicit in ICC, item

characteristics are person-invariant

2.1 particularly important in cross-cultural settings

• 3. (Conditional) local independence:

3.1 Item responses are independent across individuals (no cheating!) 3.2 Conditional on ability, item responses are locally independent across questions for the same individual

Under these assumptions, can recover estimates of ability and item characteristics given matrix of responses by individuals

Appendix: Item Response Theory

Core Assumptions

• 1. Unidimensionality - A single latent individual-specific trait

determines performance on the test

• 2. No Differential Item Functioning: Implicit in ICC, item

characteristics are person-invariant

2.1 particularly important in cross-cultural settings

• 3. (Conditional) local independence:

3.1 Item responses are independent across individuals (no cheating!) 3.2 Conditional on ability, item responses are locally independent across questions for the same individual

Under these assumptions, can recover estimates of ability and item characteristics given matrix of responses by individuals

Appendix: Item Response Theory

Core Assumptions

• 1. Unidimensionality - A single latent individual-specific trait

determines performance on the test

• 2. No Differential Item Functioning: Implicit in ICC, item

characteristics are person-invariant

2.1 particularly important in cross-cultural settings

• 3. (Conditional) local independence:

3.1 Item responses are independent across individuals (no cheating!) 3.2 Conditional on ability, item responses are locally independent across questions for the same individual

Under these assumptions, can recover estimates of ability and item characteristics given matrix of responses by individuals

Appendix: Item Response Theory

How does linking work?

◮ IRT only identifies the latent ability up to a linear

transformation

◮ need to fix the scale somewhere ◮ e.g. fix min and max. GRE used to be from 200 to 800

(130-170 now)

◮ or fix mean and SD. PISA and TIMSS have mean of 500 and

SD of 100

◮ Item characteristics are fixed and can be used to link across

samples

◮ common items serve as ‘anchors’ which bring two assessments

• n a common scale

◮ only a subset of items need to be common

◮ Without sufficient common items:

◮ Still can do IRT but scores not on comparable scales ◮ important because then you can’t use panel methods such as

differencing or fixed effects without very strong assumptions about the two distributions

Appendix: Item Response Theory

How does linking work?

◮ IRT only identifies the latent ability up to a linear

transformation

◮ need to fix the scale somewhere ◮ e.g. fix min and max. GRE used to be from 200 to 800

(130-170 now)

◮ or fix mean and SD. PISA and TIMSS have mean of 500 and

SD of 100

◮ Item characteristics are fixed and can be used to link across

samples

◮ common items serve as ‘anchors’ which bring two assessments

• n a common scale

◮ only a subset of items need to be common

◮ Without sufficient common items:

◮ Still can do IRT but scores not on comparable scales ◮ important because then you can’t use panel methods such as

differencing or fixed effects without very strong assumptions about the two distributions

Appendix: Item Response Theory

How does linking work?

◮ IRT only identifies the latent ability up to a linear

transformation

◮ need to fix the scale somewhere ◮ e.g. fix min and max. GRE used to be from 200 to 800

(130-170 now)

◮ or fix mean and SD. PISA and TIMSS have mean of 500 and

SD of 100

◮ Item characteristics are fixed and can be used to link across

samples

◮ common items serve as ‘anchors’ which bring two assessments

• n a common scale

◮ only a subset of items need to be common

◮ Without sufficient common items:

◮ Still can do IRT but scores not on comparable scales ◮ important because then you can’t use panel methods such as

differencing or fixed effects without very strong assumptions about the two distributions

Appendix: Item Response Theory

How does linking work?

◮ IRT only identifies the latent ability up to a linear

transformation

◮ need to fix the scale somewhere ◮ e.g. fix min and max. GRE used to be from 200 to 800

(130-170 now)

◮ or fix mean and SD. PISA and TIMSS have mean of 500 and

SD of 100

◮ Item characteristics are fixed and can be used to link across

samples

◮ common items serve as ‘anchors’ which bring two assessments

• n a common scale

◮ only a subset of items need to be common

◮ Without sufficient common items:

◮ Still can do IRT but scores not on comparable scales ◮ important because then you can’t use panel methods such as

differencing or fixed effects without very strong assumptions about the two distributions

Appendix: Item Response Theory

How does linking work?

◮ IRT only identifies the latent ability up to a linear

transformation

◮ need to fix the scale somewhere ◮ e.g. fix min and max. GRE used to be from 200 to 800

(130-170 now)

◮ or fix mean and SD. PISA and TIMSS have mean of 500 and

SD of 100

◮ Item characteristics are fixed and can be used to link across

samples

◮ common items serve as ‘anchors’ which bring two assessments

• n a common scale

◮ only a subset of items need to be common

◮ Without sufficient common items:

◮ Still can do IRT but scores not on comparable scales ◮ important because then you can’t use panel methods such as

differencing or fixed effects without very strong assumptions about the two distributions

Appendix: Item Response Theory

How does linking work?

◮ IRT only identifies the latent ability up to a linear

transformation

◮ need to fix the scale somewhere ◮ e.g. fix min and max. GRE used to be from 200 to 800

(130-170 now)

◮ or fix mean and SD. PISA and TIMSS have mean of 500 and

SD of 100

◮ Item characteristics are fixed and can be used to link across

samples

◮ common items serve as ‘anchors’ which bring two assessments

• n a common scale

◮ only a subset of items need to be common

◮ Without sufficient common items:

◮ Still can do IRT but scores not on comparable scales ◮ important because then you can’t use panel methods such as

differencing or fixed effects without very strong assumptions about the two distributions

Appendix: Item Response Theory

How does linking work?

◮ IRT only identifies the latent ability up to a linear

transformation

◮ need to fix the scale somewhere ◮ e.g. fix min and max. GRE used to be from 200 to 800

(130-170 now)

◮ or fix mean and SD. PISA and TIMSS have mean of 500 and

SD of 100

◮ Item characteristics are fixed and can be used to link across

samples

◮ common items serve as ‘anchors’ which bring two assessments

• n a common scale

◮ only a subset of items need to be common

◮ Without sufficient common items:

◮ Still can do IRT but scores not on comparable scales ◮ important because then you can’t use panel methods such as

differencing or fixed effects without very strong assumptions about the two distributions

Appendix: Item Response Theory

How does linking work?

◮ IRT only identifies the latent ability up to a linear

transformation

◮ need to fix the scale somewhere ◮ e.g. fix min and max. GRE used to be from 200 to 800

(130-170 now)

◮ or fix mean and SD. PISA and TIMSS have mean of 500 and

SD of 100

◮ Item characteristics are fixed and can be used to link across

samples

◮ common items serve as ‘anchors’ which bring two assessments

• n a common scale

◮ only a subset of items need to be common

◮ Without sufficient common items:

◮ Still can do IRT but scores not on comparable scales ◮ important because then you can’t use panel methods such as

differencing or fixed effects without very strong assumptions about the two distributions

Appendix: Item Response Theory

How does linking work?

◮ IRT only identifies the latent ability up to a linear

transformation

◮ need to fix the scale somewhere ◮ e.g. fix min and max. GRE used to be from 200 to 800

(130-170 now)

◮ or fix mean and SD. PISA and TIMSS have mean of 500 and

SD of 100

◮ Item characteristics are fixed and can be used to link across

samples

◮ common items serve as ‘anchors’ which bring two assessments

• n a common scale

◮ only a subset of items need to be common

◮ Without sufficient common items:

◮ Still can do IRT but scores not on comparable scales ◮ important because then you can’t use panel methods such as

differencing or fixed effects without very strong assumptions about the two distributions

Appendix: Item Response Theory

How does linking work?

◮ IRT only identifies the latent ability up to a linear

transformation

◮ need to fix the scale somewhere ◮ e.g. fix min and max. GRE used to be from 200 to 800

(130-170 now)

◮ or fix mean and SD. PISA and TIMSS have mean of 500 and

SD of 100

◮ Item characteristics are fixed and can be used to link across

samples

◮ common items serve as ‘anchors’ which bring two assessments

• n a common scale

◮ only a subset of items need to be common

◮ Without sufficient common items:

◮ Still can do IRT but scores not on comparable scales ◮ important because then you can’t use panel methods such as

differencing or fixed effects without very strong assumptions about the two distributions

Appendix: Differential Item Functioning

When it’s not a problem

C:/Users/pemb2850/Dropbox/Learning levels -

Appendix: Differential Item Functioning

When it IS a problem

C:/Users/pemb2850/Dropbox/Learning levels -

Introduction Background Data and specification Results Conclusion

The impact of an aggregate economic shock

• n schooling outcomes in Indonesia

Anisha Sharma

University of Oxford

March 22, 2015

Introduction Background Data and specification Results Conclusion

Question

How does an aggregate economic shock affect the decision to go to school?

Introduction Background Data and specification Results Conclusion

Motivation

The decision to enrol in further schooling depends on both direct and indirect costs of schooling.

Introduction Background Data and specification Results Conclusion

Motivation

The decision to enrol in further schooling depends on both direct and indirect costs of schooling. An economy-wide fall in real wages affects the wage rate earned by children, as well as household income.

Introduction Background Data and specification Results Conclusion

Motivation

In developed countries, borrowing constraints are less binding and the substitution effect of lower wages dominates. Kane 1994, Betts and MacFarland 1995, Goldin 1999.

Introduction Background Data and specification Results Conclusion

Motivation

In developed countries, borrowing constraints are less binding and the substitution effect of lower wages dominates. Kane 1994, Betts and MacFarland 1995, Goldin 1999. In developing countries, the evidence is less clear. Countercyclical in countries such as Argentina, Mexico; but procyclical in Cote D’Ivoire, Malawi (Ferreira and Schady 2009).

Introduction Background Data and specification Results Conclusion

What does this paper do

Estimates the effect of an aggregate economic shock: when most of the literature focuses on idiosyncratic variation in household income. Estimates the long-term effect of school disruptions: where there is limited access to long-run panel datasets in developing countries.

Introduction Background Data and specification Results Conclusion

What does this paper do

This paper Exploits regional heterogeneity in the impact of a recession to identify its effect on schooling outcomes and child employment. Controls for changes in the quality of schooling during this period. Examines the effect of the recession on long-run schooling and labour market outcomes.

Introduction Background Data and specification Results Conclusion

What does this paper do

This paper Exploits regional heterogeneity in the impact of a recession to identify its effect on schooling outcomes and child employment. Controls for changes in the quality of schooling during this period. Examines the effect of the recession on long-run schooling and labour market outcomes. The empirical context is Indonesia in the aftermath of the East Asian economic crisis of 1998.

Introduction Background Data and specification Results Conclusion

Background

In January 1998, Indonesian rupiah collapsed by 250% against the dollar, precipitating a deep and long-drawn economic crisis. GDP contracted by 13.2%, real wages by 40%, HH spending by 20%, human capital investments by 40%. Frankenberg et al (2003), Thomas et al (2004).

Introduction Background Data and specification Results Conclusion

Background

High inflation at 80% in 1998 led to a sharp drop in real wages. Rice price increases drove inflation, increasing by 120% in 1998 alone and 100% higher by 2000.

Median household in the sample spends 54% of budget on food, and 20% on rice.

There was substantial geographic variation in rice price increases across communities.

Introduction Background Data and specification Results Conclusion

Rice prices rose rapidly...

100 200 300 400 500 Scaled price of rice/tonne and the CPI 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 year Rice price CPI

Source: IMF World Economic Database and the UN FAO

Introduction Background Data and specification Results Conclusion

...but with variation across communities

5 10 15 20 Frequency - communities Mean +1 s.d.

• 1 s.d.

+2 s.d.

• 2 s.d.

50 100 150 200 % Increase in rice price

Note: Epanechnikov kernel density function overlaid on histogram

Introduction Background Data and specification Results Conclusion

(107,188] (84,107] (67,84] [38,67] No data

Introduction Background Data and specification Results Conclusion

Impact of rising prices

For net consumers of rice, higher rice prices will lead to: A fall in disposable household income, reducing the expenditure on schooling. A fall in real child wages, reducing the marginal cost of staying in school. Net effect on schooling outcomes is theoretically ambiguous.

Introduction Background Data and specification Results Conclusion

Data: Source and key variables

The data is drawn from 3 waves of the Indonesian Family Life Survey, which follows 7,224 origin households from 1993 to 2007, representing 83% of the Indonesian population.

Introduction Background Data and specification Results Conclusion

Data: Source and key variables

The data is drawn from 3 waves of the Indonesian Family Life Survey, which follows 7,224 origin households from 1993 to 2007, representing 83% of the Indonesian population. 100 urban districts form the relevant community. First, consider school attendance of 6,324 children aged between 7 and 15 in 1997 and 2000. Second, consider eventual schooling achievement of 2,845 children observed in 1997 and 2007.

Introduction Background Data and specification Results Conclusion

Variation in schooling outcomes

All children 7-12 yr-olds 13-15 yr-olds 1997 2000 1997 2000 1997 2000 Age 11.41 11.16 9.60 9.46 14.04 14.10 (2.62) (2.66) (1.76) (1.69) (0.80) (0.81) Male 0.50 0.50 0.52 0.50 0.48 0.50 (0.50) (0.50) (0.50) (0.50) (0.50) (0.50) Enrolled in school 0.90 0.88 0.95 0.94 0.83 0.79 (0.30) (0.32) (0.21) (0.24) (0.38) (0.41) Schoolhours/week 31.73 25.70 29.71 22.76 34.96 31.27 (6.81) (12.71) (6.92) (12.08) (5.19) (12.01) In labour market 0.02 0.10 0.01 0.05 0.05 0.18 (0.16) (0.30) (0.09) (0.22) (0.22) (0.39) Observations 3,156 3,317 1,865 2,100 1,291 1,217

Introduction Background Data and specification Results Conclusion

Specification

yijt = βlnPjt + X′

ijtγ + λj + dt + εijt

(1)

Introduction Background Data and specification Results Conclusion

Specification

yijt = βlnPjt + X′

ijtγ + λj + dt + εijt

(1) yijt is schooling outcome for student i, living in community j, at time t. Pjt is the price of rice in j at t. Xijt includes individual characteristics, season effects. λj is a community fixed effect.

Introduction Background Data and specification Results Conclusion

Specification

yijt = βlnPjt + X′

ijtγ + λj + dt + εijt

(1) yijt is schooling outcome for student i, living in community j, at time t. Pjt is the price of rice in j at t. Xijt includes individual characteristics, season effects. λj is a community fixed effect. β is an estimate of the within-community effect of an increase in the price of rice on schooling outcomes. Standard errors clustered by community-year.

Introduction Background Data and specification Results Conclusion

Results: impact of rice prices on school enrolment

(1) (2) (3) Ln rice price –0.103** –0.103* –0.105** (0.046) (0.058) (0.045) Year = 2000 0.036 0.006 0.008 (0.024) (0.035) (0.030) Season indicators Yes Yes Yes Child characteristics Yes Yes Yes Province-year effects Yes Yes Yes Community social programmes No Yes Yes Infrastructure improvements No Yes Yes El Ni˜ no shock No Yes Yes Household characteristics No No Yes Community fixed effects Yes Yes Yes Observations 6,473 6,473 6,473 Adj.R2 0.143 0.143 0.305

Introduction Background Data and specification Results Conclusion

Methodological framework

Results should be robust to: Time-invariant community characteristics (community fixed effects). Changes in school quality (funding to schools). Changes in community infrastructure (indicators for key institutions). Regional price variation (province-time fixed effects). Household characteristics (household fixed effects).

Introduction Background Data and specification Results Conclusion

Results: impact of rice prices on school enrolment

(1) (2) (3) Ln rice price –0.103** –0.103* –0.105** (0.046) (0.058) (0.045) Year = 2000 0.036 0.006 0.008 (0.024) (0.035) (0.030) Season indicators Yes Yes Yes Child characteristics Yes Yes Yes Province-year effects Yes Yes Yes Community social programmes No Yes Yes Infrastructure improvements No Yes Yes El Ni˜ no shock No Yes Yes Household characteristics No No Yes Community fixed effects Yes Yes Yes Observations 6,473 6,473 6,473 Adj.R2 0.143 0.143 0.305

Introduction Background Data and specification Results Conclusion

Results: impact on hours spent at school

(1) (2) (3) Ln rice price –16.143*** –10.589*** –10.682*** (3.278) (3.225) (3.153) Year = 2000 –5.161*** –14.297*** –14.116*** (1.590) (2.783) (2.757) Season indicators Yes Yes Yes Child characteristics Yes Yes Yes Province-year effects Yes Yes Yes Community social programmes No Yes Yes Infrastructure improvements No Yes Yes El Ni˜ no shock No Yes Yes Household characteristics No No Yes Community fixed effects Yes Yes Yes Observations 6,473 6,473 6,473 Adj.R2 0.207 0.217 0.304

Introduction Background Data and specification Results Conclusion

Results: impact on labour market participation

(1) (2) (3) Ln rice price –0.077* –0.166*** –0.170*** (0.045) (0.042) (0.040) Year = 2000 0.118*** 0.179*** 0.178*** (0.024) (0.026) (0.025) Season indicators Yes Yes Yes Child characteristics Yes Yes Yes Province-year effects Yes Yes Yes Community social programmes No Yes Yes Infrastructure improvements No Yes Yes El Ni˜ no shock No Yes Yes Household characteristics No No Yes Community fixed effects Yes Yes Yes Observations 6,473 6,473 6,473 Adj.R2 0.097 0.099 0.111

Introduction Background Data and specification Results Conclusion

Results: household fixed effects

Dependent variable School enrolment School hours Child labour Ln rice price –0.158** –13.819*** –0.097* (0.063) (4.404) (0.055) Year = 2000 –0.059 –17.230*** 0.223*** (0.046) (3.779) (0.049) Season indicators Yes Yes Yes Child characteristics Yes Yes Yes Province-year effects Yes Yes Yes Community social programmes Yes Yes Yes Infrastructure improvements Yes Yes Yes El Ni˜ no shock Yes Yes Yes Household characteristics Yes Yes Yes Household fixed effects Yes Yes Yes Observations 4,802 4,802 4,802 Adj.R2 0.342 0.355 0.194

Introduction Background Data and specification Results Conclusion

Heterogeneity by age

Employment – full-time or part-time – is very low among younger children (7-12 yrs) and relatively higher among older children (13-15 yrs).

Introduction Background Data and specification Results Conclusion

Heterogeneity by age

Employment – full-time or part-time – is very low among younger children (7-12 yrs) and relatively higher among older children (13-15 yrs). Older children (13-15 yrs) are likely to be more sensitive to changes in the child wage rate.

Introduction Background Data and specification Results Conclusion

Results: Heterogeneity by age

Dependent variable School enrolment School hours Child labour Ln rice price × 7-12 years old –0.136*** –11.551*** –0.140*** (0.045) (3.167) (0.039) Ln rice price × 13-15 years old –0.041 –8.659** –0.225*** (0.054) (3.409) (0.050) 7-12 years old 0.662** 21.529 –0.584** (0.295) (13.988) (0.254) Year = 2000 0.003 –14.118*** 0.173*** (0.029) (2.673) (0.027) Season indicators Yes Yes Yes Child characteristics Yes Yes Yes Province-year effects Yes Yes Yes Community social programmes Yes Yes Yes Infrastructure improvements Yes Yes Yes El Ni˜ no shock Yes Yes Yes Household characteristics Yes Yes Yes Community fixed effects Yes Yes Yes Observations 6,473 6,473 6,473 Adj.R2 0.286 0.282 0.099

Introduction Background Data and specification Results Conclusion

Results: summary

School attendance declines at the extensive and intensive margins; labour market participation declines as well! For a 1 st. dev. increase in the price of rice, enrolment declines by 2 pp; weekly attendance by 2.7 hours, employment by 3 pp. For a 2 st. dev increase, enrolment declines by 3 pp; weekly attendance by 5 hours, employment by 5 pp. Younger children show larger declines in school attendance;

• lder children show larger declines in labour market

participation.

Introduction Background Data and specification Results Conclusion

Did these disruptions have a long-run impact?

Adverse long-term consequences do not necessarily follow from short-run schooling disruptions: perhaps less able students are dropping out or children are catching up over time. Estimate the long-term impact of the recession by using data

• n 2,845 children observed in 1997 (7-15 yrs) and 2007 (17-25

yrs).

Introduction Background Data and specification Results Conclusion

Main specification

The main specification is: yijt+1 = αyijt + β∆lnPjt + X′

ijtγ + εijt+1

(2)

Introduction Background Data and specification Results Conclusion

Main specification

The main specification is: yijt+1 = αyijt + β∆lnPjt + X′

ijtγ + εijt+1

(2) Where the variables indicate: yijt+10 is schooling achievement for student i, who lived in community j, measured at time t + 10. ∆lnPjt is the price shock experienced by community j between t and t + 3. Xijt includes controls for individual and baseline household characteristics.

Introduction Background Data and specification Results Conclusion

Main specification

The main specification is: yijt+1 = αyijt + β∆lnPjt + X′

ijtγ + εijt+1

(2) Where the variables indicate: yijt+10 is schooling achievement for student i, who lived in community j, measured at time t + 10. ∆lnPjt is the price shock experienced by community j between t and t + 3. Xijt includes controls for individual and baseline household characteristics. β1 is an estimate of the long term impact of the economic crisis

• n schooling achievement measured in 2007.

Introduction Background Data and specification Results Conclusion

Long-term schooling outcomes improve

Dependent variable Years of schooling

• Grad. jr.

school

• Grad. sr.

school Grad. college ∆ Ln rice price 1.054* 0.119 0.116 0.082 (0.545) (0.081) (0.074) (0.058) Baseline outcomes Yes Yes Yes Yes Season indicators Yes Yes Yes Yes Child characteristics Yes Yes Yes Yes Province effects Yes Yes Yes Yes Community social programmes Yes Yes Yes Yes Infrastructure improvements Yes Yes Yes Yes El Ni˜ no shock Yes Yes Yes Yes Household characteristics Yes Yes Yes Yes Observations 2,942 2,942 2,942 2,942 Adj.R2 0.446 0.235 0.326 0.157

Introduction Background Data and specification Results Conclusion

Results: summary

Higher rice prices lead to marginally higher years of schooling after ten years. A 1 st. dev. increase in the price of rice increases total years of schooling by 0.2 (every fifth child has an additional year of schooling.

Introduction Background Data and specification Results Conclusion

Results: summary

Higher rice prices lead to marginally higher years of schooling after ten years. A 1 st. dev. increase in the price of rice increases total years of schooling by 0.2 (every fifth child has an additional year of schooling. Results are robust to correction for attrition.

Introduction Background Data and specification Results Conclusion

Conclusion

Fears of a widespread collapse in demand have proved to be unfounded. Sustaining quality of schooling might be more important. Creating employment opportunities and providing relevant vocational training will mitigate the labour market effects of an economic crisis.

Introduction Background Data and specification Results Conclusion

Thank you!

▲♦♥❣ ❚❡r♠ P❡rs✐st❡♥❝❡ ♦❢ ❈♦❧♦♥✐❛❧ ❊❞✉❝❛t✐♦♥❄

❆ ❉✐s❝♦♥t✐♥✉✐t② ❆♥❛❧②s✐s ❛t t❤❡ ❇♦r❞❡r ❜❡t✇❡❡♥ ❋r❡♥❝❤✲ ❛♥❞ ❊♥❣❧✐s❤✲s♣❡❛❦✐♥❣ ❈❛♠❡r♦♦♥ ❨❛♥♥✐❝❦ ❉✉♣r❛③ ✭P❙❊✮

❈❙❆❊ ❈♦♥❢❡r❡♥❝❡

✷✷♥❞ ▼❛r❝❤ ✷✵✶✺

❉♦❡s ❝♦❧♦♥✐❛❧ ❤✐st♦r② ♠❛tt❡r ❢♦r ❞❡✈❡❧♦♣♠❡♥t❄

◮ ❘❡❝❡♥t ❧✐t❡r❛t✉r❡ ❤❛s str❡ss❡❞ t❤❡ ✐♠♣♦rt❛♥❝❡ ♦❢ ❝♦❧♦♥✐❛❧

❤✐st♦r② ✐♥ ✉♥❞❡rst❛♥❞✐♥❣ ♣r❡s❡♥t✲❞❛② ❞✐✛❡r❡♥❝❡s ✐♥ ❞❡✈❡❧♦♣♠❡♥t✳ ▲❡❣❛❧ ♦r✐❣✐♥s ✭▲❛ P♦rt❛ ❡t ❛❧✳✱ ✶✾✾✽✱ ✶✾✾✾✮✱ ✐♥st✐t✉t✐♦♥s ✭❆❝❡♠♦❣❧✉ ❡t ❛❧✳✱ ✷✵✵✶✮✱ ❤✉♠❛♥ ❝❛♣✐t❛❧ ✭●❧❛❡s❡r ❡t ❛❧✳✱ ✷✵✵✹❀ ❇♦❧t ❛♥❞ ❇❡③❡♠❡r✱ ✷✵✵✾✮✳

◮ ■♥ ❆❢r✐❝❛✱ ❇r✐t✐s❤ ❝♦❧♦♥✐❛❧ ❧❡❣❛❝② s❛✐❞ t♦ ❜❡ ♠♦r❡ ❢❛✈♦r❛❜❧❡ t❤❛♥

♦t❤❡rs✱ ❡s♣❡❝✐❛❧❧② ❛s ❢❛r ❛s ❡❞✉❝❛t✐♦♥ ❛♥❞ ❤✉♠❛♥ ❝❛♣✐t❛❧ ❛r❡ ❝♦♥❝❡r♥❡❞✳

◮ ✶✽✼✵✲✶✾✹✵✿ ♣r✐♠❛r② ❡♥r♦❧❧♠❡♥t r❛t❡s ❤✐❣❤❡r ✐♥ ❇r✐t✐s❤ ❆❢r✐❝❛

✭❇❡♥❛✈♦t ❛♥❞ ❘✐❞❞❧❡✱ ✶✾✽✽✮✳

◮ ❚❤❡ ❞✐✛❡r❡♥❝❡ s❡❡♠s t♦ ❤❛✈❡ ♣❡rs✐st❡❞ ✭❇r♦✇♥✱ ✷✵✵✵✿

❈♦❣♥❡❛✉✱ ✷✵✵✸❀ ●❛r♥✐❡r ❛♥❞ ❙❝❤❛❢❡r✱ ✷✵✵✻✮✳

◮ ●r✐❡r ✭✶✾✾✾✮✿ ✐♠♣❛❝t ♦❢ ❝♦❧♦♥✐③❛t✐♦♥ ♦♥ ❡❞✉❝❛t✐♦♥ ❛♥❞ t❤❡♥

❣r♦✇t❤✳

❉✐✛❡r❡♥❝❡s ❜❡t✇❡❡♥ ❇r✐t✐s❤ ❛♥❞ ❋r❡♥❝❤ ❝♦❧♦♥✐❛❧ ❡❞✉❝❛t✐♦♥ ♣♦❧✐❝✐❡s✮

❇r✐t✐s❤ ❋r❡♥❝❤ ❝♦❧♦♥✐❛❧ ❡❞✉❝❛t✐♦♥ ❝♦❧♦♥✐❛❧ ❡❞✉❝❛t✐♦♥ ❈❤r✐st✐❛♥ ♠✐ss✐♦♥s ❆❣❡♥❝② ✜♥❛♥❝❡❞ ❜② ❣r❛♥ts✲✐♥✲❛✐❞ ❝♦❧♦♥✐❛❧ ❣♦✈❡r♥♠❡♥t ❝♦♠♣❡t✐t✐♦♥ ❚❡❛❝❤❡rs ❢❡✇ ❇r✐t✐s❤ t❡❛❝❤❡rs ❛ ❧♦t ♦❢ ❋r❡♥❝❤ t❡❛❝❤❡rs ▲❛♥❣✉❛❣❡ ♦❢ ✈❡r♥❛❝✉❧❛r ✐♥ ❛❧✇❛②s ✐♥str✉❝t✐♦♥ ✜rst ❣r❛❞❡s ❋r❡♥❝❤ ❝♦♥✈❡rs✐♦♥ tr❛✐♥✐♥❣ ♦❢ ❛ s♠❛❧❧ P✉r♣♦s❡ r❡❛❝❤✐♥❣ ❛ ❧❛r❣❡ ❛❞♠✐♥✐str❛t✐✈❡ ♥✉♠❜❡r ♦❢ ❝❤✐❧❞r❡♥ ❡❧✐t❡

❙♦✉r❝❡✿ ♣r❡✈✐♦✉s❧② ❝✐t❡❞ ✇♦r❦s✱ ●✐✛♦r❞ ❛♥❞ ❲❡✐s❦❡❧ ✭✶✾✼✶✮✱ ●❛❧❧❡❣♦ ❛♥❞ ❲♦♦❞❜❡rr② ✭✷✵✶✵✮✳

▼❡❛♥ ❣r♦ss ♣r✐♠❛r② ❡♥r♦❧❧♠❡♥t r❛t✐♦s ✐♥ ❙✉❜✲❙❛❤❛r❛♥ ❆❢r✐❝❛✱ ✶✾✵✵✲✷✵✶✵

❋r♦♠ ❝r♦ss ❝♦✉♥tr② t♦ ❜♦r❞❡r ❞✐s❝♦♥t✐♥✉✐t②

◮ Pr♦❜❧❡♠ ♦❢ ❝r♦ss ❝♦✉♥tr②✿ s❡❧❡❝t✐♦♥

◮ ❚❤❡ ✢❛❣ ❢♦❧❧♦✇❡❞ t❤❡ tr❛❞❡✳ ◮ ❚❤❡ ✢❛❣ ❢♦❧❧♦✇❡❞ t❤❡ ❝r♦ss✳

◮ ❙♣❛t✐❛❧ ❞✐s❝♦♥t✐♥✉✐t② ❛♥❛❧②s✐s✿

◮ ❇♦✐❧s ❞♦✇♥ t♦ ❝♦♠♣❛r✐♥❣ r❡❣✐♦♥s ♦♥ ❜♦t❤ s✐❞❡s ♦❢ ❛ ❜♦r❞❡r✱

✈❡r② ❝❧♦s❡ t♦ t❤❡ ❜♦r❞❡r✳

◮ ■❞❡♥t✐❢②✐♥❣ ❛ss✉♠♣t✐♦♥✿ t❤❡ ❢❛❝t t❤❛t ❛ ✈✐❧❧❛❣❡ ❢❡❧❧ ♦♥ ♦♥❡ s✐❞❡

♦r t❤❡ ♦t❤❡r ♦❢ t❤❡ ❜♦r❞❡r ✐s q✉❛s✐ r❛♥❞♦♠ ✰ ♥♦ s❡❧❡❝t✐✈❡ ♠✐❣r❛t✐♦♥s ❛❝r♦ss t❤❡ ❜♦r❞❡r✳

◮ ◆❛t✉r❛❧ ❡①♣❡r✐♠❡♥t✿ ♣❛rt✐t✐♦♥ ♦❢ ●❡r♠❛♥ ❑❛♠❡r✉♥ ❜❡t✇❡❡♥

t❤❡ ❇r✐t✐s❤ ❛♥❞ t❤❡ ❋r❡♥❝❤ ❛❢t❡r ❲❲■✳

❉✐✈✐s✐♦♥ ❛♥❞ ❘❡✉♥✐✜❝❛t✐♦♥✿ t❤❡ ◆❛t✉r❛❧ ❊①♣❡r✐♠❡♥t

❚❤❡ ❡✈♦❧✉t✐♦♥ ♦❢ ❈❛♠❡r♦♦♥✬s ❜♦✉♥❞❛r✐❡s

❚❤❡ ❜♦r❞❡r

▲♦❝❛❧ r❛♥❞♦♠♥❡ss ♦❢ t❤❡ ❜♦r❞❡r

❆❧❢r❡❞ ▼✐❧♥❡r✱ r❡♣r❡s❡♥t✐♥❣ t❤❡ ❯❑ ❛t t❤❡ P❛r✐s P❡❛❝❡ ❈♦♥❢❡r❡♥❝❡✿ ❚❤❡ ❜♦✉♥❞❛r✐❡s ❜❡t✇❡❡♥ t❤❡ ❞✐✛❡r❡♥t s♣❤❡r❡s ♦❢ ♦❝❝✉♣❛t✐♦♥ ❛r❡ ❤❛♣❤❛③❛r❞ ❛♥❞✱ ❛s ❛ ♣❡r♠❛♥❡♥t ❛rr❛♥❣❡♠❡♥t✱ ✇♦✉❧❞ ❜❡ q✉✐t❡ ✐♥t♦❧❡r❛❜❧❡✳ ❚❤❡② ❝✉t ❛❝r♦ss tr✐❜❛❧ ❛♥❞ ❛❞♠✐♥✐str❛t✐✈❡ ❞✐✈✐s✐♦♥s✱ t❛❦❡ ♥♦ ❛❝❝♦✉♥t ♦❢ ❡❝♦♥♦♠✐❝ ❝♦♥❞✐t✐♦♥s ❛♥❞ ❛r❡✱ ✐♥ ❡✈❡r② ✇❛②✱ ♦❜❥❡❝t✐♦♥❛❜❧❡✳ ✭◗✉♦t❡❞ ✐♥ ▲♦✉✐s✱ ✶✾✻✼✳✮

❆ s✐♠♣❧✐✜❡❞ ♠❛♣ ♦❢ ❡t❤♥✐❝ ❣r♦✉♣s ✐♥ ❲❡st ❈❛♠❡r♦♦♥

❙♦✉r❝❡s✿ ▼✉r❞♦❝❦ ✭✶✾✺✾✮ ❛♥❞ ❖❘❙❚❖▼ ✭✶✾✼✸✱ ✶✾✼✹✮✳ ❚❤✐s ✐s ❛ s✐♠♣❧✐✜❡❞ ♠❛♣✱ s♦♠❡ ❣r♦✉♣s ❛r❡ ♥♦t r❡♣r❡s❡♥t❡❞✳ ❚♦✇♥s ❛r❡ ♣❧❛❝❡s ♦❢ ❣r❡❛t ❞✐✈❡rs✐t②✱ ♣❛rt✐❝✉❧❛r❧② ✐♥ t❤❡ ❙♦✉t❤✳

◆♦ ❞✐s❝♦♥t✐♥✉✐t② ✐♥ ❡①♦❣❡♥♦✉s ❣❡♦❣r❛♣❤✐❝ ✈❛r✐❛❜❧❡s

❇❛♥❞✇✐❞t❤ ✺✵❦♠ ✶✵✵✲❦♠ ✷✵✵✲❦♠ ✷✵✵✲❦♠ ❞✐s❝♦♥t✐♥✉✐t② ❞✐s❝♦♥t✐♥✉✐t② ❞✐s❝♦♥t✐♥✉✐t② ♠❡❛♥ ❊♥❣❧✐s❤ ♠❡❛♥ ❋r❡♥❝❤ ♣♦❧②♥♦♠✐❛❧ ♦❢ ♣♦❧②♥♦♠✐❛❧ ♣♦❧②♥♦♠✐❛❧ s♣❡❛❦✐♥❣ s✐❞❡ s♣❡❛❦✐♥❣ s✐❞❡ ❞✐✛❡r❡♥❝❡ ♦❢ ♦r❞❡r ✶ ♦❢ ♦r❞❡r ✷ ♦❢ ♦r❞❡r ✸ ♠❡❛♥ ✶✱✵✶✺ ✶✱✵✵✺ ✾✳✽✷✵✵ ✲✽✻✳✽✹✷✼ ✹✵✳✽✸✺✸ ✶✶✳✸✶✼✽ ❡❧❡✈❛t✐♦♥ ✭✵✳✾✹✽✽✮ ✭✵✳✺✵✾✹✮ ✭✵✳✻✾✹✷✮ ✭✵✳✾✵✾✽✮ ♠❡❛♥ ✷✷✳✵✺ ✷✷✳✹✷ ✲✵✳✸✼✸✼ ✵✳✶✶✶✺ ✲✵✳✺✶✹✼ ✲✵✳✸✼✷✻ t❡♠♣❡r❛t✉r❡ ✭✵✳✺✻✸✺✮ ✭✵✳✽✼✾✾✮ ✭✵✳✸✻✷✹✮ ✭✵✳✺✵✵✹✮ ♠❡❛♥ ♠♦♥t❤❧② ✶✾✹✳✼ ✶✾✷✳✹ ✷✳✸✸✺✶ ✶✳✼✾✾✺ ✸✳✵✸✹✺ ✸✳✷✸✽✵ ♣r❡❝✐♣✐t❛t✐♦♥ ✭✵✳✽✷✺✼✮ ✭✵✳✻✺✻✶✮ ✭✵✳✸✺✶✶✮ ✭✵✳✸✶✼✵✮ ❖❜s❡r✈❛t✐♦♥s ✷✶ ✷✼ ✹✽ ✽✺ ✶✷✶ ✶✷✶

❘♦❜✉st st❛♥❞❛r❞ ❡rr♦rs✳ ♣ ✈❛❧✉❡s ✐♥ ♣❛r❡♥t❤❡s❡s✳ ✯ s✐❣♥✐✜❝❛♥t ❛t t❤❡ ✶✵✪ ❧❡✈❡❧✳ ✯✯ ❛t t❤❡ ✺✪ ❧❡✈❡❧✳ ✯✯✯ ❛t t❤❡ ✶✪ ❧❡✈❡❧✳ ❉♦✉❛❧❛ ✐s ❛❧✇❛②s ❡①❝❧✉❞❡❞ ❜❡❝❛✉s❡ ✐t ✐s ❡①❝❧✉❞❡❞ ❢r♦♠ ❛❧❧ ♦t❤❡r r❡❣r❡ss✐♦♥s ✭✐♥❝❧✉❞✐♥❣ t❤❡ ❞✐str✐❝t ❞♦❡s ♥♦t ❝❤❛♥❣❡ t❤❡ r❡s✉❧ts✮✳ ❈♦♥tr♦❧❧✐♥❣ ❢♦r ✺ ❜♦r❞❡r ❞✉♠♠✐❡s✳ ❊❧❡✈❛t✐♦♥ ✐♥ ♠❡t❡rs✱ t❡♠♣❡r❛t✉r❡ ✐♥ ❞❡❣r❡❡s ❈❡❧s✐✉s✱ ♣r❡❝✐♣✐t❛t✐♦♥ ✐♥ ♠♠✳

❊❝♦♥♦♠❡tr✐❝ s♣❡❝✐✜❝❛t✐♦♥

◮ ❚❤❡ ❢♦❧❧♦✇✐♥❣ ♠♦❞❡❧ ✐s ❡st✐♠❛t❡❞ ❜② ❖▲❙✿

yj = τBR + P(xj, yj) + βBj + uj j ✐♥❞❡①❡s ✈✐❧❧❛❣❡s ✭♦r ❞✐str✐❝ts✮✳ B❂✈❡❝t♦r ♦❢ ❜♦r❞❡r s❡❣♠❡♥t ❞✉♠♠✐❡s✳

◮ ❊❛❝❤ ♦❜s❡r✈❛t✐♦♥ ✐s ✇❡✐❣❤t❡❞ t♦ ❛❝❝♦✉♥t ❢♦r t❤❡ s✐③❡ ♦❢ ❡❛❝❤

✈✐❧❧❛❣❡ ✭♦r ❞✐str✐❝t✮✳

◮ ❉✐✛❡r❡♥t ❜❛♥❞✇✐❞t❤ ❛♥❞ ❞✐✛❡r❡♥t ♣♦❧②♥♦♠✐❛❧ ♦r❞❡rs✳ ◆♦

✉♥✐✈❡rs❛❧ s♦❧✉t✐♦♥✿ ❝♦♥✈✐♥❝✐♥❣ r❡s✉❧ts ❛r❡ t❤♦s❡ t❤❛t ❛r❡ r♦❜✉st t♦ ❛ ✈❛r✐❡t② ♦❢ s♣❡❝✐✜❝❛t✐♦♥s✳

P♦ss✐❜✐❧✐t② ♦❢ ❤❡t❡r♦❣❡♥❡♦✉s ❡✛❡❝ts ❛❧♦♥❣ t❤❡ ❜♦r❞❡r

❉❛t❛

◮ ❈❡♥s✉s ❞❛t❛

◮ ❈❛♠❡r♦♦♥✐❛♥ ❝❡♥s✉s❡s✿ ✶✾✼✻✱ ✶✾✽✼ ❛♥❞ ✷✵✵✺✳ ✶✾✽✼✿ ♠✐ss✐♥❣

❞✐str✐❝ts ♥❡❛r t❤❡ ❜♦r❞❡r ✰ ❜❛❞ r❡♣✉t❛t✐♦♥✳ ✷✵✵✺✿ ❣❡♦❧♦❝❛t✐♦♥ ♦♥❧② ❛t t❤❡ ❞✐str✐❝t ❧❡✈❡❧✳

◮ ▼❡❛s✉r❡ ♦❢ ❡❞✉❝❛t✐♦♥❛❧ ❛tt❛✐♥♠❡♥t✿ ❧❛st ❣r❛❞❡ ❛tt❡♥❞❡❞✳ ◮ ■ ❦♥♦✇ t❤❡ ❞✐str✐❝t ♦❢ ❜✐rt❤✳ ◮ P♦♣✉❧❛t✐♦♥ ❞✐✈✐❞❡❞ ✐♥ ✶✵✲②❡❛r ❝♦❤♦rts✱ ❝❡♥t❡r❡❞ ❛r♦✉♥❞ ✶✵ ✭❛❣❡

❤❡❛♣✐♥❣ ❞✐✛❡r❡♥t ♦♥ ❜♦t❤ s✐❞❡s ✰ ❛❣❡ ❤❡❛♣✐♥❣ ❝♦rr❡❧❛t❡❞ ✇✐t❤ ❡❞✉❝❛t✐♦♥

❛❣❡❤❡❛♣✐♥❣ ✮✳ Pr♦❜❧❡♠ ♦❢ s❡❧❡❝t✐♦♥ ❜② ♠♦rt❛❧✐t②✳ ◮ ✶✾✼✻✿ ✈✐❧❧❛❣❡s ❣❡♦❧♦❝❛t❡❞✱ r❡❛ttr✐❜✉t✐♦♥ ♦❢ ♠✐❣r❛♥ts ✐♥ t❤❡✐r

✈✐❧❧❛❣❡ ♦❢ ❜✐rt❤✳

◮ P❆❙❊❈ s❝❤♦♦❧ s✉r✈❡② ❞❛t❛ ✭✷✵✵✹✲✷✵✵✺✮✳ ◮ ❉❛t❛s❡t ♦❢ ❛❧❧ ❈❛♠❡r♦♦♥✐❛♥ ♣r✐♠❛r② s❝❤♦♦❧s ✐♥ ✷✵✵✸ ✇✐t❤ ❞❛t❡s

♦❢ ♦♣❡♥✐♥❣✳

▼❛❧❡ s❝❤♦♦❧ ❛tt❡♥❞❛♥❝❡✿ ❞✐✛❡r❡♥❝❡ ✐♥ ♠❡❛♥s ♦♥ ❛ ✶✵✵❦♠ ❜❛♥❞✇✐❞t❤

❉✐✛❡r❡♥❝❡ ✐♥ ♠❡❛♥ ♦♥ ❛ ✶✵✵✲❦♠ ❜❛♥❞✇✐❞t❤ ♦❢ t❤❡ ❙♦✉t❤❡r♥ ♣❛rt ♦❢ t❤❡ ❜♦r❞❡r✳ ❚❤❡ ❞❡♣❡♥❞❡♥t ✈❛r✐❛❜❧❡ ✐s ❛ ✪ ❛t t❤❡ ✈✐❧❧❛❣❡ ❧❡✈❡❧✳ ❘♦❜✉st ❙❊✳

▼❛❧❡ s❝❤♦♦❧ ❛tt❡♥❞❛♥❝❡✿ ❝♦♥tr♦❧❧✐♥❣ ❢♦r ❞❡t❡r♠✐♥❛♥ts ♦❢ ❞❡♠❛♥❞ ❢♦r ❡❞✉❝❛t✐♦♥ ❛♥❞ ♠✐ss✐♦♥ ❧♦❝❛t✐♦♥ ❜❡❢♦r❡ ♣❛rt✐t✐♦♥

❊✛❡❝t ♦❢ ❜❡❧♦♥❣✐♥❣ t♦ t❤❡ ❇r✐t✐s❤ s✐❞❡ ✐♥ ❛ r❡❣r❡ss✐♦♥ ✇✐t❤ ❝♦♥tr♦❧s✳ ❚❤❡ ❞❡♣❡♥❞❡♥t ✈❛r✐❛❜❧❡ ✐s ❛ ✪ ❛t t❤❡ ✈✐❧❧❛❣❡ ❧❡✈❡❧✳ ❘♦❜✉st ❙❊✳

❇♦r❞❡r ❞✐s❝♦♥t✐♥✉✐t② ✐♥ ♠❛❧❡ s❝❤♦♦❧ ❛tt❡♥❞❛♥❝❡

❉✐s❝♦♥t✐♥✉✐t② ❡st✐♠❛t❡❞ ♦♥ ❛ ✶✵✲❦♠ ❜❛♥❞✇✐❞t❤ ♦❢ t❤❡ ❙♦✉t❤❡r♥ ♣❛rt ♦❢ t❤❡ ❜♦r❞❡r ❝♦♥tr♦❧❧✐♥❣ ❢♦r ❛ ♣♦❧②♥♦♠✐❛❧ ♦❢ ♦r❞❡r ✷ ✐♥ ❧❛t✐t✉❞❡ ❛♥❞ ❧♦♥❣✐t✉❞❡ ❛♥❞ ✸ ❜♦r❞❡r s❡❣♠❡♥t ❞✉♠♠✐❡s✳ ❚❤❡ ❞❡♣❡♥❞❡♥t ✈❛r✐❛❜❧❡ ✐s ❛ ✪ ❛t t❤❡ ✈✐❧❧❛❣❡ ❧❡✈❡❧✳ ❘♦❜✉st ❙❊✳

❇♦r❞❡r ❞✐s❝♦♥t✐♥✉✐t② ✐♥ ♠❛❧❡ ♣r✐♠❛r② s❝❤♦♦❧ ❝♦♠♣❧❡t✐♦♥

❉✐s❝♦♥t✐♥✉✐t② ❡st✐♠❛t❡❞ ♦♥ ❛ ✶✵✲❦♠ ❜❛♥❞✇✐❞t❤ ♦❢ t❤❡ ❙♦✉t❤❡r♥ ♣❛rt ♦❢ t❤❡ ❜♦r❞❡r ❝♦♥tr♦❧❧✐♥❣ ❢♦r ❛ ♣♦❧②♥♦♠✐❛❧ ♦❢ ♦r❞❡r ✷ ✐♥ ❧❛t✐t✉❞❡ ❛♥❞ ❧♦♥❣✐t✉❞❡ ❛♥❞ ✸ ❜♦r❞❡r s❡❣♠❡♥t ❞✉♠♠✐❡s✳ ❚❤❡ ❞❡♣❡♥❞❡♥t ✈❛r✐❛❜❧❡ ✐s ❛ ✪ ❛t t❤❡ ✈✐❧❧❛❣❡ ❧❡✈❡❧✳ ❘♦❜✉st ❙❊✳

❇♦r❞❡r ❞✐s❝♦♥t✐♥✉✐t② ✐♥ ♠❛❧❡ s❡❝♦♥❞❛r② s❝❤♦♦❧ ❝♦♠♣❧❡t✐♦♥

❉✐s❝♦♥t✐♥✉✐t② ❡st✐♠❛t❡❞ ♦♥ ❛ ✶✵✲❦♠ ❜❛♥❞✇✐❞t❤ ♦❢ t❤❡ ❙♦✉t❤❡r♥ ♣❛rt ♦❢ t❤❡ ❜♦r❞❡r ❝♦♥tr♦❧❧✐♥❣ ❢♦r ❛ ♣♦❧②♥♦♠✐❛❧ ♦❢ ♦r❞❡r ✷ ✐♥ ❧❛t✐t✉❞❡ ❛♥❞ ❧♦♥❣✐t✉❞❡ ❛♥❞ ✸ ❜♦r❞❡r s❡❣♠❡♥t ❞✉♠♠✐❡s✳ ❚❤❡ ❞❡♣❡♥❞❡♥t ✈❛r✐❛❜❧❡ ✐s ❛ ✪ ❛t t❤❡ ✈✐❧❧❛❣❡ ❧❡✈❡❧✳ ❘♦❜✉st ❙❊✳

❘❡s✉❧ts ✉s✐♥❣ t❤❡ ✷✵✵✺ ❝❡♥s✉s ✭❞✐str✐❝t ❧❡✈❡❧✮✱ ❝♦❤♦rts ❜♦r♥ ❜❡t✇❡❡♥ ✶✾✼✶ ❛♥❞ ✶✾✽✵

✭✶✮ ✭✷✮ ✭✸✮ ✭✹✮ ✭✺✮ ✭✻✮ ♠❡❛♥s ♦♥ ✺✵✲❦♠ ❡st✐♠❛t❡❞ ❞✐s❝♦♥t✐♥✉✐t✐❡s ❜❛♥❞✇✐❞t❤ ❊♥❣❧✐s❤✲ ❋r❡♥❝❤✲ ✺✵✲❦♠ ❜❛♥❞✇✐❞t❤ ✶✵✵✲❦♠ ❜✇ ✷✵✵✲❦♠ ❜✇ s♣❡❛❦✐♥❣ s♣❡❛❦✐♥❣ ♥♦ ✭①✱②✮ ♣♦❧②♥♦♠✐❛❧ ♣♦❧②♥♦♠✐❛❧ ♣♦❧②♥♦♠✐❛❧ s✐❞❡ s✐❞❡ ❝♦♥tr♦❧s ♦❢ ♦r❞❡r ✶ ♦❢ ♦r❞❡r ✷ ♦❢ ♦r❞❡r ✸ ♠❛❧❡ s❡❝♦♥❞❛r② ✵✳✷✹✵✵ ✵✳✷✵✶✼ ✵✳✵✺✸✻✯✯✯ ✵✳✵✾✺✾✯✯✯ ✵✳✵✾✻✺✯✯✯ ✵✳✵✾✷✸✯✯✯ ❝♦♠♣❧❡t✐♦♥ ✭✵✳✵✵✵✸✮ ✭✵✳✵✵✵✽✮ ✭✵✳✵✵✵✶✮ ✭✵✳✵✵✵✵✮ ❢❡♠❛❧❡ s❡❝♦♥❞❛r② ✵✳✶✾✺✺ ✵✳✶✶✾✹ ✵✳✵✽✹✻✯✯✯ ✵✳✵✽✻✺✯✯ ✵✳✵✾✹✵✯✯✯ ✵✳✵✾✷✷✯✯✯ ❝♦♠♣❧❡t✐♦♥ ✭✵✳✵✵✵✻✮ ✭✵✳✵✶✺✾✮ ✭✵✳✵✵✶✷✮ ✭✵✳✵✵✵✹✮ ♠❛❧❡ ♣❡r❝❡♥t❛❣❡ ✵✳✽✻✸✹ ✵✳✻✺✾✼ ✵✳✷✷✸✽✯✯✯ ✵✳✶✽✾✻✯✯✯ ✵✳✶✺✷✾✯✯✯ ✵✳✷✷✹✹✯✯✯ ♦❢ ❈❤r✐st✐❛♥s ✭✵✳✵✵✵✶✮ ✭✵✳✵✵✷✼✮ ✭✵✳✵✵✶✶✮ ✭✵✳✵✵✵✵✮ ❢❡♠❛❧❡ ♣❡r❝❡♥t❛❣❡ ✵✳✾✵✶✶ ✵✳✻✾✶✵ ✵✳✷✸✾✻✯✯✯ ✵✳✷✶✾✶✯✯✯ ✵✳✶✽✻✸✯✯✯ ✵✳✷✺✸✸✯✯✯ ♦❢ ❈❤r✐st✐❛♥s ✭✵✳✵✵✵✵✮ ✭✵✳✵✵✷✵✮ ✭✵✳✵✵✵✸✮ ✭✵✳✵✵✵✵✮ ❖❜s❡r✈❛t✐♦♥s ✷✶ ✷✺ ✹✻ ✽✶ ✶✶✺ ✶✶✺

❈♦❤♦rt ❜♦r♥ ❜❡t✇❡❡♥ ✶✾✼✶ ❛♥❞ ✶✾✽✵✳ ❉❡♣❡♥❞❡♥t ✈❛r✐❛❜❧❡s ❛r❡ ♣❡r❝❡♥t❛❣❡s ❛t t❤❡ ❞✐str✐❝t ❧❡✈❡❧✳ ❉✐str✐❝ts ♦❢ ❉♦✉❛❧❛ ❛❧✇❛②s ❡①❝❧✉❞❡❞ ❢r♦♠ t❤❡ ❛♥❛❧②s✐s✳ ❆❧❧ r❡❣r❡ss✐♦♥s ❝♦♥tr♦❧ ❢♦r ✺ ❜♦r❞❡r s❡❣♠❡♥t ❞✉♠♠✐❡s✳ ❘♦❜✉st st❛♥❞❛r❞ ❡rr♦rs✳ ♣ ✈❛❧✉❡s ✐♥ ♣❛r❡♥t❤❡s❡s✳ ✯ s✐❣♥✐✜❝❛♥t ❛t t❤❡ ✶✵✪ ❧❡✈❡❧✳ ✯✯ ❛t t❤❡ ✺✪ ❧❡✈❡❧✳ ✯✯✯ ❛t t❤❡ ✶✪ ❧❡✈❡❧✳ r❡s✉❧ts ✉s✐♥❣ P❆❙❊❈ s❝❤♦♦❧ s✉r✈❡② ❞❛t❛

❘♦❜✉st♥❡ss ❝❤❡❝❦s

◮ ❘❡s✉❧ts r♦❜✉st t♦ ❛ ✈❛r✐❡t② ♦❢ s♣❡❝✐✜❝❛t✐♦♥s ✭❞✐✛❡r❡♥t

❜❛♥❞✇✐❞t❤ ❛♥❞ ♣♦❧②♥♦♠✐❛❧ ♦r❞❡rs✮✳

t❛❜❧❡s ✶✾✼✻ s❝❤♦♦❧ ❛tt❡♥❞❛♥❝❡ t❛❜❧❡ ✶✾✼✻ ♣r✐♠❛r② ❝♦♠♣❧❡t✐♦♥

◮ ❘❡s✉❧ts r♦❜✉st t♦ ✉s✐♥❣ ✶✾✽✼ ❝❡♥s✉s✳ ◮ ❙✐♠✐❧❛r r❡s✉❧ts ♦♥ t❤❡ ❈❡♥tr❛❧ ❜♦r❞❡r s❡❝t✐♦♥✳ ◮ P❧❛❝❡❜♦ ❜♦r❞❡r

P❧❛❝❡❜♦ ❜♦r❞❡rs

◮ ❙❡❧❡❝t✐♦♥ ❜② ♠♦rt❛❧✐t② ✉♥❧✐❦❡❧② t♦ ❡①♣❧❛✐♥ r❡s✉❧ts✳

❙❡❧❡❝t✐♦♥ ❜② ♠♦rt❛❧✐t②

▼❡❝❤❛♥✐s♠s

◮ ■ ✐❞❡♥t✐❢② t❤❡ ❢❛❝t ♦❢ r❛♥❞♦♠❧② ❢❛❧❧✐♥❣ ♦♥ ♦♥❡ s✐❞❡ ♦r t❤❡ ♦t❤❡r

♦❢ t❤❡ ❜♦r❞❡r✳ ❙❤❡❡r ❢❛❝t ♦❢ ❜❡✐♥❣ ❛tt❛❝❤❡ t♦ ❛ r✐❝❤❡r✴♣♦♦r❡r ❝♦❧♦♥② ♠✐❣❤t ❤❛✈❡ ❛♥ ✐♠♣❛❝t ✭❛❝❝❡ss t♦ ❞✐✛❡r❡♥t ♠❛r❦❡ts✱ r❡❞✐str✐❜✉t✐♦♥ t❤r♦✉❣❤ ♣✉❜❧✐❝ ❣♦♦❞s✮✳

◮ ❚❤❡ ♣❛rt✐t✐♦♥ ❢❛✈♦r❡❞ t❤❡ ❋r❡♥❝❤ s✐❞❡ ✭r❛✐❧✇❛②✱ ♣♦rt ♦❢

❉♦✉❛❧❛✮✳ ❈❛♠❡r♦♦♥✿ ♥❡❣❧❡❝t❡❞ ♣❛rt ♦❢ ◆✐❣❡r✐❛✳ ❙✐♠♣❧❡ ❞✐✛❡r❡♥❝❡s ✐♥ ♠❡❛♥s ❛❧✇❛②s ❢❛✈♦r t❤❡ ❋r❡♥❝❤ s✐❞❡✳ ❯♥❧✐❦❡❧✐♥❡ss ♦❢ ♠❡❝❤❛♥✐s♠s ✉♥r❡❧❛t❡❞ t♦ ♣♦❧✐❝✐❡s✳

◮ ■ ♠❛❦❡ t❤❡ ❝❛s❡ ❢♦r ❛♥❞ ❡❞✉❝❛t✐♦♥ s✉♣♣❧② ❝❤❛♥♥❡❧✿ t❤❡

❡❞✉❝❛t✐♦♥ ♣♦❧✐❝✐❡s ✐♠♣❧❡♠❡♥t❡❞ ❜② t❤❡ ❇r✐t✐s❤ ❛♥❞ t❤❡ ❋r❡♥❝❤ ✐♥ t❤❡✐r r❡s♣❡❝t✐✈❡ ♠❛♥❞❛t❡s ❝r❡❛t❡❞ ❛ ❞✐✈❡r❣❡♥❝❡ ✐♥ ❡❞✉❝❛t✐♦♥ ❛t t❤❡ ❜♦r❞❡r✱ ❞✐✈❡r❣❡♥❝❡ t❤❛t q✉✐❝❦❧② ❞✐s❛♣♣❡❛r❡❞ ✇❤❡♥ t❤❡ ❋r❡♥❝❤ st❛rt❡❞ ✐♥✈❡st✐♥❣ ✐♥ t❤❡ ❡❞✉❝❛t✐♦♥ s②st❡♠ ✐♥ t❤❡ ❧❛st ❞❡❝❛❞❡ ♦❢ ❝♦❧♦♥✐③❛t✐♦♥✳

❈♦❧♦♥✐❛❧ ❡❞✉❝❛t✐♦♥ ✐♥ ❋r❡♥❝❤ ❛♥❞ ❇r✐t✐s❤ ❈❛♠❡r♦♦♥s ✐♥ t❤❡ ✐♥t❡r✇❛r ♣❡r✐♦❞

◮ ❉✐✛❡r❡♥t ♣♦❧✐❝✐❡s t♦✇❛r❞s ✏❤❡❞❣❡ s❝❤♦♦❧s✑ ✭✈❡r② ❧♦✇ q✉❛❧✐t②

♠✐ss✐♦♥ s❝❤♦♦❧s✮✿

◮ ❇r✐t✐s❤ ❈❛♠❡r♦♦♥✿ ❝❧❡❛r ❞✐st✐♥❝t✐♦♥ ❜❡t✇❡❡♥ ❝❛t❡❝❤✐s♠s ✭❝♦✉❧❞

♥♦t ✉♥❞❡rt❛❦❡ ♥♦♥✲r❡❧✐❣✐♦✉s ❡❞✉❝❛t✐♦♥✮ ❛♥❞ ♦t❤❡r s❝❤♦♦❧s ✭✐♥s♣❡❝t❡❞ ❛♥❞ s✉❜s✐❞✐③❡❞ ♦♥ t❤❡ ❜❛s✐s ♦❢ ❡✣❝✐❡♥❝②✮✳ ✶✾✸✵✿ s✉❜s✐❞✐❡s r❡♣r❡s❡♥t❡❞ ✼✵✪ ♦❢ t❤❡ ✇❛❣❡ ❜✐❧❧ ♦❢ s✉❜s✐❞✐③❡❞ ♠✐ss✐♦♥ s❝❤♦♦❧s✳

◮ ❋r❡♥❝❤ ❈❛♠❡r♦♦♥✿ s✉❜s✐❞✐❡s t♦ ♣r✐✈❛t❡ s❝❤♦♦❧s ✇❤♦ ❢♦❧❧♦✇❡❞

✈❡r② str✐♥❣❡♥t r❡q✉✐r❡♠❡♥ts✿ ✷✳✸✪ ♦❢ t❤❡ ✇❛❣❡ ❜✐❧❧ ✐♥ ✶✾✸✵✳ ❖t❤❡r ♠✐ss✐♦♥ s❝❤♦♦❧s ❝♦♠♣❧❡t❡❧② ✉♥s✉♣❡r✈✐s❡❞ ✭♥♦t ❝❧♦s❡❞ ❞♦✇♥✮✿ ♠✐① ♦❢ ❝❛t❡❝❤✐s♠s ❛♥❞ ❢♦r♠❛❧ s❝❤♦♦❧s✳

◮ ❇♦t❤ ❝♦❧♦♥✐③❡rs ✐♥✈❡st ✐♥ ❛ s♠❛❧❧ ♣✉❜❧✐❝ s❡❝t♦r✳ ▼♦r❡ ❧♦❝❛❧ ✐♥

❇r✐t✐s❤ ❈❛♠❡r♦♦♥ ✭◆❛t✐✈❡ ❆❞♠✐♥✐str❛t✐♦♥ s❝❤♦♦❧s✱ ♥♦ ❇r✐t✐s❤ t❡❛❝❤❡rs✳✳✳✮

❚②♣❡s ♦❢ s❝❤♦♦❧s

❆❢t❡r ❲❲■■

◮ ❆❢t❡r ❲❲✷ ✭❇r❛③③❛✈✐❧❧❡ ❈♦♥❢❡r❡♥❝❡✮✱ ❡❞✉❝❛t✐♦♥❛❧ ❡✛♦rt

r❛❞✐❝❛❧❧② ✐♥❝r❡❛s❡❞ ✐♥ ❋r❡♥❝❤ ❈❛♠❡r♦♦♥✿

◮ ■♥❝r❡❛s❡ ✐♥ ♣r✐✈❛t❡ s❡❝t♦r s✉❜s✐❞✐❡s ✭✼✽✪ ♦❢ ♣r✐✈❛t❡ s❡❝t♦r✬s

✇❛❣❡ ❜✐❧❧ ✐♥ ✶✾✺✺✮✳

◮ ▼❛ss✐✈❡ ♣r♦❣r❛♠ t♦ ❜✉✐❧❞ ♣✉❜❧✐❝ s❝❤♦♦❧s ❛♥❞ ❤✐r❡ ♣✉❜❧✐❝ s❝❤♦♦❧

t❡❛❝❤❡rs✳

◮ ■♠♣♦rt❛♥❝❡ ♦❢ t❡❛❝❤✐♥❣ ✐♥ ❋r❡♥❝❤ str❡ss❡❞✳

◮ ■♥ ❇r✐t✐s❤ ❈❛♠❡r♦♦♥✱ s✉❜st❛♥t✐❛❧ ✐♥❝r❡❛s❡ ✐♥ ❣r❛♥ts✲✐♥✲❛✐❞✱

♠♦❞❡st ✐♥✈❡st♠❡♥t ✐♥ ♣✉❜❧✐❝ s❡❝t♦r✳

❉❛t❛ ♦♥ ✜♥❛♥❝✐♥❣ ❉✐s❝♦♥t✐♥✉✐t② ✐♥ ♥✉♠❜❡r ♦❢ s❝❤♦♦❧s ♣❡r s❝❤♦♦❧✲❛❣❡ ❝❤✐❧❞

❊①t❡r♥❛❧ ✈❛❧✐❞✐t②

◮ ✶✾✷✵s ❛♥❞ ✶✾✸✵s✿ ❈♦❣♥❡❛✉ ❛♥❞ ▼♦r❛❞✐ ✭✷✵✶✹✮ t❡❧❧ ❛ s✐♠✐❧❛r

st♦r② ❢♦r ❚♦❣♦✳ ❙❛♠❡ ❦✐♥❞ ♦❢ ❞✐✈❡r❣❡♥❝❡ ✐♥ ❡❞✉❝❛t✐♦♥ ✐♥ ✷ ✈❡r② ❞✐✛❡r❡♥t s❡tt✐♥❣s✳

◮ ▲❛t❡ ❝♦❧♦♥✐❛❧ ♣❡r✐♦❞✿ ✐♥❝r❡❛s❡ ✐♥ ❡❞✉❝❛t✐♦♥ ❡①♣❡♥❞✐t✉r❡ ✐♥ t❤❡

❋r❡♥❝❤ ♣❛rt ✐s ♥♦t ❛ ❈❛♠❡r♦♦♥✐❛♥ ✐❞✐♦s②♥❝r❛s② ✭❍✉✐❧❧❡r②✱ ✷✵✵✾✮✳ ❚♦❣♦ ✿ s♦♠❡ ♣❡rs✐st❡♥❝❡ ❢♦r ❝♦❤♦rts ❜♦r♥ ❜❡t✇❡❡♥ ✶✾✹✵ ❛♥❞ ✶✾✼✾✱ ❜✉t ♥♦t ✐♥ s❝❤♦♦❧ ❛tt❡♥❞❛♥❝❡✳ ❈ôt❡ ❞✬■✈♦✐r❡✴●❤❛♥❛✿ s✐♠✐❧❛r t♦ ❈❛♠❡r♦✉♥ ✭✐♥✈❡rs✐♦♥ ♦❢ t❤❡ ❞✐s❝♦♥t✐♥✉✐t② ❛❢t❡r ❲❲■■✮✳

◮ ❇❛❝❦ t♦ ❝r♦ss ❝♦✉♥tr②✿

❈r♦ss ❝♦✉♥tr②

❚❛❦❡ ❤♦♠❡ ♠❡ss❛❣❡

◮ ■♥ ❙✉❜✲❙❛❤❛r❛♥ ❆❢r✐❝❛✱ ❜❡❢♦r❡ ❲❲■■✱❇r✐t✐s❤ ❝♦❧♦♥✐❛❧ ❡❞✉❝❛t✐♦♥

✇❛s ♠♦r❡ ❡✣❝✐❡♥t ❛t r❡❛❝❤✐♥❣ ❛ ❧❛r❣❡ ♥✉♠❜❡r ♦❢ st✉❞❡♥ts✳ ■❞❡♥t✐✜❝❛t✐♦♥ ♦❢ ❛ ♣♦s✐t✐✈❡ ❇r✐t✐s❤ ❝♦❧♦♥✐③❡r ❡✛❡❝t ✐♥ ❛ s❡tt✐♥❣ ✇❤❡r❡ ♠❡❛♥ ❞✐✛❡r❡♥❝❡ ❢❛✈♦r❡❞ t❤❡ ❋r❡♥❝❤ s✐❞❡ ♦❢ t❤❡ ❜♦r❞❡r✳

◮ ◆♦t ❧❛✐ss❡③✲❢❛✐r❡ ❱❙ ❝♦❧♦♥✐❛❧ ❣♦✈❡r♥♠❡♥t ❝♦♥tr♦❧ ❜✉t

❣♦✈❡r♥♠❡♥t ♣r♦✈✐❞✐♥❣ r✐❣❤t ✐♥❝❡♥t✐✈❡s t♦ ♠✐ss✐♦♥s ✭s❡❡ ❇❡❧❣✐❛♥ ❛♥❞ P♦rt✉❣✉❡s❡ ❝♦❧♦♥✐❡s✮✳

◮ ❉❡❝♦♠♣r❡ss✐♥❣ ❤✐st♦r②✿ ❝♦❧♦♥✐❛❧ ♣♦❧✐❝✐❡s ❡✈♦❧✈❡❞ ❞✉r✐♥❣ t❤❡

❝♦❧♦♥✐❛❧ ♣❡r✐♦❞✳ ❚❤❡ ♣♦s✐t✐✈❡ ❇r✐t✐s❤ ❝♦❧♦♥✐③❡r ❡✛❡❝t q✉✐❝❦❧② ❞✐s❛♣♣❡❛rs ❛❢t❡r ❲❲■■ ✇❤❡♥ t❤❡ ❋r❡♥❝❤ st❛rt ✐♥✈❡st✐♥❣ ✐♥ t❤❡ ❡❞✉❝❛t✐♦♥ s②st❡♠✳ ❚❤✐s ♣❤❡♥♦♠❡♥♦♥ ❤❛s s♦♠❡ ❡①t❡r♥❛❧ ✈❛❧✐❞✐t②✳

◮ ❘❡❧✐❣✐♦♥ ✐s ❢♦✉♥❞ t♦ ❜❡ t❤❡ ❢❡❛t✉r❡ ♦❢ t❤❡ ✐♥✐t✐❛❧ s❤♦❝❦ t❤❛t

♣❡rs✐st❡❞ t❤❡ ♠♦st✳

❆♣♣❡♥❞✐❝❡s

❈♦❧♦♥✐❛❧ ❡❞✉❝❛t✐♦♥ ✐♥ ❋r❡♥❝❤ ❛♥❞ ❇r✐t✐s❤ ❈❛♠❡r♦♦♥s ✐♥ t❤❡ ✐♥t❡r✇❛r ♣❡r✐♦❞

❚❛❜❧❡✿ ❉✐✛❡r❡♥t t②♣❡s ♦❢ s❝❤♦♦❧s ✐♥ t❤❡ ✷ ❈❛♠❡r♦♦♥s

❋r❡♥❝❤ ❈❛♠❡r♦♦♥ ❇r✐t✐s❤ ❈❛♠❡r♦♦♥ ❈❛t❡❝❤✐s♠s ❤❡❞❣❡ s❝❤♦♦❧s é❝♦❧❡s ♥♦♥ r❡❝♦♥♥✉❡s ❋♦r♠❛❧ ♣r✐✈❛t❡ s❝❤♦♦❧s ✉♥❛ss✐st❡❞ ❛ss✐st❡❞ é❝♦❧❡s r❡❝♦♥♥✉❡s P✉❜❧✐❝ s❝❤♦♦❧s ◆✳❆✳ s❝❤♦♦❧s é❝♦❧❡s ♦✣❝✐❡❧❧❡s ❣♦✈❡r♥♠❡♥t s❝❤♦♦❧s

❇❛❝❦ t♦ ❝♦❧♦♥✐❛❧ ❡❞✉❝❛t✐♦♥

❆❣❡ ❤❡❛♣✐♥❣ ✐♥ t❤❡ ✶✾✼✻ ❝❡♥s✉s

❇❛❝❦ t♦ ❝❡♥s✉s ❞❛t❛

❙♦✉t❤❡r♥ ❜♦r❞❡r s❡❝t✐♦♥✿ ♣❡r❝❡♥t❛❣❡ ♦❢ ♠❡♥ ✇❤♦ ❛tt❡♥❞❡❞ s❝❤♦♦❧ ✭✶✮

✭✶✮ ✭✷✮ ✭✸✮ ✭✹✮ ✭✺✮ ✭✻✮ ✭✼✮ ♠❡❛♥s ♦♥ ✶✵✲❦♠ ❡st✐♠❛t❡❞ ❞✐s❝♦♥t✐♥✉✐t✐❡s ❜❛♥❞✇✐❞t❤ ❝♦❤♦rt ✶✵✲❦♠ ❜❛♥❞✇✐❞t❤ ✷✺✲❦♠ ❜✇ ✺✵✲❦♠ ❜✇ ❜♦r♥ ❇r✐t✐s❤ ❋r❡♥❝❤ ♥♦ ✭①✱②✮ ♣♦❧②♥♦♠✐❛❧ ♣♦❧②♥♦♠✐❛❧ ♣♦❧②♥♦♠✐❛❧ ♣♦❧②♥♦♠✐❛❧ ❜❡t✇❡❡♥ s✐❞❡ s✐❞❡ ❝♦♥tr♦❧s ♦❢ ♦r❞❡r ✶ ♦❢ ♦r❞❡r ✷ ♦❢ ♦r❞❡r ✸ ♦❢ ♦r❞❡r ✺ ✶✽✽✷ ✫ ✵✳✵✹✵✼ ✵✳✵✽✹✻ ✲✵✳✵✷✼✻ ✲✵✳✵✺✵✶ ✲✵✳✵✶✻✸ ✲✵✳✵✷✸✽ ✲✵✳✵✺✼✹ ✶✽✾✶ ✭✵✳✺✺✽✸✮ ✭✵✳✶✵✹✹✮ ✭✵✳✹✻✼✶✮ ✭✵✳✺✽✺✸✮ ✭✵✳✷✾✾✹✮ ❬✷✸❪ ❬✼❪ ❬✸✵❪ ❬✸✵❪ ❬✸✵❪ ❬✺✾❪ ❬✽✼❪ ✶✽✾✷ ✫ ✵✳✶✷✽✸ ✵✳✶✷✺✸ ✵✳✵✵✵✸ ✲✵✳✵✶✶✶ ✲✵✳✵✵✾✷ ✵✳✵✶✽✷ ✲✵✳✵✽✸✸ ✶✾✵✶ ✭✵✳✾✾✼✵✮ ✭✵✳✽✾✻✻✮ ✭✵✳✾✷✶✸✮ ✭✵✳✽✷✼✽✮ ✭✵✳✷✽✻✷✮ ❬✸✷❪ ❬✷✹❪ ❬✺✻❪ ❬✺✻❪ ❬✺✻❪ ❬✶✷✸❪ ❬✶✽✹❪ ✶✾✵✷ ✫ ✵✳✶✼✸✸ ✵✳✶✼✽✻ ✵✳✵✶✸✻ ✵✳✵✺✶✻ ✵✳✵✺✸✾ ✵✳✵✸✽✶ ✲✵✳✵✵✺✾ ✶✾✶✶ ✭✵✳✽✵✵✻✮ ✭✵✳✸✵✽✽✮ ✭✵✳✸✷✾✷✮ ✭✵✳✸✺✾✷✮ ✭✵✳✽✼✾✶✮ ❬✸✾❪ ❬✸✷❪ ❬✼✶❪ ❬✼✶❪ ❬✼✶❪ ❬✶✻✵❪ ❬✷✺✼❪

❘♦❜✉st st❛♥❞❛r❞ ❡rr♦rs✳ ♣ ✈❛❧✉❡s ✐♥ ♣❛r❡♥t❤❡s❡s✳ ◆✉♠❜❡r ♦❢ ✈✐❧❧❛❣❡s ✐♥ t❤❡ r❡❣r❡ss✐♦♥ ❜❡t✇❡❡♥ ❜r❛❝❡s✳ ✯ s✐❣♥✐✜❝❛♥t ❛t t❤❡ ✶✵✪ ❧❡✈❡❧✳ ✯✯ ❛t t❤❡ ✺✪ ❧❡✈❡❧✳ ✯✯✯ ❛t t❤❡ ✶✪ ❧❡✈❡❧✳

❜❛❝❦ t♦ r♦❜✉st♥❡ss

❙♦✉t❤❡r♥ ❜♦r❞❡r s❡❝t✐♦♥✿ ♣❡r❝❡♥t❛❣❡ ♦❢ ♠❡♥ ✇❤♦ ❛tt❡♥❞❡❞ s❝❤♦♦❧ ✭✷✮

✭✶✮ ✭✷✮ ✭✸✮ ✭✹✮ ✭✺✮ ✭✻✮ ✭✼✮ ♠❡❛♥s ♦♥ ✶✵✲❦♠ ❡st✐♠❛t❡❞ ❞✐s❝♦♥t✐♥✉✐t✐❡s ❜❛♥❞✇✐❞t❤ ❝♦❤♦rt ✶✵✲❦♠ ❜❛♥❞✇✐❞t❤ ✷✺✲❦♠ ❜✇ ✺✵✲❦♠ ❜✇ ❜♦r♥ ❇r✐t✐s❤ ❋r❡♥❝❤ ♥♦ ✭①✱②✮ ♣♦❧②♥♦♠✐❛❧ ♣♦❧②♥♦♠✐❛❧ ♣♦❧②♥♦♠✐❛❧ ♣♦❧②♥♦♠✐❛❧ ❜❡t✇❡❡♥ s✐❞❡ s✐❞❡ ❝♦♥tr♦❧s ♦❢ ♦r❞❡r ✶ ♦❢ ♦r❞❡r ✷ ♦❢ ♦r❞❡r ✸ ♦❢ ♦r❞❡r ✺ ✶✾✶✷ ✫ ✵✳✸✷✸✽ ✵✳✷✹✷✸ ✵✳✵✽✾✺✯✯ ✵✳✶✸✻✹✯✯✯ ✵✳✶✸✸✽✯✯✯ ✵✳✵✾✺✻✯✯ ✵✳✵✾✵✸✯✯ ✶✾✷✶ ✭✵✳✵✸✷✵✮ ✭✵✳✵✵✶✸✮ ✭✵✳✵✵✷✷✮ ✭✵✳✵✶✶✸✮ ✭✵✳✵✶✺✹✮ ❬✹✽❪ ❬✸✾❪ ❬✽✼❪ ❬✽✼❪ ❬✽✼❪ ❬✶✽✼❪ ❬✸✵✹❪ ✶✾✷✷ ✫ ✵✳✺✵✾✸ ✵✳✹✸✻✾ ✵✳✵✼✼✺✯ ✵✳✶✶✸✾✯✯✯ ✵✳✶✶✶✶✯✯ ✵✳✵✾✶✶✯✯ ✵✳✵✼✼✵✯✯ ✶✾✸✶ ✭✵✳✵✺✾✻✮ ✭✵✳✵✵✻✶✮ ✭✵✳✵✶✸✻✮ ✭✵✳✵✶✷✶✮ ✭✵✳✵✹✾✸✮ ❬✺✷❪ ❬✹✵❪ ❬✾✷❪ ❬✾✷❪ ❬✾✷❪ ❬✶✾✽❪ ❬✸✷✶❪

❘♦❜✉st st❛♥❞❛r❞ ❡rr♦rs✳ ♣ ✈❛❧✉❡s ✐♥ ♣❛r❡♥t❤❡s❡s✳ ◆✉♠❜❡r ♦❢ ✈✐❧❧❛❣❡s ✐♥ t❤❡ r❡❣r❡ss✐♦♥ ❜❡t✇❡❡♥ ❜r❛❝❡s✳ ✯ s✐❣♥✐✜❝❛♥t ❛t t❤❡ ✶✵✪ ❧❡✈❡❧✳ ✯✯ ❛t t❤❡ ✺✪ ❧❡✈❡❧✳ ✯✯✯ ❛t t❤❡ ✶✪ ❧❡✈❡❧✳

❜❛❝❦ t♦ r♦❜✉st♥❡ss

❙♦✉t❤❡r♥ ❜♦r❞❡r s❡❝t✐♦♥✿ ♣❡r❝❡♥t❛❣❡ ♦❢ ♠❡♥ ✇❤♦ ❛tt❡♥❞❡❞ s❝❤♦♦❧ ✭✸✮

✭✶✮ ✭✷✮ ✭✸✮ ✭✹✮ ✭✺✮ ✭✻✮ ✭✼✮ ♠❡❛♥s ♦♥ ✶✵✲❦♠ ❡st✐♠❛t❡❞ ❞✐s❝♦♥t✐♥✉✐t✐❡s ❜❛♥❞✇✐❞t❤ ❝♦❤♦rt ✶✵✲❦♠ ❜❛♥❞✇✐❞t❤ ✷✺✲❦♠ ❜✇ ✺✵✲❦♠ ❜✇ ❜♦r♥ ❇r✐t✐s❤ ❋r❡♥❝❤ ♥♦ ✭①✱②✮ ♣♦❧②♥♦♠✐❛❧ ♣♦❧②♥♦♠✐❛❧ ♣♦❧②♥♦♠✐❛❧ ♣♦❧②♥♦♠✐❛❧ ❜❡t✇❡❡♥ s✐❞❡ s✐❞❡ ❝♦♥tr♦❧s ♦❢ ♦r❞❡r ✶ ♦❢ ♦r❞❡r ✷ ♦❢ ♦r❞❡r ✸ ♦❢ ♦r❞❡r ✺ ✶✾✸✷ ✫ ✵✳✼✵✼✵ ✵✳✼✵✻✵ ✵✳✵✵✾✶ ✵✳✵✺✶✺ ✵✳✵✹✻✸ ✵✳✵✹✻✽ ✵✳✵✸✷✺ ✶✾✹✶ ✭✵✳✼✾✼✽✮ ✭✵✳✶✹✺✹✮ ✭✵✳✶✼✵✵✮ ✭✵✳✶✷✺✻✮ ✭✵✳✷✽✶✼✮ ❬✺✸❪ ❬✹✸❪ ❬✾✻❪ ❬✾✻❪ ❬✾✻❪ ❬✷✵✹❪ ❬✸✷✺❪ ✶✾✹✷ ✫ ✵✳✽✼✼✹ ✵✳✾✸✸✺ ✲✵✳✵✹✾✺✯✯ ✲✵✳✵✵✶✹ ✲✵✳✵✵✻✶ ✲✵✳✵✶✺✸ ✲✵✳✵✷✷✻ ✶✾✺✶ ✭✵✳✵✶✺✷✮ ✭✵✳✾✷✽✼✮ ✭✵✳✻✼✺✼✮ ✭✵✳✷✺✻✻✮ ✭✵✳✶✶✵✸✮ ❬✺✻❪ ❬✹✹❪ ❬✶✵✵❪ ❬✶✵✵❪ ❬✶✵✵❪ ❬✷✶✷❪ ❬✸✸✵❪ ✶✾✺✷ ✫ ✵✳✾✻✽✷ ✵✳✾✼✸✾ ✵✳✵✵✵✸ ✵✳✵✵✹✶ ✵✳✵✵✷✵ ✲✵✳✵✵✵✹ ✲✵✳✵✵✻✵ ✶✾✻✶ ✭✵✳✾✺✵✼✮ ✭✵✳✺✶✻✻✮ ✭✵✳✼✼✵✼✮ ✭✵✳✾✹✷✸✮ ✭✵✳✷✾✶✶✮ ❬✺✼❪ ❬✹✻❪ ❬✶✵✸❪ ❬✶✵✸❪ ❬✶✵✸❪ ❬✷✷✵❪ ❬✸✹✹❪

❘♦❜✉st st❛♥❞❛r❞ ❡rr♦rs✳ ♣ ✈❛❧✉❡s ✐♥ ♣❛r❡♥t❤❡s❡s✳ ◆✉♠❜❡r ♦❢ ✈✐❧❧❛❣❡s ✐♥ t❤❡ r❡❣r❡ss✐♦♥ ❜❡t✇❡❡♥ ❜r❛❝❡s✳ ✯ s✐❣♥✐✜❝❛♥t ❛t t❤❡ ✶✵✪ ❧❡✈❡❧✳ ✯✯ ❛t t❤❡ ✺✪ ❧❡✈❡❧✳ ✯✯✯ ❛t t❤❡ ✶✪ ❧❡✈❡❧✳

❜❛❝❦ t♦ r♦❜✉st♥❡ss

❙♦✉t❤❡r♥ ❜♦r❞❡r s❡❝t✐♦♥✿ ♣❡r❝❡♥t❛❣❡ ♦❢ ♠❡♥ ✇❤♦ ❝♦♠♣❧❡t❡❞ ♣r✐♠❛r② ✭✶✮

✭✶✮ ✭✷✮ ✭✸✮ ✭✹✮ ✭✺✮ ✭✻✮ ✭✼✮ ♠❡❛♥s ♦♥ ✶✵✲❦♠ ❡st✐♠❛t❡❞ ❞✐s❝♦♥t✐♥✉✐t✐❡s ❜❛♥❞✇✐❞t❤ ❝♦❤♦rt ✶✵✲❦♠ ❜❛♥❞✇✐❞t❤ ✷✺✲❦♠ ❜✇ ✺✵✲❦♠ ❜✇ ❜♦r♥ ❇r✐t✐s❤ ❋r❡♥❝❤ ♥♦ ✭①✱②✮ ♣♦❧②♥♦♠✐❛❧ ♣♦❧②♥♦♠✐❛❧ ♣♦❧②♥♦♠✐❛❧ ♣♦❧②♥♦♠✐❛❧ ❜❡t✇❡❡♥ s✐❞❡ s✐❞❡ ❝♦♥tr♦❧s ♦❢ ♦r❞❡r ✶ ♦❢ ♦r❞❡r ✷ ♦❢ ♦r❞❡r ✸ ♦❢ ♦r❞❡r ✺ ✶✽✽✷ ✫ ✵✳✵✵✷✷ ✵✳✵✵✺✻ ✲✵✳✵✵✶✷ ✲✵✳✵✵✷✷ ✲✵✳✵✵✵✽ ✲✵✳✵✶✾✶ ✲✵✳✵✷✵✼ ✶✽✾✶ ✭✵✳✻✶✷✾✮ ✭✵✳✶✺✽✻✮ ✭✵✳✺✻✺✽✮ ✭✵✳✹✶✵✻✮ ✭✵✳✸✼✸✼✮ ❬✷✸❪ ❬✼❪ ❬✸✵❪ ❬✸✵❪ ❬✸✵❪ ❬✺✾❪ ❬✽✼❪ ✶✽✾✷ ✫ ✵✳✵✷✼✸ ✵✳✵✷✹✹ ✵✳✵✵✷✺ ✵✳✵✵✻✽ ✵✳✵✶✶✹ ✵✳✵✶✷✶ ✲✵✳✵✵✵✸ ✶✾✵✶ ✭✵✳✽✻✸✽✮ ✭✵✳✼✸✵✶✮ ✭✵✳✺✻✽✹✮ ✭✵✳✻✺✹✸✮ ✭✵✳✾✽✽✽✮ ❬✸✷❪ ❬✷✹❪ ❬✺✻❪ ❬✺✻❪ ❬✺✻❪ ❬✶✷✸❪ ❬✶✽✹❪ ✶✾✵✷ ✫ ✵✳✵✹✹✸ ✵✳✵✸✼✺ ✵✳✵✶✷✵ ✵✳✵✶✽✻ ✵✳✵✷✵✺ ✵✳✵✷✶✼ ✵✳✵✵✸✻ ✶✾✶✶ ✭✵✳✹✶✾✼✮ ✭✵✳✹✹✺✵✮ ✭✵✳✹✷✷✵✮ ✭✵✳✶✽✶✾✮ ✭✵✳✽✶✹✻✮ ❬✸✾❪ ❬✸✷❪ ❬✼✶❪ ❬✼✶❪ ❬✼✶❪ ❬✶✻✵❪ ❬✷✺✼❪

❘♦❜✉st st❛♥❞❛r❞ ❡rr♦rs✳ ♣ ✈❛❧✉❡s ✐♥ ♣❛r❡♥t❤❡s❡s✳ ◆✉♠❜❡r ♦❢ ✈✐❧❧❛❣❡s ✐♥ t❤❡ r❡❣r❡ss✐♦♥ ❜❡t✇❡❡♥ ❜r❛❝❡s✳ ✯ s✐❣♥✐✜❝❛♥t ❛t t❤❡ ✶✵✪ ❧❡✈❡❧✳ ✯✯ ❛t t❤❡ ✺✪ ❧❡✈❡❧✳ ✯✯✯ ❛t t❤❡ ✶✪ ❧❡✈❡❧✳

❜❛❝❦ t♦ r♦❜✉st♥❡ss

❙♦✉t❤❡r♥ ❜♦r❞❡r s❡❝t✐♦♥✿ ♣❡r❝❡♥t❛❣❡ ♦❢ ♠❡♥ ✇❤♦ ❝♦♠♣❧❡t❡❞ ♣r✐♠❛r② ✭✷✮

✭✶✮ ✭✷✮ ✭✸✮ ✭✹✮ ✭✺✮ ✭✻✮ ✭✼✮ ♠❡❛♥s ♦♥ ✶✵✲❦♠ ❡st✐♠❛t❡❞ ❞✐s❝♦♥t✐♥✉✐t✐❡s ❜❛♥❞✇✐❞t❤ ❝♦❤♦rt ✶✵✲❦♠ ❜❛♥❞✇✐❞t❤ ✷✺✲❦♠ ❜✇ ✺✵✲❦♠ ❜✇ ❜♦r♥ ❇r✐t✐s❤ ❋r❡♥❝❤ ♥♦ ✭①✱②✮ ♣♦❧②♥♦♠✐❛❧ ♣♦❧②♥♦♠✐❛❧ ♣♦❧②♥♦♠✐❛❧ ♣♦❧②♥♦♠✐❛❧ ❜❡t✇❡❡♥ s✐❞❡ s✐❞❡ ❝♦♥tr♦❧s ♦❢ ♦r❞❡r ✶ ♦❢ ♦r❞❡r ✷ ♦❢ ♦r❞❡r ✸ ♦❢ ♦r❞❡r ✺ ✶✾✶✷ ✫ ✵✳✵✼✽✽ ✵✳✵✺✹✶ ✵✳✵✷✽✺ ✵✳✵✺✶✶✯✯✯ ✵✳✵✹✾✽✯✯✯ ✵✳✵✺✺✶✯✯✯ ✵✳✵✹✸✶✯✯✯ ✶✾✷✶ ✭✵✳✶✸✶✵✮ ✭✵✳✵✵✷✶✮ ✭✵✳✵✵✹✺✮ ✭✵✳✵✵✶✶✮ ✭✵✳✵✵✾✼✮ ❬✹✽❪ ❬✸✾❪ ❬✽✼❪ ❬✽✼❪ ❬✽✼❪ ❬✶✽✼❪ ❬✸✵✹❪ ✶✾✷✷ ✫ ✵✳✷✵✹✾ ✵✳✶✸✹✼ ✵✳✵✼✹✾✯✯✯ ✵✳✵✼✽✹✯✯✯ ✵✳✵✽✵✶✯✯✯ ✵✳✵✾✻✹✯✯✯ ✵✳✵✽✷✶✯✯✯ ✶✾✸✶ ✭✵✳✵✵✵✽✮ ✭✵✳✵✵✵✵✮ ✭✵✳✵✵✵✵✮ ✭✵✳✵✵✵✵✮ ✭✵✳✵✵✵✵✮ ❬✺✷❪ ❬✹✵❪ ❬✾✷❪ ❬✾✷❪ ❬✾✷❪ ❬✶✾✽❪ ❬✸✷✶❪

❘♦❜✉st st❛♥❞❛r❞ ❡rr♦rs✳ ♣ ✈❛❧✉❡s ✐♥ ♣❛r❡♥t❤❡s❡s✳ ◆✉♠❜❡r ♦❢ ✈✐❧❧❛❣❡s ✐♥ t❤❡ r❡❣r❡ss✐♦♥ ❜❡t✇❡❡♥ ❜r❛❝❡s✳ ✯ s✐❣♥✐✜❝❛♥t ❛t t❤❡ ✶✵✪ ❧❡✈❡❧✳ ✯✯ ❛t t❤❡ ✺✪ ❧❡✈❡❧✳ ✯✯✯ ❛t t❤❡ ✶✪ ❧❡✈❡❧✳

❜❛❝❦ t♦ r♦❜✉st♥❡ss

❙♦✉t❤❡r♥ ❜♦r❞❡r s❡❝t✐♦♥✿ ♣❡r❝❡♥t❛❣❡ ♦❢ ♠❡♥ ✇❤♦ ❝♦♠♣❧❡t❡❞ ♣r✐♠❛r② ✭✸✮

✭✶✮ ✭✷✮ ✭✸✮ ✭✹✮ ✭✺✮ ✭✻✮ ✭✼✮ ♠❡❛♥s ♦♥ ✶✵✲❦♠ ❡st✐♠❛t❡❞ ❞✐s❝♦♥t✐♥✉✐t✐❡s ❜❛♥❞✇✐❞t❤ ❝♦❤♦rt ✶✵✲❦♠ ❜❛♥❞✇✐❞t❤ ✷✺✲❦♠ ❜✇ ✺✵✲❦♠ ❜✇ ❜♦r♥ ❇r✐t✐s❤ ❋r❡♥❝❤ ♥♦ ✭①✱②✮ ♣♦❧②♥♦♠✐❛❧ ♣♦❧②♥♦♠✐❛❧ ♣♦❧②♥♦♠✐❛❧ ♣♦❧②♥♦♠✐❛❧ ❜❡t✇❡❡♥ s✐❞❡ s✐❞❡ ❝♦♥tr♦❧s ♦❢ ♦r❞❡r ✶ ♦❢ ♦r❞❡r ✷ ♦❢ ♦r❞❡r ✸ ♦❢ ♦r❞❡r ✺ ✶✾✸✷ ✫ ✵✳✸✻✸✸ ✵✳✸✻✵✺ ✵✳✵✶✷✹ ✵✳✵✶✼✽ ✵✳✵✷✵✵ ✵✳✵✹✸✷ ✵✳✵✶✶✶ ✶✾✹✶ ✭✵✳✻✽✷✼✮ ✭✵✳✹✾✻✺✮ ✭✵✳✹✸✻✵✮ ✭✵✳✶✹✽✼✮ ✭✵✳✻✾✾✽✮ ❬✺✸❪ ❬✹✸❪ ❬✾✻❪ ❬✾✻❪ ❬✾✻❪ ❬✷✵✹❪ ❬✸✷✺❪ ✶✾✹✷ ✫ ✵✳✻✺✵✷ ✵✳✼✻✾✵ ✲✵✳✶✵✾✻✯✯✯ ✲✵✳✵✷✻✻ ✲✵✳✵✸✺✸ ✲✵✳✵✸✻✾ ✲✵✳✵✻✷✸✯✯✯ ✶✾✺✶ ✭✵✳✵✵✶✶✮ ✭✵✳✷✸✽✾✮ ✭✵✳✶✷✽✾✮ ✭✵✳✶✵✵✸✮ ✭✵✳✵✵✼✻✮ ❬✺✻❪ ❬✹✹❪ ❬✶✵✵❪ ❬✶✵✵❪ ❬✶✵✵❪ ❬✷✶✷❪ ❬✸✸✵❪ ✶✾✺✷ ✫ ✵✳✼✹✵✼ ✵✳✽✶✵✻ ✲✵✳✵✺✹✾✯✯✯ ✲✵✳✵✷✸✹ ✲✵✳✵✸✶✼✯ ✲✵✳✵✸✼✾✯✯ ✲✵✳✵✻✸✵✯✯✯ ✶✾✻✶ ✭✵✳✵✵✺✷✮ ✭✵✳✷✷✻✶✮ ✭✵✳✵✾✷✵✮ ✭✵✳✵✶✻✼✮ ✭✵✳✵✵✵✷✮ ❬✺✼❪ ❬✹✻❪ ❬✶✵✸❪ ❬✶✵✸❪ ❬✶✵✸❪ ❬✷✷✵❪ ❬✸✹✹❪

❘♦❜✉st st❛♥❞❛r❞ ❡rr♦rs✳ ♣ ✈❛❧✉❡s ✐♥ ♣❛r❡♥t❤❡s❡s✳ ◆✉♠❜❡r ♦❢ ✈✐❧❧❛❣❡s ✐♥ t❤❡ r❡❣r❡ss✐♦♥ ❜❡t✇❡❡♥ ❜r❛❝❡s✳ ✯ s✐❣♥✐✜❝❛♥t ❛t t❤❡ ✶✵✪ ❧❡✈❡❧✳ ✯✯ ❛t t❤❡ ✺✪ ❧❡✈❡❧✳ ✯✯✯ ❛t t❤❡ ✶✪ ❧❡✈❡❧✳

❜❛❝❦ t♦ r♦❜✉st♥❡ss

❘❡♣❡t✐t✐♦♥ ✐s ♠♦r❡ ♣r❡✈❛❧❡♥t ✐♥ t❤❡ ❋r❡♥❝❤ s②st❡♠ ✭P❆❙❊❈ ✷✵✵✹✲✷✵✵✺ s❝❤♦♦❧ s✉r✈❡②✮

✭✶✮ ✭✷✮ ✭✸✮ ✭✹✮ ✭✺✮ ✭✻✮ ♠❡❛♥s ♦♥ ✺✵✲❦♠ ❡st✐♠❛t❡❞ ❞✐s❝♦♥t✐♥✉✐t✐❡s ❜❛♥❞✇✐❞t❤ ❊♥❣❧✐s❤✲ ❋r❡♥❝❤✲ ✺✵✲❦♠ ❜❛♥❞✇✐❞t❤ ✶✵✵✲❦♠ ❜✇ ✷✵✵✲❦♠ ❜✇ s♣❡❛❦✐♥❣ s♣❡❛❦✐♥❣ ♥♦ ✭①✱②✮ ♣♦❧②♥♦♠✐❛❧ ♣♦❧②♥♦♠✐❛❧ ♣♦❧②♥♦♠✐❛❧ s✐❞❡ s✐❞❡ ❝♦♥tr♦❧s ♦❢ ♦r❞❡r ✶ ♦❢ ♦r❞❡r ✷ ♦❢ ♦r❞❡r ✸ ❛❣❡ ✐♥ ❝❧❛ss ✷ ✻✳✼✷✽✸ ✻✳✾✾✷✼ ✲✵✳✹✶✸✽✯ ✲✵✳✹✺✺✵ ✲✵✳✷✼✻✵ ✲✵✳✷✺✷✽ ✭✵✳✵✼✸✻✮ ✭✵✳✷✺✽✹✮ ✭✵✳✹✽✷✾✮ ✭✵✳✹✷✷✺✮ ❛❣❡ ✐♥ ❝❧❛ss ✺ ✶✵✳✹✵✺✹ ✶✵✳✽✹✹✸ ✲✵✳✺✼✸✶✯✯ ✲✵✳✼✼✶✺✯ ✲✵✳✺✹✹✸✯ ✲✵✳✼✺✵✷✯✯ ✭✵✳✵✷✼✷✮ ✭✵✳✵✺✸✹✮ ✭✵✳✵✽✷✶✮ ✭✵✳✵✸✷✵✮ ❤❛s ❡✈❡r r❡♣❡❛t❡❞ ✵✳✹✵✸✽ ✵✳✹✻✾✼ ✲✵✳✷✵✵✹✯✯✯ ✵✳✵✸✽✽ ✲✵✳✶✶✷✽ ✲✵✳✵✾✶✼ ✭✇❤❡♥ ✐♥ ❝❧❛ss ✷✮ ✭✵✳✵✵✸✻✮ ✭✵✳✼✺✵✺✮ ✭✵✳✸✾✾✹✮ ✭✵✳✺✷✸✺✮ ❤❛s ❡✈❡r r❡♣❡❛t❡❞ ✵✳✺✺✽✵ ✵✳✻✽✾✷ ✲✵✳✶✺✾✾✯✯ ✲✵✳✶✽✺✽✯ ✲✵✳✶✷✵✼ ✲✵✳✶✼✸✸✯✯ ✭✇❤❡♥ ✐♥ ❝❧❛ss ✺✮ ✭✵✳✵✶✵✽✮ ✭✵✳✵✼✺✽✮ ✭✵✳✶✻✶✼✮ ✭✵✳✵✷✷✽✮ ◆✉♠❜❡r ♦❢ s❝❤♦♦❧s ✷✹ ✷✹ ✹✽ ✹✽ ✼✵ ✽✷

❘♦❜✉st st❛♥❞❛r❞ ❡rr♦rs ❝❧✉st❡r ❛t t❤❡ s❝❤♦♦❧ ❧❡✈❡❧✳ ♣ ✈❛❧✉❡s ✐♥ ♣❛r❡♥t❤❡s❡s✳ ✯ s✐❣♥✐✜❝❛♥t ❛t t❤❡ ✶✵✪ ❧❡✈❡❧✳ ✯✯ ❛t t❤❡ ✺✪ ❧❡✈❡❧✳ ✯✯✯ ❛t t❤❡ ✶✪ ❧❡✈❡❧✳ ❆❧❧ r❡❣r❡ss✐♦♥s ❝♦♥tr♦❧ ❢♦r ❣❡♥❞❡r✳ ❆❧❧ r❡❣r❡ss✐♦♥s✱ ❝♦♥tr♦❧ ❢♦r ✺ ❜♦r❞❡r s❡❣♠❡♥t ❞✉♠♠✐❡s✳ ❝❡♥s✉s✷✵✵✺

P❧❛❝❡❜♦ ❜♦r❞❡rs

❋✐❣✉r❡✿ ♠❛❧❡ s❝❤♦♦❧ ❛tt❡♥❞❛♥❝❡✱ ❝♦❤♦rt ✶✾✷✷✲✶✾✸✶

❜❛❝❦ t♦ r♦❜✉st♥❡ss

P❧❛❝❡❜♦ ❜♦r❞❡rs

❋✐❣✉r❡✿ ♠❛❧❡ ♣r✐♠❛r② s❝❤♦♦❧ ❝♦♠♣❧❡t✐♦♥✱ ❝♦❤♦rt ✶✾✷✷✲✶✾✸✶

❜❛❝❦ t♦ r♦❜✉st♥❡ss

P❧❛❝❡❜♦ ❜♦r❞❡rs

❋✐❣✉r❡✿ ♠❛❧❡ s❡❝♦♥❞❛r② s❝❤♦♦❧ ❝♦♠♣❧❡t✐♦♥✱ ❝♦❤♦rt ✶✾✼✶✲✶✾✽✵

❜❛❝❦ t♦ r♦❜✉st♥❡ss

P❧❛❝❡❜♦ ❜♦r❞❡rs

❋✐❣✉r❡✿ ♠❛❧❡ ♣❡r❝❡♥t❛❣❡ ♦❢ ❈❤r✐st✐❛♥s✱ ❝♦❤♦rt ✶✾✼✶✲✶✾✽✵

❜❛❝❦ t♦ r♦❜✉st♥❡ss

❙❡❧❡❝t✐♦♥ ❜② ♠♦rt❛❧✐t② ❛♥❞ t❤❡ ❯P❈ ✇❛r

◮ ❙❡❧❡❝t✐♦♥ ❜② ♠♦rt❛❧✐t② ✐s ❛ ❝♦♥❝❡r♥ ✐❢ ♠♦rt❛❧✐t② r❛t❡s ✇❡r❡

❞✐✛❡r❡♥t ♦♥ ❜♦t❤ s✐❞❡s ♦❢ t❤❡ ❜♦r❞❡r ❛♥❞ ♠♦rt❛❧✐t② s❡❧❡❝t❡❞ t❤❡ ❧❡ss✴♠♦r❡ ❡❞✉❝❛t❡❞✳

◮ ❉✐✛❡r❡♥❝❡s ✐♥ ♠♦rt❛❧✐t② ♥❡❡❞ t♦ ❜❡ ✈❡r② ❤✐❣❤ ❢♦r ✐t t♦ ❜❡ ❛

❝♦♥❝❡r♥✳

◮ ▼♦rt❛❧✐t② r❛t❡s ✈❡r② ❤✐❣❤ ❞✉r✐♥❣ ❈❛♠❡r♦♦♥✬s ✐♥❞❡♣❡♥❞❡♥❝❡ ✇❛r

✭✶✾✺✺✲✶✾✼✶✮✳

◮ ❍♦✇❡✈❡r✱ ❝❛♥♥♦t ❡①♣❧❛✐♥ t❤❡ r❡s✉❧ts

◮ ❉✐s❝♦♥t✐♥✉✐t② ❢❛✈♦r✐♥❣ t❤❡ ❇r✐t✐s❤ s✐❞❡ ❜❡❢♦r❡ ❲❲■■✿ ❡✛❡❝t

✇❡❛❦❡r ❢♦r ❝♦❤♦rts ✇❤♦ ✇❡r❡ ♦❢ ✜❣❤t✐♥❣ ❛❣❡✱ ✇❡❛❦❡r ✐♥ t❤❡ r❡❣✐♦♥ ✇❡r❡ t❤❡ ❞❡❛t❤ t♦❧❧ ✇❛s t❤❡ ❤✐❣❤❡st ✭❇❛♠✐❧❡❦❡✿ ❝❡♥tr❛❧ ❜♦r❞❡r s❡❣♠❡♥t✮✳

◮ ❉✐s❝♦♥t✐♥✉✐t② ❢❛✈♦r✐♥❣ t❤❡ ❋r❡♥❝❤ s✐❞❡ ❛❢t❡r ❲❲■■✿ ✐s ♦❜s❡r✈❡❞

❢♦r ❝♦❤♦rts ❜♦r♥ ❛❢t❡r t❤❡ ✭♠♦st ✐♥t❡♥s❡ ♣❛rt ♦❢✮ t❤❡ ❝♦♥✢✐❝t✳

❜❛❝❦ t♦ r♦❜✉st♥❡ss

❙t❛t✐st✐❝s ❖♥ t❤❡ ❋✐♥❛♥❝✐♥❣ ♦❢ ❊❞✉❝❛t✐♦♥ ✐♥ t❤❡ ✷ ❈❛♠❡r♦♦♥s

✶✾✷✺ ✶✾✸✵ ✶✾✸✺ ✶✾✸✽ ✶✾✺✵ ✶✾✺✺ P✉❜❧✐❝ ❡①♣❡♥❞✐t✉r❡ ✐♥ ❡❞✉❝❛t✐♦♥ ♣❡r s❝❤♦♦❧✲❛❣❡ ❝❤✐❧❞✱ ✶✾✷✺ s❤✐❧❧✐♥❣s(✶)(✷)

• ♦✈❡r♥♠❡♥t s❝❤♦♦❧s

❋r✳ ✵✳✷✼ ✵✳✻✺ ✶✳✶✷ ✵✳✻✹ ✶✵✳✺✼ ✷✵✳✵✻ ❇r✳ ✶✳✵✺ ✶✳✽✾ ✶✳✸✹ ✷✳✵✷ ✺✳✺✵ ✸✳✼✼ Pr✐✈❛t❡ s❝❤♦♦❧s ✭s✉❜s✐❞✐❡s✮ ❋r✳ ✵✳✵✷ ✵✳✵✹ ✵✳✵✺ ✵✳✵✹ ✸✳✶✻ ✾✳✵✷ ❇r✳ ✵✳✵✶ ✵✳✵✺ ✵✳✶✾ ✵✳✸✷ ✸✳✷✺ ✻✳✸✺ ❚♦t❛❧ ❡①♣❡♥❞✐t✉r❡ ❛♥❞ s❤❛r❡ ♦❢ ❡❞✉❝❛t✐♦♥ ❚♦t❛❧ ❡①♣❡♥❞✐t✉r❡ ♣❡r ❝❛♣✐t❛ ✭✶✾✷✺ ➾✮ ❋r✳ ✵✳✶✻✶✷ ✵✳✹✵✽✷ ✵✳✹✺✽✼ ✵✳✸✶✽✾ ✶✳✼✼✹✶ ✹✳✵✼✶✽ ❇r✳ ✵✳✷✷✵✷ ✵✳✷✽✹✾ ✵✳✷✹✺✵ ✵✳✸✶✸✶ ✵✳✼✹✾✶ ✵✳✽✷✼✹ ❙❤❛r❡ ♦❢ ❡❞✉❝❛t✐♦♥ ❋r✳ ✶✳✽✼✪ ✶✳✻✼✪ ✷✳✺✻✪ ✷✳✵✼✪ ✼✳✼✹✪ ✻✳✺✶✪ ❇r✳ ✺✳✶✸✪ ✻✳✽✸✪ ✻✳✷✼✪ ✼✳✸✵✪ ✶✶✳✻✽✪ ✽✳✸✻✪

❜❛❝❦

❉✐s❝♦♥t✐♥✉✐t② ✐♥ ♥✉♠❜❡r ♦❢ s❝❤♦♦❧s ♣❡r ✶✵✵ ❝❤✐❧❞r❡♥

❉✐s❝♦♥t✐♥✉✐t✐❡s ❡st✐♠❛t❡❞ ♦♥ ❛ ✷✵✵✲❦♠ ❜❛♥❞✇✐❞t❤ ❛❝r♦ss t❤❡ ❜♦r❞❡r ❝♦♥tr♦❧❧✐♥❣ ❢♦r ❛ ♣♦❧②♥♦♠✐❛❧ ♦❢ ♦r❞❡r ✸ ✐♥ ❧❛t ❛♥❞ ❧♦♥❣ ❛♥❞ ❝♦♥tr♦❧❧✐♥❣ ❢♦r ✺ ❜♦r❞❡r s❡❣♠❡♥t ❞✉♠♠✐❡s✳ ❚❤❡ ❞❡♣❡♥❞❡♥t ✈❛r✐❛❜❧❡ ✐s ❛t t❤❡ ✶✾✼✻ ❞✐str✐❝t ❧❡✈❡❧ ✭✺✸ ❞✐str✐❝ts ✲❉♦✉❛❧❛ ❡①❝❧✉❞❡❞✮✳ ❘♦❜✉st ❙❊✳ ❜❛❝❦

P✉❜❧✐❝ ✈❡rs✉s ♣r✐✈❛t❡ s❝❤♦♦❧s

❉✐s❝♦♥t✐♥✉✐t✐❡s ❡st✐♠❛t❡❞ ♦♥ ❛ ✷✵✵✲❦♠ ❜❛♥❞✇✐❞t❤ ❛❝r♦ss t❤❡ ❜♦r❞❡r ❝♦♥tr♦❧❧✐♥❣ ❢♦r ❛ ♣♦❧②♥♦♠✐❛❧ ♦❢ ♦r❞❡r ✸ ✐♥ ❧❛t ❛♥❞ ❧♦♥❣ ❛♥❞ ❝♦♥tr♦❧❧✐♥❣ ❢♦r ✺ ❜♦r❞❡r s❡❣♠❡♥t ❞✉♠♠✐❡s✳ ❚❤❡ ❞❡♣❡♥❞❡♥t ✈❛r✐❛❜❧❡ ✐s ❛t t❤❡ ✶✾✼✻ ❞✐str✐❝t ❧❡✈❡❧ ✭✺✸ ❞✐str✐❝ts ✲❉♦✉❛❧❛ ❡①❝❧✉❞❡❞✮✳ ❘♦❜✉st ❙❊✳ ❜❛❝❦

❈r♦ss ❝♦✉♥tr② r❡❣r❡ss✐♦♥ ♦♥ ❡♥r♦❧❧♠❡♥t r❛t❡s

❇❛❝❦ t♦ ❡①t❡r♥❛❧ ✈❛❧✐❞✐t②