Rethinking the value of advanced maths participation Andy Noyes - - PowerPoint PPT Presentation

SLIDE 1

Rethinking the value of advanced maths participation

Andy Noyes & Mike Adkins, University of Nottingham

 http://www.revamp-nottingham.org  andy.noyes@nottingham.ac.uk

SLIDE 2

Outline

Project rationale Major findings

1

Economic value: Wage premiums from A level mathematics at age 34

2

Valued by: Completion of A level mathematics

3

Academic value: The role of A level mathematics in Biology and Chemistry degree outcomes

4

Political value: tracking the policy discourse surrounding the 10% premium

In progress

1

Perceived value: End user attitudes to post-16 mathematics

Future research avenues

Andy Noyes & Mike Adkins (UoN) REVAMP May 13, 2015 2 / 39

SLIDE 3

Project Rationale

The level of participation in advanced mathematics courses has been raising concerns for several years. Recent international comparisons show England to have one of the lowest levels of post-16 mathematics

• engagement. This, together with sustained pressure from stakeholders, has

led to the Secretary of State’s call for most young people to be studying mathematics up to 18 by the end of the decade. REVAMP weaves together four strands of quantitative analysis and one qualitative policy analysis strand to understand the current and changing attitudes to, participation in, and value of A level mathematics.

Andy Noyes & Mike Adkins (UoN) REVAMP May 13, 2015 3 / 39

SLIDE 4

REVAMP Work Packages

Research questions

WP1: Is there still a ‘return’ to A level mathematics? Do Dolton and Vignoles’ findings hold in more recent datasets? WP2: Who is doing A level Mathematics now? How have participation patterns changed; by social category, by school type, etc? WP3: What is the relationship between A level participation and attainment and degree outcomes? WP4: How have mathematics education reports/policy/etc, taken up the economic and other value discourses since Curriculum 2000? WP5: What do 17 year olds think is the value of post-16 advanced mathematical study and how does it relate to their current and future choices and aspirations?

Andy Noyes & Mike Adkins (UoN) REVAMP May 13, 2015 4 / 39

SLIDE 5

Work Package 1: Economic Value

Background: Economic Return to Maths Discourse

Liz Truss(18th September 2013), Parliamentary Under Secretary of State for Education and Childcare, argued that ”Maths, for example, is the only school subject which has been proven to add to earnings, by up to 10% at A level, even when every other factor is taken into account. Pupils who are ahead of their peers in maths at age 10 tend to be earning 7% more at the age of 30. Those working in science or technological careers are paid, on average, 19% more than other professions...” (CBI talk on improving education and curriculum reform)

Andy Noyes & Mike Adkins (UoN) REVAMP May 13, 2015 5 / 39

SLIDE 6

Work Package 1: Economic Value

Economic Return to Maths Discourse

Nick Gibb (10th September 2014) and Nicky Morgan (10th November 2014) have continued this argument: ”Those who do Maths A level will go on to earn 10% more” (IET Skills event) ”And yet maths, as we all know, is the subject that employers value most, helping young people develop skills which are vital to almost any career. And you don’t just have to take my word for it - studies show that pupils who study maths to A level will earn 10% more over their lifetime” (Your Life Campaign Launch).

Andy Noyes & Mike Adkins (UoN) REVAMP May 13, 2015 6 / 39

SLIDE 7

Work Package 1: Economic Value

Approach

Research question: Is there still a ‘return’ to A level mathematics? Do Dolton and Vignoles’ findings hold in more recent datasets? Data: We used the British Cohort Study wave 7 (age 34) along with education data from wave 6 (age 30) and ability scores from wave 3 (age 10) to estimate average earnings conditional on a range of demographic, education, work experience and ability score predictors. Sample: 2027 male and female respondents born in 1970 and age 34 in 2004. Selection criteria is that each individual must have taken part in one adult wave of the BCS, completed at least one A level and be in full or part-time work. Multiple Imputation: Given the vast improvements in computing power since the original study, we chose improve the handling of missing data through the technique of multiple imputation chained equations.

Andy Noyes & Mike Adkins (UoN) REVAMP May 13, 2015 7 / 39

SLIDE 8

Work Package 1: Economic Value

Repeat study econometric model

(log)yi = α + βFemalei

1

+ βMarriedi

2

+ βChildreni

3

+ βManagerial−Technicali

4

+ βSkilled_Non−Manuali

5

+ βSkilled−Manuali

6

+ βPart−Skilledi

7

+ βUnskilledi

8

+ βOthersi

9

+ βEast_Midsi

10

+ βEast_Englandi

11

+ βNorth_Easti

12

+ βNorth_Westi

13

+ βSouth_Easti

14

+ βSouth_Westi

15

+ βWest_Midsi

16

+ βYorkshirei

17

+ βScotlandi

18

+ βWalesi

19

+ βDegreei

20

+ βNVQi

21

+ βProfi

22

+ βHE_Diplomai

23

+ βMaths&Computingi

24

+ βSciencei

25

+ βHumanitiesi

26

+ βSocial_sciencei

27

+ βOtheri

28

+ βPart_timei

29

+ βWork_Expi

30

+ βWork_Exp2

i

31

+ βTenurei

32

+ βUnemploymenti

33

+ βAge10_Mathsi

34

+ βAge10_Readingi

35

+ εi εi ∼ N(0, σ2)

Andy Noyes & Mike Adkins (UoN) REVAMP May 13, 2015 8 / 39

SLIDE 9

Work Package 1: Economic value

Data level Model 5

Andy Noyes & Mike Adkins (UoN) REVAMP May 13, 2015 9 / 39

SLIDE 10

Work Package 1: Economic value

Data level Model 6

Andy Noyes & Mike Adkins (UoN) REVAMP May 13, 2015 10 / 39

SLIDE 11

Work Package 1: Economic value

Covariates vs. Log of earnings

Andy Noyes & Mike Adkins (UoN) REVAMP May 13, 2015 11 / 39

SLIDE 12

Work Package 1: Economic value

Predictions

Andy Noyes & Mike Adkins (UoN) REVAMP May 13, 2015 12 / 39

SLIDE 13

Work Package 1: Economic value

Predictions Our first scenario looks at the gender differences based on an individual that is married, has at least one child, works a full-time professional job in London, has a degree and a mathematics and computing A-level. This individual has below average scores for work experience, unemployment time, current tenure, and age 10 ability scores. Our second scenario looks at specific differences in earnings when an individual has or does not have an A-level in mathematics and computing. This proposed individual is married, has at least one child, works in a full-time professional job in London, as a degree, a science A-level, humanities A-level and a social science A-level, has average work experience, unemployment time, current tenure and ability scores. Scenario 1: Predicted difference between men and women with Maths and Computing A level is £15200 and £11000. Scenario 2: Predicted difference using model five between men with and without Maths and Computing A level is £5500. Scenario 2: Predicted difference using model five between women with and without Maths and Computing A level is £6400. Scenario 2: Predicted difference using model six between men with and without Maths and Computing A level is £4550. Scenario 2: Predicted difference using model six between women with and without Maths and Computing A level is £4160.

Andy Noyes & Mike Adkins (UoN) REVAMP May 13, 2015 13 / 39

SLIDE 14

Work Package 2: Valued by

Background:

In December 2012, the Advisory Committee on Mathematics Education (ACME) launched their strategy to tackle mathematics provision and

• participation. In the foreword, Professor Stephen Sparks noted the

numbers studying AS and A level mathematics have been rising steadily (after the drop in participation post Curriculum 2000), along with numbers studying AS and A level Further Mathematics. However, at least 250,000 achieving a grade in GCSE mathematics choose not to study any maths after GCSE.

Andy Noyes & Mike Adkins (UoN) REVAMP May 13, 2015 14 / 39

SLIDE 15

Work Package 2: Valued by

Approach: Research question: Who is doing A level Mathematics now? How have participation patterns changed; by social category, by school type, etc? Data: Using the National Pupil Database, we took a cohort based approach and followed the 2002/3 to 2009/10 year groups from their KS4 results to their KS5 A level outcomes from 2003/4 to 2012/13 linking datasets through the unique anonymous pupil identification number. This was cleaned extensively over several months. Sample: We are using data from the entire population. The multilevel model paper focuses on the population in London schools.

Andy Noyes & Mike Adkins (UoN) REVAMP May 13, 2015 15 / 39

SLIDE 16

Work Package 2: Valued by

Descriptives

Andy Noyes & Mike Adkins (UoN) REVAMP May 13, 2015 16 / 39

SLIDE 17

Work Package 3: Academic value

Background: Scientists Need Better Maths Skills? Three major reports have called for stronger maths skills amongst undergraduate scientists:

1

The Royal Society in their report on UK first degrees in STM produced a list of skills for UK graduates to develop which included: ...[the] ability to think mathematically, to process, present and quantitatively analyse numerical and other scientific data...(2006: 56)”

2

The Royal Society in their State of the Nation report argued: ”that while there was considerable variation in entry requirements, one powerful message coming through was that those who aspire to study university STEM qualifications need to take mathematics in addition to science subjects(2011:15)”.

3

The House of Lords Select Committee on Science and Technology from report stated that: ”the number of pupils studying maths post-16 is insufficient to meet the level of numeracy needed in modern society, and the level at which the subject is taught does not meet the requirements needed to study STEM subjects at undergraduate level”(2012:18).

Andy Noyes & Mike Adkins (UoN) REVAMP May 13, 2015 17 / 39

SLIDE 18

Work Package 3: Academic value

Approach

Research question: What is the relationship between A level Mathematics participation and attainment and degree outcomes? Data: We took a cohort based approach and followed the 2005/2006 GCSE group in England and linked this through the unique anonymous pupil identification number to KS5 A level outcomes and also the Higher Education Statistics Agency Data from 2008/2009 to 2012/13. This was cleaned extensively over several months and finally subset into different subjects Sample: 7402 Biology and 2548 Chemistry graduates with a first class or upper-second class honours degree from a UK university from the 2005/2006 GCSE cohort who completed their studies by 2012/13. Multilevel Structure: Students at level 1 are clustered in universities (level 2). There are approximately 98 level 2 units for the Biology model and 63 level 2 units for the Chemistry model.

Andy Noyes & Mike Adkins (UoN) REVAMP May 13, 2015 18 / 39

SLIDE 19

Work Package 3: Academic value

Approach

Missing cases: We imputed 10 datasets using the joint modelling approach which assumes a MVN distribution for all variables and allows two-level models to be specified. This was carried out using the REALCOM-IMPUTE software. Each of the 10 copies were saved separately using the STATA add-on and imported into R. Model Fit: Bayesian multilevel/hierarchical modelling via Markov Chain Monte Carlo (STAN MC). 10 chains of 20,000 iterations (for Biology) and 10,000 iterations (for Chemistry) were run (each with its

• wn imputed dataset) and chains merged into one object. These were

then thinned to provide a posterior sample of 1000 for each parameter.

Andy Noyes & Mike Adkins (UoN) REVAMP May 13, 2015 19 / 39

SLIDE 20

Work Package 3: Academic value

Binary Logistic Multilevel Model Specification [1]: Data level Pr(yi = 1) = logit−1(α + βfemalei

1

+ βeth_Blacki

2

+ βeth_Asiani

3

+ βeth_Chinesei

4

+ βeth_Mixedi

5

+ βeth_Otheri

6

+ βGCSE_Math_Pointsi

7

+ β

GCSE_Math_Points2

i

8

+ βGCSE_English_Pointsi

9

+ β

GCSE_English_Points2

i

10

+ βAve_GCSE_Pointsi

11

+ βQCA_KS5_Pointsi

12

+ β

QCA_KS5_Points2

i

13

+ βKS5_Maths_Pointsi

14

+ β

KS5_Maths_Points2

i

15

+ βKS5_Chem_Pointsi

16

+ β

KS5_Chem_Points2

i

17

+ βKS5_Bio_Pointsi

18

+ β

KS5_Bio_Points2

i

19

+ βKS5_Phys_Pointsi

20

+ β

KS5_Phys_Points2

i

21

+ +βIDACI_Scorei

22

+ αuniversity

j Andy Noyes & Mike Adkins (UoN) REVAMP May 13, 2015 20 / 39

SLIDE 21

Work Package 3: Academic value

Model [2]: Group level and priors

Group level:

         αj β1j β7j β8j β14j β15j          ∼ N                        ,          σ2

α

ρσασβ1 σ2β1 ρσασβ7 ρσβ1 σβ7 σ2β7 ρσασβ8 ρσβ1 σβ8 ρσβ7 σβ8 σ2β8 ρσασβ14 ρσβ1 σβ14 ρσβ7 σβ14 ρσβ8 σβ14 σ2β14 ρσασβ15 ρσβ1 σβ15 ρσβ7 σβ15 ρσβ8 σβ15 ρσβ14 σβ15 σ2β15                   , for j =1,…,J Priors: For the Bayesian priors we have gone with a weakly informative approach which intentionally includes less information than we have available, but provides enough to improve computation, allowing the data to speak for itself… α ∼ N (0, 5); β ∼ N (0, 5)forβ1, . . . , β22/23; σ2

α ∼ Chalf (0, 2.5);

σ2

β ∼ Chalf (0, 2.5);

ρ ∼ lkj(1.5) Andy Noyes & Mike Adkins (UoN) REVAMP May 13, 2015 21 / 39

SLIDE 22

Work Package 3: Academic value

Data level estimates (Biology)

Andy Noyes & Mike Adkins (UoN) REVAMP May 13, 2015 22 / 39

SLIDE 23

Work Package 3: Academic value

Data level estimates (Chemistry)

Andy Noyes & Mike Adkins (UoN) REVAMP May 13, 2015 23 / 39

SLIDE 24

Work Package 3: Academic value

University level variation in the Russell Group (Biology)

Andy Noyes & Mike Adkins (UoN) REVAMP May 13, 2015 24 / 39

SLIDE 25

Work Package 3: Academic value

University level variation in the Russell Group (Biology)

Andy Noyes & Mike Adkins (UoN) REVAMP May 13, 2015 25 / 39

SLIDE 26

Work Package 3: Academic value

University level variation in the Russell Group (Chemistry)

Andy Noyes & Mike Adkins (UoN) REVAMP May 13, 2015 26 / 39

SLIDE 27

Work Package 3: Academic value

University level variation in the Russell Group (Chemistry)

Andy Noyes & Mike Adkins (UoN) REVAMP May 13, 2015 27 / 39

SLIDE 28

Work Package 4: Political value

Tracing the emergence of research ideas in policy discourse: ‘It takes an extraordinary concatenation of circumstances for research to influence policy directly’ (Weiss, 1991, in Whitty, 2006, 171) Ball and Exley’s (2010) notion of policy interlockers and networks is important for our analysis: “policy networks are relatively unstable structures of positions and sites - think tanks, social enterprises and advisers - but are also flows of ideas and people…within the capillaries of these networks, ideas have careers and are diffused.” (p.155)

Andy Noyes & Mike Adkins (UoN) REVAMP May 13, 2015 28 / 39

SLIDE 29

Work Package 4: Political value

Tracking the research

1999: New Labour Government (from 1997) has continued neoliberal education policy direction from previous Conservative government; national review of 16-18 qualifications led by Lord Dearing (Dearing 1996) has precipitated major reform, i.e. Curriculum 2000; Peter Dolton and Anna Vignoles’ research (at the LSE) indicates wage premium for A level mathematics (Dolton and Vignoles 1999); long-term decline in advanced mathematics participation is causing concern (Hawkes and Savage 1999) 2000-2004: Curriculum 2000 has major impact on qualification landscape; there is a ‘disastrous’ (Smith 2004) reduction in A level mathematics participation which requires immediate government remediation through curriculum adjustment; increasing number of reports on importance of STEM to economic security (Roberts 2002); Alison Wolf cites Dolton and Vignoles’ research in ‘Does Education Matter’ (Wolf 2002). 2004: Tomlinson Report (DfES 2004) recommendations on 14-19 education rejected; Smith Report (Smith 2004) on post-14 mathematics published and will lead to flurry of activity; A level mathematics participation is rising gradually..

Andy Noyes & Mike Adkins (UoN) REVAMP May 13, 2015 29 / 39

SLIDE 30

Work Package 4: Political value

Tracking the research 2004-2010: The Maths Pathways Project (following ‘Smith’) leads to lengthy programme of curriculum and qualification development but ultimately has little impact; The REFORM group, a right-leaning think-tank, publishes ‘The Value of Mathematics’ (Kounine et al. 2008. Liz Truss, recently appointed Deputy Director

• f REFORM, is co-author); A level mathematics numbers still rising, mainly as a

result of larger cohorts and higher proportion of top grades at GCSE. 2010: General election and new coalition government; Wolf Report (Wolf 2011)

• n vocational education commissioned; the Nuffield Foundation publishes widely

cited ‘Outliers’ report (Hodgen et al. 2010) on post-16 mathematics participation. 2011: The Conservative-commissioned Vorderman Report (2011) ‘A world-class mathematics education for all our young people’ is published; Secretary of State for Education, Michael Gove (Gove 2011) speaks at the Royal Society setting out vision that “within a decade the vast majority of pupils are studying maths right through to the age of 18”. 2012: Elizabeth Truss becomes junior minister at the Department for Education; during short term in office advocates strongly for mathematics, particularly A level but also new Core Maths qualifications

Andy Noyes & Mike Adkins (UoN) REVAMP May 13, 2015 30 / 39

SLIDE 31

Work Package 4: Political value

Six conditions for adoption

1

The main research findings are simple and simplifiable

2

The research is persuasive

3

Key connections are made

4

The research harmonises with policy values

5

The research must be workable

6

The research needs an interested champion for whom it is politically beneficial

Andy Noyes & Mike Adkins (UoN) REVAMP May 13, 2015 31 / 39

SLIDE 32

Work Package 4: Political value

Four problems

1

Decontextualisation: ignorance of the historical, economic and cultural context

Nowhere in the policy discourse associated with this research is there any acknowledgement of the historical context (i.e. that the research participants are now 57). Social research is historically and culturally framed and losing sight of this framing increases the risk of

• misapplication. This problem also faces researchers in their use of work

from another time and place.

Andy Noyes & Mike Adkins (UoN) REVAMP May 13, 2015 32 / 39

SLIDE 33

Work Package 4: Political value

Four problems

2

Partiality: only using the convenient parts of the research

The elision of maths and computing in the original research has been

• lost. Although this is not surprising given the Conservative (Truss’s)

education agenda it could be considered a missed opportunity at a time when ministers are concerned to re-establish computing (i.e. programming) in the curriculum. Partiality can also consist in selective use of statistical results: The ranges reported in the original work have disappeared: the ‘return’ is now a fixed 10%, or ‘around 10%’ and reflects a bias towards a more politically expedient result. Moreover, another aspect of exaggeration can be seen in the tendency to strip out the inherent uncertainty - e.g. the standard errors.

Andy Noyes & Mike Adkins (UoN) REVAMP May 13, 2015 33 / 39

SLIDE 34

Work Package 4: Political value

Four problems

3

Overgeneralisation:

To time: Nicky Morgan’s claim about earnings over lifetimes is an example of overgeneralization and needs challenging. All that can be said is that in 1991, amongst a small sample of people born in 1958, those who had completed an A level in maths or computing in 1975 were earning, on average, between 7 and 10% more than their A level peers who had not taken mathematics. To other subjects: Dolton and Vignoles’ work identified mathematics as unique amongst A levels. In particular, science A levels did not have the same effect (although it is likely that physics and biology have quite different effects) However, this is inconvenient for the STEM agenda.

Andy Noyes & Mike Adkins (UoN) REVAMP May 13, 2015 34 / 39

SLIDE 35

Work Package 4: Political value

Four problems

4

Misinterpretation:

This arises as the flipside of research being simple and simplifiable; findings can be misunderstood and inadvertently misrepresented. In this present case, the notion of causality is a pertinent example. Politicians are concerned with the exercise and maintenance of power, using research to change behavior, and are less concerned about the theoretical explanations underlying phenomena. What cannot be implied from this research is that a young person aged 16 in 2015 who is persuaded to change their A-level choices to include mathematics on the basis of this research will, as a result, be earning 7-10% more than their non-A level mathematics peers in 2033.

Andy Noyes & Mike Adkins (UoN) REVAMP May 13, 2015 35 / 39

SLIDE 36

In Progress

Work Package 5: Perceived value[1] The project includes a national survey of 17-year olds The timing for the survey is important given that schools are now faced with the question of whether they will

• ffer Core Maths qualifications from 2015

Data linking: NPD agreement on minimum requirements for data linking Match to 2016 2017 A level NPD Timeline: Stage Time Plan Development/piloting Jan-July 14 Makes use of some TIMSS items; plan for linking to NPD Recruitment Sept-Nov 14 Random samples contacted; 116 institutions recruited with possible sample of 14,000 Survey January 15 Returns to date suggest just over 8,000 Data entry February-March 15 Sub-contracted by Seymour Research Analysis June 15

Table: School survey timeline

Andy Noyes & Mike Adkins (UoN) REVAMP May 13, 2015 36 / 39

SLIDE 37

In Progress

Work Package 5: Perceived value[2]

Andy Noyes & Mike Adkins (UoN) REVAMP May 13, 2015 37 / 39

SLIDE 38

In Progress

Work Package 5: Perceived value[3]

Andy Noyes & Mike Adkins (UoN) REVAMP May 13, 2015 38 / 39

SLIDE 39

Where to next?

Future research avenues

Linked data - Analysis of linked data during 2016/17 Life course - Preliminary work has been carried out on estimating the impact of mathematics and computing A level on earnings over the life course and this will be presented at the Centre for Longitudinal Studies conference in March this year (and expanded substantially by the proposed PRiSM project). Predicting A level outcomes - potentially via an ordinal logistic/proportional odds multilevel model, but this may be better served by a different data source. PRiSM - Pipeline and Return to Science and Mathematics Quantitative literacy

Andy Noyes & Mike Adkins (UoN) REVAMP May 13, 2015 39 / 39