Awarding Grades Summer 2020 Edgbarrow School Hello, my name is - - PDF document
Awarding Grades Summer 2020 Edgbarrow School Hello, my name is Phil Marshall; I am a teacher at Edgbarrow School and I am part of the senior team responsible for whole-school data. You may have heard me talk about data at one of our information
Edgbarrow School
Hello, my name is Phil Marshall; I am a teacher at Edgbarrow School and I am part of the senior team responsible for whole-school data. You may have heard me talk about data at one of our information evenings. I also teach maths, computer science and occasionally physics, so I may have taught someone in your family over the last few years. 1
In this video we are going to explain the process for awarding grades for GCSE, A-level and vocational courses, both the school part of the process and how the grades will be awarded at a national level. 2
We know that there are a number of concerns about the grades this year and many students are frustrated that, having worked hard for two years, they will not get to “prove themselves” in an exam. Teachers are equally concerned and frustrated; we believe that some students will be pleasantly surprised by their grades this year, but equally we know that there will be students who are disappointed and don’t get the grades they perhaps hoped or expected to achieve. For us, this is not new. Whilst we know that the system this year is not perfect, there are many people who would argue that exams are not perfect
just one bad day and get a lower grade than expected. There are almost always surprises and disappointments, whatever the system for awarding grades. So the highs and lows of results day are familiar to us and whatever grades you receive and whatever your plans for the future, your teachers, tutors, heads of year and senior teachers will be available over the results period to support and advise you, just as they always would. We also know that colleges, universities and employers are going to have to be sensible when it comes to considering grades this year. 3
“choosing” grades for students
…Exam boards award grades
National Subject
School Student
The process of awarding grades this year has lots of steps and uses data about the student, the school, each subject and national data. This is a national process and is not about individual teachers “choosing” grades for students. So the basic process is that schools will submit data to the exam boards and the exam boards will use our data and also national data to award the grades for each subject as fairly as they can given the circumstances. Now, there are lots of people involved in the school part of the process, and we are using a wide range of data about our students to ensure fairness. We have taken advice about our process from many sources, including the Department for Education, OfQual who maintain standards for qualifications, the Association of School and College Leaders and ALPS, who provide data support to
sources at the end of the video if you are interested. 4
This process has lots of steps, so we will try to break it down for you; my email address is also at the end of the video if you want to ask any questions, although please be aware that there is some data we are not allowed to share yet as it would be considered malpractice by the exam boards. The process makes more sense if we start with the big picture and then work in to the student data. 4
0.00 2.00 4.00 6.00 8.00 10.00 12.00 14.00 16.00 18.00 20.00 1 2 3 4 5 6 7 8 9
Percentage
Grade
Grades Awarded Nationally in 2019
In order to understand what we mean by “fair” grades, we need to consider the national data. Here, *roughly*, is the distribution of GCSE grades last year. As you can see, most grades we around a 5, and between 5% and 10% of grades were a 9. 5
0.00 5.00 10.00 15.00 20.00 25.00 1 2 3 4 5 6 7 8 9
Percentage Grade
Example of "unfair" data
National Example
So let’s imagine if the national grades this year looked like this light blue set, with far more higher grades and nearly double the number of grade 7s. If those were the results from a set of exams, we would say that those grades looked unlikely and we would probably say that the exams had got easier, rather than that students had suddenly performed that much better. We might consider that these inflated grades would be unfair to students and unhelpful to employers or colleges trying to choose between students. In fact the exam boards already manage this each year by adjusting the grade boundaries of the exams so the grades awarded each year are broadly similar to previous years. This is called having “comparable outcomes” and it means that colleges, universities, employers and so on know that a student with a grade 7 one year is similar to a student with a grade 7 in another year. Even without exams this year, this principle of keeping results similar to previous years will help to ensure fairness across the country. That said, there have been calls to be more relaxed about that this year to reduce the impact on students. Part of the issue is that we can recognise when something looks unfair, but it is far 6
harder to define clearly what we might accept as fair. Therefore the exam boards won’t make the final decisions on some of these details until they have actually seen all the school data. So what follows is the guidance we have, on the understanding that there are some things we can’t know yet. 6
0.00 2.00 4.00 6.00 8.00 10.00 12.00 14.00 16.00 18.00 20.00 1 2 3 4 5 6 7 8 9 Percentage Grade
Grades Awarded Nationally in 2019
0.00 2.00 4.00 6.00 8.00 10.00 12.00 14.00 16.00 18.00 20.00 1 2 3 4 5 6 7 8 9
Percentage Grade
Subject 1
0.00 5.00 10.00 15.00 20.00 1 2 3 4 5 6 7 8 9 Percentage Grade
Subject 2
0.00 5.00 10.00 15.00 20.00 25.00 1 2 3 4 5 6 7 8 9 Percentage Grade
Subject 3
0.00 5.00 10.00 15.00 20.00 25.00 1 2 3 4 5 6 7 8 9 Percentage Grade
Subject 4
Of course, each subject is different. If this was the national picture, each subject has its own national distribution of results, like this. This is important as it is the subject-level data that will be used to make sure that the grades are fair. 7
0.00 5.00 10.00 15.00 20.00 25.00 30.00 35.00 1 2 3 4 5 6 7 8 9
Percentage Grade
National Subject Data vs Edgbarrow Data
National Edgbarrow
We also know that there are differences between schools. For example, here is the national data in dark blue for one subject… and the data for the same subject at Edgbarrow School in light blue As you can see, our most common grade is a 5 in this subject and we have more of the high grades. Overall, we do better than the national average in this subject. Now, in general, we would expect our subject results to be slightly better than the national averages because our Progress8 score is positive. Last year our Progess8 was +0.37 and this means that, on average, our students scored just over a third of a grade better than the national results. Of course, just like the national data, this varies by subject within a school. This is the most important data, as it is the subject results for each school over the last few years that will be used by the exam boards to moderate and standardise our results. If this seems slightly strange, it is worth knowing that the majority of schools across the country have “stable results”, that means they don’t change much from one year to the next, or only change in line with national changes. 8
We would therefore expect that the subject results for each school in 2020 will be similar to the subject results achieved in 2019. If we submit grades that are too generous, or too harsh compared to previous years, the exam boards will change them, just like they do at the moment when they moderate coursework and projects. Of course, this is not perfect but this may be the “least bad” way to ensure fairness across the country, and we know that there are still discussions happening about exactly how this will work at the national level. There are more checks as well and we will come to those shortly. 8
Student Rank Student A 3 Student B 8 Student C 6 Student D 10 Student E 1 Student F 4 Student G 2 Student H 7 Student J 5 Student K 9 Student Rank Student E 1 Student G 2 Student A 3 Student F 4 Student J 5 Student C 6 Student H 7 Student B 8 Student K 9 Student D 10 People
Data
So, how does this work in practice? Let’s consider a small option subject with 10 students. The first step is for staff to rank the students in the subject. In this case, the students are ranked from 1 to 10. This is across the whole subject not across a class, so there is a single ranking of all the students for each subject. This ranking is not based on one set of data, or the opinion of a single member of
subject and staff considered the exam marks that we think students might have achieved in the summer. There were lots of people involved in the process and lots of (virtual) meetings to agree the ranking. This is a real balance of data, our knowledge
The ranking is really important, as the exam boards may change grades, but we believe that they won’t change the ranking, and we’ll see why that is important shortly. 9
Student Rank Centre-Assessed Grade Student E 1 9 Student G 2 7 Student A 3 7 Student F 4 6 Student J 5 6 Student C 6 6 Student H 7 5 Student B 8 5 Student K 9 5 Student D 10 4 People
Data
completed
0.00 5.00 10.00 15.00 20.00 25.00 1 2 3 4 5 6 7 8 9
Percentage Grade
Subject model for our students
Once we have agreed the ranking, we then assigned a centre-assessed grade to each student in the subject. Again, this is not based on a single piece of data. This grade takes into account data from many sources, plus the professional judgement of many staff. The grade is not just the grade from the last grade report, nor is it just the most recent mock grade. Crucially, these grades include an element of modelling using our subject results from previous years and data about each student that predicts roughly what grades students might have achieved this year. So we are already trying to standardise the grades internally in the same way that the exam boards will, because know that if we don’t, they will change the grades to ensure fairness. 10
Student Rank Centre-Assessed Grade Last grade report Student E 1 9 8+ Student G 2 7 7+ Student A 3 7 7- Student F 4 6 7- Student J 5 6 6+ Student C 6 6 6= Student H 7 5 6- Student B 8 5 5+ Student K 9 5 5- Student D 10 4 5-
A key point is that the final grades may well be different to the grades on the last grade report. Also, a student may have had the same predicted grades as someone else in March, but then end up with different grades to them following the national moderation. Predictions always carry an element of uncertainty. For example, a student predicted a “4-” back in March was always at risk of getting a 3 in the summer, even before the significant changes in circumstances. We know that students do not all achieve the grades they are predicted – some grades go up and others go down, and that happens every year in an exam series. To put this in context, if the exam boards just awarded students the grades from their last grade report, our Progress8 this year would be about +0.8 – which would be more than double last year’s result. Therefore, as we know the subject results this year need to be broadly in line with previous years, those grades would seem
results this year are similar to a typical exam series. We have to reflect this in our centre-assessed grades, using the advice and data that 11
has become available since the grade reports were published. So, the centre-assessed grades may be different to the last grade report while still being as positive and generous as we can be given the national guidelines. 11
Whole-school data Subject results over time Data for groups of students Data for individual students Centre-Assessed Grades To help make the grades as fair as possible, we did lots of internal standardisation of all the centre-assessed grades: * We looked at the results from a whole-school level, to make sure the big picture made sense * We looked at the subject results compared to previous years, as we know this will be important * We looked at the results for groups of students to avoid internal bias, * We looked at the results for individual students, to ensure that the spread of grades for each student was believable 12
Student Subject 1 Subject 2 Subject 3 Subject 4 Subject 5 Subject 6 Subject 7 Subject 8 Subject 9 Subject 10 Last grade report 3+ 4- 4- 4- 3+ 4- 4- 4- 4- 3+ “Believable” grades 3 3 4 3 4 4 4 3 4 3 “Unlikely” grades 3 3 3 3 3 3 3 3 3 3
For example, we might have a student who was expected to get a mix of 4s and 3s. Therefore if the subject results came out like the middle row, we would probably think these grades were believable. However, if our centre-assessed grades came out like the bottom row we want to would have another look at the data for their subjects because that seems too low. This has been a long process, to make sure the data we are sending to the exam boards seems fair and believable at every level of analysis. 13
Student Rank Grade Student A 3 7 Student F 4 6 Student Rank Grade Student A 3 7 Student F 4 7 Student Rank Grade Student A 3 7 Student F 4 6 Student Rank Grade Student A 3 6 Student F 4 6 Student Rank Grade Student A 3 6 Student F 4 7
We have now submitted our rankings and centre-assessed grades to the exam boards and they will use the historical data for each subject and their own statistical models to moderate the grades from each school. Grades may go down, or up. We don’t know quite how that will work, but we know there will be a lot of scrutiny of the results and pressure to make sure that students are not disadvantages by circumstances outside their control. What we do know is that although the exam boards may change the grades, they won’t change the rank order of the students. So student F, above, can’t get a higher grade than student A. This is just like grade boundaries in an exam – if the grade boundaries are changed then grades may change, but the order of the students stays the same. 14
a) Less emphasis on modelling as groups are smaller
a) Student level b) Subject level c) School level
The process for A levels is basically the same as GCSEs. It was a long process to ensure fairness at every level as far as we can. For reference, 6th form grade reports tend to be closer to final outcomes, although there is still some variation. 15
not an overall grade
Vocational courses are slightly different, as they often have far more coursework and students may already have results from several modules. Therefore the data we submitted was slightly different. For modules that were due to be completed this summer, the process is similar to
16
Results day
Appeals process
evidence to the exam boards, unless there has been a clerical error There may be opportunities to sit exams next year in some subjects, but we don’t have any firm details yet.
Now we wait for results day. We will communicate the next steps by email and via
On results day please talk to your teachers, tutors, heads of year and the senior team about any concerns. Whatever grades you get, we will support you with your next
There is a limited appeals process. We may be able to check for clerical errors, for example if you think we have submitted the wrong data to the exam boards. However, we don’t expect to be able to challenge grades or submit additional evidence to the exam boards. There may be opportunities to sit exams in some subjects next year, but we don’t yet know how this will work. We will keep you updated as we learn more and we will advise students about this depending upon their individual circumstances. 17
qualifications-in-summer-2020
centre-assessed-judgement-grades-in-2020-a-checklist/
assessment/Examinations-and-assessment/Centre-assessed-Grading- 2020-Technical-guidance
If you want to read the publicly available guidance, you can find various resources by following these links, which are also available in the video description. 18
If you have any questions, please feel free to email me. Remember that I cannot release specific data yet about the rankings or the centre-assessed grades, as that would be malpractice. In the meantime, thank you for watching, we hope you have a good summer and we’ll see you on results day. Stay safe everyone. 19