SLIDE 1
1
Good afternoon colleagues. In this presentation, we present a model of assessment which is in some respects an extension of my PhD thesis, but has also been the product of discussions with my co-author.
SLIDE 2 We present a model for assessment against the backdrop of the educational crises which are evident in this country. We present assessment in the larger context and describe some dangers and challenges. Then some measurement principles and technological resources, features of Rasch measurement
- theory. Finally we present an example of assessment and extrapolate from this
example to an extended model.
2
SLIDE 3
“Educational crises” has been presented by Jonathan Jansen. The crisis of the youth identity is part of this bigger crisis. Crisis in our view is not altogether negative: it rather presents a ferment from which change can emerge. The response to these crises by the education department and the government has in my view not addressed the problem. By trying to exert stricter controls and cracking down on freedom of expression they are working against democratic principles.
3
SLIDE 4
While some may assert that education supports democracy (and blame the lack of democracy on lack of education), it is more apt to say that at best we have two contrasting aims. Robert Young in a book “Habermas and our children’s future” notes that the authority and conformity bred in our schools leads to blind followers rather than critical thinkers. Recently, PDU told this story. In his visit to schools around the country, he asks What is democracy? The answer came, Mandela!. He said yes and what else. They said No apartheid. And what else. He said we will have achieved democracy when each one of you is the most important person in this country. Perhaps this is what we should strive for. A society where each person, including both teachers and learners are accorded their rightful place in society.
4
SLIDE 5 The fact is that our assessment practices have not always accorded teachers and learners the respect they deserve. From history we know that the purposes of the earliest forms of assessment were that of discrimination and
- selection. We have evolved to a view of assessment where the development of
each individual is important. The dangers however still lurk. We still have blunt instruments that spew out a single number, and some of our systemic assessment practices leave teachers bewildered and with little useful information. The challenge for researchers is to adhere to sound scientific practice (more of which will be explained) and for teachers to demand relevant contextual information
5
SLIDE 6
We do need accountability it seems but we also need a formative component where teachers can explore safely within the confines of their classroom, and in relation to the learners, strategies which she thinks may work. Alongside both these components we need professional development (also looking at the work of Bennett & Gitomer from the ETS) .
6
SLIDE 7
7
SLIDE 8
8
SLIDE 9
But if one starts with the idea of a concept, for example ratio. And then adds to this primary concept, a number of related concepts: fraction, proportion, rate, percent, probability … and the set of problem situations which require these concepts for their solution we have a
9
SLIDE 10
But if one starts with the idea of a concept, for example ratio. And then adds to these concepts, a number of related concepts, fraction, proportion, rate, percent, probability … and the set of problem situations which require these concepts for their solution we have a
10
SLIDE 11
But if one starts with the idea of a concept, for example ratio. And then adds to these concepts, a number of related concepts, fraction, proportion, rate, percent, probability … and the set of problem situations which require these concepts for their solution we have a
11
SLIDE 12
But if one starts with the idea of a concept, for example ratio. And then adds to these concepts, a number of related concepts, fraction, proportion, rate, percent, probability … and the set of problem situations which require these concepts for their solution we have a
12
SLIDE 13
Here we have the person-item map where persons and items are located on the same scale. The five items discussed in the paper are located at graded difficulty levels. Levels aligned with logits are demarcated in bands.
13
SLIDE 14 This table again depicts the 5 items of graded difficulty level in the vertical
- dimension. After the level and the description, five analytic categories are
- shown. Hierarchical development can be identified. Here we focus on one
category, mathematical structure. Vergnaud investigates the underlying mathematical structure of the elements of the mcf. The underlying structure of many of these concepts may be depicted as “measure spaces”. While the underlying structure may be the same, the position of the unknown in the item problem may determine the item difficulty. The problem in item 1: For each bottle collected by Zanele, Mishack collects three. If Zanele collects 9, how many will Mishack have collected. This problem is solved through multiplication and may be solved in two ways. Item 20? The items 5 and 10 are more complex. Item 30?
14
SLIDE 15 This table again depicts the 5 items of graded difficulty level in the vertical
- dimension. After the level and the description, five analytic categories are
- shown. Hierarchical development can be identified. Here we focus on one
category, mathematical structure. Vergnaud investigates the underlying mathematical structure of the elements of the mcf. The underlying structure of many of these concepts may be depicted as “measure spaces”. While the underlying structure may be the same, the position of the unknown in the problem may determine the difficulty. The problem in item 1. For each bottle collected by Zanele, Mishack collects three. If Zanele collects 9, how many will Mishack have collected. This problem is solved through multiplication and may be solved in two ways. The items 5 and 10 are more complex. The category response process is explored in more depth in the thesis.
15
SLIDE 16 This table again depicts the 5 items of graded difficulty level in the vertical
- dimension. After the level and the description, five analytic categories are
- shown. Hierarchical development can be identified. Here we focus on one
category, mathematical structure. Vergnaud investigates the underlying mathematical structure of the elements of the mcf. The underlying structure of many of these concepts may be depicted as “measure spaces”. While the underlying structure may be the same, the position of the unknown in the problem may determine the difficulty. The problem in item 1. For each bottle collected by Zanele, Mishack collects three. If Zanele collects 9, how many will Mishack have collected. This problem is solved through multiplication and may be solved in two ways. The items 5 and 10 are more complex. The category response process is explored in more depth in the thesis.
16
SLIDE 17 This table again depicts the 5 items of graded difficulty level in the vertical
- dimension. After the level and the description, five analytic categories are
- shown. Hierarchical development can be identified. Here we focus on one
category, mathematical structure. Vergnaud investigates the underlying mathematical structure of the elements of the mcf. The underlying structure of many of these concepts may be depicted as “measure spaces”. While the underlying structure may be the same, the position of the unknown in the problem may determine the difficulty. The problem in item 1. For each bottle collected by Zanele, Mishack collects three. If Zanele collects 9, how many will Mishack have collected. This problem is solved through multiplication and may be solved in two ways. The items 5 and 10 are more complex. The category response process is explored in more depth in the thesis.
17
SLIDE 18 This table again depicts the 5 items of graded difficulty level in the vertical
- dimension. After the level and the description, five analytic categories are
- shown. Hierarchical development can be identified. Here we focus on one
category, mathematical structure. Vergnaud investigates the underlying mathematical structure of the elements of the mcf. The underlying structure of many of these concepts may be depicted as “measure spaces”. While the underlying structure may be the same, the position of the unknown in the problem may determine the difficulty. The problem in item 1. For each bottle collected by Zanele, Mishack collects three. If Zanele collects 9, how many will Mishack have collected. This problem is solved through multiplication and may be solved in two ways. The items 5 and 10 are more complex. The category response process is explored in more depth in the thesis.
18
SLIDE 19
The empiricaldifferences in difficulty level are shown here. Item 1 and Item 30.
19
SLIDE 20
20
SLIDE 21
This table provides a composite summary of item analyses and interview analyses. A diagonal line across this table shows a plausible zone of proximal development for a class comprising the current proficiencies of learners in this cohort.
21
SLIDE 22
This table provides a composite summary of item analyses and interview analyses. A diagonal line across this table shows a plausible zone of proximal development for a class comprising the current proficiencies of learners in this cohort.
22
SLIDE 23
23
SLIDE 24
24
SLIDE 25 Many of you will have seen the person item map, also called a variable map,
- riginally conceptualised by Wright (1978). This history of this powerful tool is
explained by Wilson (2011). These levels are demarcated for the purposes of analysis. Griffin (2007) works with clusters of items rather than levels. If what we have here are good “measures” based on sound theory of the construct underpinning the development of the instrument then we may be able to invoke Vygotsky’s zone of proximal development, the zone where
- ptimal learning takes place. For example learners needs at particular levels
may be identified.
25
SLIDE 26 However, the seed which resulted in the technical and methodological resource I am going to share with you was planted by the principal of a school not very far from here. After administering some tests at his school, he commented “I hope we are going to get more than numbers.” The presentation today is my response to him. A development in response to the principal’s request is the table presented
- here. This percent correct (colour-coded) chart comes from real data but has
been adapted to preserve anonymity. I will come back to this chart. But firstly lets investigate the different aspects.
26
SLIDE 27
I have here an excerpt with only the first two items, geometry and algebra.
27
SLIDE 28
We have three schools, each with two classes.
28
SLIDE 29
The numbers in the classes range from 20 to 30.
29
SLIDE 30
The numbers and item descriptions are given here
30
SLIDE 31
The item difficulty in terms of logits (obtained from Rasch measures) are in the second column
31
SLIDE 32
Item 1 – over 90% correct Item 2 – 50% correct
32
SLIDE 33
Where C1 does better on Item 1, C2 does better on item 2
33
SLIDE 34
It appears looking only at this data that the classes are divided according to ability.
34
SLIDE 35
The overall learner mean is added as a reference point.
35
SLIDE 36
36
SLIDE 37
The question many will ask, especially maths teachers, is what information can you obtain from only one or two items. This test has about 55 items. With the common request to cover the curriculum it is difficult to get more than a “hypothesis to be explored”.
37
SLIDE 38
38
SLIDE 39
Some proficiency here
39
SLIDE 40
Work to be done
40
SLIDE 41
An interesting contrast
41
SLIDE 42
42
SLIDE 43
The information that we gain is only as good as the theory that informed the test instrument. An analysis of the proficiency zones elicits the following
43
SLIDE 44
In general, the Level one items require recall, and the level 6 items require more advanced mathematical thinking.
44
SLIDE 45
45
SLIDE 46
We come to the big question.
46
SLIDE 47
A model of assessment in the early stages (also examining the work of Bennett & Gitomer from the ETS) is in process. Three components
47
SLIDE 48
48
SLIDE 49
49
SLIDE 50
50
SLIDE 51
51
SLIDE 52
52
