Purpose of Assessment Combined approach to improve and Feedback - - PowerPoint PPT Presentation

Purpose of Assessment and Feedback

Combined approach to improve teaching and learning whilst informing reporting and accountability Lenore Adie Associate Professor in Teacher Education and Assessment

Rowntree, 1987

How shall we know them?

Assessment Purpose

Formative

Progressing learning during teaching

Next-step learning

Content + Process Learning goals

Feedback

Range of variables

Dialogic Summative

Reporting

Next-step learning

Content + Process Learning goals

Feedback Reporting

Range of variables

Why are you assessing?

How do you want them to know it? Exactly what knowledge do you want students to have? What task(s) will the students perform to communicate their knowledge?

How will you analyse and interpret evidence? What will you accept as evidence that a student has the desired knowledge?

Assessment purpose - Assessment design

Adapted from Pellegrino, J.W., DiBello, L.V. & Goldman, S.R. (2016)

Educational assessment is at heart an exercise in evidentiary reasoning.

(Mislevy & Riconscente, 2005, p. iv)

design of formative assessment

• pportunities

high quality, multiple data sources attend to descriptive features of student work connect the data with instructional plans - differentiation of instruction Appraise the information for validity and relevance formative feedback is a finer grain size

Evidentiary decision-making

The indispensable conditions for improvement are that the student comes to hold a concept of quality roughly similar to that held by the teacher, is able to monitor continuously the quality of what is being produced during the act

• f production itself, and has a repertoire of

alternative moves or strategies from which to draw at any given point. In other words, students have to be able to judge the quality of what they are producing and be able to regulate what they are doing during the doing of it.

Sadler (1989. p. 121)

Transparency

Fairness Fine-grained details Engendering trust Motivation Right to privacy

Sadler (1985)

3 difficulties in defining criteria

Lack of distinct boundaries Role of experience

• n interpretation

Unanticipated qualities

How to enable students to be better learners through knowledge of criteria and developing expertise in their use?

reduce trial and error attempts in students’ efforts to produce ‘good work’ promote practices and provide artefacts that develop evaluative experience and expertise

4 enabling conditions for including students in assessment practices

Teachers’ assessment identities Students’ assessment identities Social moderation The role of artefacts

Teacher assessment identity

Knowledge and skills

Beliefs, confidence, role, feelings

Development of expertise

Professional standards

Evidentiary decision making

Use of data and evidence; artefacts.

Moderation

Looney, Cumming, van der Kleij & Harris (2017)

(Adie, 2016)

Graduate (G) Proficient (P) Highly Accomplished (HA) Lead (L)

5.2 Provide Feedback to Students on Learning

Students School Purpose Content Action

Effective Range P u r p

• s

e Informed, Timely Judgements

Model L: Initiate L: About Current Needs in order to Progress Learning HA: Select HA: Provide P: Demonstrate an Understanding G: Programs L: Exemplary Practice L: Support Colleagues L: About Achievement Relative to Learning Goals P: About Learning G: + Targeted + Effective Timely, Appropriate Feedback HA: P: G:

(Adie, Stobart & Cumming, forthcoming)

Artefacts

Exemplar Cognitive commentary

Overall this student’s work sample is best matched with a ‘Developing +’ standard. This student generally expressed a point of view, used the exposition framework, provided some supporting evidence for the viewpoint and included some relevant details to provide information about life in the Gold Fields which are matched with the ‘Consolidating’ standard. However, the student needed to have written a final statement that referred to the viewpoint, explained the reason for her point of view in the thesis and made use of more specific conjunctions to strengthen the argument to achieve an overall ‘Consolidating’ standard. Because this student did not offer a conclusion but demonstrated a ‘Consolidating’ standard against 4 criteria, on balance, this work has been rated at no more or no less than a Developing +’ standard.

From the M eeting in the M iddle project (W yatt-Sm ith & Bridges, 2007, p. 8) Adie & W illis (2016).

Moderation insights

… you never really understand what they’re [criteria and standards] about until you are grading or you are using them. So, until you see them in

• peration it’s hard to

know, but there is a danger of being too detailed and almost verbose with what you’re trying to do.

(Klenowski & Wyatt-Smith, 2010, p. 34)

Teacher 1: …what I have found is that it is impossible to design criteria that work all the time and what you need to do is that you need to use them more as a … Teacher 2: …a guideline? Teacher 1: …a rough guideline and then you need to see.

(Wyatt-Smith, Klenowski, & Gunn, 2010, p. 68)

Student assessment identity

Knowledge and skills

Beliefs, confidence, role, feelings Development of expertise

Self-assessment; Peer assessment

Evidentiary decision making

Use of data and evidence; artefacts.

Feedback as a dialogic process

Van der Kleij, Adie, Cumming, 2016

Artefacts Criteria Self-assessment

Panadero, Alonso-Tapia, and Huertas, 2012; Fletcher, 2016 Chang, Liang, & Chen, 2013; Wyatt-Smith & Bridges, 2007

higher-performing students may be more inaccurate in SA

(Hosein & Harle 2018)

A

Define the criteria by which students assess their work

B

Teach students how to apply the criteria

C

Give students feedback on their self-assessments

D

Give students help in using self-assessment data to improve performance

E

Provide sufficient time for revision after self- assessment

F

Do not turn self- assessment into self-evaluation by counting it toward a grade Self-assessment: Implications for practice

Panadero, Jonsson, & Strijbos, 2016, p. 306

Peer assessment

Guiding questions, prompts

Rotsaert, Panadero, Schellens, & Raes (in press); Gan & Hattie (2014)

Rubrics, Exemplars - Knowledge of standards and criteria

Gan & Hattie (2014); Rotsaert, Panadero, Schellens, & Raes (in press)

Co-constructing and deconstructing criteria

Leenknecht & Prins (in press)

Psychologically safe conditions — Trust

Harris & Brown (2013); Panadero, (2016)

Guidance and instruction

Tsivitanidou, Zacharia & Hovardas, (2011); Hovardas et

• al. (2014)

Practice

(Adie, Stobart & Cumming, forthcoming)

Site validity

Localised knowledge, situated contexts

System validity

Political accountability / improvement agendas, generalisable across sites

(Freebody & Wyatt-Smith, 2004; Adie & Wyatt-Smith, 2018 in-press)

Rowntree, 1987

How shall we know them?

Theoretical framework

Robinson & Taylor (2007, p.8) four core values of student voice A conception of communication as dialogue Dialogue enables development

• f shared

understanding The requirement for participation and democratic inclusivity Participation and democratic inclusivity require all voices to be listened to; and acceptance of diversity The recognition that power relations are unequal and problematic Equal power means equitable

• pportunities

to actively express ideas and to shape consequences The possibility for change and transformation Transformation actively seeks (student) voice, takes it seriously and uses it to creatively solve problems

Theoretical ways forward - Student Voice

Lundy (2007)

Space to express a view Coaching to communicate their learning Action on views expressed Audience to hear views

Reconceptualising reporting

teacher

• wnership of

data

reinterpretation through dialogue

participation and inclusiveness mutual partnership expertise of teacher, student, and family reporting progressing learning assessment communicating

Combined approach to improve teaching and learning whilst informing reporting and accountability Dialogic feedback System and site validity Reporting: Student voice Student self- and peer assessment How shall we know them? Assessment design Standards-referenced assessment as an interconnected system

Assessment as shared inquiry

Assessment identity

References

Adie, L. (2016). Working in a system of standards-referenced assessment: Traversing the intersections. In H. Fehring (Ed.). Assessment into Practice. Primary English Teaching Association Australia (PETAA). Adie, L. & Willis, J. (2016). Making meaning of assessment policy through teacher assessment conversations. In D. Laveault and L. Allal (Eds.). Assessment for Learning: Overcoming the Challenges of Implementation. (pp. 35-53). Dordrecht, the Netherlands: Springer. Adie, L., & Wyatt-Smith, C. (in-press 2018). What is the potential of standards, validation and moderation for demonstrating quality in initial teacher education? In C. Wyatt-Smith & L. Adie (Eds.), Teacher education, learning innovation and accountability. Springer. Chang, C.-C., Liang, C., & Chen, Y.-H. (2013). Is learner self-assessment reliable and valid in a web-based portfolio environment for high school students? Computers & Education, 60(1), 325– 334. Fletcher, A. K. (2016). Exceeding expectations: Scaffolding agentic engagement through assessment as learning. Educational Research, 58(4), 400–419. Freebody, P., & Wyatt-Smith, C. (2004). The assessment of literacy: Working the zone between system and site validity. Journal of Educational Enquiry, 5(2), 30-49. Gan, M. S., & Hattie, J. (2014). Prompting secondary students’ use of criteria, feedback specificity and feedback levels during an investigative task. Instructional Science, 42(6), 861–878. Harris, L. R., & Brown, G. T. L. (2013). Opportunities and obstacles to consider when using peer- and self-assessment to improve student learning: Case studies into teachers' implementation. Teaching and Teacher Education, 36(0), 101-111. Hovardas, T., Tsivitanidou, O. E., & Zacharia, Z. C. (2014). Peer versus expert feedback: An investigation of the quality of peer feedback among secondary school students. Computers & Education, 71(Supplement C), 133–152. Klenowski, V. & Wyatt-Smith, C. (2010). Standards, teacher judgement and moderation in contexts of national curriculum and assessment reform. Assessment Matters, 2, 107-131. Leenknecht, M. J. M., & Prins, F. J. (in press). Formative peer assessment in primary school: The effects of involving pupils in setting assessment criteria on their appraisal and feedback style. European Journal of Psychology of Education, 1–16. doi:10.1007/s10212-017-0340-2 Looney, A., Cumming, J., Van Der Kleij, F., & Harris, K. (2017). Reconceptualising the role of teachers as assessors: Teacher assessment identity. Assessment in Education: Principles, Policy & Practice, 1–26. https://doi.org/10.1080/0969594X.2016.1268090 Lundy, L. (2007). ‘Voice’ is not enough: conceptualising Article 12 of the United Nations Convention on the Rights of the Child. British Educational Research Journal, 33(6), 927–942. Mislevy, R. J., & Riconscente, M. M. (2005). Evidence-centered assessment design: Layers, structures, and terminology. Menlo Park, CA: SRI International. Panadero, E. (2016). Is it safe? Social, interpersonal, and human effects of peer assessment: A review and future directions. In G. T. L. Brown & L. R. Harris (Eds.), Handbook of human and social conditions in assessment (pp. 247–266). New York, NY: Routledge. Panadero, E., Alonso-Tapia, J., & Huertas, J. A. (2012). Rubrics and self-assessment scripts effects on self-regulation, learning and self-efficacy in secondary education. Learning and Individual Differences, 22(6), 806–813. Panadero, E., Jonsson, A., & Strijbos, J.-W. (2016). Scaffolding self-regulated learning through self-assessment and peer assessment: Guidelines for classroom implementation. In D. Laveault &

• L. Allal (Eds.), Assessment for learning: Meeting the challenge of implementation (pp. 311–326): Cham, Switzerland: Springer International.

Pellegrino, J.W., DiBello, L.V. & Goldman, S.R. (2016). A framework for conceptualizing and evaluating the validity of instructionally relevant assessments. Educational Psychologist, 51(1), 59-81. Robinson, C., & Taylor, C. (2007). Theorizing student voice: values and perspectives. Improving Schools, 10(1), 5–17. Rotsaert, T., Panadero, E., Schellens, T., & Raes, A. (2017). “Now you know what you’re doing right and wrong!” Peer feedback quality in synchronous peer assessment in secondary education. European Journal of Psychology of Education, 1–21. doi:10.1007/s10212-017-0329-x Rowntree, D. (1987). Assessing students: How shall we know them? (Rev. ed.). London: New York: Kogan Page; Nichols Pub. Sadler, D. R. (1985). The origins and functions of evaluative criteria. Educational Theory, 35(3), 285–297. Sadler, D. R. (1989). Formative assessment and the design of instructional systems. Instructional Science, 18, 119–144. Tsivitanidou, O. E., Zacharia, Z. C., & Hovardas, T. (2011). Investigating secondary school students’ unmediated peer assessment skills. Learning and Instruction, 21(4), 506–519. Van der Kleij, F., Adie, L., & Cumming, J. (2017). Using video technology to enable student voice in assessment feedback. British Journal of Educational Technology, 48(5), 1092-1105. Wyatt-Smith, C. M., & Bridges, S. (2007). Meeting in the middle – assessment, pedagogy, learning and educational disadvantage. Evaluation Study for the Department of Education, Science and Training on Literacy and Numeracy in the Middle years of Schooling Initiative Strand A, Queensland Project Report. Wyatt-Smith, C., Klenowski, V., & Gunn, S. (2010). The centrality of teachers’ judgment practice in assessment: A study of standards in moderation. Assessment in Education: Principles, Policy & Practice, 17(1), 59–75.

Associate Professor Lenore Adie

Lenore.adie@acu.eduau