Accrediting outcomes Evidencing the skills necessary for - - PowerPoint PPT Presentation

▶
accrediting outcomes
SMART_READER_LITE
LIVE PREVIEW

Accrediting outcomes Evidencing the skills necessary for - - PowerPoint PPT Presentation

Sep 06, 2023 272 likes •593 views

1 P Accrediting outcomes Evidencing the skills necessary for employability Daniel Southam @danielsoutham d.southam@curtin.edu.au 2 P Todays outline The need proposition for changing accreditation processes Development of a

slide-1
SLIDE 1 P

Accrediting outcomes

Evidencing the skills necessary for employability

Daniel Southam

@danielsoutham d.southam@curtin.edu.au

1

slide-2
SLIDE 2 P

Today’s outline

✓ The need proposition for changing accreditation processes ✓ Development of a new accreditation process using shared outcomes ✓ Some observations about the self-reporting of shared outcomes in chemistry curricula ✓ Links between outcomes, assessment, and the employability of chemistry graduates

2

slide-3
SLIDE 3 P 3

Disrupted or disruptive?

What role does a professional body have in a post-professional world? How do science and scientists remain employable?

slide-4
SLIDE 4 P

Susceptibility to computerisation

4

Occupation Prob. Occupation Prob. Medical Scientists 0.0045 Environmental Scientists 0.033 Teachers and Instructors 0.0095 Mathematicians 0.047 Pharmacists, Microbiologists 0.012 Chemists, Physicists 0.1 Biological Scientists 0.015 Actuaries 0.21 Conservation Scientists 0.016 Physical Scientists 0.43 Soil and Plant Scientists, Materials Scientists 0.021 Geoscientists 0.63 Biochemists and Biophysicists 0.027 Atmospheric and Space Scientists 0.67 Frey, C. B., & Osborne, M. A. (2013). The Future of Employment: How susceptible are jobs to computerisation? Oxford Martin School.
slide-5
SLIDE 5 P

Drivers for change

✓ Move from environmental “inputs” to student “outcomes” ✓ Harness the community consensus threshold learning outcomes ✓ Adapt to modern curricular practice in the sector ✓ Align with the Higher Education Standards processes

5

  • |
slide-6
SLIDE 6 P

The landscape

6

What are (collectively) the graduate
  • utcomes that we
want? How will we (collectively) know that students have met these outcomes?
  • Higher Education Standards Framework (Threshold Standards) 2015, pt A, s 1.4.
slide-7
SLIDE 7 Yates, B. J.; Jones, S.; Kelder, J. Science Learning and Teaching Academic Standards Statement; ALTC: Melbourne, 2011 P

Threshold Learning Outcomes

7

  • 1
Understanding science
  • 2
Scientific knowledge ? 3 Inquiry & problem solving
  • 4
Communication
  • 5
Personal & professional responsibility
slide-8
SLIDE 8 Here are some potential outcomes of a chemistry degree. Are they desirable for the future employability of a chemistry graduate? Tick the ones you think are desirable and then click “submit answer” when you’re done. P

Question 1

8

Enter the room code Go to: po.st/acsme Room for app: ACSME Pyke, S. M.; O’Brien, G.; Yates, B. J.; Buntine, M. A. Chemistry Academic Standards Statement; OLT and RACI, 2014.
slide-9
SLIDE 9 P

Question 1 (n = 113)

Which outcomes are desirable for the future employability of a chemistry graduate?

9

Understanding science Scientific knowledge Inquiry & problem solving Communication Personal & professional responsibility 0% 20% 40% 60% 80% 100% 1.1 1.2 1.3 2.1 2.2 3.1 3.2 3.3 3.4 3.5 4.1 4.2 5.1 5.2 5.3
slide-10
SLIDE 10 P

Accreditation process

Desired characteristics

✓ Outcomes-focussed ✓ Data-driven ✓ Shared understanding ✓ Benchmarked ✓ Referenced

10

  • |
slide-11
SLIDE 11 P

Chemistry accreditation

Establishing a framework 11

Allows systematic comparison of diverse curricula INTENDED A posteriori idealised curriculum CTLOs IMPLEMENTED Operational curriculum Curriculum map aligned to TLOs PERCEIVED Experienced curriculum externally validated by panel ACHIEVED Attained curriculum referenced & benchmarked Developed by Rosier and Keeves (1991) from the work of the International Association of Evaluation of Educational Achievement (Goodlad 1966) and extended by Treagust (1986)
  • |
slide-12
SLIDE 12 P ✓ Data is collected for each component of a degree ✓ The nomenclature in the survey is adapted for the local context ✓ The institution to be accredited complete a survey for each component

12

Data is collected from an
  • nline survey
  • Step 1

po.st/demo.survey View a demo at: GO

slide-13
SLIDE 13 P

Learning outcomes

Three step mapping process 13

Assessments Which assessment types are explicitly assessing the learning
  • utcomes?
Engagement Which learning outcome(s) are taught, practised and/or assessed? Ranking For each assessment type rank their development of each learning outcome MAPPING
slide-14
SLIDE 14 P ✓ Aggregates data to chart the
  • perational curriculum
✓ Permits a shared understanding between host and panel for a purposeful accreditation visit

14

|

This data is used to prepare a curriculum map

Step 2 po.st/demo.report View a demo at: GO

slide-15
SLIDE 15
  • P
✓ Visit to the host institution by a panel of trained accreditors ✓ Information from the curriculum map is used to guide the discussion ✓ The panel reflect on development of the outcomes

15

A visit to the host institution by a panel validates this map

Step 3

STRENGTHS What elements are strong, important or influential that demonstrate good practices at the institution, in curriculum and expressed through assessment? INSIGHTS What elements were you unaware of that you now think are important or influential at the institution, in curriculum and expressed through assessment? IMPROVEMENTS What elements of assessment practice or curriculum need change, and how might the institution change them?
slide-16
SLIDE 16
  • P
✓ This report collects together the curriculum map and the panel’s findings ✓ Validates the (mis)alignment between the implemented curriculum and perceived curriculum

16

The panel prepares a report for endorsement

Step 4

Commendations Affirmations Recommendations
slide-17
SLIDE 17 P

Aggregated data

17

There are three potentially interesting elements to these data
  • ^
1 ^ 2 3 ^ Learning activities Assessment tasks Learning outcomes Number Institutions 7 Campuses 10 Degrees 11 Units/courses 129 Assessment tasks 452
slide-18
SLIDE 18 P

Observations

✓ Students are crucial to elucidating the outcomes of their experiences ✓ Staff are committed to students and improving their experiences ✓ Each panel operated differently, responding to institutional culture ✓ All panels were able to judge whether TLOs as a whole were attained ✓ There were differing interpretations of some TLOs between institutions ✓ There is some desire to accredit programs beyond undergraduate courses and in transnational institutions or partnerships

18

slide-19
SLIDE 19 P

Question 2

What was the most frequently encountered (taught, practised or assessed) outcome? Tick one outcome and then click “submit answer”

19

Go to: po.st/acsme Room for app: ACSME
slide-20
SLIDE 20
  • P

Question 2 (n = 94)

What was the most frequently encountered (taught, practised or assessed) outcome?

20

Understanding science Scientific knowledge Inquiry & problem solving Communication Personal & professional responsibility 0% 20% 40% 60% 80% 100% 1.1 1.2 1.3 2.1 2.2 3.1 3.2 3.3 3.4 3.5 4.1 4.2 5.1 5.2 5.3
slide-21
SLIDE 21 P

Question 3

Other than knowledge of the principles and concepts of chemistry, what was the most frequently encountered (taught, practised or assessed)
  • utcome?
Tick one outcome and then click “submit answer”

21

Go to: po.st/acsme Room for app: ACSME
slide-22
SLIDE 22
  • P

Question 3 (n = 103)

Other than knowledge of the principles and concepts of chemistry, what was the most frequently encountered (taught, practised or assessed) outcome?

22

Understanding science Scientific knowledge Inquiry & problem solving Communication Personal & professional responsibility 0% 20% 40% 60% 80% 100% 1.1 1.2 1.3 2.1 2.2 3.1 3.2 3.3 3.4 3.5 4.1 4.2 5.1 5.2 5.3
slide-23
SLIDE 23 P

Question 4

What was the most infrequently encountered (taught, practised or assessed) outcome? Tick one outcome and then click “submit answer”

23

Go to: po.st/acsme Room for app: ACSME
slide-24
SLIDE 24
  • P

Question 4 (n = 98)

What was the most infrequently encountered (taught, practised or assessed) outcome?

24

Understanding science Scientific knowledge Inquiry & problem solving Communication Personal & professional responsibility 0% 20% 40% 60% 80% 100% 1.1 1.2 1.3 2.1 2.2 3.1 3.2 3.3 3.4 3.5 4.1 4.2 5.1 5.2 5.3
slide-25
SLIDE 25 P

CTLO Network Map

Unit level (n = 129) 25

1.1 1.2 1.3 2.1 2.2 3.1 3.2 3.3 3.4 3.5 4.1 4.2 5.1 5.2 5.3 CTLO3.5: Demonstrating the cooperativity and effectiveness of working in a team environment CTLO1.3: Understanding and being able to articulate aspects of the place and importance
  • f chemistry in the local and global community
CTLO3.3: Applying recognised methods and appropriate practical techniques and tools, and being able to adapt these techniques… CTLO2.1: Demonstrating a knowledge of, and applying the principles and concepts of chemistry Average weight low high
slide-26
SLIDE 26 P

Question 5

Other than knowledge of the principles and concepts of chemistry, what is the most often assessed outcome? Tick one outcome and then click “submit answer”

26

Go to: po.st/acsme Room for app: ACSME
slide-27
SLIDE 27
  • P

Question 5 (n = 100)

Other than knowledge of the principles and concepts of chemistry, what is the most often assessed outcome?

27

Understanding science Scientific knowledge Inquiry & problem solving Communication Personal & professional responsibility 0% 20% 40% 60% 80% 100% 1.1 1.2 1.3 2.1 2.2 3.1 3.2 3.3 3.4 3.5 4.1 4.2 5.1 5.2 5.3
slide-28
SLIDE 28 P

CTLO Network Map

Assessment level (n = 452)

  • Almost every task (86%)
submitted will assess chemistry knowledge
  • 57% of tasks assessing content
knowledge simultaneously assess demonstration of self- directed learning

28

1.1 1.2 1.3 2.1 2.2 3.1 3.2 3.3 3.4 3.5 4.1 4.2 5.1 5.2 5.3 Average weight low high
slide-29
SLIDE 29 P

Outcomes

29

STRENGTH

88% 86% 92%

2.1

Demonstrating a knowledge of, and applying the principles and concepts
  • f chemistry

INSIGHT

88% 56% 68%
  • 5.1
Demonstrating a capacity for self-directed learning

IMPROVEMENT

57% 27% 43%
  • 1.3
Understanding and being able to articulate aspects of the place and importance
  • f chemistry in the local and
global community
slide-30
SLIDE 30 RACI:
  • Presidents: Paul Bernhardt, Mark
Buntine, Peter Junk
  • National Office: Caroline Lewington,
Mary Pappa, Roger Stapleford, Robyn Taylor
  • Accreditation Committee: Jason
Harper, Gwen Lawrie, Glennys O’Brien, Matt Sykes, Chris Thompson Chemistry TLO development
  • Mark Buntine, Glennys O’Brien,
Simon Pyke, Brian Yates Trial and pilot phase institutions and panel members/chairs P

Acknowledgements

@danielsoutham d.southam@curtin.edu.au

30 po.st/demo.survey po.st/demo.report

slide-31
SLIDE 31 P

References

Frey, C. B., & Osborne, M. A. (2013). The Future of Employment: How susceptible are jobs to computerisation? Oxford Martin School. Goodlad, J. I. (1966). The Development of a Conceptual System for Dealing with Problems of Curriculum and Instruction. University of California, Los Angeles. Higher Education Standards Framework (Threshold Standards) 2015, pt A, s 1.4. Pyke, S. M.; O’Brien, G.; Yates, B. J.; Buntine, M. A. Chemistry Academic Standards Statement; Office for Learning and Teaching and The Royal Australian Chemical Institute, 2014. Rosier, M. J.; Keeves, J. P . The IEA study of science I: Science education and curricula in twenty-three countries; Pergamon Press, 1991. Treagust, D. F. Exemplary Practice in High School Biology Classes. NARST Annual Meeting, San Francisco, 1986. Yates, B. J.; Jones, S.; Kelder, J. Science Learning and Teaching Academic Standards Statement; Australian Learning and Teaching Council: Melbourne, 2011.

31