Integrating LLN and the ACSF into our Training & Assessment - - PowerPoint PPT Presentation

Integrating LLN and the ACSF into our Training & Assessment

To be effective as trainers/assessors we need to have an understanding of the significance of language, literacy and numeracy (LLN) in learning and assessment processes.

Numeracy

LLN

Reading Writing Oral communication

The LLN demands of the workplace for which the learner is being prepared The LLN demands we impose through

• ur training and

assessment materials and resources The LLN skills of the learner

Three intersecting sets of LLN demands

• n the learner

What do we need to know about LLN?

The session will also consider how aspects

• f the very complex

Australian Core Skills Framework can be used by the general trainer/assessor to analyse, determine and cater for each of these demands.

How do we establish the level of these demands?

A complex framework, designed primarily for “LLN specialists” BUT, also a framework that can help the “ordinary” trainer/assessor

ACSF Overview

The ACSF describes each of the five core skills [learning, reading, writing,

• ral communication, numeracy] across three interactive dimensions.

5 Core Skills

Five levels of performance ranging from 1 (low level performance) to 5 (high level performance) The five levels of performance represent milestones along a continuum of performance.

ACSF PERFORMANCE INDICATORS Core Skill Indicator Number Description Learning .01 Active awareness of self as a learner, planning and management of learning .02 Acquisition and application of practical strategies that facilitate learning Reading .03 Audience, purpose and meaning-making .04 Reading strategies Writing .05 Audience, purpose and meaning-making .06 The mechanics of writing Oral Communication .07 Speaking .08 Listening Numeracy .09 Identifying mathematical information and meaning in activities and texts .10 Using and applying mathematical knowledge and problem solving processes .11 Communicating and representing mathematics

Indicator numbers are consistent across core skill levels e.g. Reading

1.03 / 1.04 2.03 / 2.04 3.03 / 3.04 4.03 / 4.04 5.03 / 5.04

Core skills are colour coded

ACSF Overview

Applying the ACSF

What are important reading tasks carried out by an assessor?

FOR EXAMPLE Reading and interpreting competency standards Reading attendance sheets Reading and interpreting assessment tools Reading and analysing documents supplied as evidence Reading and analysing evidence presented in different media Reading learner records sheets

ACSF Reading Levels

• The first Reading Indicator addresses audience, purpose and meaning-making.
• The second Reading Indicator addresses reading strategies.
• Specific Indicator statements describe exit performance at each level.

Reading attendance sheets 2.03 Reading learner records sheets 2.03

Reading strategies Audience, purpose and meaning making

Reading and interpreting competency standards 3.03 Reading and analysing documents supplied as evidence 3.03 Reading and interpreting assessment tools 3.03 Reading and analysing evidence presented in different media 3.04

Applying the ACSF

What are important writing tasks carried

• ut by an assessor?

FOR EXAMPLE Completing learner records Filling out previously prepared checklists Writing judgements about evidence provided Using various communication strategies to record outcomes (e.g. text, charts, electronic)

The first Writing Indicator addresses audience, purpose and meaning-making. The second Writing Indicator addresses the mechanics of writing. Specific Indicator statements describe exit performance at each level.

ACSF Writing Levels

Completing learner records 1.05 Filling out previously prepared checklists 2.05/2.06 Writing judgements about evidence provided 4.05 Using various communication strategies to record

• utcomes (e.g. text, charts, electronic) 4.06

Establishing LLN Skill Levels

• ral communication skills,

and numeracy skills, required in your job. We could do a similar mapping of the learning skills,

Note that we will probably end up with a “spiky profile”. “Spiky” Profiles

This concept of “spiky profiles” draws our attention to the lack of correspondence between AQF levels and ACSF levels.

ACSF AQF

More Profiles

We could do an analysis of the LLN demands of the training and assessment materials and processes we provide as part

• f our training context.

Hopefully we would find some match between these “spiky profiles” and the “spiky profile” of the LLN demands of the actual job.

Having established the LLN demands of the work environment and of the training and assessment context, we now need some understanding of the LLN skills of our learner – another “spiky profile”.

Learner LLN Skills

In particular, we need to be aware

• f mismatches between the

learner “spiky profile” and the workplace and training contexts “spiky profiles”.

As “non-specialists” in LLN we are not able to carry out a fine-grained analysis of each

• learner. However, we can use

LLN assessment tools that will help us make useful judgements.

Identifying Learner Skills

These tools need to better focused than generalised reading, writing and maths tests. Such tests don’t tell us if a learner will be able to deal with the LLN demands of the workplace and training contexts.

Go back to the list of reading tasks we identified as being required by an assessor. Let us assume that one of these was reading a report to gain some required information. How could we establish if someone else would have the necessary reading skill to complete this task?

Developing a Better LLN Test

So we could build up a test of reading tasks directly related to the workplace and training contexts.

We could do the same thing with writing tasks, oral communication tasks, and numeracy tasks. We would then have a set of assessment tasks that would allow us to make judgements of the extent to which our learners could cope with these tasks. The outcomes

• f these tests

would identify where the LLN skills of learners are adequate and where there are gaps in these skills.

Developing a Better LLN Test

So What?

This information might help us make adjustments to our delivery and assessment processes so that we could include the development of some of the “gap” skills, or refer the learner to more specialised help.

It might also help us identify where our own training and assessment materials and/or processes do not match the LLN demands of the workplace/unit of competency.

Validated LLN Assessment Tools

Some of these will exist in your industry area. If not, existing tools may be able to be contextualised to address the LLN demands of your industry area.

If these LLN assessment tools are to have real value it is important that they are validated (with our peers and/or with specialists and/or with industry.)

Fortunately, a considerable amount of work has already been done in developing sets

• f such validated tools.

Some of these will work in your industry area. If not, existing tools may be able to be contextualised to address the LLN demands

• f your industry area.

Validated LLN Tools - Precision

http://www.precisionconsultancy.com.au/acs_framework

Validated LLN tools - VET-FEE HELP

'The testing tool must be an approved testing tool. The approved testing tool, available since 1 January 2016 and specified in the VET Guidelines is the Core Skills Profile for Adults (CSPA).’

Department of Education and Training, Update on VET FEE−HELP Reforms, 8 December 2015

Assessment of LLN testing instruments and processes for VET FEE-HELP providers Report (Stage 2)

• The provider must arrange to have their LLN assessment tool independently assessed.
• An independent assessor who undertakes the audits of any LLN testing instruments

should be an independent body or approved independent and specialist personnel, such as an auditor with VET experience for details.

• The independent assessment must directly respond to the specifications and

requirements for approval.

• Completed assessment must be sent to the Secretary, DET, care of the Branch

Manager, VET FEE-HELP Branch for approval. This could be in electronic or paper form.

Very detailed requirements are included in pages 31 - 43

Validated Tools - VET-FEE HELP

Criterion 1: Validity

• The assessment is based on quality assessment development processes
• The learner undertakes the assessment independently (Also in Criterion 2)
• The tasks and questions are validly mapped against the different ACSF levels (Also in Criteria 2, 3)
• Information is provided about the performance or scores required to decide that a learner has

independently proved their competence at ACSF Exit Level (x) Criterion 2: Reliability

• Administration guidelines include details about who can mark the assessment if it is to be manually

marked and a specified set of marking information and scoring rubrics are provided

• The marking and scoring guides or rubrics can be applied consistently and reliably across different

assessors/markers

• Administration guidelines include clear details of the conditions under which the assessment should take

place (Also in Criteria 3 and 4, plus reasonable adjustment) Criterion 3: Fairness

• The assessment tasks were tested and trialled with different groups of learners and checked for

performance and feedback gathered from different learners Criterion 4: Well-constructed

• The LLN assessment tool is well constructed with quality presentation and quality and clear sets of

instructions for both the learners being assessed and for the administrators of the assessment This list of criteria has been adapted from the ACER list for developing tools for VET FEE-HELP purposes. It is provided as a guide for developing less formal tools. NOTE: VET FEE-HELP tools MUST meet ALL of the original requirements.

Validating your own LLN Assessment Tools

1 2 3

We have discussed LLN and ACSF We have identified the LLN demands of workplaces and training materials and training processes. We have looked at ways of identifying learner LLN levels, using validated tools.

We have put all of this together

Dave Tout

Numeracy: A Core and Crucial Skill in VET