low level EAP be assessed? Bruce Howell, University of Reading BH - - PowerPoint PPT Presentation

▶
low level eap be
SMART_READER_LITE
LIVE PREVIEW

low level EAP be assessed? Bruce Howell, University of Reading BH - - PowerPoint PPT Presentation

Apr 26, 2023 137 likes •383 views

How should, and how can, low level EAP be assessed? Bruce Howell, University of Reading BH How should, and how can, low level EAP be assessed? BALEAP SIG Bristol Feb 2011 Why assess? (e.g. Reading) EACH COURSE: beginning:

slide-1
SLIDE 1

BH How should, and how can, ‘low level EAP’ be assessed? BALEAP SIG Bristol Feb 2011

Bruce Howell, University of Reading

How should, and how can, ‘low level EAP’ be assessed?

slide-2
SLIDE 2

BH How should, and how can, ‘low level EAP’ be assessed? BALEAP SIG Bristol Feb 2011

Why assess? (e.g. Reading) Oct-Dec

  • EACH

COURSE:

  • beginning:

placement

  • end: progress

(proficiency)

Jan-Mar Apr-Jun Jul-Sep

slide-3
SLIDE 3

BH How should, and how can, ‘low level EAP’ be assessed? BALEAP SIG Bristol Feb 2011

‘Placement’ – getting on the right track

  • 1. "is this candidate at least the minimum level to

join the pre-sessional EAP programme?"

  • 2. "which (e.g. writing) group within the EAP pre-

sessional programme should this student be in?"

  • 3. ‚what specific area needs to be addressed

within the EAP in-sessional programme?‛ (Read 2008)

slide-4
SLIDE 4

BH How should, and how can, ‘low level EAP’ be assessed? BALEAP SIG Bristol Feb 2011

‘Placement’ – getting on the right track

  • 1. programme screening
  • yes/no
  • 2. class placement
  • multi-level
  • 3. diagnostic
  • descriptions not numbers?
slide-5
SLIDE 5

BH How should, and how can, ‘low level EAP’ be assessed? BALEAP SIG Bristol Feb 2011

‚It must be practical‛ (1)

  • time

restraints

  • e.g. British

Council (O’Sullivan 2010)

slide-6
SLIDE 6

BH How should, and how can, ‘low level EAP’ be assessed? BALEAP SIG Bristol Feb 2011

‚It must be practical‛ (2)

  • 10 minutes Dictation (gap-fill sentences)
  • 10 minutes Kanji ( spelling)
  • 30 minutes Writing
  • 5 minutes Speaking (interview)
  • 3 minutes self-assessment questionnaire

(Toyoda & Hashimoto 2001) ** The more data the better ** BUT a minimum of data may be enough for your situation

slide-7
SLIDE 7

BH How should, and how can, ‘low level EAP’ be assessed? BALEAP SIG Bristol Feb 2011

But what about EAP?

  • TEEP: listening and reading into writing; task-based

speaking; academic perspective on topics

  • process approach can be replicated in (placement) tests

(Lee 2005; Plakans 2009)

  • …but is this necessary for low levels?
  • experience of TEEP: below ~4.5 can ‚blow it completely‛
  • so perhaps the approach should be:

– general language competence – university context

slide-8
SLIDE 8

BH How should, and how can, ‘low level EAP’ be assessed? BALEAP SIG Bristol Feb 2011

How should ‘low level’ EAP be assessed?

  • Reading Writing Listening Speaking

– yes but … perhaps not possible

  • what about just a simple grammar test?

– NO! – ‚use of a single test format is inadequate‛ (Green and Weir 2004)

  • why not just use international tests or school results?

– not always available – expensive – bland, generalised – not always trustworthy, despite claims (Ypsilandis & Nantaridou 2007)

slide-9
SLIDE 9

BH How should, and how can, ‘low level EAP’ be assessed? BALEAP SIG Bristol Feb 2011

Popular suggestions

  • grammar test
  • vocabulary test
  • independent writing test
  • independent reading test
  • interview
slide-10
SLIDE 10

BH How should, and how can, ‘low level EAP’ be assessed? BALEAP SIG Bristol Feb 2011

More suggestions…

  • linked reading  speaking task

– ‘two-in-one’ – going beyond formulaic language

  • dictation

– ‚high correlation with grammatical knowledge AND oral/aural proficiency‛ (Ford-Niwa 1998, cited in Toyoda & Hashimoto 2001)

  • a staged/modular procedure:

– not all sections are needed (Fulcher 1997) – a pre-test screening section?

  • self-assessment
slide-11
SLIDE 11

BH How should, and how can, ‘low level EAP’ be assessed? BALEAP SIG Bristol Feb 2011

DELNA (Read 2008)

slide-12
SLIDE 12

BH How should, and how can, ‘low level EAP’ be assessed? BALEAP SIG Bristol Feb 2011

Online OPT

slide-13
SLIDE 13

BH How should, and how can, ‘low level EAP’ be assessed? BALEAP SIG Bristol Feb 2011

British Council (O’Sullivan 2010)

slide-14
SLIDE 14

BH How should, and how can, ‘low level EAP’ be assessed? BALEAP SIG Bristol Feb 2011

Self-assessment

  • does it work?
  • research gives differing results (Dandenault 1997)

for: against:

  • practicality – saves

time/money

  • involves the learner

in the decision- making process

  • increases self-

awareness

  • advantages have a place in

classrooms, not tests

  • unconvincing for teachers
  • inviting exaggeration ('cheating'?)
  • influence of personality factors
  • tendency to err towards the

'average' – not to rank high or low

slide-15
SLIDE 15

BH How should, and how can, ‘low level EAP’ be assessed? BALEAP SIG Bristol Feb 2011

…need to ask meaningful questions

slide-16
SLIDE 16

BH How should, and how can, ‘low level EAP’ be assessed? BALEAP SIG Bristol Feb 2011

…don’t be too simplistic!

(Gere et al 2010)

slide-17
SLIDE 17

BH How should, and how can, ‘low level EAP’ be assessed? BALEAP SIG Bristol Feb 2011

How can ‘lower level’ EAP be assessed (practical solution)

  • depends on your context!
slide-18
SLIDE 18

BH How should, and how can, ‘low level EAP’ be assessed? BALEAP SIG Bristol Feb 2011

Computer-based?

  • … if you have the resources!

for: against:

  • simple, from the

administration point of view

  • data collection

automatic

  • scoring can be instant
  • certificates can be

automatic

  • attractive for some
  • restrictive of item type
  • impersonal (unless

complemented with interview etc.)

  • off-putting for some
slide-19
SLIDE 19

BH How should, and how can, ‘low level EAP’ be assessed? BALEAP SIG Bristol Feb 2011

How to check your test is working

  • trial/pre-test it first – if possible

– imperative (Fulcher 1997)

  • get teachers’ views on suitabilty
  • ask students about their experience
  • analyse score data – eliminate problem items
  • review cut level(s) – are they living up to

expectations?

  • how does it compare to scores on IELTS etc?

– (concurrent validity)

slide-20
SLIDE 20

BH How should, and how can, ‘low level EAP’ be assessed? BALEAP SIG Bristol Feb 2011

… but remember

  • we cannot assume the test ‘is not working’

just because students do not increase their level in a linear way

  • depends on:

– motivation/completion of work/attendance – aptitude for languages – external factors

  • e.g. distractions outside class

– etc.

slide-21
SLIDE 21

BH How should, and how can, ‘low level EAP’ be assessed? BALEAP SIG Bristol Feb 2011

Most important

  • assessments must have a system

– essential to have stakeholders on board (Read 2008) – must be easily understood by all – follow-up plans (e.g. split profile) – who does the decision-making?

  • advise potential students so that they can make

the right decision

  • programme screening test:

– short test may be OK, but it’s high-stakes … – so it has to have proven reliability and good security

slide-22
SLIDE 22

BH How should, and how can, ‘low level EAP’ be assessed? BALEAP SIG Bristol Feb 2011

Bibliography

  • …on request!
slide-23
SLIDE 23

BH How should, and how can, ‘low level EAP’ be assessed? BALEAP SIG Bristol Feb 2011

What do YOU think?