09/06/2015 Assessing Language Skills in Young Children: Identifying - - PDF document

09/06/2015 1

Assessing Language Skills in Young Children: Identifying the issues for professionals when assessing language skills. Julie Dockrell Language, Literacy and Numeracy Centre: research and practice Purpose

• Raise key issues which

practitioners could consider when they examine young children’s language skills

Plan of presentation

• 1. Language and literacy
• 2. Language system
• 3. Why assess language skills
• 4. Screening v. assessment
• 5. Ways forward
• 6. Key practitioner messages

3

09/06/2015 2 Language and Literacy

• Language underpins literacy

– Word decoding – Reading comprehension – Spelling – Text production

• Literacy supports the development of oral language

– Word – Sentence – Text level – Genres

4

The effects of poor language

5

Hirsch, 1996

The Matthew Effect: performance differences between good and poor readers may increase

• ver time (Stanovich, 1986)

Exposure to orthography benefits vocabulary acquisition

• For typically

developing children

• & those with

developmental challenges

6

Ricketts, J., Dockrell, J., Patel, N., Charman, T., & Lindsay, G. (2015). Do children with specific language impairment and autism spectrum disorders benefit from the presence of orthography when learning new spoken words?. Journal of Experimental Child Psychology, 43–61. 10.1016/j.jecp.2015.01.015

09/06/2015 3

Bidirectional relationship

7

Mutually supportive or mutually limiting?

Which aspects of the language system for

• 1. Which aspects of reading?
• 2. Which aspects of writing?
• 3. Which languages/orthographies?

Today’s question which form of assessment?

KEY QUESTIONS

The language system -1

• Essential to understand

– To identify strengths and needs – Distinguish between structural and pragmatic aspects of language

• Both develop through an interaction between

– The intrinsic capacities of the child and the context in which he is developing

(Thomas, 2010).

– Also complex interplays between the subcomponents of the language system

(Dixon & Marchman, 2007; Tomblin & Zhang, 2006).

• Draw on other cognitive skills to support language learning e.g.

memory

9

09/06/2015 4 The language system -2

Structural aspects of the system

• Lexicon (vocabulary),
• Syntax (the rules for combining words into phrases and sentences),
• Morphology (the rules for constructing larger words out of smaller

units of meaning),

• Phonology (the sounds that make up words and the rules that

combine sounds) Pragmatics (the rules of social communication). SO – when we think about language assessment we need to think about which skills at which point in development and in relation to which literacy dimensions

10

Language delays and difficulties (1)

• Occur for a range of (not mutually exclusive reason)

– Social disadvantage

• Long-standing acknowledgement that poor language skills are associated with social

disadvantage

• Prevalence rates of language delays in disadvantaged populations are high, but rates of

identification are often low (King et al., 2005).

• Moreover, the poorest outcomes are disproportionately associated with the most

socially and economically disadvantaged (Washbrook & Waldfogel, 2010)

– Different dialects and bilingualism

• Growing concern that children from ethnic minority groups are over-represented in the

caseloads of speech and language therapists and are over-identified generally as having speech language and communication needs (Dockrell, Lindsay, Roulstone & Law, 2014).

• Awareness that non-standard varieties of English differ from the Standard English that

language assessments are designed to test.

• Children should not be viewed as having a speech or language disorder because they

speak a variety of English other than the standard dialect 11

Language delays and difficulties (2)

• Hearing impairment

– Children who experience deafness, and even mild or unilateral hearing impairment, typically experience delays in receptive and expressive language development.

• Unexplained difficulties to the language system

– Large group of children who experience language delays for no obvious reason.

– Discrepancy criteria (cognitive referencing) used in the past (language skills and non-verbal ability)

• concerns about measurement and the determination of the appropriate

formula for the discrepancy (Aram, Morris & Hall, 1992; Plante, 1998).

• Language problems may also impact on children’s performance on non-verbal

tasks, thereby affecting assessments of non-verbal ability.

• DSM-5 does not include a discrepancy criterion for language disorders.
• No differences in response to oral language intervention have been found for

children with and without discrepancies between their verbal and non-verbal performance (Bowyer-Crane et al., 2011; Friel-Patti, 1999).

12

09/06/2015 5

Why assess children’s language skills?

• Part of the curriculum to monitor progress

– How did you assess speaking and listening

• Screening
• Pre-intervention and post-intervention measures to evaluate

the impact of oral language interventions e.g. Talk of the Town.

• Identify potential targets to support attainment and access to

the curriculum

13

Test properties

• Reliable

– If you give it twice would you get the same result

• Valid

– Measures what it is suppose to measure – name of the test won’t tell you enough

• Fit for purpose

– Time, child and location constraints

• Standardized on an

appropriate population

– Number of children – Social context – Recent – Standardization sample representative

• Bus story is a test of narrative

recall

• 3;6 and 7.
• assessor tells a story about a

naughty bus and the child is asked to repeat it

• No restrictions
• Scoring challenges
• Standardization 513 children

south east of England

• Reliability N = 13
• Validity on 27 children

14

Screening versus assessment

SCREENING Process to identify whether or not a child is functioning at an expected level

– Sensitivity accurately identifies children as cases who have language problems – Specificity measure does not identify as cases children who do not have a language problem.

Trade off between the two, depending on the purpose of the screening. ASSESSMENT

• Characterise nature and extent
• f the problem

– What – How severe

• Guided by

– Initial evaluation of the child – Theoretical orientation – Developmental level – Practical constraints

15

09/06/2015 6

Screen identifies language problems correctly No language problems correctly

THE SCREENING TRADEOFF

Screening

• Many tests do not meet these basic criteria for screening purposes.
• Studies have consistently raised concerns about the ability of screening

tests to detect children with concurrent language problems, that is problems at the time of testing (de Koning et al., 2004; Laing, Law, Levin, & Logan, 2002).

• Screening measures to predict the likelihood of a child experiencing

language difficulties in the future is fraught with difficulties.

– Studies that have attempted this have been unsuccessful in identifying language factors which predict future performance (Law, Rush, Anandan, Cox, & Wood, 2012; Nelson, Nygren, Walker, & Panoscha,

2006a; Wilson, McQuaige, Thompson, & McConnachie, 2013).

• As Snowling et al (2012) concluded, regular monitoring is preferable

because one-off screenings of aspects of development, including language and reading, have limited power to predict later performance because children’s developmental trajectories vary

17

Assessment 1:Standardised tests of oral language

• Many child language tests are commercially available

– Oral language composite scores (omnibus measures)

• Overall standard score – receptive and expressive

– Target specific components of the language system

• Phonology, vocabulary, grammar
• Can be either receptive or expressive
• Often but not always restricted in use to psychologists and speech

and language therapists

• Not all standardised in the UK
• You need to think about what you want to know and whether the

assessment is ‘fit for purpose’

18

09/06/2015 7

Assessment 2: Composite Language Measures

• CELF instruments most commonly used measures of assessing language

internationally

– focus of a number of psychometric studies (Eadie et al., 2014; Spaulding et al., 2006). – Overall acceptable levels of specificity and sensitivity data (Spaulding et al.’s, 2006)

• Recent Australian research has indicated that the CELF-P2 does not demonstrate

adequate levels of sensitivity (64%) to identify children with language disorders at age 5

(Eadie et al., 2014).

– Test-retest measures are good. – But reliability of subscales often questionable , in particular Sentence Structure in the CELF- P2 (Eigenbrood, 2007).

• Particular importance when wanting to compare across language and other

skills to have tests standardised on the sample population and following same test construction principles

– Wechsler Individual Achievement Test (3rd edition: WIAT-III) includes both a listening comprehension and oral expression scale. – The WIAT-III subtests have strong psychometric properties and there is evidence to support the use of subtests with special populations (Miller, 2010). – The WIAT-III has the added advantage of comparing across composite scores such as oral language and reading comprehension. 19

Assessment 3:Single elements of the language system

• Single measures of language

– Inadequate for determining whether a child is developing typically or is experiencing a delay at any age, and they become less reliable the younger the child (Thal & Katch, 1996). – When the measures are reliable and valid when combined with other forms of assessment, provide a profile of a child’s strengths and needs.

• Vocabulary BPVSII

– Concurrent validity with other language measures is not high – Vocabulary scores cannot be used as though they were indicators of general language ability (Gray, Plante, Vance, & Henrichsen, 1999; Spaulding, Hosmer, & Schechtman, 2013) – Some children with language disorders vocabulary scores can be well within the norm, despite wider problems with receptive and expressive language (Friberg, 2010; Spaulding et al., 2013). – Should not be used as the sole measure to identify children with language difficulties (Longo, 2005).

20

Assessment 3:Single elements of the language system

• Sentence repetition (sentence imitation and sentence recall)

– Long history in psycholinguistic research (Rodd & Braine, 1971) and language assessment (Schwartz & Daly, 1978). – Assumption is that children will only be able to repeat structures that are part of their language system.

• Many studies have shown that sentence repetition is significantly less accurate in

children with developmental language disorders (see Conti-Ramsden, Botting & Faragher, 2001;

Riches, Loucas, Baird, Charman & Simonoff, 2010,

• Children who are not native speakers of the test language (Komeili & Marshall, 2013).
• Conventional language tests elicit production and test

comprehension using artificial tasks. By contrast, narrative tasks provide a more naturalistic setting to examine children’s language skills e.g Bus story

– Much harder to assess – Get reliable and valid results

21

09/06/2015 8

Assessment 4:Dynamic assessment

• Static tests only give a snapshot in time and do not reveal why children

perform poorly.

– might perform poorly on a test for a variety of reasons, and these differences could potentially be important for intervention.

• Interest in an interactive approach to conducting assessments that focuses
• n the child’s ability to respond to intervention, i.e. his capacity for change
• r “modifiability”.

– “dynamic assessment”, and unlike traditional testing, it employs a test, teach, retest procedure to assess the child’s learning processes. – Dynamic assessment is considered more culturally fair to those from different linguistic or cultural backgrounds (Lidz & Peña, 2009) – More sensitive for measuring change in language over time (Hasson & Botting, 2010). – Distinguish between children whose language is delayed, but whose capacity for learning language is not impaired whether monolingual or bilingual (Hasson et al., 2013; Peña, Resendiz & Gillam,

2007). –

Dynamic Assessment

• Links well with RTI models
• Can be fine tuned to language in the classroom

22

Assessment 5: Checklists language

• Checklists of early language skills, to be completed by parents and

professionals, exist.

– Many of these have not been validated psychometrically in terms of reliability and validity (see for example Mok & Lam, 2011). – The best researched are the Communication Development Inventories (see Law & Roy, 2008 for a

review).

• Using parental report data can be helpful to gain a broader perspective of a

child’s language skills and when children are difficult to assess.

– Checklists are inexpensive to use, and additional training is not required (Hall & Segarra, 2007;

Nordahl-Hansen, Kaale, & Ulvund, 2013).

– Differences between respondents, such as their background, may affect how they report their children’s language skills, care needs to be taken in interpreting the results (Pan, Rowe, Spier, &

Tamis-Lemonda, 2004).

– Importantly, given the variability in language trajectories, checklists are not reliable in identifying children who will go on to experience language delays (Law & Roy, 2008). 23

Key Practitioner message

• Language is a complex system to assess, comprising a range of

subsystems

• Regular monitoring of language is preferable; one-off screenings have

limited power to predict later performance because children’s developmental trajectories vary

• Composite language measures provide more reliable and valid

assessments of children’s language skills

• Many assessment tools are not suitable for the range of preschool children

who experience language delays and problems

• Dynamic assessment is more culturally fair to those from different linguistic
• r cultural backgrounds and may be more sensitive for measuring change in

language over time

24

09/06/2015 9

Remember

• If a child can’t say it and understand it orally

– Why would you expect them to able to write it or understand what they read

• There are an increasing number of effective

interventions that support oral language skills

• You can make your classroom/group work a

communication supporting environment but its tough!

25

Thank you for your attention

• Collaborator Dr. Chloe Marshall
• For further information
• Measurement Issues: Assessing language skills in young children

26