Understanding standardised assessments in everyday practice - - PowerPoint PPT Presentation

Understanding standardised assessments in everyday practice

Claire Parsons Assessment Solutions Consultant Pearson Clinical 17th June 2019

1

Rookwood Driving Battery (RDB)

Claire Parsons

Assessment Solutions Consultant Pearson Clinical Assessment

Aims of the workshop

3

• Explain the principles of administering and

interpreting standardised assessments

• Gain an understanding of the considerations to

be made when selecting appropriate standardised assessments

• Understand and apply the principles of

standardised assessment within the

• ccupational therapy process
• Understand the principles of test interpretation
• Gain a clearer understanding of statistical

concepts

• Describe the learning of standardised

assessments as a Continuing Profession Development (CPD) activity

Agenda

• CPD requirements UK Occupational Therapists
• The Assessment Process
• Getting Started
• Understanding scoring
• Interpretation of scores
• Questions

CPD Requirements

CPD can be defined as: “a range of learning activities through which health and care professionals maintain and develop throughout their career to ensure that they retain their capacity to practice safely, effectively and legally within their evolving scope of practice”

Health and Care Professions Council (HCPC)

CPD Requirements in the UK

• 1. Standards of proficiency

‒ Meeting the standards ‒ Scope of practice

• 2. CPD and Audit Process

‒ Maintain a record ‒ Demonstrate ‒ Contributes to quality of practice ‒ Must benefit the service user[s] ‒ Written profile

• 3. Examples of CPD activities

‒ Work based learning ‒ Professional activities ‒ Formal education ‒ Self-directed learning

Guidelines

• Principles for continuing professional development and lifelong

learning in health and social care – January 2019

• Aligned with and complimentary to the HCPC guidelines (does not

replace these)

• Underpinning concept = critical reflection

The Assessment Process

Selecting the right test

• Depends on
• Tests are designed for different purposes
• The tests should

The Assessment Process

• 1. Hypothesis testing

‒ A problem solving process ‒ Identifying strengths

• 2. Types of Assessment

‒ Screeners ‒ Non-standardised ‒ Criterion-referenced ‒ Standardised ‒ Full battery ‒ More specific ‒ Predictors for problems in everyday life

The Assessment Process

Standardised Assessment: Key Features

1. Allows you to compare your patients score to a criterion 2. Is always administered and scored in exactly the same way 3. Comparisons made using standardised scores 4. The aim is to make a decision

The Essentials of Getting Started

Where to Start…

• 1. Understand the population for which the assessment is intended
• 2. What service needs will the assessment meet
• 3. How was the assessment developed
• 4. How are assessment findings reported
• 5. Practice administration

How was the Assessment Developed…

• 1. Theoretical Underpinnings
• 2. Data Collection

Evidence of Consistency & Usefulness… Reliability

• Inter-rater reliability
• Test-retest reliability
• Parallel form reliability

Evidence of Consistency & Usefulness… Validity

• Construct validity
• Content validity
• Criterion validity
• Face validity
• Ecological validity

How we Measure Reliability and Validity: Correlation Coefficients

• Measure of the degree of relationship between 2 variables e.g. age

and scores on a test

• Range +/- 1 to 0
• +1 and –1 are perfect correlations in different directions
• 0 = no relationship between the variables
• Correlation of .6 = acceptable, .8 or higher = very good

Understanding Scoring

How are assessment findings reported?

The patient’s score is compared to the norms - the scores of the reference group The aim is to decide whether:

• patient’s performance is evidently impaired
• the patients performance is significantly different to what you

would expect in view of age, ability etc

Normal Distribution

• Distribution of test scores for large

groups of heterogeneous test takers

• Mean, median, mode should all

have same value

• 68.26% of population should fall

within 1 Standard Deviation (S.D.) of the mean

• Few extremes
• Scores must be normally distributed

for these to be useful

Normal Distribution:

Standard Scores

• Mean of 100
• r
• Mean of 10

Advantages of standard scores:

• allow comparison
• take account of standard deviation
• see where differences are extreme, abnormal or

very unlikely

• clear cut definition of impairment
• most commonly used form of scoring on tests

Age & Standard Scores - Example

Two patients with identical raw scores on the Doors and People test Patient A - aged 29 years Patient B- aged 81 years Subtest Raw score Standard Standard Score A Score B Verbal recall 14 4 9 Visual Recall 22 3 10 Verbal Recog 12 4 8 Visual recog 14 5 10

Interpreting a Composite Standard Score

Less than 70 More than 2 standard deviations from the norm Significant Impairment 70 - 85 More than 1 standard deviation from the norm Moderate impairment 85 - 115 Within 1 standard deviation of the norm Average range 115 - 130 More than 1 standard deviation from the norm Above average range Over 130 More than 2 standard deviations from the norm Very high / superior

Interpreting Subtest Level Scores

Less than 4 More than 2 standard deviations from the norm Significant Impairment 4 - 7 More than 1 standard deviation from the norm Moderate impairment 7 - 13 Within 1 standard deviation of the norm Average range 13 - 16 More than 1 standard deviation from the norm Above average range More than 16 More than 2 standard deviations from the norm Very high / superior

Percentiles

• Reflect a kind of rank ordering
• indicate the proportion of people obtaining a score EQUAL TO OR

LESS THAN a given raw score

• 1st percentile - low level
• 99th percentile - high score

Interpreting Percentiles

Less than 2 More than 2 standard deviations from the norm Significant Impairment 2 - 16 More than 1 standard deviation from the norm Moderate impairment 16 - 84 Within 1 standard deviation of the norm Average range 84 - 98 More than 1 standard deviation from the norm Above average range More than 98 More than 2 standard deviations from the norm Very high / superior

Interpretation of Scores

Defining Impairment

• With raw scores
• With standard scores
• With percentile ranks

What do scores tell us?

• What does a score in the impaired range tell us?
• What does a score in the average range tell us?

Interpretation of Scores

• A score on a test does not prove or disprove anything
• It just gives an estimate of performance in a particular area of behaviour
• Must be interpreted in the wider context of results from a range of tests
• Take into account previous abilities and achievements

Confidence Intervals

• Reflect the uncertainty in scores arising from measurement error i.e. – you

would not expect the same score on every single occasion.

• 95% confidence band = you are 95% confident that the true score will lie within

this band.

• Based on the reliability measurements of the test
• The higher the confidence interval is the wider the band will be

Composite vs Subtest Level Interpretation

• Composite more robust
• Some subtests are sufficiently robust to use in isolation
• Comparing subtest level data helps build understanding about an

individual’s profile

• Statistical difference vs clinical importance

The profile

38

Communication of Results

• Exercise caution
• Support suggestions
• Rule out
• Adapt your style of reporting
• For other professionals who are familiar with cognitive assessment
• Take appropriate steps regarding confidentiality.

Questions?

There’s so much more to learn

Find out more about us at Pearsonclinical.co.uk