ASCOT TRAINING: LEEDS AND LONDON 2013 Nick Smith, Ann Marie Towers, - - PowerPoint PPT Presentation

ASCOT TRAINING: LEEDS AND LONDON 2013

Nick Smith, Ann‐Marie Towers, Juliette Malley, Stacey Rand, Ann Netten and Julie Beadle‐Brown

Outline of training day

• Introduction to ASCOT
• Measuring outcomes using ASCOT
• Versions of ASCOT
• Self‐completion (SCT4 / SCT3)
• Lunch (12.30 ‐ 1.30)
• Face to face interview (INT4)
• Mixed methods (CH3)
• Scoring and interpreting ASCOT
• Close (3pm)

INTRODUCTION TO ASCOT

What is ASCOT?

• Adult Social Care Outcomes Toolkit

– A collection of tools for measuring service user

• utcomes across the whole of social care including

a version for use in care homes. – See website www.pssru.ac.uk/ascot

Why use ASCOT?

• Policy emphasis on outcome measurement
• Individual Social Care Related Quality of Life

(SCRQoL)

• Development involved rigorous testing
• Can be used to measure the impact of services
• Reflects preferences

One tool does not fit all

• Across settings and client groups

– Community settings – Day care – Care homes

• Using different methodologies:

– Self‐completion – Interview – Observation

29/04/2013 6

HOW ASCOT MEASURES OUTCOMES

What do we mean by outcome?

• Quality of life (very broad)
• Social care related quality of life

– Domains we can reasonably expect to be affected by social care – Broad enough to apply to everyone – Compare to the general population

• ASCOT measures SCRQoL at the level of

the individual

ASCOT domains

• Personal cleanliness and comfort
• Food and drink
• Safety
• Clean and comfortable accommodation
• Social participation and involvement
• Control over daily living
• Occupation
• Dignity

9

Level of outcome

• No needs: The individual has no or the type of temporary

trivial needs that would be expected in this area of life of someone with no impairments. – This can be split into two

• Some needs: Some needs are distinguished from no needs

by being sufficiently important or frequent to affect an individual’s quality of life – Some needs do not have a health implication

• High needs: High needs are distinguished from some needs

by having mental or physical health implications if they are not met over a period of time. This may be because of severity or number. – High needs have a health implication.

The levels…

• No needs

– Ideal state: The individual’s preferences and wishes in that area are met. – No needs: The individual’s needs are met, but not to the desired level (mustn’t grumble)

A summary of the levels

• Ideal state
• No needs
• Some needs
• High level needs

What can ASCOT measure?

• Current SCRQoL
• Expected SCRQoL
• SCRQoL gain

29/04/2013 13

Current SCRQoL

• All versions of ASCOT measure current SCRQoL
• Like any other QoL measure
• Measures SCRQoL as it currently stands.
• In most uses of ASCOT, this would usually mean

the SCRQoL the person has as a result of receiving services and support – not a functioning measure like e.g. EQ5D

• The only exception may be when you use ASCOT

to measure baseline SCRQoL before somebody starts to use a service.

Expected SCRQoL

• Only in some versions of ASCOT
• Expected SCRQoL = what we would expect

somebody’s SCRQoL to be in the absence of services and support & assuming no other help steps in.

• The definition of services and support

depends upon the context in which you are using ASCOT

29/04/2013 15

Why measure expected SCRQoL?

• To identify impact of social care interventions
• Need a counterfactual, but RCTs difficult

(unethical, expensive)

• Expected SCRQoL = the counterfactual
• No help steps in – get full value of

intervention, assumptions

29/04/2013 16

• Dignity is the one domain where we do not

measure expected SCRQoL – WHY?

29/04/2013 17

SCRQoL gain

• When we measure both expected and current

SCRQoL we can estimate the impact of the service

• Impact = current SCRQoL ‐ expected SCRQoL
• We call this SCRQoL gain

Summary

• ASCOT measures SCRQoL
• 8 domains
• 3 or 4 levels
• Current and expected SCRQoL
• Calculating impact

VERSIONS OF ASCOT

Versions of ASCOT

• Self completion tools (SCT3 and SCT4)

– 3 level version developed for day care – 4 level version

• Interview schedule (INT4)

– Structured version – 4 levels

• Care homes version (CH3)

– 3 level ratings based on

• Observations
• Interviews – residents/staff/relatives
• Why do we have so many different versions?

Comparing the ASCOT instruments

SCT4 SCT3 INT4 CH3 Format Self‐completion questionnaire Self‐completion questionnaire Face to face interview Mixed methods Number of levels 4 3 4 3 Current SCRQoL Yes Yes Yes Yes Expected SCRQoL No** Yes** Yes Yes Number of questions 9 18 23 n/a Intended use All services where service users able to self‐complete a questionnaire Developed for use in day care All services where service users able to participate in a face to face interview Residential setting such as care or nursing homes Training No No Yes, limited Yes, extensive

SELF COMPLETION (SCT3 & SCT4)

29/04/2013 23

29/04/2013 24

Examples of applications

• Adult Social Care Survey

http://www.ic.nhs.uk/article/2215/User‐Experience‐Survey‐Adult‐Social‐Care‐ Guidance‐2012‐13 http://www.ic.nhs.uk/article/2021/Website‐ Search?productid=10128&q=ascs&sort=Relevance&size=10&page=1&area=both#top

• Monitoring performance
• Assessment / review

29/04/2013 25

Group Task 1: Filling in SCT4

• In groups (4‐5), think of someone you know

who receives social care

• As a group, fill in SCT4 questionnaire for that

person

• Note down any questions / queries about

SCT4

• 15 mins for task
• Enter scores from each group into the ASCOT

data entry tool

29/04/2013 26

Summary

• Measures current SCRQoL
• Simple & quick to complete
• Algorithm for expected SCRQoL & SCRQoL gain

for SCT3 & to come for SCT4

• Any questions?

29/04/2013 27

FACE TO FACE INTERVIEW: INT4

29/04/2013 29

INT4 Interview Schedule

• Instructions for interviewer
• Introduction
• 23 questions across the 8 domains
• For each domain (except dignity)

– Current SCRQoL – Filter question

• If yes, then…

– Expected SCRQoL

• Plus some other things (more on this…)

Task 2: Filling in INT4

• In pairs, one person be interviewer and one

imagine you are a service user

• Take 15 mins to fill in INT4
• Swap around if you are finished quickly
• Note down any questions / queries about INT4
• Enter scores from pairs into the ASCOT data

entry tool

29/04/2013 31

AT1

Slide 31 AT1 I think they wil struggle to get INT4 done one way so probably ambitious to think they'll both be the interviewer in a 15 min task - or are you allowing 30 mins for this task?

Ann-Marie Towers, 12/03/2013

Points for discussion

• Interviewer instructions – training
• Introduction
• Current‐filter‐expected questions

– Defining support and services – Sensitivity

• Dignity questions
• Any other queries?

29/04/2013 32

Occupation filter question

• Filter question

Do the support and services that you get from <<EXAMPLES>> affect how you spend your time?

Things to remember: 1. Emphasise the domain of QOL

The question is asking if support and services help in (a positive or negative way) that aspect of life, not in general

2. Definition of Support and Services

29/04/2013 33

Defining Support and Services (1)

• Define support and services according to your

study

– e.g. if want impact of home care ask about home care, publicly‐funded social care ask about publicly‐funded, etc

• May be difficult to define in some instances

– e.g. where service is ill‐defined, infrequent, there is complexity & source of funding is unclear

29/04/2013 34

Defining Support and Services (2)

• Need to explore service receipt & funding in detail
• Collect information about service receipt before

ASCOT questions.

– Be careful with how you name services… – If possible use tested questions e.g. Natcen social care module (although complicated – requires CAPI)

• Use information about service receipt as prompts

within questions i.e. <<EXAMPLES>>

• Can be helpful to say what you don’t mean
• Take‐away message: Be very clear & use examples

29/04/2013 35

Example filter question from a study

Do the support and services that you get from <<the day centre, meals on wheels or your personal assistant>> affect how you spend your time? Interviewer Prompt: By support and services we mean, for example, <<the day centre, meals on wheels or your personal assistant>> Please do not include help from health professionals, such as GPs and nurses, or from friends and family.

29/04/2013 36

Occupation expected SCRQoL

Imagine you didn’t have the support and services from <<the day centre, meals on wheels or your personal assistant >> that you do now, and no other help stepped in. In that situation, which of the following would best describe how you’d spend your time? INTERVIEWER NOTE: It is important that people do not base their answers on the assumption that any help steps in, please emphasise this to interviewees. REASSURE IF NECESSARY: please be assured that this is purely imaginary and does not affect the services you receive in any way.

29/04/2013 37

Dignity question

• Follows a different structure

–Current only (effect of way helped) –No services help filter or expected

• Filter question before dignity question

–Not used in ASCOT scoring –Ensures people respond to dignity question in right way

29/04/2013 38

Summary

• Measures current and expected SCRQoL
• Can be used to measure SCRQoL gain – impact
• Careful thought needs to be given if using expected

questions to how support and services is defined

• Recommend including a module about service receipt

and funding before ASCOT questions

• Do NOT recommend using over telephone
• READ OUT introduction & prompts!
• Recommend some training for interviewers

ASCOT MIXED METHODS: CH3

Mixed Methods (CH3)

• For use in residential settings such as care

homes

• Uses mixed methods

– Observation – Face to face interviews – Measures both current and expected SCRQoL

Levels in care homes

• 3 levels of needs

– No needs – Some needs – High level needs

• Why not 4 levels?

Measuring need amongst care home residents

• In an ideal world, we’d ask them!

– Do you feel in control of your daily life?

• But sometimes people can’t tell you
• Large proportion of care home residents have severe and

complex needs (severe and profound LD, dementia, autism etc.)

• Asking their views is difficult and can be unreliable for a

variety of reasons

• Other options include

– Asking a representative (limitations to this approach) – Observation

Collecting evidence

• For each resident following data gathered

– Notes from an observation to reflect their lived experience – A structured interview with staff about the resident’s SCRQoL – A structured interview with a family member about the resident’s SCRQoL (where possible) – An interview with the resident (again where possible – format also variable) – Other data?

Hierarchy of evidence

• 1. Resident’s views
• 2. The observation
• 3. Family member interview
• 4. Staff interview

Using this version of ASCOT

• Training is highly recommended
• Need to be really familiar with the domains
• Need to understand what the levels of
• utcome look like for each domain
• Gathering evidence and making a judgement

can be tricky and requires practice.

• We run a 2 day training course at Kent

29/04/2013 46

Rest of this session

• Not going to rush through an outline of the

course

• Time better spent illustrating why this

approach can be helpful

• Show you some short video clips (some of

which we use in the course)

29/04/2013 47

Clip number 1

• Cynthia who loves to garden
• Example of really good person‐centred ‘active

support’

• About giving a person just enough support to

achieve what they want to achieve but not so much that you are doing it for them.

• What ASCOT domains might this clip feed

into?

29/04/2013 48

CLIP REMOVED

Clip number 2

• Primrose: older lady with a learning disability
• Spent most of her life living in care homes
• Had a reputation for ‘challenging behaviour’
• When Primrose is happy/occupied, she is

quiet

• When she is not happy/bored, she swears and

curses

• What ASCOT domains does this clip feed into?

29/04/2013 50

CLIP REMOVED

Clip number 3

• Sometimes people say that domestic tasks are

not ‘worthwhile’ activities

• Do all activities have to be social or ‘special’ to

be worthwhile?

• Some service users have never had the
• pportunity to try ‘normal’ domestic tasks
• Need to be given the opportunity and

assistance to try it before we decide how worthwhile it is…

29/04/2013 52

CLIP REMOVED

Summary

• Observation offers an opportunity to represent

the lived experience of residents

• Can tell you a lot about staff/resident interactions
• Gives service users who would otherwise not

have their views represented an opportunity to take part in research

• Facial expressions, gestures and body language

can tell us a lot about how people feel

29/04/2013 54

Back to ASCOT

• During the two day course we spend a lot of

time talking about what ‘good’ care looks like

• This is important when trying to judge

whether a service user has needs in a domain

• If you are interested in learning more about

this approach, send us an email (ascot@kent.ac.uk)

• Any questions?

29/04/2013 55

SCORING AND INTERPRETING ASCOT

Scoring and interpreting ASCOT

• The course has looked at the ASCOT tools and

we have seen the different ways in which we can collect data on the needs (both current and expected) in each domain.

• In each version of ASCOT ratings from each

domain can be combined to give an overall ASCOT score

An overview of scoring

• Score can be calculated from both current and

expected questions

• Score combines ratings / response options

across all eight domains

• Each rating is given a different value – known

as a weight

• Weights are added together to give a weighted

score

• Weighted score entered into a formula to give
• verall SCRQoL score

Why use weights?

• To reflect value of social care outcomes
• The different SCRQoL states described by the ASCOT

measure are not of equal value

• e.g. having complete control over your daily life is seen as being more

important than being as socially involved as you want to be

• The weights for each rating reflect the general

population’s preferences for each SCRQoL state

• Weights for each rating within each domain derive

from a study of people’s preferences

• Compared service user and general population preferences and found

no significant differences

What’s behind the weights…

29/04/2013 60

Preferences or what matters most?

29/04/2013 61

What’s behind the weights (2)…

29/04/2013 62

The meaning of SCRQoL scores

• Gives you a number between

–1.00 and ‐0.23 (CH3) –1.00 and ‐0.17 (INT4/SCT4)

• 1.00 = optimum or ‘ideal’ SCRQoL
• 0.00 = being dead
• What do negative SCRQoL scores mean?

SCRQoL gain

• In versions of ASCOT that measure both current

and expected SCRQoL, can calculate SCRQoL gain.

• SCRQoL gain = estimate of the impact of a

service(s) on SCRQoL

• The difference between current and expected

SCRQoL i.e. current SCRQoL – expected SCRQoL

• What would a negative SCRQoL gain score mean?

ASCOT data entry tools

• Each version of ASCOT has a data entry tool (MS Excel)
• If you enter data collected for an individual case it gives

you the ASCOT scores for that case

• It also gives you summary or aggregate scores for all

cases entered as well as breakdowns of each domain

• Produces spider diagrams of data from all the entered

cases

• Available on the ASCOT website

(www.pssru.ac.uk/ascot)

Understanding spider diagrams (1)

66

Understanding spider diagrams (2)

67

Thank you for listening! Please fill in your feedback forms.

Contacting us ASCOT website: www.pssru.ac.uk/ascot Forum/Twitter ASCOT mailbox: ascot@kent.ac.uk More details in ASCOT publications – see website

The SCRQoL score formula

• SCT4/INT4

– SCRQoL = (0.203 x weighted score) – 0.466

• CH3

– SCRQoL = (0.228 x weighted score) – 0.586

• SCT3*

– SCRQoL (current) = weighted score/41.70