Measuring & Using Patient Experiences: Surveying Adult and Child - - PowerPoint PPT Presentation

Measuring & Using Patient Experiences: Surveying Adult and Child Patients

Measuring & using patient experience: surveying adults and children

What we will cover:

About Picker Institute Europe Introduction Conducting and using surveys Measuring children’s experiences Using survey results for service improvement Conclusions

How it will work

Each segment will be introduced with a brief presentation Then there will be a discussion We will try to make it as interactive as possible First of all I will introduce the Picker Institute

Picker Institute Europe 4

An international charity dedicated to ensuring the highest quality health and social care for all

Our work is underpinned by a commitment to the Picker Institute principles

• f person centred care. Our core aims are to:
• Influence policy and practice so health and

social care systems are always centred around people’s needs and preferences;

• Inspire the delivery of the highest quality care,

developing tools and services which enable all experiences to be better understood; and

• Empower those working in health and social

care to improve experiences by effectively measuring and acting upon people’s feedback.

www.pickereurope.org

www.pickereurope.org

Picker Institute Europe

Set up 15 years ago by Picker Institute Inc Employs some 60 staff across sites in Oxford and Hamburg Undertakes NHS staff survey (one of the biggest in the world) Undertakes patient surveys in a range of countries Applied research on person centred care, often in partnership with Oxford University Service improvement work including Always EventsTM with IHI

www.pickereurope.org 7

Why do surveys?

Without measurement you cannot manage and evaluate performance and improvement Without measuring patient experience you cannot judge whether you are meeting customer needs – patient experience is core to quality Surveys are quantifiable and trends can be seen over time They are potentially representative It is also possible to judge in what ways they might not be representative They are a tool to get stakeholders on board and for accountability

www.pickereurope.org 8

Our team

Chris Graham, Director of Research and Policy Bridget Hopwood, Director of Health Experiences Myself – CEO of the Picker Group in Europe

www.pickereurope.org 9

Hi, it is our first time in Dallas!

ICEBREAKER

www.pickereurope.org 10

Conducting and using patient experience surveys

Chris Graham Director of Research & Policy @ChrisGrahamUK

Why survey patients?

Strengths Weaknesses Can produce reliable, generalizable estimates Care needed to ensure response is representative Allow for robust comparisons between wards/sites/institutions Standardisation can limit ability to focus on specific local issues …and between different patient groups ‘One size fits all’ approach may miss issues important to minorities Can track trends over time Statistical results require some (limited) expertise to use Inclusive; wide opportunity to respond Don’t tell individual stories Cost effective, high volume data

Picker Institute Europe 13

When you had important questions to ask a nurse, did you get answers that you could understand?

Picker Institute Europe 14

Source: NHS England & Quality Health. 2014 National Cancer Patient Experience Survey National Report. https://www.quality- health.co.uk/resources/surveys/national-cancer-experience-survey/2014-national-cancer-patient-experience-survey/2014-national-cancer-patient-experience- survey-national-reports (p52) 15

Surveys don't tell individual stories - but they do get representative feedback, including from people who wouldn't otherwise volunteer it

Picker Institute Europe 7

What makes a good survey?

Good surveys need to consider: Who we survey & how they are selected (sampling) How responses are sought (survey mode) What we ask (question design) These are interrelated, and decisions will be based on: Why we want feedback: what’s the purpose of the survey?

Picker Institute Europe 17

Survey ‘modes’

Choosing the right survey mode

Choice of mode will be influenced by: Budget Survey purpose How quickly the results are needed Audience There’s no one ‘best’ mode Results from different modes may not be comparable

Picker Institute Europe 19

Mode effects – timing matters

58% 75% 37% 20% 6% 6%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Postal Handout

"Did you have confidence & trust in the nurses treating you?" Yes, always Yes, sometimes No

NHS Inpatient Survey

Patients receive questionnaires by post 2-6 months post discharge What are the advantages and disadvantages of this?

Picker Institute Europe 22

Possible problems Strengths Memory? Allows time for reflection Practical problems – moving etc? Practical benefits – data availability Delay in getting feedback to hospital ‘Landmark’/ ‘milestone’ view Wide range; differential recall? Easily replicable & widely applicable

www.pickereurope.org

Sampling

What is sampling?

Surveying entire populations is often impractical Sampling is an intuitive way to make estimates

24

Sampling

A sample is drawn from a larger pool (or ‘frame’) of people If this is biased, then your results will be biased too – no matter how many people you get responses from

Picker Institute Europe 25

Self-selecting samples

Question: why do people respond to surveys? Interest in the subject Have a particular experience they want to share For money Sense of duty/social responsibility Feeling valued If the reasons are related to what you’re measuring, results are likely to be biased.

Picker Institute Europe 26

Sampling: summary

1.

Start from an appropriate sample frame

2.

Be wary of self-selection

3.

Be mindful of how individuals are selected

4.

Don’t get too hung up on the sample size – unless it’s very small!

Picker Institute Europe 27

www.pickereurope.org

Question design

Survey design: Principles of good practice

Keep questions simple and focussed Avoid jargon Do not use ‘double-barrelled’ questions Responses options should... Form balanced scales Be unambiguous and mutually exclusive Allow respondents to say if they don’t know Focus on recent experiences (within a year) Avoid complex layouts – eg matrices

Questionnaire length

General assumption is that longer questionnaires discourage responses but evidence is patchy Salience may be more important. Underlying question is: What motivates an individual to respond to a survey? Routing can be a way of speeding up the questionnaire – but it might not help if the questionnaire still looks very long

Picker Institute Europe 30

Summary – questionnaire design

Questions should be simple, direct, and address a single concept Response options should be mutually exclusive and comprehensive Give people the opportunity to say ‘not applicable’ Avoid loaded or leading statements Test questions if you possibly can Limited use of routing can be helpful Be mindful of the possibility of contextual effects – eg question/response order

Picker Institute Europe 31

Picker Institute Europe 32

Discussion

Questions

Where are surveys currently used within your

• rganisation, if anywhere?

Think about patient, carer, and staff feedback What is the aim of the surveys? How could this approach be improved to focus on the strengths of surveys? How are results used? Comparisons, trends, etc? Do these uses fit the methodology?

Picker Institute Europe 33

Measuring children’s experiences of care

Bridget Hopwood Director of Health Experiences

Picker Institute Europe 34

Few paediatric surveys exist  parental surveys “typically used as proxies” Gathering national patient experience data from the children themselves not been a priority Distinct gap between what we know about adults’ experiences and that of children and young people’s

• wn experiences

Picker Institute Europe 35

Children should be seen and not heard?

No decision about me, without me” [Harvey Picker] NHS Constitution – all patients (and families or carers where appropriate) have a right to be involved Society becoming more child-focused in general Evidence that it works…..

“Children are more likely to be less stressed if their views are taken into account, and they are allowed to participate in their health care in accordance to their rights.”

[Tiina Pelander; Children’s best and worst experiences during hospitalisation; 2010]

Picker Institute Europe 36

Why ask the children?

Picker Institute Europe 37

Why ask the children?

“At the end of the day, it’s my leg so I would rather be asked than my Mum or Dad….”

Because they want us to….. Unique perspective…..

“...this man had no legs

and he was really old and he just left the door open and I had to see this man lying there with no legs” “Not much to do except eat and watch people die”

Achieving Patient-Centred Healthcare: Indicators of Progress and Success

CYP Patient experience

Problems/challenges with designing surveys for children

Age range variety – target age? ‘Children’ classed as aged 0-17yrs Different development rates / reading ages / comprehension Whose opinion counts? Parents or children or both? Different experiences, both important Focus on what matters Ethics

Picker Institute Europe 39

Survey co- design with children and young people

www.pickereurope.org

Survey Design Process

Picker Institute Europe 41

Qualitative Scoping

• Focus groups

and/or depth interviews to scope main issues/views Questionnaire Design

• Priority areas

identified in qualitative stage

• Format, length

and design appropriate to target audience Cognitive Testing

• Test for

interpretation, recall, responses

• ptions,

format

• Amend and

retest Pilot and Data Collection

• Select

appropriate methodology for population

• Test data

collection approach (pilot)

• Implement

surveys/ data gather

Children and Young People Surveys

Focus on areas important to children

Hospital environment Entertainment People (and their characteristics)

Parents views also important

Parking Time-specific information Directions Appointment booking Information

Use of illustrations and colour

Picker Institute Europe 43

Cognitive Testing

Picker Institute Europe 44

• Question wording
• Question length
• Number of response options
• Questionnaire length
• Design: Colours and illustrations

Appealing to children and appropriate to their ability Maintain interest Maximise child involvement and completion

Examples of our feedback tools - children and young people

Picker Institute Europe 45

Children’s FFT Paediatric outpatient survey

Acute hospital care

Inpatient & day case (now national CQC) Outpatients (DH Innovation in

• utcomes competition winner)

Emergency Department Neonatal (parents’ experiences) Transition to adult services Friends and Family Test form

Children’s community – SaLT;

OT; physio

Chronic conditions

Allergies Sickle cell disease

Who completed the children’s questionnaire? (2008)

Methods: Age Range

Age of patient (yrs) % surveys completed

National Children’s Inpatient & Day Case Survey 2014

www.pickereurope.org

Background

CYP part of the annual National Patient Survey Programme for first time in 10 years Based on existing Picker Institute Paediatric IP & DC survey – developed in 2008, revised 2011, conducted annually since (voluntary) Survey developed with young patients and parents CQC licenced survey from Picker in 2014 to use as part of the national patient survey programme  some amendments (questions, format)

Picker Institute Europe 48

Achieving Patient-Centred Healthcare: Indicators of Progress and Success

National CYP Survey – Methods

Identical questions and methodology for all NHS trusts: Dissent process Postal survey: initial mailing plus two reminders M1 – questionnaire, cover letter, freepost envelope, multi- language sheet M2 – reminder letter (non-responders) M3 – questionnaire, final reminder letter, freepost envelope, multi-language sheet Online option (CYP only) Sample: between 300 and 850 recent patients aged 0-15 years, August discharges (count back into July /forward into Sept if needed) Freephone and Language-Line available

The Questionnaires x 3

Parent (0-7)

Picker Institute Europe 50

Children (8-11) Young People (12-15)

Responses from 18, 736 children and young people under 16 years who received inpatient or day care in 137 NHS acute NHS trusts during August 2014 Overall response rate of 27%

Achieving Patient-Centred Healthcare: Indicators of Progress and Success

National Children’s Inpatient & Day Case Survey 2014 - Background

26% 29% 30%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 0 - 7 years (n= 12,235) 8 - 11 years (n=3,108) 12 - 15 years (n=3,393)

Response Rate by survey type

Feeling Safe

89% of 8-15 year olds said that they felt safe on the ward all the time. 91% of parents (0-7 years) felt their child was safe on the ward all the time

Play

42% of children 8 – 11 years old said staff did not play or do any activities with them while in hospital. 27% of parents of children 0 – 7 years old stated that staff did not play with their child and they would have liked this.

Involvement

• Only just over half (57%) of young people 12 – 15 years old were fully involved in

decisions about their care and treatment. More than one in eight (13%) said that they were not involved at all.

• 35% of parents were not fully encouraged to be involved in decisions about their child’s

care and treatment.

Achieving Patient-Centred Healthcare: Indicators of Progress and Success

National CYP Survey – Key Results

Focussing improvement:

Most Important - highest correlation

Picker Institute Europe 53

Question Label Importance Correlation P62 Child not looked after 1.000 0.932 P63 Not treated with respect .911 0.859 C61 Staff unfriendly .852 0.810 P58 Parent not listened to .831 0.793 P25 Low confidence in staff .827 0.789 P60 Staff unfriendly .820 0.784 P8 Child not safe .773 0.745 C59 Child not listened to .769 0.742 P24 Care plan not agreed .754 0.730 P17 Poor information about care .731 0.711

Children

• Friendly staff
• Being listened to

Parents

• Staff – friendly, treating with respect
• Feeling child was safe

Achieving Patient-Centred Healthcare: Indicators of Progress and Success

Priorities – most important

Least important: lowest correlation

Picker Institute Europe 55

Question Label Importance Correlation P3 Admission date changed .000 0.108 P36+ Parents couldn't stay overnight .047 0.147 P2 No choice admission date .071 0.166 P35 No hot drink facility .084 0.177 C11 Lacked things to do .257 0.319 P12+ Poor food .257 0.320 C15+ Staff didn't play enough .276 0.336 C13+ Poor food .303 0.358 C21 Staff talked in front of patient .331 0.381 P10 Lacked appropriate playthings .333 0.382

Children

• Things to do / entertainment
• Staff playing with them
• Food

Parents

• Admission dates
• Food
• Hot drinks

Achieving Patient-Centred Healthcare: Indicators of Progress and Success

Priorities – least important

Pictures (8-11 years)

Picker Institute Europe 57

Achieving Patient-Centred Healthcare: Indicators of Progress and Success

http://www.cqc.org.uk/sites/default/files/20150619_cypsurvey_results_cypversion.pdf

Success stories – using the data

Picker Institute Europe 59

Partnership working with Sheffield Children’s Hospital Ongoing measurement, improvement and monitoring success Building services around people’s experiences capital building programme informed by the survey findings. Info gained re sign-posting, parking, disabled access, privacy & dignity needs, parent accommodation have informed their design. Publicly available action plan on website

Picker Institute Europe 60

“Participation encourages integration

and inclusion, lets youth feel valued and ultimately leads to progress. It is right and should not be tokenistic …the benefits are clear and the importance is imperative."

www.rcpch.ac.uk/participation

Using survey results for service improvement

Dr Andrew McCulloch Chief Executive

Picker Institute Europe 61

Using survey results for service improvement

Present the data in a way which people can understand Discuss at Board and Ward level Involve patients and carers as well as front line staff in understanding what the data mean Prioritise what is important to patients and staff Prioritise some quick wins Turn areas of concern into concrete objectives Use the evidence base on improvement methodology Make the solution fit the problem!

www.pickereurope.org 62

Your survey data: turning ‘areas’ into

• bjectives

Objectives = ‘standards’

• what do you want to achieve?
• how will care be for patients / service-users?

Looking at all the areas you just identified:

• what do you want to increase?
• what do you want to decrease?

www.pickereurope.org

Patient Experience Review:

www.pickereurope.org 64

Review Analysis Staff and patient engagement Reporting and implementation

Review

Patient experience data sources, collection methods and formats; results reporting and information dissemination: systems and pathways; awareness, interpretation and use of patient experience data from ward to Board.

www.pickereurope.org 65

Analysis

Data integration Thematic analysis From data to information Different ways to present information

www.pickereurope.org 66

www.pickereurope.org 67

Using the NHS National Maternity survey results as a catalyst for improving culture and experience…

www.pickereurope.org 68

St George’s University Hospitals NHS Foundation Trust is a referral unit for high risk and potentially complex cases, such as geriatric and diabetic pregnancies, and diverse deliveries, including increasing numbers from women who had experienced FGM From 2010-12 the trust received poor results in their national maternity experience survey, scoring in the lower half of trusts surveyed nationally. The trust focused on using the experiences in those in their care to engage staff in the cultural and process changes needed to improve standards: The results were used to highlight key areas of concern with regard to care quality including:

• involvement
• support
• information
• communication

A focus on people before process…

www.pickereurope.org 69

Staff identified 4 key measures of success that would support them to deliver the type of care they aspired to and that initially drove them to choose St. Georges as a place to work: “Positive talk,” goal setting

• Together, the staff established a ‘first or best position’ – a goal for the organisation to

work towards, based on the unit’s evidenced strengths and shared aspirations… ‘St George’s Maternity Unit is the UK’s centre of excellence for woman-centred maternity care’

• As a team, the staff were trained in models of positive thinking and behaviour, known

as ‘mental models of change’.

“Living Leadership” at all levels

• The trust invested in leadership training and focusing on behavioural, learning and

management styles

• Front line staff lead special ward rounds to identify, share and praise examples of

positive staff behaviours and care-giving.

• In the culture of continuous improvement staff gave themselves permission to develop

and implement their own ongoing improvement ideas.

…

www.pickereurope.org 70

“Easy for staff better for patients,” streamlining processes

• Processes such as admittance bed management and discharge were reviewed

using both staff and patient experiences

• Standardised action planning processes were introduced to aid clarity and

consistency:

• Data (collecting and sharing information about the area under review
• analysis (analysis of the information under review
• decision-making (deciding on analysis-based actions)
• action-taking (taking the right action)

“Maintaining Momentum” Responding to feedback and celebrating success

• Introduction of new real time or near real time feedback collection at a ward level
• Results and actions/improvement stories are, made available for all to see and

celebrate,

• Themes for improvement are identified and fed back to staff and displayed

under a “you said we did” approach

In the 2013 NHS National Maternity survey the trust has show statistically significant

www.pickereurope.org 71

In the 2013 NHS National Maternity survey the trust has show statistically significant improvement in key

• 9.5% improvement in involving women in decisions about antenatal care
• 6.8% improvement in having skin to skin contact with babies soon after birth
• 14.0% improvement in giving enough information in post natal hospital care
• 25.1% improvement in giving enough support and encouragement about feeding
• 26.3% improvement in giving mothers enough information about their own

recovery after birth

Picker Institute Europe 72

Discussion

Discussion

What are the issues for you, in your setting, for using patient experience surveys for quality improvement? Think about barriers and enablers Who needs to be involved? Success stories?

Picker Institute Europe 73

Picker Institute Europe 74

Closing thoughts

Key learning

Why and when surveys are valuable… …and, by extension, when they’re not! Characteristics of a good survey: who, how, what Children can feedback about their experiences – but care is needed to ask the right questions Don’t stop at measurement!

Picker Institute Europe 75

Picker Institute Europe Buxton Court 3 West Way Oxford OX2 0JB Tel: + 44 (0) 1865 208100 Fax: + 44 (0) 1865 208101 Info@pickereurope.ac.uk www.pickereurope.org

Charity registered in England and Wales: 1081688 Charity registered in Scotland: SC045048 Company limited by guarantee registered in England and Wales

Dr Andrew McCulloch andrew.mcculloch@pickereuro pe.ac.uk

Next…

Exhibitor Reception 5:00-6:30 PM Foyers

www.theberylinstitute.org 77