Dr Carol Rivas, Sydney Anstee Dr Daria Tkacz, Dr Don Cruickshank - - PowerPoint PPT Presentation

PRESENT: Pragmatic engineering of themes from large volume text to improve healthcare http://www.present.org.uk/ CPES – Cancer Patient Experience Survey - Freetext comments

Dr Carol Rivas, Sydney Anstee Dr Daria Tkacz, Dr Don Cruickshank

What is CPES?

• Cancer Patient Experience Survey
• The English Cancer Patient Experience Survey (CPES) is commissioned

by NHS England and administered on their behalf by an external survey provider organisation (Quality Health).

• The survey provides insights into the care experienced by cancer

patients across England who were treated as day cases or inpatients. Further details, including access to the latest report can be found at:

• https://www.quality-health.co.uk/surveys/national-cancer-patient-

experience-survey

2

Current process of freetext from CPES

59 Tick box questions & 3 freetext comment boxes in the survey

75,000 freetext comments per year

(approx.)

A new and timely process needed

75,000 freetext comments

(approx.)

Rapid process Accurate representation of patients views Visual summary Useable in Service Improvement

Representative of Patients views and Usable for HCPs

• Literature review
• Discrete choice experiment survey
• Concept mapping workshops with patients
• Individual interviews with patients
• TEXT ENGINEERING – sensitivity developments
• PRESENT dashboard ready for Proof of Concept

testing……..

Development of the Dashboard

Thematic summary – Dashboard

• We used freetext comments from 2013 CPES Wales

to populate the Dashboard

Representative of Patients views and Usable for HCPs

2 hours (max) video/ audio-recorded interviews in usual work place Objective: find defects and errors from the users’ point of view In order to: improve the quality and effectiveness of a product

Structured Walkthrough Interview aims:

• Is the dashboard Usable? (standard usability principles, Nielson’s Heuristic

Evaluation)

• Can role relevant tasks be accomplished? (based on Goal Directed Design,

Alan Cooper)

• Implementability of the dashboard? (Normalisation, impact, evaluation,

Carl May)

• Diffusion of Innovation: self-assess where they & organisations fit on

innovator-adopter scales

Structured Walkthrough with Health Care Professionals (HCPs)

DASHBOARD – proof of concept

Walkthroughs conducted: 23rd March - 21st April 2017 (15 planned, 13 completed) Professional roles we planned to recruit from:

Locations Gender

Total Completed Lead GPs

London, Leeds and Southampton 1 male 2 female

3

Ward managers; Nurse director

London London 1 female 1 male

2

Associate directors, lead clinicians, Members of the Trust Board Southampton and Leeds 2 male

2

Lead commissioners in CCGs

Southampton 1 male

1

Lead members of multidisciplinary treatment teams (MDT) Southampton, London and Leeds 2 female 1 male

3

Other – Data Analyst in NHS Improvement (A&E); Patient Experience Leads 1 Southampton; London 1 male 1 female

2 N= 13

6 HCPs agreed to participate, were booked, but later cancelled: All Nurse directors and managers London, Leeds and Southampton

WALKTHROUGH INTERVIEW PARTICIPANTS

Positives Challenges and Opportunities Usability

Overall Intuitive - Simple ‘easy on the eye’ Themes/comments good Infographic – easier to interpret Navigation – add some tools Severity: Not market-ready yet

Goals

Needs are easy to identify Justifications for improvements Successes (morale and sharing) Timely data Text sensitivity Export/ report function Implementation Value and purpose Likely support of use (Ind &Org) Possible new initiatives Could freetext from other sources be included Who will fund and support Should be open to all & not heavily protected

Other

Tensions between Performance Targets and Patient Experience

Overall

“Could revolutionise”; “be phenomenal”; “replace all the others”…. If Changes can be made

ANY QUESTIONS?

• For further contact - Carol Rivas: c.a.rivas@soton.ac.uk
• http://www.present.org.uk/

Funding acknowledgement

This presentation summarises independent research funded by the National Institute for Health Research (Health Services and Delivery Research, 14/156/15 - PRESENT: Patient Reported Experience Survey Engineering of Natural Text: developing practical automated analysis and dashboard representations of cancer survey freetext answers). The views expressed herein are those of the author(s) and not necessarily those

• f the NHS, the National Institute for Health Research or the Department
• f Health.

Nor does it necessarily represent the views of any of the collaborators whose logos are used herein.

http://present.org.uk/ http://dashboard.present.org.uk/ds_final/index.html

The team

13

Dr Carol Rivas, University of Southampton Dr Daria Tkacz, University of Southampton Dr Don Cruickshank, University of Southampton Sydney Anstee, University of Southampton Dr Richard Wagland, University of Southampton Dr Mike Bracher, University of Southampton Professor Carl May, University of Southampton Dr Richard Giordano, University of Southampton Professor Dame Jessica Corner, University of Nottingham David Simpson Professor Theresa Wiseman, University of Southampton Dr Adam Glaser, University of Leeds Dr Georgios Lyratzopoulos, University of Cambridge Jennie Whitford, Macmillan Cancer Support Hanna Hine, Macmillan Cancer Support Dr Shelley Dolan, Clinical Director, London Cancer Alliance Lesley Turner, patient advisor and our other collaborators, advisory group members and patients