Evaluation of a Clinical Simulation-based Method for EHR-platforms - - PDF document

Evaluation of a Clinical Simulation-based Method for EHR-platforms

• Istanbul, September 1st 2014

Sanne Jensen

Authors: Sanne Jensen, Stine Loft Rasmussen, Karen Marie Lyng

Background

• Procurement of new EHR-platform 2012 – 2013
• 2 Danish regions covering half the Danish population
• 20 hospitals with approx. 40,000 healthcare professionals
• Broad functionality to support clinical and administrative

core processes

• Strategic demand of simulation-based assessment:
• shall cover usability & human factor issues
• must cover demands from various different end-users,

specialties and cultures

• shall meet transparency demands of procurement in a public

tender according to EU rules

Purpose of new EHR-platform

Main topic is increased effectiveness in quality of care

• 1. continuity of care and patient safety
• 2. streamlining of clinical processes and workflow
• 3. patient and employee satisfaction

Furthermore

• Support cross-functional work processes
• Support hand-over of responsibility

• Clinicians using it-systems in realistic environments
• Clinical simulation may be used for various

purposes and in all stages of the lifecycle of clinical information systems

Clinical simulation

Challenges to qualitative approach

Methodological requirements:

• Comparable assessment results
• Uniformly assessment of the EHR-platforms
• Transparent process
• Easily collected and rapidly analyzed results

Size of procurement:

• All aspects of EHR-platform should be covered
• Many clinical specialties and professionals
• All types of users should be considered

Data collection: Observations and questionnaires

• Patient safety
• Ease of use
• Usefulness

Questions related to specific tasks and scenarios 1. Broken hip 2. Appendicitis 3. Depression 4. Chronicle Obstructive Pulmonary Disease (COLD) 5. Breast cancer 6. Child with failure to thrive

Green team

• 3 physicians
• surgery, internal medicine , psychiatry
• 3 nurses
• surgery, internal medicine , psychiatry

Red team

• 3 physicians
• surgery, internal medicine , psychiatry
• 2 nurses
• surgery, internal medicine
• 1 medical secretary

Blue team

• 3 physicians
• surgery, internal medicine , pediatric
• 2 nurses
• surgery, internal medicine
• 1 medical secretary

Overview of assessment

Navn (Sidehoved/fod)

1. 1st visit in outpatient department 2. Interruption and shift between patients 3. Assignment to hospital (physician) 4. Initial nursing assessment 5. Patient care 6. Discharge from hospital (physician) 7. Shift of rounds (nurses) 8. Medical supervision 9. Ward shift 10. Discharge from hospital (nurse) 11. Dispensation & administration of drugs 12. Outpatient control Overall metrics for assessment

• 1. support of key areas
• 2. user experience of systems
• 3. patient safety

Specific parts of daily work practice, that will be assessed:

• 4. documentation
• 5. overview (patient-

information and tasks)

• 6. plans
• 7. decision support

Detailed plan

Evaluation of assessment method

• Purpose:
• What is the eligibility of the method?
• What are the advantages/disadvantages?
• Reveal possible issues to be improved
• Method:
• Observations
• Semi-structured interviews

Navn (Sidehoved/fod) Titel/beskrivelse (Sidehoved/fod)

Result: assessment of user satisfaction

Pros

• User involvement
• Subjective evaluation is easily done
• Possible to reflect on and assess each of the

platforms and differences Cons

• Objective assessment is difficult
• Difficult to assess minor variances in ease of use
• Interruptions influenced fidelity and realism

Navn (Sidehoved/fod) Titel/beskrivelse (Sidehoved/fod)

Result – assessment of usefulness

Pros

• Efficient method to examine vendors descriptions
• Deep insight in the EHR-platform was gained
• Assessment across specialties and healthcare groups
• Differences in clinical needs became obvious
• Broad insight due to structural training

Cons

• Not part of the assessment
• System integrations
• Longitudinal time factor

Navn (Sidehoved/fod)

Result – assessment of patient safety

Pros

• Clinical simulation can reveal patient safety issues

not evident in textual descriptions

• Detailed use of technology is visualized by clinical

simulation Cons

• Patient safety experts should have observed the

simulations

Navn (Sidehoved/fod) Titel/beskrivelse (Sidehoved/fod)

Result – general issues

Pros

• Use of ”patients” supported realism in scenarios a

facilitated a smooth flow in the simulation

• Vendors found they were treated equally and fair

Cons

• Lack of complexity in patient cases
• Fewer patients than in every-day work
• Vendors would have preferred to use their own test

data

Navn (Sidehoved/fod) Titel/beskrivelse (Sidehoved/fod)

Discussion

• Assessment of qualitative aspects were possible
• Metrics of assessments must be very specific
• Users get first-hand experience
• Support of work practice were assessed
• Difficult balance between transparency and

clinical relevance

• A lot of training is needed
• Resource demanding

Navn (Sidehoved/fod) Titel/beskrivelse (Sidehoved/fod)

Conclusion

Clinical simulation:

• is beneficial for gaining insight in user

satisfaction, usefulness and patient safety

• illuminates relation between users, technology

and work practice

• assessed systematically each of the platforms

and their differences

• supplements other assessment activities

Navn (Sidehoved/fod)

Thank you for your attention