Usability of reports generated by a computerised dose prediction - PowerPoint PPT Presentation

Usability of reports generated by a computerised dose prediction software Melissa Baysari, Joanne Chan, Jane Carland, Sophie Stocker, Maria Moran, Richard Day Macquarie University; UNSW Australia; St Vincent’s Hospital, Sydney

Therapeutic drug monitoring (TDM) TDM involves monitoring drug concentrations in the blood and adjusting doses accordingly, in order to achieve the optimal effect of that drug TDM is most beneficial in patients with altered pharmacokinetic parameters, such as - those with impaired renal function - those that are critically ill - those who are obese, etc. BUT, maintaining optimal drug concentrations is a highly complex task Computerised dose prediction software can assist clinicians in this process AUSTRALIAN INSTITUTE OF HEALTH INNOVATION FACULTY OF 2 MEDICINE AND HEALTH SCIENCES

Computerised dose prediction Patient data is inputted (by a pharmacologist, pharmacist, TDM team?) or data is automatically extracted from EMR Algorithm (integrating relevant patient factors) Report, including dose recommendation Delivered to prescriber (e.g. as a link in EMR) Guide prescriber decision making AUSTRALIAN INSTITUTE OF HEALTH INNOVATION FACULTY OF 3 MEDICINE AND HEALTH SCIENCES

Computerised dose prediction Research has shown that computerised decision support can improve dosing: Dose calculators, order sets, alerts But limited evidence on dose prediction software in practice • Research showing that recommendations from software are likely to lead to drug concentrations within the therapeutic range a These systems are not yet widely used in clinical practice - Why? • Expensive? Resource intensive (who enters the data?)? • Hard to use or interpret? a Clin Pharmacokinet 2013, 52:59-68; 2016, 55:1295-1335 AUSTRALIAN INSTITUTE OF HEALTH INNOVATION FACULTY OF 4 MEDICINE AND HEALTH SCIENCES

Report for prescribers Dose prediction software can be highly sophisticated and accurate in predicting individualised dosing regimens, but if the information in reports is not understood by doctors, the benefits of the software are not achieved Previous research? Some research on what features of guidelines are liked and used Very limited evaluations of reports from dose prediction software AUSTRALIAN INSTITUTE OF HEALTH INNOVATION FACULTY OF 5 MEDICINE AND HEALTH SCIENCES

Previous evaluation of reports • In one study, 12 dosing systems were assessed on a range of criteria by a pharmacist and 2 clinical pharmacologists • 1 criterion related to the report generated for prescribers • Raters assigned a score to each report based on - readability - the inclusion of a relevant graph Reports scored - the inclusion of a free-text field - whether it displayed a user’s identity 1 - 4.3 out of 5 - whether it could be customised - whether it could be converted to other formats AUSTRALIAN INSTITUTE OF HEALTH INNOVATION FACULTY OF 6 Fuchs et al. Clinical Pharmacokinetics 52 (2013) 9-22 MEDICINE AND HEALTH SCIENCES

Study rationale and aim We know very little about how the reports are viewed by end-users (i.e. prescribers) So we set out to assess the perceived usability of a report generated from a dose prediction system findings communicated to vendor lead to improvements in report content and design AUSTRALIAN INSTITUTE OF HEALTH INNOVATION FACULTY OF 7 MEDICINE AND HEALTH SCIENCES

Study site and dosing decision support 320-bed teaching hospital in Sydney The hospital had electronic medication management, ordering and reporting of lab and imaging tests. Patient notes were not electronic Hospital planned to implement DoseMe , after some additional testing and piloting DoseMe uses Bayesian forecasting to provide individualised dosing recommendations AUSTRALIAN INSTITUTE OF HEALTH INNOVATION FACULTY OF 8 MEDICINE AND HEALTH SCIENCES

Method Research team reviewed the report from DoseMe (clinical pharmacologists, ID physician, pharmacists, medical student, human factors researcher) Made some changes to the report to improve readability Tested this new report with prescribers AUSTRALIAN INSTITUTE OF HEALTH INNOVATION FACULTY OF 9 MEDICINE AND HEALTH SCIENCES

Original report • Lots of information, including historical information • Limited white space, with some text overlapping • Inconsistent alignment of text AUSTRALIAN INSTITUTE OF HEALTH INNOVATION FACULTY OF 10 MEDICINE AND HEALTH SCIENCES

Patient information Main recommendation More detailed information Drug exposure graph 11

Participants and procedure 15 prescribers from a variety of wards (1 intern, 7 residents, 7 registrars) Prescribers were presented with a mock DoseMe report and asked: 1. What is your first impression of this report? 2. What information do you find useful and not useful? 3. Is there anything you find confusing? 4. What other information should be provided? 5. What sections of the report (box, text, graph) would you like provided to you? 6. What improvements could be made to the report, in terms of both content and layout? AUSTRALIAN INSTITUTE OF HEALTH INNOVATION FACULTY OF 12 MEDICINE AND HEALTH SCIENCES

Results

Positive perception: Summary box Prescribers liked that the top box provided a clear, straightforward recommendation and drew prescribers’ attention to the most important piece of information I like that, getting a big red box with what to do next AUSTRALIAN INSTITUTE OF HEALTH INNOVATION FACULTY OF 14 MEDICINE AND HEALTH SCIENCES

Positive perception: Drug exposure graph Seen to be a good way to visualise dosing, especially for patients on long-term courses of a drug I’m probably a more visual person so I like seeing this laid out more, in a more visual way AUSTRALIAN INSTITUTE OF HEALTH INNOVATION FACULTY OF 15 MEDICINE AND HEALTH SCIENCES

Positive perception: Drug exposure graph Also seen to be more accessible than text-based information when time pressured I think people could easily misinterpret it if they're rushing through so I think the graph makes it a lot easier to see "oh, okay, we must increase to get to the target outcome” AUSTRALIAN INSTITUTE OF HEALTH INNOVATION FACULTY OF 16 MEDICINE AND HEALTH SCIENCES

Negative perception: Unfamiliar and ambiguous terminology Prescribers were unfamiliar with the term AUC 12 - a key concept in the report Some questioned the value of Area Under the Curve (AUC) to non specialist doctors But I don’t know how many clinicians would be familiar with using it (AUC) other than infectious disease (ID) physicians or specific microbiology/ID doctors …Because we don’t really use the area -under-the-curves in sort of a practical, clinical sense on a regular basis AUSTRALIAN INSTITUTE OF HEALTH INNOVATION FACULTY OF 17 MEDICINE AND HEALTH SCIENCES

Negative perception: Unnecessary information Although most prescribers saw value in including the drug exposure graph, it was also noted that the graph shows the same information as the text above it, which is confusing I think it would be useful just to see that, actually see that the graph has represented what’s written down. Because some people may think that they’re getting two amounts of information at first glance AUSTRALIAN INSTITUTE OF HEALTH INNOVATION FACULTY OF 18 MEDICINE AND HEALTH SCIENCES

Overall perceptions Despite displaying complex information, most prescribers viewed the report as useful and appreciated being shown the data to support the recommendation in the red box It's always good if the clinician can look at something and say that it sounds reasonable or not…that you're completely relying on a computer - generated system to give you information in case there's an error within it... I think it is nice to know on what basis that was done and that it gives you at least a feeling of justification AUSTRALIAN INSTITUTE OF HEALTH INNOVATION FACULTY OF 19 MEDICINE AND HEALTH SCIENCES

Overall perceptions But …a number of prescribers anticipated that in the context of a fast-paced time-poor environment, the upper box would likely be the only part of the report that is utilised I'm in a hurry, I'm super busy, I’m already doing 14 -hour days. I've got 25 patients and they're all really sick. I literally am racing through the bloods…I literally just want to know what to do next and how to do it as quickly as possible AUSTRALIAN INSTITUTE OF HEALTH INNOVATION FACULTY OF 20 MEDICINE AND HEALTH SCIENCES

Discussion

Main findings • The mock report was effective in communicating the recommended dose of a drug, but this recommendation was presented alongside information that was not understood or unlikely to be used by prescribers • The aspects of the report viewed negatively by end-users related to a lack of familiarity with TDM terminology • If specialised terms (like AUC) are to be used in outputs for prescribers, they need to be accompanied by training and/or the report should include clear definitions (e.g. when you hover over them with your mouse) AUSTRALIAN INSTITUTE OF HEALTH INNOVATION FACULTY OF 22 MEDICINE AND HEALTH SCIENCES