HERDING CATS: CORRALLING THE SUBJECTIVITY IN HTA Integration of a simple framework for translating evidence into policy at a hospital-based health technology assessment unit Nisha Almeida, PhD Epidemiologist, Technology Assessment Unit of the MUHC April 16 th , 2019
Disclosure • I have no actual or potential conflict of interest in relation to this topic or presentation. 2
Overview • Conception of a simple framework to aid in the developing of recommendations • How do we deal with the subjectivity inherent in HTA decision-making? • ignore it • informally incorporate it into the decision- making process by consulting relevant stakeholders • explicitly incorporate it into the decision- making process by recording value judgements of different stakeholders • convert value judgements of stakeholders into an objective scoring system 3
Background TAU composition and process Decision-making process • Research Staff • Approved • Approved for Evaluation (Epidemiologist, • Policy Committee Health economist) • Not approved (physicians, nurses, allied health professionals, administration, and patients) Evaluation of Recommendation evidence 4
Background HTA process Observational RCTs studies Effectiveness Decision- Evidence Recommendation Safety Making Process Local Cost data 5
Background Pilot exercise • Are just 3 factors (efficacy, cost, 8 safety) sufficient to inform the 7 final decision? 6 • Reviewed 24 past TAU reports 5 Frequency 4 • Correlated the overall strength of 3 factors (efficacy, cost, safety) 3 with the final recommendation 2 1 0 APPROVED APPROVED FOR NOT APPROVED EVALUATION Recommendation type Weak Moderate Strong 6
Background Factors important to decision-making GRADE 1 OHTAC 2 Domain Sub-domain Domain Comment Effectiveness The larger the difference between the desirable Overall clinical Balance between and undesirable effects, the higher the likelihood benefit desirable and Safety that a strong recommendation is warranted. The undesirable narrower the gradient, the higher the likelihood Burden of illness that a weak recommendation is warranted effects Need The higher the quality of evidence, the higher the Quality of Feasibility of Economic feasibility likelihood that a strong recommendation is evidence warranted adoption Organizational feasibility The more values and preferences vary, or the Values and greater the uncertainty in values and preferences, Value for money Economic evaluations preferences the higher the likelihood that a weak recommendation is warranted Consistency with Expected societal values expected societal and The higher the costs of an intervention — that is, Expected ethical values Costs (resource the greater the resources consumed — the lower ethical values allocation) the likelihood that a strong recommendation is warranted 1 Grading of Recommendations Assessment, Development 2 Ontario Health Technology Appraisal Committee and Evaluation, McMaster University 7
Goal Creating a decision-making framework To create a decision-aid tool that is : • easy to use • specific to a hospital setting • captures the subjectivity inherent in HTA 8
Objectives Objectives Identify relevant criteria that stakeholders in the decision-making 1. process recognize as important to shape a recommendation; Determine the extent to which the expert community agrees on the 2. importance of the identified criteria; Incorporate the identified domains into a decision-aid tool ; 3. Pilot test the tool among TAU Policy Committee members 4. 9
Objectives Objective 1: Identifying relevant criteria • Clinical benefit • Cost OHTAC • Feasibility • Ethics • Two TAU members independently reviewed a random Review of sample of past reports past TAU • to identify criteria that played a role in the decision- reports making process • special consideration to a survey of hospitals in Literature France, based on an extensive literature review that search included 47 decision-criteria grouped under 8 domains 10
Objective 1: Initial list of decision criteria Results • Magnitude of effectiveness • Quality of evidence for effectiveness Clinical Benefit • Safety • Burden of illness to patient • Absence of alternative treatment options • Costs related to the technology Value for money • Increased hospital efficiency • External financial support • Ease of implementation Feasibility • Prior hospital experience with the technology • Need for evidence of effectiveness in the local setting • Disease is a public health priority • Disease is rare Ethics and Values • Benefit of technology to society • Impact on delivery of equitable care • Impact on patient-important outcomes • Impact of technology on attracting new patients and/or health professionals Strategic concerns • Impact on creating research opportunities and external collaborations • Ability to offer a cutting-edge technology or new alternative treatment • Availability of the technology in other local centres 11
Objective 2 Objective 2: Determining the importance of relevant domains Respondent representation % • Surveyed 61 panelists MUHC member 73 • 52 completed responses (85%) HTA member 31 Administrator 23 Patient perspective 8 12
Reorganization of domains New version Old version • Magnitude of effectiveness • Magnitude of effectiveness Clinical Benefit • Quality of evidence for effectiveness • Quality of evidence for effectiveness Clinical Benefit • Safety • Safety • Burden of illness to patient • Budget impact (Net cost) • Absence of alternative treatment options • Costs avoided /increased hospital efficiency Value for money • Budget impact on other departments • Costs related to the technology • Cost-effectiveness Value for money • Increased hospital efficiency • Availability of local expertise • External financial support • Disruptiveness • Need to generate local evidence Feasibility • Impact on cross-institution collaboration • Ease of implementation • Personnel satisfaction • Prior hospital experience with the technology Feasibility • Impact of innovativeness of the technology • Need for evidence of effectiveness in the local setting • Impact on patient convenience Impact on patient • Personal utility: patient values and preference • Disease is a public health priority • Patient-reported outcomes (QoL) • Disease is rare Ethics and Values • Benefit of technology to society • Benefit to society (reduces health care costs) • Impact on delivery of equitable care Impact on health • Burden on other health care centres • Impact on patient-important outcomes system • Need: unnecessary duplication • Impact of technology on attracting new patients and/or health professionals • Stakeholder pressure Strategic • Impact on creating research opportunities Strategic • No. of patients affected by technology and external collaborations concerns • Availability of external funding concerns • Ability to offer a cutting-edge technology or new alternative treatment Ethical • Availability of the technology in other local considerations centres 13
Objective 3 Objective 3. Creating the framework After HTA Before committee At committee completed meeting meeting • By TAU • By Policy • By Policy research staff committee committee members members • Rate favourability of • Rate • Deliberate each domain importance of results from each domain tool and make recommendati on 14
Objective 4 Pilot testing the tool To evaluate the use of a hydrogel spacer as part of the external beam radiation therapy protocol Policy in patients with localized prostrate cancer at the Question MUHC Population Patients with localized prostrate cancer receiving radiotherapy Intervention External beam radiotherapy (EBRT) + spacer gel Comparator EBRT alone Outcome Acute and late QoL (using EPIC), and adverse events (using RTOG or CTCAE) 15
Objective 3 Creating the framework GOAL: To ensure a systematic By TAU After HTA approach to considering research staff completed different factors relevant to the decision-making process. • TAU researchers complete the HTA and record their findings for each criterion • They also rate whether the findings are favourable: • Yes • No • Maybe • Need more information 16
Objective 3 Creating the framework Before By Policy committee committee meeting members • Each committee member sent a link to the decision-aid tool to be completed online, along with the technology assessment report • Policy committee members rate the importance of each criterion based on their values and preferences and within the context of the technology 17
Illustration • SurveyGizmo • https://s-6ff987-i.sgizmo.com/s3/i-19Jk34IBN7VJczN0rk- 3120373/?sguid=19Jk34IBN7VJczN0rk 18
Objective 3 Creating the framework GOAL: • To provide a visual means for arriving at a final recommendation, • by juxtaposing the importance rating for each domain against the results of the health technology assessment. • For a technology to be approved, a majority of criteria considered important should also have received favourable findings. • Encourages participation free of group pressure as recommendation made anonymously through our online tool ahead of the committee meeting. 19
Recommend
More recommend
