Qualitative Evaluation of the VHA Telegenomics Clinic - - PowerPoint PPT Presentation

Qualitative Evaluation of the VHA Telegenomics Clinic Implementation Process

Health Services Research & Development Quality Enhancement Research Initiative (QUERI) Project Number: RRP 12-535 (PI: Lerner)

VA Boston Center for Healthcare Organization & Implementation Research (CHOIR)

• Barbara Lerner, PhD, MS (PI)
• Nathalie McIntosh, PhD
• Mark Meterko, PhD
• Shannon Wiltsey-Stirman, PhD

VA Genomic Medicine Service (GMS)

• Vickie Venne, MS, LGC

Background - Telegenomics

• Genomics Medicine Service

– Based at the Salt Lake City VA

• Telehealth based genetic counseling service

– Lack access to expertise in the VA and rural areas – Includes 5 licensed genetic counselors – First patient seen in 2011

• Inter-facility telehealth program

– GMS acts as remote provider with full access to EHR at participating hospitals

Coordination & Interdependencies: Telegenomics Implementation

PATIENT Chief of Pathology

GMS

Facility Technology Coordinator Privacy Officer HIMS(CAC/ADPAC) Group Provider Center Director/ Chief of Staff Telehealth Clinic Technician ITS

Presentation Objective

• Identify the factors that contribute to the efficient

implementation of the Telegenomic Clinic Program at VHA facilities.

Study Design

• Qualitative Semi-structured interviews
• Five facilities

– Facility selection process

• High vs. Low Implementation based on efficiency
• Extreme case purposeful sampling
• Efficiency measures

– Subjective

• GMS staff ratings

– Objectives

• # of contacts GMS staff had and the facility to address problems
• # of system tests required prior to launch

Study Facilities

Quartile Facilities Efficiency Level Bottom 4th A Inefficient Bottom 4th B Inefficient Top 1st C Efficient Top 1st D Efficient Top 1st E Efficient

Study Design

• Semi-structured interviews with key informants
• 1. Facility Telehealth Coordinator (FTC)
• 2. Chief of Staff
• 3. Chief of Pathology
• 4. Information Privacy Officer (ISO)
• 5. Information Technology Technician
• 6. Telehealth Clinical Technologist (TCT)
• 7. Clinical Application Coordinator (CAC)
• 8. Referring Providers

Interviews/facility: Range: 5-8 Mean: 7.2 Total: 36

Consolidated Framework for Implementation Research (CFIR)

Domains

Intervention Characteristics Characteristics

• f the

Individuals Inner Setting Outer Setting Process of Implementation

Constructs

Evidentiary support Knowledge Structural characteristics Patient needs & resources Planning Relative advantage Self-efficacy Networks & communication Organizational connectedness Engaging Adaptability Stage of change Culture Peer pressure Executing Trialability Identification with

• rganization

Climate External policies & incentives Reflecting & evaluating Complexity Readiness for implementation

Damschroder, et al. Implementation Science, 2009

Data Collection & Analysis

Damschroder, L & Lowery, J. Evaluation of a large-scale weight management program using the consolidated framework for implementation research (CFIR). Implementation Science 2013.

Dichotomized Score Interview Coding

(Coder 1)

Interviews Conducted Scored

(Coder 2)

CFIR Scoring Consensus Scored

(Coder 1)

Interview Coding

(Coder 2)

Coding & Rating the Constructs

Damschroder & Lowery, Implementation Science 2013.

Rating Criteria

• 2
• Negative influence;
• At least two interviewees describe explicit

examples

• 1
• Negative influence in the organization;
• General statements but without concrete

examples

• No evidence of positive or negative influence;
• Credible or reliable interviewees contradict each
• ther

+1

• Positive influence;
• General statements but without concrete

examples +2

• Positive influence;
• At least two interviewees describe explicit

examples Missing

• Not asked about construct; or responses did not

correspond to the intended construct

1

Damschroder & Lowery, Implementation Science 2013.

Dichotomized Rating Results

Inefficient Facilities Efficient Facilities Domain Construct Site A Site B Site C Site D Site E Inner Setting

Readiness for Implementation

1 1 1

Networks & Communication

1 1 1

Implementation Climate

1 1 1 1

Process of Implementation

Engaging

1 1 1

Planning

1 1 1

Executing

1 1

Inefficient Facilities Efficient Facilities Domain Construct Site A Site B Site C Site D Site E Inner Setting

Readiness for Implementation

1 1 1

Networks & Communication

1 1 1

Implementation Climate

1 1 1 1

Process of Implementation

Engaging

1 1 1

Planning

1 1 1

Executing

1 1

Dichotomized Rating Results

Tangible and immediate indicators of organizational commitment to its decision to implement an intervention.

Inefficient Facilities Efficient Facilities Domain Construct Site A Site B Site C Site D Site E Inner Setting

Readiness for Implementation

1 1 1

Networks & Communication

1 1 1

Implementation Climate

1 1 1 1

Process of Implementation

Engaging

1 1 1

Planning

1 1 1

Executing

1 1

Dichotomized Rating Results

“I cannot stress to anybody in telehealth how important it is to be friends with everybody. You need to know your CACS, your ADPACs, the HMS Department head. If they know you …and respect you, it's going to make your life so much

• easier. You've gotta be buddies with them. Write your

thank-you cards. Do your SPOT Awards. I'm not above bribing people. I bring people food.” (High performing FTC) “Good working relationships with other departments is

• essential. The secret is baking. Cookies are on the

agenda!” (High performing FTC) The nature and quality of webs of social networks and the nature of quality of formal and informal communications within an organization.

Inefficient Facilities Efficient Facilities Domain Construct Site A Site B Site C Site D Site E Inner Setting

Readiness for Implementation

1 1 1

Networks & Communication

1 1 1

Implementation Climate

1 1 1 1

Process of Implementation

Engaging

1 1 1

Planning

1 1 1

Executing

1 1

Dichotomized Rating Results

The absorptive capacity for change, shared receptivity of involved individuals to an intervention.

Inefficient Facilities Efficient Facilities Domain Construct Site A Site B Site C Site D Site E Inner Setting

Readiness for Implementation

1 1 1

Networks & Communication

1 1 1

Implementation Climate

1 1 1 1

Process of Implementation

Engaging

1 1 1

Planning

1 1 1

Executing

1 1

Dichotomized Rating Results

Attracting and involving appropriate individuals in the implementation through a strategy of social marketing, education, training, etc.

Inefficient Facilities Efficient Facilities Domain Construct Site A Site B Site C Site D Site E Inner Setting

Readiness for Implementation

1 1 1

Networks & Communication

1 1 1

Implementation Climate

1 1 1 1

Process of Implementation

Engaging

1 1 1

Planning

1 1 1

Executing

1 1

Dichotomized Rating Results

The degree to which a scheme or method of behavior and tasks for implementing an intervention are developed in advance and the quality of those schemes or methods.

Inefficient Facilities Efficient Facilities Domain Construct Site A Site B Site C Site D Site E Inner Setting

Readiness for Implementation

1 1 1

Networks & Communication

1 1 1

Implementation Climate

1 1 1 1

Process of Implementation

Engaging

1 1 1

Planning

1 1 1

Executing

1 1

Dichotomized Rating Results

Carrying out or accomplishing the implementation according to plan.

Domains

Intervention Characteristics Characteristics

• f the

Individuals Inner Setting Outer Setting Process of Implementation

Constructs

Evidentiary support Knowledge Structural characteristics Patient needs & resources Planning Relative advantage Self-efficacy Networks & communication Organizational connectedness Engaging Adaptability Stage of change Culture Peer pressure Executing Trialability Identification with organization Climate External policies/ incentives Reflecting & evaluating Complexity Readiness for implementation

Conclusions Primary Drivers of Effective Implementation

Recommendations

• Programmable & Feedback Coordination Theory

– Programmable Coordination

1. Creating templates and instructions for tasks, 2. Explicit understanding of the role and expectations of each team member

– Feedback Coordination

1. Kick-off meeting to with all are involved in the process 2. Regular meetings & communication with all parties

Young & Charns, Health Services Research. 1998

Limitations

• Limited number of facilities evaluated
• Retrospective evaluation
• In one facility the original FTC was no longer

available to interview

Thank You!

• Facilities and Interviewees
• Research Team

Nathalie McIntosh, PhD Mark Meterko, PhD Shannon Wiltsey-Stirman, PhD Vickie Venne, MS, LGC

• Genomic Medicine Service
• barbara.lerner@va.gov
• VA HSR&D/QUERI

Quality Enhancement Research Initiative (QUERI) Project Number: RRP 12-535 (PI: Lerner)

Inefficient Facilities Efficient Facilities Domain Construct Site A Site B Site C Site D Site E Inner Setting

Readiness for Implementation

1 1 1

Networks & Communication

1 1 1

Implementation Climate

1 1 1 1

Process of Implementation

Engaging

1 1 1

Planning

1 1 1

Executing

1 1

Dichotomized Rating Results

Tangible and immediate indicators of organizational commitment to its decision to implement an intervention. “If the providers weren't interested or if it was a struggle to get them to think of patients to refer I don't know if we would have done telegenomics. If the providers aren't interested in doing it, we don't even pursue it.” (High performing FTC)

Inefficient Facilities Efficient Facilities Domain Construct Site A Site B Site C Site D Site E Inner Setting

Readiness for Implementation

1 1 1

Networks & Communication

1 1 1

Implementation Climate

1 1 1 1

Process of Implementation

Engaging

1 1 1

Planning

1 1 1

Executing

1 1

Dichotomized Rating Results

“This kind of telehealth program hadn’t been done here before so we had a huge learning curve. There was all the trouble-shooting and we struggled a lot.” (Low performing FTC) The absorptive capacity for change, shared receptivity of involved individuals to an intervention.

Inefficient Facilities Efficient Facilities Domain Construct Site A Site B Site C Site D Site E Inner Setting

Readiness for Implementation

1 1 1

Networks & Communication

1 1 1

Implementation Climate

1 1 1 1

Process of Implementation

Engaging

1 1 1

Planning

1 1 1

Executing

1 1

Dichotomized Rating Results

“I had a strong marketing campaign that focused on early adopters who were positive about the program.” (High performing FTC) “ We weren’t made aware of how much pent up demand there was from the providers and it came as a huge

• surprise. If we had only been included it would have gone

a lot smoother.” (Low performing Pathology Chief) Attracting and involving appropriate individuals in the implementation through a strategy of social marketing, education, training, etc.

Inefficient Facilities Efficient Facilities Domain Construct Site A Site B Site C Site D Site E Inner Setting

Readiness for Implementation

1 1 1

Networks & Communication

1 1 1

Implementation Climate

1 1 1 1

Process of Implementation

Engaging

1 1 1

Planning

1 1 1

Executing

1 1

Dichotomized Rating Results

“We didn't quite understand the process that was going to

• happen. So we got a lot of things set up and we told GMS

that we were ready. They said we set it up incorrectly for the program so we had to revise that and that took a while.” (Low Performing FTC) “ I Spoke with GMS prior to implementation to determine the optimal way to set up telegenomics. I also met with each department to assess what they wanted and expected prior to setting up the program.” (High performing FTC) The degree to which a scheme or method of behavior and tasks for implementing an intervention are developed in advance and the quality of those schemes or methods.

Inefficient Facilities Efficient Facilities Domain Construct Site A Site B Site C Site D Site E Inner Setting

Readiness for Implementation

1 1 1

Networks & Communication

1 1 1

Implementation Climate

1 1 1 1

Process of Implementation

Engaging

1 1 1

Planning

1 1 1

Executing

1 1

Dichotomized Rating Results

Carrying out or accomplishing the implementation according to plan.

“I blame myself for this that I wasn't as on top of it as I could have been -- but again, everybody has other things they're doing as well, including me…”

(Low performing FTC)