PE PEPR PReC eC Partnered Evidence-based Policy Resource Center - - PowerPoint PPT Presentation

VETERANS HEALTH ADMINISTRATION

Design Choices in Randomized Partnered Evaluations: Veteran-Directed Home and Community Based Services

Melissa M Garrido, PhD

@GarridoMelissa

Partnered Evidence-based Policy Resource Center

PE PEPR PReC eC

VETERANS HEALTH ADMINISTRATION

• HSR&D / QUERI funded resource center
• Objective: Provide timely, rigorous data analysis to support

the development of high-priority policy, planning, and management initiatives and quantitative program evaluations with strong potential to improve the quality and efficiency of VA healthcare

• Director: Austin Frakt, PhD

Overview of PEPReC

2

Partnered Evidence-based Policy Resource Center

PEPR PEPReC eC

VETERANS HEALTH ADMINISTRATION

• Type of randomized design

– Level of randomization? – Timing and distribution of exposure to the intervention?

• Choice of allocation technique

– Simple or restricted randomization?

Design Choices in Randomized Program Evaluations

3

Partnered Evidence-based Policy Resource Center

PEPR PEPReC eC

VETERANS HEALTH ADMINISTRATION

• Goal: Reduce risk of unwanted placement in nursing home or
• ther long-term care facility

Veteran-Directed Home and Community Based Services (VD-HCBS)

4

Partnered Evidence-based Policy Resource Center

PEPR PEPReC eC

VETERANS HEALTH ADMINISTRATION

VD-HCBS Evaluation Goals

5

Partnered Evidence-based Policy Resource Center

PEPR PEPReC eC

• Rigorously evaluate effects of VD-HCBS on health outcomes

and costs

• Work with operations and research partners to produce

evidence that will inform GEC’s decisions about the best ways to prevent unnecessary institutionalization of older Veterans

VETERANS HEALTH ADMINISTRATION

• Considerations

– Feasibility – Contamination of study groups – Statistical power

• Our choice: Cluster (medical

center) randomization

Choice: Level of Randomization Individual vs. Cluster

6

Partnered Evidence-based Policy Resource Center

PEPR PEPReC eC

VETERANS HEALTH ADMINISTRATION

• Considerations

– Equipoise – Feasibility – Statistical power

• Our choice: Stepped wedge

Choice: Timing and Distribution of Exposure to Intervention Parallel vs. Stepped Wedge

7

Partnered Evidence-based Policy Resource Center

PEPR PEPReC eC

Clusters Time 1 Time 2 Time 3 Time 4 Time 5 1-7 8-14 15-21 22-28 Clusters Time 1 Time 2 Time 3 Time 4 Time 5 1-7 8-14 15-21 22-28

VETERANS HEALTH ADMINISTRATION

VD-HCBS Evaluation: Cluster Randomization

8

Partnered Evidence-based Policy Resource Center

PEPR PEPReC eC

VAMCs not currently participating in VD-HCBS (n=77) Early enrollment Excluded:

• Recently or nearly completed readiness review
• Insufficient buy-in from VAMC stakeholders

From eligible VAMCs, GEC will identify ~14 sites willing and able to implement VD-HCBS and refer Veterans within the next six months Late enrollment Repeated every 6-10 months from 2017-2019 Randomization

VETERANS HEALTH ADMINISTRATION

VAMCs 3/2017 6/2017 9/2017 12/2017 3/2018 6/2018 9/2018 12/2018 3/2019 6/2019 9/2019 12/2019 1-7 8-14 15-21 22-28 29-35 36-42 43-49 50-56 57-63 64-70 71-77

VD-HCBS Evaluation: Stepped Wedge Design

9

Partnered Evidence-based Policy Resource Center

PEPR PEPReC eC

Start times and exact number of sites in each step subject to change

Every eligible site will participate in VD-HCBS during the evaluation

VETERANS HEALTH ADMINISTRATION

• Considerations

– Number of units – Existence of potential confounders – Knowledge of partial or full list of participating units

• Our choice: Restricted randomization (covariate constrained

randomization)

Choice: Allocation Technique Simple vs. Restricted Randomization

10

Partnered Evidence-based Policy Resource Center

PEPR PEPReC eC

VETERANS HEALTH ADMINISTRATION

Eligible Sites

Simple Randomization

11

Partnered Evidence-based Policy Resource Center

PEPR PEPReC eC

Simple Randomization

Early Enrollment Late Enrollment

VETERANS HEALTH ADMINISTRATION

Eligible Sites

Matching

12

Partnered Evidence-based Policy Resource Center

PEPR PEPReC eC

Matching

Early Enrollment Late Enrollment Matched Pairs

Random Assignment within Pairs

VETERANS HEALTH ADMINISTRATION

Eligible Sites

Stratification

13

Partnered Evidence-based Policy Resource Center

PEPR PEPReC eC

Stratification

Early Enrollment Late Enrollment Strata

Random Assignment within Strata

VETERANS HEALTH ADMINISTRATION

Eligible Sites

Covariate Constrained Randomization

14

Partnered Evidence-based Policy Resource Center

PEPR PEPReC eC

Covariate Constrained Randomization

Early Enrollment Late Enrollment Early Enrollment Late Enrollment Early Enrollment Late Enrollment Option 1 Option 2 Option 3

VETERANS HEALTH ADMINISTRATION

VD-HCBS Evaluation: Covariate Constrained Randomization

15

Partnered Evidence-based Policy Resource Center

PEPR PEPReC eC

Size of patient population CAN scores Jen Frailty Index scores Prospective NOSOS scores VAMC spending on HCBS VAMC has CLC on campus Market penetration

• f HCBS

Urban/rural location State participation in early participant- directed care initiatives State Medicaid spending on HCBS

Patient Case-Mix Site-Specific Patterns of Caring for Older Patients State or County Access to HCBS

VETERANS HEALTH ADMINISTRATION

• Randomized partnered program evaluations require decisions

about unit of randomization, timing of and exposure to intervention, and allocation technique

• Decisions need to take into account feasibility, partners’

practical concerns, and ability to generate rigorous evidence

Summary

16

Partnered Evidence-based Policy Resource Center

PEPR PEPReC eC

VETERANS HEALTH ADMINISTRATION

• VD-HCBS Evaluation Team

– VHA Office of Geriatrics & Extended Care – Administration for Community Living – PEPReC – Center of Innovation in Long Term Services and Supports – Center for Health Services Research in Primary Care – The Lewin Group – Applied Self Direction

• Funding

– QUERI: PEC 16-001 / HSR&D: SDR 16-196, CDA 11-201/CDP 12-255

Acknowledgements

17

Partnered Evidence-based Policy Resource Center

PEPR PEPReC eC

VETERANS HEALTH ADMINISTRATION

Questions?

melissa.garrido@va.gov http://www.isrctn.com/ISRCTN12228144

The views expressed in this presentation do not necessarily reflect the position or policy of the Department of Veterans Affairs or the United States government.

18

Partnered Evidence-based Policy Resource Center

PEPR PEPReC eC