zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBA "Reviewed - - PowerPoint PPT Presentation

zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBA

zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBA 2D Barcode Pilot Lessons Learned & Early Findings

Marshall Gaddis Deloitte Consulting

44

"Reviewed November 2014"

zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBA

Outline

Lessons Learned

• Vaccination Data
• User Experience

Analysis Methodology Early Pilot Findings

• Vaccination Data Overview
• Pilot Supply Chain
• Metrics
• Next Steps

zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBA C3699AA – HOSP (P)C3699AA

1.PEDIARIX-VFC 2.DTAP/IPV/HEP B 3.VFC: DTAP-POLIO- HEP B … 101.DTAP-HEP 8-IPV

PEDIARIX EMR IIS

1.Connected 2.Transfer 3.Double Entry

• 4. IIS Only

Lessons Learned: Vaccination Data

• 1. Data cleansing
• Intentionally added information
• Reduction in data quality
• 2. Consistency between EMR

and IIS data

• Reporting requirements
• Non-identical datasets
• 3. Name standardization
• As written at administration
• Any field with free-form entry

zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBA

Inventory Administration Inventory Administration Only small percent 2D vaccines Difficulties scanning Inconsistent supply 2D vaccines Different process Only small percent 2D vaccines Inconsistent supply 2D vaccines Having to open box to scan Different process other vaccines Difficulties scanning Minor Moderate Major 50 100 150 Respondents (#)

Lessons Learned: User Experience

• 1. Common Challenges
• Within administration and inventory

Preliminary results

zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBA zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBA

Administration: 2D Scanning Frequency 50 (N=152) Respondents (%) 20 10 Less than 1 to 3 4 to 6 7 to 9 10 or more 1 Frequency of scanning per week 40 30

Lessons Learned: User Experience (cont.)

• 2. Incorporating Scanners with

EMRs

• Configuration and capability
• Inventory versus Administration
• 3. Pilot Supply Chain
• Infrequent 2D product
• Decreases likelihood to scan

Preliminary results

Analysis Methodology

zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBA

to

zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBA

Early Pilot Findings: Vaccination Data Overview

Data V

• lume
• Nearly one million records

analyzed

• Robust estimates
• More 2D

records per site in EMR

EMR IIS Baseline Learning Baseline Learning Records 297,000 718,000 344,000 960,000 2D Records 146 32,000 58 26,000 Providers 160 118 204 204

Note: Figures rounded. After exclusions: missing vaccination date, invalid vaccine type (e.g., PPD test), withdrawn site

Site distribution

• Subset of sites provided all

phases

• Before-and-after comparison
• Use for analyses

EMR IIS

7* sites 84 sites 112* sites

*Note: Numbers above represent count of sites with both Baseline and Learning data for either EMR or IIS. IIS data came from different sets of 204 sites in Baseline vs. Learning.

Preliminary results

zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBA

zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBA

Early Pilot Findings: Pilot Supply Chain

• Percent of each vaccine that has converted to 2D barcodes in EMR data

Note: Data shown are from EMR vaccinations

Preliminary results

zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBA

100% 90% 80% 70% 60% 50%

288,000 667,000 265,000 648,000

Complete Accurate Base Learn

Number of records written vertically

100% 90% 80% 70% 60% 50%

192,000 674,000 289,000 816,000

Complete Accurate Base Learn

Number of records written vertically

Early Pilot Findings: Metrics

Data elements

• Lot Number, Expiration Date, Product Code

Metrics

• Completeness and Accuracy

Upward trend before looking at potential confounders

• Isolate effect of technology from other causes
• Impact of percentage increase on one million records

EMR Lot Metrics

(where lot number is scanned)

IIS Lot Metrics

(where lot number is scanned) Preliminary results

zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBA Early Pilot Findings: Next Steps

Maturity Data Collection

• All sites finish by end of April

Analysis

• Calculate effect of technology after removing potential confounders (e.g., Site

variation, observation bias, seasonality)

• Examine across demographic variables
• Statistical tests and adjustments (e.g., Bonferroni)

Learning Data Maturity Data Baseline Data Awardee Recruiting Immunizer Recruiting Immunizer Installation Installation Scheduling Pilot WFA 1 WFA 2

Sep 2011 May 2013 Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May

Questions?