130 5 Evaluation July 18 , 20 17 Erica Smith, MS Evaluation, - - PowerPoint PPT Presentation

130 5 Evaluation

July 18 , 20 17 Erica Smith, MS Evaluation, Epidemiology, and Data Team Manager Center for Chronic Disese Prevention and Control

Evaluation, Epidem iology, and Data Team

• Erica Smith, MS – Manager
• Carly Stokum, MPH - 1305 Epidemiologist
• Elizabeth Funsch, MPH, MA - 1422 Program Evaluator
• Alicia Vooris, MSPH - Program Evaluator
• Georgette Lavetsky, MHS - BRFSS Coordinator
• Marshall Washick - Graduate Research Assistant

Center Approach

Integration and enhanced bidirectional communication to ensure strong evaluation/ epidemiology and program understanding:

• Managers meet at least weekly
• Data team engaged in program planning and implementation
• Data team attends partnership meetings, site visits, monthly

contract monitoring calls

• Evaluation and program staff meet at least monthly

Role of the Data Team - Exam ples

• Provide overall technical support and guidance on evaluation and data
• Identify appropriate data sources and targets
• Develop reporting templates/ guidance, surveys, and evaluation tools
• Review RFPs and new contracts
• Assist with IRB processes, as needed
• Monitor performance measures and other evaluation indicators
• Develop and contribute to plans, reports, and other publications
• Support data visualization and presentation

130 5 Process Evaluation Questions

1. How has coordination with critical partners changed due to the implementation of 1305?

• b. How has working across categorical program areas enhanced

coordination with critical partners?

• 2. How has your organizational structure and approach changed due to

the implementation of 1305? b. How has working across categorical program areas increased or decreased operational efficiencies?

Process Evaluation Data Collection

• Review the work plan, budget, contracts/ MOUs, and other 1305

documents

• Review organizational chart and position recruitments
• Key informant interviews
• Preliminary findings shared with 1305 staff to cross-check data and fill

gaps in information

• 40 partners critical to successful implementation of 1305 strategies,

including 25 contracts/ MOUs

• 60% of critical partners worked across 2 or more categorical programs,

with frequent overlap between:

• School Health and DNPAO
• HDSP and Diabetes

How has coordination with critical partners changed?

How has working across categorical program s enhanced coordination with critical partners?

• Streamlined reporting
• Aligned with the way partners approach chronic disease
• Techniques to be used across strategies
• Multiple interventions in one setting
• Cross-promotion of programs
• Expanded partnerships for individual categorical programs

• Center is organized into 3 teams:
• Community-Clinical Linkages
• Health Systems
• Evaluation, Epidemiology, and Data
• 90% of 1305-funded staff

positions require work across 2 or more categorical programs

How has your organizational structure and approach changed?

Health Systems Team Manager (Kathleen Graham) Evaluation, Epidemiology, and Data Team Manager (Erica Smith) Health Systems Coordinator (vacant) Epidemiologist (Carly Stokum) School Health Coordinator (Caroline Green) Diabetes Coordinator (Sue Vaeth) Health Policy Analyst (Berit Dockter) Fiscal Officer (Robert Bauer) Partnership Coordinator (Christine Boyd)

1305-Funded Positions

• Combined funding opportunities and contracts
• Staff time
• Shared resources and materials
• Combined meetings, trainings, and events

How has working across categorical program s increased

• perational efficiencies?

• Burdensome budget requirements
• Duplicate strategies in Basic and Enhanced made reporting difficult
• Sub-contractors sometimes gave one categorical program less emphasis

(until addressed by 1305 contract monitor)

How has working across categorical program s decreased

• perational efficiencies?

Process Evaluation Data Utilization

• Shared with CDC
• Shared with Chronic Disease managers and leadership
• Informed action planning and next steps
• Planning for sustainability

DNPAO Evaluation Plan

• Domain 2, Strategy 2: Implement Food Service Guidelines/ Nutrition

Standards 1. What are the key activities and/ or resources considered critical to the successful implementation of food service guidelines? 2. What are the major facilitators and barriers in implementing this initiative? How were barriers overcome?

DNPAO Data Collection

• CDC Worksite Health Score Card data collected through Healthiest

Maryland Businesses (HMB)

• Monthly Reports and Training and Technical Assistance Logs

completed by HMB Regional Coordinators

• Additional qualitative data (e.g. Regional Coordinator group calls,

interviews, etc.)

DNPAO Data Utilization and Sharing

• Informed program planning (e.g. annual goals for Regional

Coordinators)

• Center’s worksite wellness strategic plan and sustainability
• HMB Report this fall to summarize 1305 HMB achievements
• Score Card scores, changes to scores over time, and the impact of

training and technical assistance

DNPAO Data Utilization and Sharing

• 345 businesses, reaching 275,435 Maryland employees, completed the

Score Card

• Among 104 businesses that have taken the Score Card more than once:
• 55 (53%) improved their score in the nutrition section
• 24 (44%) received TA from a Regional Coordinator on nutrition
• 59 (57%) improved their score in the physical activity section
• 25 (42%) received TA from a Regional Coordinator on physical activity

Source: HMB data 7/1/2013-1/31/17

School Health Evaluation Plan

• Domain 2, Strategy 3: Create Supportive Nutrition Environments in

Schools 1. What critical factors or activities influence the successful implementation of nutrition policy and nutrition practice?

• 2. To what extent has implementation of nutrition policies and

nutrition practices increased access to healthier foods and beverages at school?

School Health Data Collection

• Training Tracker System shared with Maryland State Department of

Education (MSDE)

• Local Health Department School and Childcare Wellness grants
• Statewide Surveillance
• School Health Profiles
• Youth Risk Behavior Survey (YRBS)
• Maryland Wellness Policies and Practices Project (MWPPP)

School Health Data Utilization and Sharing

• MWPPP data briefs and in-person feedback

sessions with each jurisdiction

• "Implementation of Local Wellness Policies

in Schools: Role of School Systems, School Health Councils, and Health Disparities” published in the Journal of School Health in October 2016

• Findings informed scope of work for 5

School and Childcare Wellness grants

School Health Data Utilization and Sharing

• MWPPP encouraged schools to establish a school based wellness team
• Percent of schools that reported having wellness teams increased from 44% in Wave I

(2012-2013) to 53% in Wave II (2014-2015)

• During the 2014-2015 school year (Wave II data collection):
• 42% of schools organized or held activities for staff members to support and promote

healthy eating and physical activity (compared to 27% in 2012-2013)

• 26% provided training or education to encourage staff to model healthy eating and

physical activity behaviors (compared to 20% in 2012-2013)

• 26% held activities involving families to support and promote healthy eating and

physical activity among students (compared to 21% in 2012-2013)

School Health Data Utilization and Sharing

• School Health Interdisciplinary Program

(SHIP) Conference

• Presentations highlighted MDH-MSDE

collaboration and included school health data

• 2014 and 2016 YRBS Reports

HDSP Evaluation Plan

• Domain 3, Strategy 1: Increase Implementation of Quality

Improvement Processes in Health Systems 1. What were the major facilitators and barriers in promoting implementation of quality improvement processes, such as use

• f EHRs in health systems? How were barriers overcome?
• 2. How has the relationship between the state health department,

health care systems, and other QI/ HIT partners in the state changed as a result of 1305?

HDSP Health System s Definition

• Local Health Departments, reaching approximately 24 private

practices

• Mid-Atlantic Association of Community Health Centers (MACHC),

reaching 15 Federally Qualified Health Centers (FQHCs)

• Medicaid, reaching 8 Managed Care Organizations (MCOs)
• Maryland Learning Collaborative, reaching 52 Patient-Centered

Medical Home practices (project w as discontinued)

HDSP Data Collection

• Contractor reporting (e.g. Plan-Do-Study-Act cycles (PDSAs),

quarterly data reports (including NQF data), final reports)

• Meeting evaluations
• Academic partner evaluation studies
• Payers (e.g. All Payer Claims Database, Medicaid, and Medicare)
• Additional qualitative data (e.g. one-on-one monthly calls, monthly

Community of Practice Calls, site visits, etc.)

HDSP Data Utilization and Sharing

• “Using Health Information Technology and Data to Improve Chronic

Disease Outcomes in Federally Qualified Health Centers in Maryland” published in Preventing Chronic Disease in December 2016

• 4 FQHCs contributing data with additional FQHCs in onboarding process
• "A Pathway for Quality Improvement in Hypertension: A Cross-Sector

Partnership between Public Health and a Statewide Learning Collaborative" submitted to CDC clearance in June 2017 for eventual submission to the journal of Health Prom otion and Practice

HDSP Data Utilization and Sharing

• “Quality Improvement in Health Systems through Collaboration

between State and Local Governments” poster presentation at the 2017 National Health Outreach Conference in March

HDSP Data Utilization and Sharing

• Million Hearts Evaluation Report and

Partner Profiles

• Used to inform current Quality Improvement

in Health Systems RFP and scopes of work

• Network Analysis
• Analysis of Quality Improvement in Health

Systems partnership networks in 7 funded jurisdictions

• Will inform technical assistance in Year 5 (e.g.

workshop on network management)

HDSP Data Utilization and Sharing

Preliminary Findings from Network Analysis of LHD QI Projects:

• 77% to 96% of respondents in each network said they understand the

skills and knowledge of their partner organizations

• Duration of partnerships varied among jurisdictions:
• Partnerships that existed for less than 1 year ranged from 2% to 38%
• Partnerships that existed for more than 10 years ranged from 15% to 40%
• Primary means of communication with partner organizations varied

among jurisdictions

Diabetes Evaluation Plan

• Domain 4, Strategy 2: Increase Use of Lifestyle Intervention Programs

in Community Settings 1. What were the major facilitators and barriers to implementing the four drivers? How were barriers overcome?

• 2. What were the key activities critical to addressing disparities in

the four drivers during the implementation phase?

Diabetes Data Collection

• CDC Diabetes Prevention Recognition Program (DPRP) Reports/ Website
• Diabetes meetings/ event evaluations
• Be Healthy Maryland website reports and utilization
• Statewide Diabetes Prevention Program (DPP) survey
• Additional qualitative data (e.g. technical assistance calls, quarterly

network call, etc.)

Diabetes BRFSS

• Prediabetes module included in 2014 and 2017
• 10.4% of adults without diabetes have been told by a health care professional they have

prediabetes or borderline diabetes (2014)

• Diabetes module included in 2015 and 2017
• 10.4% have been told by a health care professional they have diabetes (excluding

diabetes during pregnancy) (2015)

• 49.1% have ever taken a course in diabetes self-management (2015)
• Surveillance briefs

Diabetes Data Utilization and Sharing

• Developed maps to understand

geographic accessibility of DPP in Maryland

• Shared with partners, including

Maryland Medicaid and the Maryland Tobacco Quitline vendor, to inform programs

• Used to inform discussions on

future DPP capacity-building efforts and sustainability

Diabetes Data Utilization and Sharing

• Monitor DPP performance, with graphs accessible to program staff:
• Number of DPPs in Maryland increased from 12 in March 2014 to 56 in April 2017
• Percent of participants with a blood test or history of gestational diabetes increased

from 19.5% to 54.9%

• Average weight loss is 4.4% (March 2017)

Diabetes Data Utilization and Sharing

• At least 62 DPP locations statewide (some DPPs have more than one

location)

• 22 of 24 local jurisdictions (91.7%) have at least one DPP location
• Medicaid DPP demonstration project funding from NACDD

Questions?