Are we ready yet? Readiness of community-based organizations to - - PowerPoint PPT Presentation

Are we ready yet?

Readiness of community-based

• rganizations to adopt and implement

evidence-based home visiting programs

Sara rah K Kaye ye, P PhD Kaye Implementation & Evaluation, LLC Debo borah F h F. P Perry, P PhD Georgetown University Center for Child and Human Development

Disclaimer/Disclosure

This project was supported by the Health Resources and Services Administration (HRSA) of the U.S. Department of Health and Human Services (HHS) under grant number D89MC25207 Affordable Care Act–Maternal, Infant and Early Childhood Home Visiting Program (MIECHV). This information or content and conclusions are those of the authors and should not be construed as the official position or policy of, nor should any endorsements be inferred by DOH, HRSA, HHS or the U.S. Government. None of the authors have conflicts of interest to disclose.

Maternal, Infant, Early Childhood Home Visiting (MIECHV)

• Nationwide expansion of home visiting
• 18 evidence-based models approved by HRSA; selected by states
• Targets high risk service populations

MIECHV Benchmark Domains:

1. Improvement in maternal and newborn health 2. Reduction in child injuries, abuse, and neglect 3. Improved school readiness and achievement 4. Reduction in crime or domestic violence 5. Improved family economic self-sufficiency 6. Improved coordination and referral for other community resources and supports

USDHHS, HRSA, MCHB. https://mchb.hrsa.gov/maternal-child-health-initiatives/home-visiting-overview

Implications for capacities of community-based providers:

• Recruitment and retention of

high-risk families

• Assessment of child family

needs

• Referral networks
• Data collection and reporting

Evaluation Context R Reach (and Retention) E Effectiveness A Adoption I Implementation M Maintenance

Glasgow et al. (1998).

Aims of the Adoption Study

POLICY/PRACTICE Understand strengths and limitations of local community-based organizations to inform city- wide capacity-building efforts Think about what providers need to implement a new EBHV model: Healthy Families America (HFA) RESEARCH/EVALUATION Pilot community-engaged research methodology to study influences on community-based organizations’ decisions to adopt EBHV and progress through implementation stages Collect information about organizational-level factors to inform and contextualize other sub- studies Engage and build relationships with provider

• rganizations

Evaluation Methods

All community-based child & family serving organizations

• Key informant interviews
• Guided by TCU Program

Change Model

Phase 1: Adoption

Selected MIECHV local implementing agencies

• All staff interviews
• Document Reviews
• Focused on HFA essential

elements

Phase 2: Implementation

Request for Applications

Community Engagement Methods

Design

• Selected guiding theory of organizational readiness
• Reviewed and provided feedback on interview guides

Recruitment

• Identified organizations and key informants to

include in sampling frame

• Participated in qualitative interviews

Analysis

• Reviewed findings during member checking
• Identified actionable recommendations

Sample Characteristics

PHASE 1 Key informants in organizations who provide child and family services in the community (nstaff=12, norg=10, RR=67%) 2 providers under MIECHV 5 providers of EBHV through other funding sources 3 have never provided EBHV 4 had completed the HFA affiliate process PHASE 2 All program staff in organizations awarded EBHV contracts following RFA process (nstaff=12, norg=2, RR=100%) 1 Federally qualified community health center providing comprehensive maternal/child health care and social services 1 small non-profit providing residential and case management services primarily for pregnant and parenting teens

Phase 1

READINESS FOR ADOPTION OF HFA UNDER MIECHV

Phase 1: Readiness for Adoption

Adoption Planning and Preparation Implementation Practice Improvement

Motivation Resources Staff Attributes Program Climate Costs

Organizational Readiness & Functioning TCU Program Model for Change—Adapted for DC EBHV

Adoption Study Phase 1

Simpson, D.D. & Flynn, P.M. (2007) Moving innovations into treatment: a stage-based approach to program

• change. Journal of Substance Abuse Treatment, 33(2), 111-120.

TCU Program Model for Change – Constructs

Phase 1: Qualitative Analysis

• Interviews audio recorded and professionally transcribed verbatim
• TCU organizational readiness and functioning constructs used as sensitizing concepts during

coding

• Qualitatively coded in Nvivo to identify themes by one analyst (Co-PI)
• Codes with coded quotes reviewed with PI, and codes were refined
• Comparative analysis of perspectives based on sample characteristics (e.g., size, experience)
• First draft of findings shared with interview participants as part of member checking
• Second draft of findings shared with Community Advisory Board to identify actionable

recommendations

Results: Motivation

Influences on organizational decision making Internal External Pragmatic  Sustaining programs

• Self-assessed capacity
• Meeting funding agency expectations
• Trend toward “evidence based”
• Responsiveness to funding
• pportunities

Altruistic

• Fit with mission, values, target

population

• Client needs
• Meeting community needs,
• Responsiveness to target population:

changing demographics Analytic

• Feedback from evaluations
• Feedback from clients
• Feedback from stakeholders
• Community needs assessment
• Analysis of client needs

Results: Organizational Capacity

Construct Findings Resources

• Wide variation in IT infrastructure and data collection/reporting experience
• Office tech was sufficient; mobile tech was limited
• Office space was mostly sufficient (with creativity)
• Internal training resources typically a challenge; orgs partner and share
• Some internal QA/QI/Evaluation, some contracted

Staff attributes

• Range: 1.5 case managers to 102 medical field staff; most have 3-9 home visitors
• Partner with local universities for additional support
• Many staff do not have college degrees, but many are working toward degrees
• Some hire graduates of the home visiting programs
• Important to “be from the community” and “ability to go into the home and be

comfortable and engaging” Costs

• Frequently raised as an area of concern—at start up and ongoing maintenance
• Training was identified as a major expenditure; few trainers available

Results: Planning and Preparation

All organizations, even with mature programs, identified planning and preparation steps that would need to be taken to successfully implement or scale-up with new MIECHV funding

• Recruiting and hiring staff
• Training staff
• Supporting staff, especially paraprofessionals

Phase 1: Adoption and Implementation

Adoption Implementation No Adoption No application to RFA Adoption Application for RFA Planning and Preparation Completed the HFA affiliate process Implementation Providing HFA services Practice Improvement Using feedback to improve HFA delivery 2 8 4 1 8 2 1 1

Provider-Recommended Supports for Increasing Readiness

• Provide funding for EBHV
• Provide training citywide
• Support a network of organizations and meaningful referral process
• Support data collection, data reporting, and evaluation support – and allow organizations to

access their own data

• Develop opportunities for cross-agency peer support for home visitors
• Reinforce outreach attempts through citywide marketing to “get the word out” about home

visiting

Phase 2

READINESS FOR IMPLEMENTATION OF HFA UNDER MIECHV

Phase 2 Mixed Methods Analysis

• Guided by HFA essential elements; considers how TCU readiness and functioning constructs

supports or hinders successful achievement of essential elements

• Open-ended questions asked during semi-structured interviews with all staff and supervisors/

program managers

• Recorded interviews professionally transcribed
• Researchers operationalized indicators of fully ready, ready, and approaching readiness for all

essential elements

• Interview transcripts and supporting documents (e.g., implementation plan, job descriptions,

policies/procedures) and coded by essential element and TCU construct; single analyst (Co-PI)

• Codes and quoted text reviewed with PI
• Both researchers assigned readiness ratings to both organizations for each HFA essential

element

Readiness Rating Scale

Symbol Rating Description

+

Fully Ready

Staff can articulate processes, procedures, and how they support outcomes in a way that is consistent with the program theory of change. The

• rganizational infrastructure has been developed to include supports which

facilitate high-fidelity implementation.



Ready

Staff understand their role in supporting implementation. The organization has sufficient resources to support staff in fulfilling roles and responsibilities.

• Approaching

Readiness

Staff have a general understanding of requirements, but do not demonstrate an in-depth understanding of processes. Organizational resources do not provide the infrastructure necessary for quality and consistency in implementation.

Example

HFA Essential Element 1: Target population and service initiation

+

Fully Ready

The organization has a strategic and comprehensive plan for recruiting

• participants. Formal MOUs are established with organizations that serve a

large percent of the target population. There is a formal process in place for

• rganizations to screen and refer families to HFA.



Ready

The organization receives referrals through a network of organizations who work with the target population. Outreach is conducted with some subgroups of eligible populations.

• Approaching

Readiness

Most referrals will be made by word of mouth. There is not a plan in place to conduct outreach to eligible populations. The outreach and recruitment plan is not formalized.

HFA Critical Element Organization A Organization B Target population and service initiation

+



Standardized intake/eligibility determination

+

• Enrollment and retention

+

• Service intensity

+

• Culturally sensitive services

+



Supporting parents and parent-child development

+



Link to medical and other providers

+

• Limited caseloads

+



Staff selection

+

• Training

+



Supervision

+

Trustworthiness of Readiness Ratings

Inter-rater reliability = 92% agreement Criterion validity – organization A was a mature implementation site, organization B was a new start up implementation site Predictive validity – organization A continued through implementation, organization B did not

Study Limitations

• Key informant interviews are not ideal for assessing organizational readiness and functioning
• Missing information about important influence on adoption and implementation: program

climate

• Limited information about organizations that did not participate in Phase 1 interviews; type and

extent of selection bias is unknown

Discussion/Implications

• Community-based social service organizations in this sample all described some degree of

motivation to implement EBHV, yet demonstrated substantial variation in readiness to adopt and implement EBHV

• Different implementation strategies/supports might be needed to operate through different

causal mechanisms to address different targets in order to increase readiness of providers in ways that are most relevant and meaningful for them. Critical case examples:

• Home-based medical service provider with 102 staff interested in making philosophical shift toward

prevention through 2 generation parenting support programs

• Residential/group home provider with 4 staff and semester-long interns who want to expand into home

visiting

• Experienced HFA provider organization that would be increasing capacity and addressing new

requirements based on funding source

Contact

Sarah Kaye, PhD

Kaye Implementation & Evaluation, LLC sarah@kayeimplementation.com

Deborah F. Perry, PhD

Georgetown University Center for Child and Human Development Deborah.Perry@Georgetown.edu