Planning & Program Implementation National Replication - - PowerPoint PPT Presentation

Planning & Program Implementation National Replication

9/30/2015

GOALS FOR THE PRESENTATION

• Introduction of a framework to use when considering replication
• Share web site resources
• Share printed questionnaires to complete when doing your assessment
• Introduce various work products to support replication
• EBAssests
• EBAnalytics
• EBAccountability
• Discuss success and challenges when implementing
• Considerations when implementing

What Type of Cook are You? Do You Follow the Recipe? Do You Improvise or Substitute? Do You Need Pictures? Do You Measure?

Website Resources

1. NREPP (NATIONAL REGISTRY OF EVIDENCE BASED PROGRAMS AND PRACTICES)

• ex. IDENTIFIED PROBLEM IN COMMUNITY IS: DELINQUENCY AND CRIME AMONG YOUNG ADULTS

http://www.nrepp.samhsa.gov/ > enter “crime” in the Find An Intervention box> 18 results, then narrow search by Age {select “18‐25”}; by Outcome Categories {select Crime/Delinquency}; by Geographic Locations {select Urban & Rural} , now narrowed to 2 models to research.

2. THE CALIFORNIA EVIDENCE‐BASED CLEARING FOR CHILD WELFARE

• ex. IDENTIFIED PROBLEM IN COMMUNITY IS: LACK OF SUPPORT FOR PARENTS

http://www.cebc4cw.org/ > Select & Implement Programs>Topic Areas>Alpha Listing yields 5 top rated models to research.

3. BLUEPRINTS FOR HEALTHY YOUTH DEVELOPMENT

• ex. IDENTIFIED PROBLEM IN COMMUNITY IS: BULLYING

http://www.blueprintsprograms.com/ > Program Search>Search All Criteria at Once>Problem Behavior>now narrow search by Program Outcomes {select “Bullying”}; and by Target Population {select Late Childhood & Early Adolescence}, now narrowed to 3 models to research.

4. SAMHSA (SUBSTANCE ABUSE AND MENTAL HEALTH SERVICES ADMINISTRATION)

• ex. IDENTIFIED PROBLEM IN COMMUNITY IS: SUBSTANCE ABUSE

http://www.samhsa.gov/ >Programs & Campaigns>EBP Webguide>Mental Health Treatment>Effective Child Therapy:Evidence‐based Mental Health Treatment for Children & Adolescents>>Effective Child Therapy>Professionals and Educators>About Specific Treatments>Specific Evidence‐based Treatment Programs>Substance Abuse yields 6 programs, 5 with Internet links.

WHAT OTHER RESOURCES ARE THERE

Identifying and Selecting Evidence‐Based Programs and Practices: Questions To Consider

• Outcome Identification
• EBP Identification
• Evidence for Effectiveness
• Organizational and Community Fit
• Capacity and Resources
• Monitoring and Sustainability

Based in theory , collecting quality data, has logic model , and monitoring fidelity with positive findings . Research Design : At least 1 rigorous study

2;

with positive findings Replication : Not Replicated ; and Sustainability : Outcomes are not sustained for 1 year

Innovative Practice Cannot Rate

Emerging

Promising Effective

1RCT – Randomized Controlled Trial 2Rigorous Study

– Randomized Controlled Trial or Quasi ‐Experimental Study

Exemplary

(Evidence ‐Based) Research Design : At least 1 RCT

1 plus

another rigorous study

2; both with positive

findings Replication : Replicated with positive findings ; and Sustainability : Outcomes are sustained for at least a 1 year Research Design : At least 1 rigorous study

2;

with positive findings Replication : Replicated with positive findings ;

• r

Sustainability : Outcomes are sustained for at least a 1 year Research Design : Pre‐Post study or Post study; with positive findings Replication : Not Replicated ; and Sustainability : Outcomes are notsustained for 1 year k

Questions To Ask as You Explore the Possible Use of an Intervention

• Implementations
• Adaptations
• Staffing
• Quality Assurance Mechanisms
• Training and Technical Assistance
• Costs

Assessment Organizational and System Readiness Implementation

Assessment Organizational and System Readiness Implementation

Administrative Oversight Service Provider Oversight Referral/Utilization Management Stakeholder Management Model Fidelity Promotion Data Collection and Evaluation Funding and Sustainability Guidance

Assessment (Month 1)

Administrative Oversight

• SWOT Analysis
• How will the Project be Managed
• Understand how the Target Population will get linked to the service

Service Provider Oversight

• Procure for the service delivery requesting provider proposals
• Develop organizational readiness checklist for the selected provider
• Identify current referral stream for selected provider and cross check for target population

Referral/Utilization Management

• Identify outcome targets for # served, length of treatment & utilization
• Meet with primary referral source and share referral process

Stakeholder Management

• Identity each key stakeholder and determine support for the service while assess knowledge of

the evidence based intervention

Model Fidelity Promotion

• Meet with selected service provider and assess experience and capacity with EBP’s including

quality assurance, data collection, evaluation, program improvement plans, staff retention and incentives, hiring/correction action/firing practices

Data Collection and Evaluation

• Set benchmarks for outcomes
• Identify sources for obtaining outcome data
• Determine need for outside evaluation

Funding & Sustainability Guidance

• Meet with funder to identify current funding amounts, restrictions, review reimbursement

procedures and insure all costs are covered including training, licensing, travel etc.

• Determine if Federal funds or other funds could be used in the future for the intervention

Organizational & System Readiness (Months 2‐4)

Administrative Oversight

• Finalize /Order referral, reporting and information forms that will need to

collected for the target population

• Draft initial policy and procedure manual

Service Provider Oversight

• Contract for services from funder drafted along with securing supporting

documentation (i.e. background screening, insurance certificates etc.)

• Assist with hiring
• Complete review of EBP readiness checklist
• Set up Monthly check in calls with administration for progress

Referral/Utilization Management

• Identify POC for referral agencies
• Train referral agencies on referral procedures

Stakeholder Management

• Train stakeholders on the model, expectations and benefits
• Establish outcome reporting process and content

Model Fidelity Promotion

• Schedule initial model training
• Establish benchmarks for fidelity & potential monetary incentives
• Establish supervision procedures
• Establish monthly calls with model experts

Data Collection & Evaluation

• Formalize data collection, benchmarks and reporting format

Funding and Sustainability Guidance

• Finalize budgets and look for cost savings if able to bundle service costs
• Develop long term sustainability funding for the EBP

Implementation (Month 5 and on)

Administrative Oversight

• Ensure Provider is complying with Policy and Procedure Manual
• Revise Manual as needed

Service Provider Oversight

• Monitor model compliance by randomly attending supervision and file review
• Meet monthly with Program Director and Agency administration to review

implementation of EBP

• Track staff turnover
• Larger rollouts convene monthly Provider calls

Referral/Utilization Management

• Monitor referral, appropriateness and utilization numbers monthly
• Meet with referral sources as needed

Stakeholder Management

• Share outcome data as agreed upon frequency and content
• Participate in Community Meetings dealing with the relevant service delivery needs

Model Fidelity Promotion

• Review and monitor fidelity reports and treatment pacing information
• Facilitate monthly calls with model experts

Data Collection and Evaluation

• Review monthly accuracy of data reported from demographics to dates of service
• Report on identified outcomes
• Meet with outside evaluator to ensure data set quarterly

Funding and Sustainability Guidance

• Monitor expense and make recommendations for overages or under utilization
• Pursue long term funding as needed

• Importance of hiring right (the first time)

PROBLEM: > 50% turnover and average 85 day vacancy rate identified in 2010

• WHAT WE DID:
• Turnover analysis of every contracted agency for last 4 years (2007‐2010)
• HR assessment of each agency’s recruitment, hiring & retention practices
• Researched model requirements for staff competencies
• WHAT WE FOUND:
• Many therapists seemed unprepared for the EBP realities such as in‐home therapy, intense oversight &

scrutiny, and being held accountable for outcomes

• Models had very specific requirements and expectations
• SOLUTIONS:
• In collaboration with the model experts and with the EBP program managers and team supervisors, we created

a list of EBP Core Competencies based on the Alaskan Core Competencies as identified in a survey done by the Annapolis Coalition in 2010. We added several descriptors to each Core Competency

• In collaboration with the model experts and the agencies, we wrote a comprehensive model specific Hiring

Manual

• Paradigm of screening in is flipped to screening out…Initial Screening is designed to screen out
• Structured Interviewing with scoring is stressed

RESULTS: 27% turnover and average 45 day vacancy rate achieved by the end of 2011

CORE COMPETENCIES FOR MASTERS LEVEL EBP'S with descriptors

BASIC KNOWLEDGE ABOUT CLINICAL PRACTICES/THEORIES (i.e. CBT) DESIRE TO LEARN/SELF‐ MOTIVATED EMOTIONAL INTELLIGENCE/ABILITY & TO ENGAGE/ GOOD COMMUNICATION SKILLS COPING SKILLS ANALYTICAL THINKER/ABILITY TO INTEGRATE PROFESSIONALISM CULTURAL COMPETENCY/DIVERSITY SENSITIVITY BASIC THERAPY SKILLS

COGNITIVE/BEHAVIORAL APPROACH DESIRE FOR LIFE LONG LEARNING ABILITY TO ENGAGE SELF CARE/STRESS MANAGEMENT ABILITY TO SHIFT TO PROCESS CONSCIENTIOUSNESS TO DO UNSTRUCTURED WORK INDEPENDENTLY AND WELL. RELATE WITHOUT PATRONIZING ABILITY TO CLARIFY AWARENESS OF FAMILY SYSTEMS WILLINGNESS TO ACCEPT FEEDBACK AND ADJUST ACCORDINGLY BELIEF THAT FAMILIES ARE DOING THE BEST THEY CAN, GIVEN THE SKILLS THEY HAVE. ABILITY TO MANAGE SITUATIONS WITH FAMILIES AS BARRIERS ARE PRESENTED PROBE, NOT TAKE THINGS AT FACE VALUE RESPECT AND UNDERSTANDING FOR TEAMWORK AND COMPANY SYSTEMS. BEING COMFORTABLE WITH CULTURAL DIFFERENCES BEING BRIEF AND DIRECT *PRAGMATIC FAMILY THERAPIES EMBRACE TRAINING FAMILY AND CHILD ADVOCATE/ CHEERLEADER ABILITY TO MANAGE CRISIS SITUATIONS CALMLY CAN CONCEPTUALIZE AND TRANSLATE MENTAL MAPS/CONSTRUCTS FOR FAMILIES GOOD TEAM PLAYER NON‐JUDGMENTAL ABILITY TO REFRAME *SOCIAL ECOLOGICAL THEORY AMBITION TO DO THE BEST STRENGTH FOCUSED TAKING RESPONSIBILITY PRODUCING TIMELY AND USEFUL NOTES & DOCUMENTATION UP FOR THE CHALLENGE ABILITY TO "SELL" THERAPY IDEAS TO FAMILIES BEING DEPENDABLE & RELIABLE HAVING PROFESSIONAL WORKING RELATIONSHIP WITH EXTERNAL STAKEHOLDERS. USE SETBACKS TO GROW ABILITY TO EMPATHIZE ETHICAL AND TRUSTWORTHY BEING ORGANIZED

COLOR CODES

FINE‐TUNED LISTENING SKILLS *PATTERN DETECTION SKILLS GOOD TIME MANAGEMENT SKILLS

SKILLS & KNOWLEDGE‐ measured by Interview Questions

SHOWS COMPASSIONATE AND TACTFUL CANDOR WHEN COMMUNICATING WITH FAMILY *LOGICAL THINKING ABILITY *HIGH LEVEL OF SELF AWARENESS

BEHAVIOR/DRIVES‐ measured by Behavioral Inventory testing

USES HUMOR APPROPRIATELY (IN A NURTURING HEALING WAY) *FLEXIBLE, CREATIVE, OPEN‐ MINDED

BOTH‐ measured by Interview Questions and Behavioral Inventory testing Sources: Annapolis Study, BSFT, EBA, FFT, MST, Florida Redirections Providers

62.0% 84.4% 84.2% 79.6% 76.1% 75.6% 72.9% 70.1% 70.0% 68.6% 66.3% 65.3% 65.2% 63.6% 63.4% 61.5% 61.1% 60.7% 60.6% 58.9% 58.9% 55.3% 54.6% 47.8% 47.0% 39.0% 18.2% 8.3% ‐6.7% ‐15.2% ‐20.0% 0.0% 20.0% 40.0% 60.0% 80.0% 100.0%

Reduction Percentages by Grantee

Reductions in STP Admissions & Felony Commitments from 2012 Baseline (EBA Counties in Green) 15% Reduction

‐40.0% 84.4% 84.2% 70.1% 70.0% 66.3% 63.6% 61.1% 58.9% 55.3% 39.0% ‐20.0% 0.0% 20.0% 40.0% 60.0% 80.0% 100.0%

Reduction Percentages for EBA‐Related Grantees

65.3% Average

79.6% 76.1% 75.6% 72.9% 68.6% 65.3% 65.2% 63.4% 61.5% 60.7% 60.6% 58.9% 54.6% 47.8% 47.0% 18.2% 8.3% ‐6.7% ‐15.2% ‐20.0% 0.0% 20.0% 40.0% 60.0% 80.0% 100.0%

Reduction Percentages for Non EBA‐Related Grantees

50.7% Average

njaner@ebanetwork.com mmcalister@ebanetwork.com