HeadStart Kent Seminar 4: What have we learnt? Where are we going? - - PowerPoint PPT Presentation

HeadStart Kent

@HeadStartKent #headstartmatters #bounceback

Seminar 4: What have we learnt? Where are we going?

Time Item 9.30am – 9.40am Welcome and Introductions 9.40am – 10am Overview of HeadStart Phase Three 10am – 10.30am Learning from Phase Two and the Knowledge Seminars 10.30am – 11am Creating a Theory of Change: the process so far and plan 11am – 11.15am Break 11.15am – 12pm Activity: Theory of Change for HeadStart Kent 12pm – 12.15pm The importance of providers developing their own TOC 12.15 – 12.30pm Next Steps for Knowledge Seminars: thoughts from the group

@HeadStartKent #headstartmatters #bounceback

Plan for the Seminar

The HeadStart Vision

for Phase Three

Angela Ford

@HeadStartKent #headstartmatters #bounceback

THE HEADSTART MISSION

The Big Lottery Funded HeadStart programme aims to improve the mental well-being

• f at-risk* 10 to 16 year-olds by investing up to £75m in up to 12 local partnerships to

facilitate and support: 1.the implementation of a locally developed, cross-disciplinary, multi-layered and integrated prevention strategy, with the young person and their needs at its core 2.the development of the necessary local conditions to enable that strategy to become sustainable in time 3.the development of a more robust evidence-base around ‘what works’ in the area of mental well-being to be pro-actively shared beyond HeadStart with the aim of contributing to the national and local policy debate.

* At risk of developing mental ill-being

4

THE SPECIFIC ROLE OF RESILIENCE

Particularly relevant in the context of HeadStart given that many children in our target population might face more adversity than the norm WELL-BEING Resilience

5

Socially significant improvement of the mental well-being of at- risk young people Reduction in the onset of diagnosable mental health disorders Improved engagement in school and improved academic attainment Reduced engagement in ‘risky’ behaviour: e.g.;

• Substance abuse
• Criminality
• Teenage pregnancy

Main programme outcome Longer-term outcomes

6 Baseline age 10 HeadStart exit

Improved employabilit y

1 2 3 4 5

Positive transitions: e.g.; Focus of HeadStart activity Outcomes embedded within the local systems

6

WORKING TOWARDS A COMMON OUTCOMES FRAMEWORK

THE FOUNDATIONS

7

By May 2016 we want to be able to take a firm decision

• n which

partnerships, if any, to fund, with clarity

• n what we will be

funding and to what end

• Confidence in the proposed

programme

• Confidence in the local

leadership

• Confidence in sustainability of

programme beyond the Big Lottery Fund

• Confidence in the

implementation

• Confidence in the willingness /

ability to engage

• Robust programme strategy:
• Local translation of HS mission
• Clarity on target population
• Combination of robust interventions and

approaches

• Articulation of integrated client journey
• Clarity of short, medium and long-term
• utcomes
• Strong and committed governance in place
• Strong day-to-day management in place
• Clear and robust articulation of how programme could

continue to be funded after HS

• Strategy for achieving sustainability beyond HS
• Robust implementation plan for the first 18 months of

the programme

• Detailed budget based on clarity of current spend

against the proposed programme versus required spend

• Basic delivery infrastructure in place
• Evidence of willingness and ability to engage with

wide range of stakeholders: young people, community, other partnerships, external experts, S&D, Big Lottery Fund We want…. We need from the partnerships...

BUILDING BLOCKS OF AN EFFECTIVE PROGRAMME STRAGEGY

8

Mission Target Population Outcomes Programme Design

Definition

• Purpose of the programme,

what it is meant to achieve and how it will achieve it What good looks like

• Clear statement of intent
• Basis for determining whether

the partnership has succeeded

• Descriptive and unambiguous –

not a Vision

• Aspirational…but achievable
• Holistic and enduring

Definition

• The group of young people to

which the partnership will holds itself accountable

• Expressed as a set of

enrolment criteria

• E. g. Age of entry into the

programme, socio-economic profile

What good looks like

• Clear operational guidelines
• Clear thresholds

Definition

• Specific differences we look for the

programme as well as the individual interventions to make What goods look like:

• Clear basis to ascertain whether the

programme / the intervention has achieved its aim (after the fact)

• Clear basis to ascertain whether the

programme / the intervention is achieving its aim (while it is on-going) Definition

• Documentation of which interventions,

by whom, when, how often, how long…what exactly will happen in order to achieve outcomes (including the system mechanisms) What good looks like

• Shared understanding of what the

“programme” means for everyone in the partnership

• Clear rationale for how the

“programme” will lead to the outcomes

• Provides a clear basis for recruiting,

training and manage staff (competencies)

The Big Lottery Fund’s HeadStart programme aims to improve the mental well-being of at- risk 10 to 16 year-olds by investing up to £75m in up to 12 local partnerships to facilitate and support: 1.the implementation of a locally developed, cross-disciplinary, multi-layered and integrated prevention strategy, with the young person and their needs at its core 2.the development of the necessary local conditions to enable that strategy to become sustainable in time 3.the development of a more robust evidence-base around ‘what works’ in the area of mental well-being to be pro-actively shared beyond HS with the aim of contributing to the national and local policy debate

MISSION

9 Each of the partnerships to translate the overall HS mission into a statement of intent for their local programme strategy

• Clear articulation of what

the programme is meant to achieve and how it is going to achieve that

• Common interpretation

across the partnership

• Drives decision-making

Big Lottery Fund mission for HeadStart Local mission for HeadStart

OUTCOMES THE RESILIENCE FRAMEWORK CAN HELP DEFINE SOME OF THE SHORT-TERM OUTCOMES

10

• Access to a trusted adult available over time
• Access to and engagement with activities, hobbies, sports, etc.
• Improved problem-solving skills (how to measure?)
• Access to a safe space (what does that mean in practice?)
• A realistic yet aspirational plan for the future
• Improved self-management skills (how to measure?)
• Access to and engagement with activities that contribute to others (e.g.;

volunteering, peer mentoring)

Source: BoingBoing

PROGRAMME DESIGN DIFFERENT PROGRAMME ‘LAYERS’

11

Interventions aimed at improving the ability of schools / other services to support the mental well-being of children/ young people Universal and targeted interventions aimed at improving parent’s ability to support the mental well-being of their children Universal as well as targeted interventions specifically aimed at improving the mental well- being of young people (in school, in alternative provision, in the community) Awareness raising Professionals (including teachers) / practitioners / volunteers Parents / carers Young people Activity that does as it says

• n the tin

Integration into a cohesive journey with the young people at its core Again, each of the partnerships will need to decide how to balance activity across those different ‘layers’ of activity

LEADERSHIP WITH A BIG ‘L’

12

The people and the structures in place to drive the programme and realise its full potential

• Commitment and engagement from the ‘top’
• Commitment and engagement from all key stakeholders

(organisations and relevant individuals)

• Clear accountability
• Common purpose with clarity on individual roles and responsibility

amongst all partners

• Clear decision-making processes based around the presentation of

insights

• Strategic thinking
• Change agent(s)
• Solid day-to-day programme management
• ‘Young people in the lead’

SUSTAINABILITY IN THE CONTEXT OF PHASE 3

13

We are not expecting ...

• A firm commitment from the

partnerships by July 2016 to fully replace BLF funding to cover HS activity once the initial funding commitment has run out We are expecting...

• An understanding amongst the partnership that

embedding HS within the local context long-term is absolutely part of our aims for HS – if this doesn’t happen in the long-term, we will not consider the programme a success

• A partnership governance structure that facilitates

sustainability by including key decision-makers and providing links into the wider agendas at local level

• An understanding amongst the partnership of the

existing relevant funding pots, including an understanding of the current spend against HS activity

• An understanding of what would need to happen for

funds to be able to shift to HS activity: e.g.;

• What evidence
• Milestones to achieve
• Leverage explicitly considered when developing the

budget but

ELEMENTS OF SUPPORT TO BE PROVIDED

14 Elements Support with strategy development Fund mapping Support with the development of implementation plan / budget Development of common outcome / measurement framework – support with the set-up of local measurement frameworks What Facilitation of series of structured workshops to support the partnership in the development of their Phase 3 programme:

• Clarity on target population
• Clarity on desired outcomes to achieve
• Clarity on programme journey for young people
• Which interventions
• How to integrate
• What will be different

Support the sites to identify all the relevant funding streams for HS-related activities (including existing activity) to feed into the budgeting process : Review and critique of the plans developed by the partnerships

• Facilitate a process with the partnerships to identify, across

the locally developed outcome frameworks, the common themes and decide, for those common outcomes, common measurement

• For those common outcomes and measurements, develop a

common framework, going back to input, outputs, etc Who Facilitated by new S&D in close partnership with existing S&D (to provide subject knowledge) and HS team New S&D

• New S&D in close partnership

with existing S&D (to provide subject knowledge) and HS team

• At final stage of programme,

potentially involve expert advisors

• Anna Freud and new S&D

‘YOUNG PEOPLE IN THE LEAD’

• There is a wide recognition amongst all parties involved in HeadStart (the

partnerships, their wider stakeholders, S&D, the HS team) that young people need to be at the core of HeadStart for the programme to be able to achieve its aims

• There has been various efforts at all levels, some more successful then others, to

ensure young people are meaningfully involved in all stages of the work (needs analysis, prioritisation of solution, design, implementation, monitoring)

• We see this as an area for continuous improvement and we will look to actively

engage young people in the next few months, working with the partnerships, to ensure we continue t develop our common understanding of what ‘young people in the lead’ should look like in practice, from the perspective of the young people...

15

16

July/August 15

Workshop: academic resilience Phase 3 Options identified Lottery workshop 28th Exec Group

September 15

Theory of Change drafted 8th - Knowledge seminar TOC Activity Young people District Councils Health and Wellbeing Board

October 15 TOC and cohort

• ptions to

Shadow board Programme Board 1/10 Common Measurement Work Shop 13/10 Kent Support and Development workshops

Nov/Dec 15

Draft Strategy and Implementation plans formed Sustainability checked Wider refinement 21 Dec result of workshops and coproduction

January 2016

Bid finalised, and signed off. Costings Shadow Board Programme Boards

February 16

26th Bid submitted to Big Lottery

March - July 16

Young People’s Shadow Board and Programme Board in April Phase 2 completed - 31st July Building blocks for phase 3:

• Confidence in proposed programme
• Confidence in local leadership
• Confidence in sustainability of programme

beyond BIG

• Confidence in the implementation
• Risk and Issues identified and Mitigated

Proposed Draft Time Line for Phase 3 Bid Submission

Dates KCC Meetings BIG Meetings Young People Meetings 28/8/15 HSK Exec. Group 8/9/15 HSK Knowledge Seminar 24/9/15 Children’s Health & Wellbeing Board 1/10/15 BIG (CMF) London 7/10/15 HSK Young People’s Shadow Board HSK Young People’s Shadow Board 13/10/15 HSK Programme Board S&D 1st Workshop 10/15 - 11/15 Consultation, schools, districts, internal, external, yp and families, market place, head teachers 22/10/15 HSK Exec Group. S&D debrief 3/11/15 Digital workshop (Manchester) 10/11/15 S & D 2nd workshop S & D 2nd workshop 23/11/15 HSK Exec. Group 25/11/15 Children’s Health and Wellbeing Board 28/11/15 HSK young person shadow board HSK young person shadow board 8/12/15 Big Lottery CMF 9/12/15 Young people in the Lead 21/12/15 HSK Exec. Group 5/1/2015 S&D 3rd workshop S&D workshop 9/1/16 HSK Young People’s Shadow Board HSK Young People’s Shadow Board 12/1/16 HSK Exec. Group S&D Final Review Meeting 20/1/16 Big Lottery CMF 28/1/16 HSK Programme Board 5/2/16 HSK Exec. Group

Learning from Phase 2 and the Seminars

@HeadStartKent #headstartmatters #bounceback

Challenges Learning from the projects and the knowledge seminars What is needed for the way forward

Learning from Phase 2 and the Seminars

@HeadStartKent #headstartmatters #bounceback

Challenges

 The Big Lottery’s lack of clarity on focus for the project and for the evaluation  Systemic and strategic issues about how the programme was initiated.  The ground work needed to ensure a coherent model and evaluation framework were implemented was not undertaken.  This has impacted on how projects were selected and rolled out and issues related to evaluation.  This has led to a scattered approach that may impact on well-being but possibly not resilience.

@HeadStartKent #headstartmatters #bounceback

Challenges

 Linking broader aims of HeadStart programme to what projects actually do and the need for projects to fit what they do with the broader aims of HeadStart has raised challenges for both parties.  The co-construction of a definition of resilience across Kent and a coherent theory

• f change that projects can be commissioned reviewed against are key.

 However there is a tension of working collaboratively with commissioned services.  Kent does need to acknowledge the challenge that poses in a market driven economy where projects are competing for commissions and where some of the stakeholders have a level of autonomy that will allow them to go in their direction in spite of a county-wide view.

@HeadStartKent #headstartmatters #bounceback

Learning from the Projects and the Knowledge Seminars

Increase in Strengths Based Approaches  A lot of new projects in place throughout Kent.  An increase in the number of environments conducive with positive wellbeing for children and young people.  Professionals from a number of different agencies are clearly moving toward a strengths based approach (rather than deficit) that is conducive with enhancing resilience processes.  A number of the projects are clearly embracing principles of co-production with the young people.

@HeadStartKent #headstartmatters #bounceback

Learning from the Projects and the Knowledge Seminars

Focus on Universal Approaches  A gap in terms of any targeted approaches focusing on children and young people who would benefit the most i.e. those at serious risk of adversity.  For example a focus could have been on children living in complex families where there are mental health problems, drug and alcohol misuse and many more. It is these families where the parents have multiple needs, experiencing risk and adversity that we have strong evidence base on how to intervene and promote resilience.  A concern is that funding has been adsorbed in to mopping up gaps in what should be universal provision.

@HeadStartKent #headstartmatters #bounceback

Learning from the Projects and the Knowledge Seminars

Talents and Interests Neglected as a Domain  Very few examples where the children and young people were offered any

• pportunity to develop new talents or interests, a critical domain.

 Highlights a consistent problem throughout HeadStart Kent in that it is not linking back the activities and services they are providing to a resilience framework and certainly not covering many domains.

@HeadStartKent #headstartmatters #bounceback

Learning from the Projects and the Knowledge Seminars

Evaluating Impact  There was very little (if any) baseline data collected at the beginning of HeadStart and where projects are collecting data this is not consistent between the partner

• rganisations preventing any meaningful comparison.

 Demonstrating impact or demonstrating outcomes with any of the programmes is going to be extremely problematic.

@HeadStartKent #headstartmatters #bounceback

Learning from the Projects and the Knowledge Seminars

Theory of Change and Use of Evidence  There seemed to be an issue of understanding the ‘what’ and ‘why’ of Theory of Change by the projects.  Most projects did not have an explicit theory of change.  This is of concern as a theory of change is vital in terms of explaining the aims of the project, its expected outcomes, how it expects to achieve these i.e. how project activities link to achieving these outcomes and how they plan to measure it.

@HeadStartKent #headstartmatters #bounceback

Learning from the Projects and the Knowledge Seminars

Theory of Change and Use of Evidence  Linked to this is the issue of evidence and how it is used.  Often projects were undertaken with no reference to the evidence on which it was based.  There seemed to be a misunderstanding that a project needed to be evidence based and therefore an innovation or new project does not need a theory of change or evidence.  Any innovation that wants to demonstrate evidence needs to be clear about how it is proposing to meet its aims and outcomes and should be using evidence to argue for the logic or rationale for why they have chosen to do things in the way they have

@HeadStartKent #headstartmatters #bounceback

Learning from the Projects and the Knowledge Seminars

Theory of Change and Use of Evidence  If there is no evidence in an area then projects need to be clear about this and the project needs to set out a clear argument for how they plan to evidence it themselves.  Innovative approaches that are less tried and tested are vital for developments in this field however they should still be grounded in a credible theoretical model of change.

@HeadStartKent #headstartmatters #bounceback

Learning from the Projects and the Knowledge Seminars

 Theory of Change and Use of Evidence  In terms of evidence for projects it seemed that often projects were not critical of the evidence bases or of their own evidence.  It is much more helpful to be tentative and critical in the conclusions made about

• ne’s evidence and be open to the fact that the field of resilience or emotional

wellbeing is a complex one and it is hard to be conclusive.  The Social Research Unit /NESTA offers a set of Standards of Evidence that projects should be encouraged to explore and use when thinking of their own evaluations.

@HeadStartKent #headstartmatters #bounceback

@HeadStartKent #headstartmatters #bounceback

Learning from the Projects and the Knowledge Seminars

The Role of the Knowledge Seminars and their Impact  There is a need to consider the role of the Knowledge Seminars.  It appears that despite them proving valuable information to the project team, in their current format they have had limited impact on the projects views and understanding of resilience.

@HeadStartKent #headstartmatters #bounceback

What is needed for the Way Forward:

 There is a need to develop an overarching framework for emotional wellbeing so staff/agencies can ‘sign up’ to this. This needs a clear conceptualisation of emotional wellbeing i.e. how is it defined and what is the theory of change.  Based on this evidence based approaches can be mapped onto this conceptualisation and projects commissioned based on this.  There needs to be a coherent system for evidence based evaluation ensuring that each element of the system is clear on how they evidence outcomes and impact

@HeadStartKent #headstartmatters #bounceback

What is needed for the Way Forward:

 An evidence matrix / outcomes framework is key to measuring emotional wellbeing in its varying forms across services.  Need to ensure that thought is given to measurement at a county level (what does this tell us about Kent’s progress in terms of improving resilience in young people), a service level (how does the service know it’s contribution to the overall picture is effective) and an individual level (how do we ensure that outcomes are meaningful for the work with individual young people).

Developing a Theory of Change

HeadStart Kent Knowledge Seminar

Eileen McKibbin Research & Evaluation Manager KCC - Strategic Business Development & Intelligence Ugochi Nwulu Public Health Researcher CHSS – University of Kent

HeadStart - National Mission

There are 3 aims; the third is -

• “the development of a more robust evidence-base around

‘what works’ in the area of mental well-being to be pro- actively shared beyond HeadStart with the aim of contributing to the national and local policy debate”

Review – theory of change

• Theories of Change explain how & why a service or programme

works (or helps identify in what ways and why it is not working as you expected)

• They explain the steps (and little steps) required to achieve
• utcomes – but also – the connections between them
• A key reason that complex programmes are difficult to evaluate is

that the theories of change that underpin them are poorly articulated (Weiss, 1995)

NESTA Standards of Evidence

Early Intervention Foundation - Standards of Evidence

What we have learned so far across Kent

• Three mechanisms for operationalising HeadStart in Kent

1. local coordinator to monitor interventions 2. no local coordinator with oversight from HS operational team 3. local coordinator with devolved budget and decision making

• Learning / developmental phase meant that a degree of flexibility was

allowed in implementations

• A great deal of good will helped with the programme!
• Acceptability of interventions better when YP/families are actively involved
• Local context plays a big role in acceptability/success of interventions

targeting community practitioners

Key lessons from our learning to take forward for Phase 3 Programme

An outcome evaluation necessitates a different approach:

• Clear Theory of Intervention / Theory of Change for evaluation of

intervention fidelity and outcomes

• Marketing of HS interventions with visible local link / coordinator increases

success of implementation Greater acceptability / durability of Phase 3 programme:

• Involvement of YP important
• Acknowledgment of ‘goodwill’ - the efforts of practitioners /teachers etc. in

helping to market/deliver interventions

• Being prepared for local contexts affecting implementation plans

@HeadStartKent #headstartmatters #bounceback

Tea break

Activity: Theory of Change for HeadStart Kent

Discussion – appreciative enquiry

Discover (what’s working well) Dream (what would be good) Design (planning and prioritising) Destiny (Actions)

The importance of providers developing their own TOC

Theory of Change to Logic Models

Your road map!

Theory of Change (ToC)

• The product of a series of critical thinking exercises providing a comprehensive

picture of short and medium term changes in a population that are needed to reach a long term goal

• Provides an honest picture of the steps required to reach a goal and the steps

required to reach it

• What can you influence? What impact can you have? Is it realistic to reach the goal

with the time and resources available?

• How is the impact / change measured
• NOT ROCKET SCIENCE BUT A STRATEGIC EXERCISE USING CRITICAL THINKING AND

EVIDENCE

Steps to creating a TOC

1. Identify long term goal 2. Conduct a ‘backwards mapping’ exercise to identify the preconditions necessary to achieve that goal 3. Identify the interventions that your service will perform to create these preconditions 4. Develop indicators for each precondition that will be used to assess the performance of the intervention 5. Write a narrative that can be used to summarise the various moving parts in your theory Adapted from www.theoryofchange.org

TOC to Logic Model

• Logic model is a visual model of the process to achieve the
• utcome
• Highlights the link between outputs and outcomes – what is

invested, what is done, and what results

• Can be more than one in an organisation
• ToC works at strategic level whereas logic model is used to

summarise the work at programme / operational level

A logic model is your program ROAD MAP

Where are you going? How will you get there? What will tell you that you’ve arrived?

What does a logic model look like?

 Graphic display of boxes and

arrows; vertical or horizontal

• Relationships, linkages

 Any shape possible

• Circular, dynamic
• Cultural adaptations; storyboards

 Level of detail

• simple
• complex

 Multiple models

Family Members Budget Car Camping Equipment Drive to camp site Set up camp Cook, play, talk, laugh, hike Family members learn about each

• ther; family

bonds; family has a good time

Example: Every day logic model – Family Camping Trip

Staff Money ey Partner ers Devel elop

• p

parent t ed curriculum Deliver ver series es of interac ractive tive sessi ssion

• ns

Parent rents s incre crease e knowl wledge edge of chil ild dev Parent rents s bett tter er underst erstand nding ng their eir own n parent renting ng style yle Parent rents s use e effec fecti tive e parent renting ng practi ctices es Impro roved chil ild- parent rent rela lati tions

• ns

Resea earc rch

INPUTS OUTPUTS OUTCOMES

Facilitate te support

• rt

groups ps

Parent rents s gain in skil ills in effec fecti tive e parent renting ng practi ctices es

Example: One component of a comprehensive parent education and support initiative

Parent rents s ident entify fy approp ropri riate te acti tion

• ns to

take Stron rong g famil milie ies

Situation:

Targe gete ted parents nts attend nd Assum umpt ptio ions ns: Exter ternal nal factor

• rs:

During a county needs assessment, majority of parents reported that they were having difficulty parenting and felt stressed as a result

INPUTS OUTPUTS OUTCOMES

Program investment s Activities Participation Short Medium What we invest What we do Who we reach What results Long- term

Programmes are not linear

Feedback loops and multi-dimensions

SMART objectives: Specific, measurable, attainable, results-oriented, timed

Who/what Change/desired effect In what By when Families participating in the Family Resource Center

increase their use of community resources and services within one year

• f joining

4 school boards adopt policies to improve student nutrition and physical activity by Dec 2014

Writing good outcomes

Where does evaluation fit?

From beginning to end

Next Steps for Knowledge Seminars: thoughts from the group

@HeadStartKent #headstartmatters #bounceback

Search ‘Kelsi HeadStart’ Email HeadStart@kent.gov.uk

Contact HeadStart Kent