Measuring the Process and I mpact of Children and Young Peoples - - PowerPoint PPT Presentation

Measuring the Process and I mpact of Children and Young People’s Participation

Mererid Lewis, Emyr Williams and Stacey Jones Participation Unit

Aims of Presentation

• To raise awareness of the National Children and Young People’s

Participation Standards for Wales.

• To look at how organisations can move forward from measuring the

process of children and young peoples participation to measuring the impact.

Guesstimates!! Trial Question- How many people had a take-away on the weekend?

1.How many people know how they could go about doing a self-assessment?

• 2. How many people in this workshop have

registered on the Standards Self- assessment website?

• 3. How many people have completed a

Self-assessment?

Measuring the Process - National Standards for Participation

These are a good quality process to help measure and promote the participation of children and young people in making decisions, planning and reviewing any action that will affect them.

• 1. I nformation- which is easy for children and young people to understand
• 2. I t’s Your Choice – enough information and time to make an informed choice
• 3. No Discrimination - every young person has the same chance to participate.
• 4. Respect- Your opinion will be taken seriously
• 5. You get something out of it- You will get something from the experience
• 6. Feedback- You will find out what difference your opinion has made
• 7. I mproving how we work- Adults will ask you how they can improve how they work for

the future

Measuring the Impact-

What’s Changed?

I mpact Assessment Project: Background

• Standards and Kite-marking
• Measuring Outcomes
• A Common Framework
• Links to other developments: WAG

Outcomes Framework, RBA and Demonstrating Success

A number of challenges need to be addressed:

• Currently No agreed indicators against which to measure effective child

participation

• There is a need to construct universally applicable indicators in order to be

able to compare initiatives in terms of the outcomes and impact they achieve.

• Children themselves have a role to play in determining the objectives in any

initiatives in which they are involved.

• The outcomes of participation are often qualitative not quantitative and

therefore hard to measure

• Long-term changes in children’s lives which cannot be measured within a

short time period.

No simple answers exist to these challenges….

• However, Standards and Tools with which to measure child

participation will:

• Help define the legislative and policy environment needed to

promote and respect children’s right to participation

• Clarify who are the duty bearers and their responsibilities
• Enable children to gain a greater understanding of what they hope to

achieve.

• Help identify what support and resources are needed to strengthen

child participation.

• Provide evidence to support the case for political commitment to the

realisation of children’s participation rights.

• Encourage donors to see the benefits of investment in strategies to

promote child participation.

Impact Assessment Project Who?

• Participation Consortium (Standards sub-

group)

• Anne Crowley,Cardiff University
• Other inputs – NYA, Save the Children

Outcomes, Indicators & Performance Measures

Outcome is a condition of well-being for children, adults, families or communities in plain language Indicator is a piece of measurable information which helps quantify achievement of an Outcome Performance Measure is a measure of how well an individual service etc is working Children born healthy Rate of low birthweight babies

Population Performance 3 types

• 1. How much did we do?
• 2. How well did we do it?
• 3. Is anyone better off? = customer results

How much did we do?

Performance Measures (RBA model)

How well did we do it? Is anyone better off?

Quantity Quality Effect Effort

# %

How much did we do?

Types of Measures Found in Each Quadrant

How well did we do it? Is anyone better off?

# Clients/customers served # Activities (by type

• f activity)

% Common measures

e.g. client staff ratio, workload ratio, staff turnover rate, staff morale, % staff fully trained, % clients seen in their own language, worker safety, unit cost

% Skills / Knowledge

(e.g. parenting skills)

# % Attitude / Opinion

(e.g. toward drugs)

# % Behavior

(e.g.school attendance)

# % Circumstance

(e.g. working, in stable housing)

# % Activity-specific measures

e.g. % timely, % clients completing activity, % correct and complete, % meeting standard

Impact Assessment Project What?

FRAMEWORK Three dimensions of change:

• changes to policy and services;
• changes to social and power relations and
• changes to the children or young people’s personal

development and well-being.

• Other dimensions of change could be added depending on the objective of

the participation activity, e.g. parents, community.

Impact Assessment Project What?

FRAMEWORK

• For each dimension of change a small

number of indicators and suggested monitoring arrangements

STEP 1 – Purpose and Measuring Success

• The first step is to decide on the participation

activity/structure/mechanism that you want to evaluate the impact of and to document the purpose(s) of this example of children or young people’s participation.

• Based on the purposes you have identified look through

the Impact Assessment Framework and decide on the most appropriate indicators to be used and add others that are distinctive to the participation you are evaluating.

• For example: if one of the purposes

identified is: to increase children or young people’s rights awareness then select indicators from the framework (under Dimension B.). If one of the purposes is: to enable children or young people to have a voice in shaping services, then some or all

• f the indicators under Dimension A.

should be selected.

STEP 2 – Fill in the specifics

• Agree the specific objectives of the activity you want to evaluate.
• For example if the participation activity relates to trying to influence a

particular policy, service or practice (that a youth forum has selected, for example) record the change that the young people want to see (see Indicator A4 in the Framework on page 5).

• Try and be as specific as possible – it will help you later when you come

back to assess the impact of children or young people’s participation.

• Young people specifying they want ‘careers advice available in informal

surroundings outside of school as well as in school and to have drop in sessions’ is going to be easier to evaluate in policy terms than young people specifying they want ‘better careers services’.

Under Dimension A: Impact on Polices/Services.

• As well as recording the changes that the

children or young people want to see, if you are using indicator A3 – record the issue(s) that the activity is hoping to raise awareness of and define who are the ‘relevant service providers’.

• Add any additional indicators selected by

children or young people.

Dimension B: Impact on Social and Power Relations.

• If you expect or want to have an impact on broader social and power relations

between adults and children/young people – have a look at the suggested indicators under Dimension B.

• This can seem the most complicated ‘dimension of change’ in the Framework.

Activities designed to challenge negative stereotypes of young people, demonstrate the valuable contribution young people can make and thereby encourage adults at all levels of society and in all roles (professional, parent, neighbour etc.) , to engage with, listen to and take into account children and young people’s views when making decisions about things that affect them, would come under this heading. The suggested indicators under Dimension B aim to help you capture evidence of change along these lines.

Dimension C: Impact on personal development and well- being.

• All the indicators under this Dimension rely on ‘before’ and ‘after’
• measurements. When looking to assess the impact of participation
• n the personal development and well-being of children you will

need to decide on which children or young people the participation is expected to impact on – usually it will be the children involved in the forum or group but you may have a wider target group.

• Again, the forum or group may wish to supplement the suggested

indicators with one’s of their own.

STEP 3 – Data Collection

• Collect the BEFORE information and store it safely.
• Decide when you are going to collect the AFTER information.
• In making this decision you need to consider:
• At what point might you expect to see change?
• Changing a policy or a service or how a service is delivered can take a long time – it

might require a policy decision, an implementation plan, training before any changes are felt on the ground.

• The availability of the data – some statistics are only published annually; surveys of

children or young people in schools cannot be undertaken during school holidays or

• ver examination periods.
• How long the group or forum and/or individual members will be around for.
• When you are reviewing the impact of the participation activity you will probably want

to try and understand how these changes have occurred and what helped or didn’t help achieve a good impact or maximum benefits.

STEP 4 – Analysis and report

• The final tasks are to make sense of all the information you have

collected and report on the evidence of the impact of children or young people’s participation on whatever change dimensions you selected.

• You will then be able to make use of this report in any ways you

wish including (we hope!) as part of your regular reviews of the way you work (in line with Participation Standard ’Improving how we work’).

Conwy Impact Assessment Pilot

Aim:

• To improve and formalise existing methods for measuring the impact that the Youth

Council has in general. Objectives:

• To introduce a robust system for measuring the difference that being involved in the

Youth Council has made to the young people taking part e.g. skills, accreditation, achievement (impact on personal development and wellbeing);

• To introduce a meaningful method for measuring what difference the Youth Council

has made to the delivery of the Children and Young People’s Plan in Conwy (impact

• n policies/services);
• To explore methods for measuring how the Youth Council’s involvement has had an

impact on the relationship between adults/services and young people (impact on social and power relations);

• To ensure that Youth Council members are involved in developing these methods,

and in evaluating their own work.

• Give priority to this objective

Baseline

“Before” measure, recorded through a log book filled in by Conwy Youth Council members, during October 2010. B2: Young people’s views on:

• (a) adults’ awareness of children’s rights
• Log book question:
• Adults and your rights - do you feel that the adults working with you

know about your rights and think that they’re important?

Adults and your rights 1 2 4 4 1 2 3 4 5 6 No - 1 2 3 4 5 Yes - 6 Number of Youth Council members

• B3: Young people’s views on whether they can change things

and influence the services and community provision that affect them (and C4)

• Log Book question:
• Your Influence - do you feel like you’re able to change things and

influence the things that affect you in Conwy?

Your influences 1 2 5 3 1 2 3 4 5 6 Not at all - 1 2 3 4 5 Yes fully - 6 Number of Youth Council members

In your groups….

• How do you currently measure impact?
• Where are you missing evidence?
• How applicable would using this

framework be in your setting?

• What are the suggestions for improvement

to the framework and/or implementation

Feedback

Contact Us-

For more information about the work that we do, please contact us – Participation Unit Save the Children Phoenix House 8 Cathedral Road Cardiff, CF11 9LJ Phone 02920 396 838 E-mail- participationunit@savethechildren.org.uk Website- www.participationworkerswales.org.uk