[PPT] - Programme BRICK Programme Evaluation: How, why and what? The plan PowerPoint Presentation

SLIDE 1

External Evaluation of the BRICK Programme

BRICK Programme Evaluation: How, why and what?

SLIDE 2

The plan …

 Practical evaluation - meeting objectives  BRICK evaluation - aims and approach  BRICK evaluation - emerging findings  Learning - other Catalyst programmes  The future - legacy and sustainability  Q&A

SLIDE 3

Practical Evaluation



Meeting objectives



Managing resources



Evaluation Plan



Research tool design - evidence gathering



Coverage - sample size



Experiencing the programme offer



Analysis



Reporting

SLIDE 4

Best elements - what has worked?



What have been the key successes?

Process of management or delivery
Impacts on organisations
Impacts on teams or individuals
Softer or less intangible benefits
Continuous learning
Ability of participants to influence the design and

delivery of the project

Case study material – assessment and marketing

SLIDE 5

Learning points - what could have worked better?



What have you all learned?

What hasn’t worked?
What would you change with the benefit of hindsight?
What are the barriers or challenges?
What are the catalysts for success?
Are there any differences between cohorts?
What are the key successes/learning points?
Have lessons been learnt and good practice shared

across partners?

SLIDE 6

BRICK Evaluation

 External evaluation by an independent consultant  Cover training and materials over the final year of the

programme as well as documentation from years 1 to 3 inclusive

 Complement internal evaluation work  Assessment of the medium/long term impact of

participating in the BRICK programme of early BRICK attendees

 Assessment of how the BRICK programme has

progressed, achieved against objectives and legacy

SLIDE 7

A. Project Management

Inception meeting Development and agreement

f an Evaluation

Plan Regular progress reporting

B. Desk Research

Document Review Data Analysis Research Tool Design

C. Fieldwork

BRICK Delegates Online survey Interviews with at least 25 participants Stakeholders Telephone interviews stakeholders and programme sponsors Workshop

bservation
D. Analysis and Reporting

Analysis Qualitative Quantitative Reporting Ten case studies Draft Report Final Report

BRICK Evaluation Methodology

SLIDE 8



Unless stated based on 71 responses to online survey of people accessing the following elements of the BRICK programme offer:  Workshops  Mentoring  BRICK +  Online resources  Heritage Helpline  Conferences



Complemented by stakeholder consultations

BRICK Evaluation - Emerging Findings

SLIDE 9

17% 13% 13% 10% 10% 7% 7% 7% 6% 4% 4% 3%

0% 5% 10% 15% 20%

In which region or country is your organisation based?

SLIDE 10

48% 18% 13% 10% 6% 3% 3%

0% 10% 20% 30% 40% 50%

How would you describe your organisation?

SLIDE 11

Prince’s Regeneration Trust website 61% Direct invitation (BRICK and/or PRT) 56% BRICKwork website 49% Word of mouth 29% Heritage Update 23% Heritage Lottery Fund 23% Architectural Heritage Fund 20% English Heritage 9% Association of Preservation Trusts 7% Local council 6% Historic Environment Scotland 4% Twitter 4% Facebook 4% Cadw 3% Historic Environment Division - DoE Northern Ireland 1% How have you received information about the BRICK Programme?

SLIDE 12

How would you rate the BRICK workshops?

61% 39% 40% 43% 40% 41% 51% 43% 38% 51% 50% 48% 40% 49% 46% 55% 2% 8% 5% 7% 20% 10% 2% 2% 2%

0% 10% 20% 30% 40% 50% 60% 70%

Excellent Good Average Poor Very Poor

SLIDE 13

Most useful aspect of the BRICK workshops? ‘Questions were answered in a way that made me sense to me instead of the terminology going over my head’ ‘Ability to meet other people involved in similar projects, and to network’ ‘Meeting with other people who were already further down the project path than ourselves, gave us hope!’ ‘The talks, on the whole, were excellent, informative, inspiring and useful’ ‘Making contact with experts’

SLIDE 14

Most useful aspect of the BRICK workshops? ‘The inspirational talks by speakers who were already actively carrying out projects’ ‘The BRICK workshop provided a good, clear and informative picture of the elements involved’ ‘I came away feeling I had tools and knew how to go about approaching funders’ ‘Talking about governance and how to handle and run

rganisations with willing volunteers and recruit people to

become involved’

SLIDE 15

Suggested improvements to the BRICK workshops? ‘more case studies from rural areas would have been more relevant’ ‘more practical workshops would have been welcomed with more relevant case studies’ ‘slightly higher level information - in addition to info for

rganisations earlier in the project process’

‘more ways of fundraising, explanation of all funding

pportunities that are available’

‘not so many topics per day to give more opportunity for discussion to learn from other experiences’

SLIDE 16

61% 39% 33% 50% 28% 17% 28% 28% 11% 11% 6% 6% 6% 11% 11% 6% 28% 17% 6%

0% 10% 20% 30% 40% 50% 60% 70%

The quality of the mentoring Enabling your project to secure funding Helping to unlock barriers to the project proceeding Increasing your confidence in undertaking a community project

Excellent Good Average Poor Very Poor Don't Know N/a

How would you rate the mentoring support? (n=18)

SLIDE 17

22% 17% 22% 33% 33% 11% 17% 28% 33% 33% 28% 28% 28% 17% 17% 28% 22% 17% 17% 39% 39%

0% 10% 20% 30% 40%

Increasing capacity to raise funding Upskilling trustees Improving

rganisational

governance Developing a sound business case Developing a project vision Marketing and PR Informing the

wnership,

repairs and reuse of the building

Substantial Significant Some Very little None N/a

Impact of mentoring support? (n=18)

SLIDE 18

Describe the impact of the mentoring programme on you and/or your organisation?

SLIDE 19

Suggested improvements to the mentoring programme ‘more time input’ ‘can you clone another 6 Laura Norris's so she can visit us more often please?’ ‘ensure that as many stages and needs are covered as possible and that communication remains open until a conclusion has been reached’ ‘Felt stretched as a programme and this was frustrating. I wonder whether we were actually the right sort of

rganisation to be accepted into the programme - the

selection process was fairly brief as I recall’

SLIDE 20

12% 12% 6% 12% 12% 13% 6% 24% 59% 65% 53% 53% 29% 25% 29% 35% 0% 10% 20% 30% 40% 50% 60% 70% Overall as a resource Ease of finding information

n resources

and events Enabling users to connect and share resources Promoting projects and celebrating success Facilitating

nline

discussions Offering and securing pro bono advice from professionals Searching for volunteers and potential trustees A source of news

Excellent Good Average Poor Very Poor Don't Know Not used