Towards a Benchmarking Raster: A selection of indicators to measure - - PowerPoint PPT Presentation

Towards a “Benchmarking Raster”: A selection of indicators to measure policies for culture and creative industries”

CREA.RE – Mid-term conference Göteborg, 20-21 March 2011

Aims of the assignment

• Create and test a “benchmarking raster” which will provide

local policy-makers with a set of indicators to measure support policies.

• Assist policy makers in setting up the ideal “creative ecology”

in non-large urban centers.

• Stimulate investment of Structural Funds into culture and

creative industries (CCIs).

Commissioned by the CREA.RE network

Common features Differences Partners represent small and medium-sized urban and rural areas. Partners represent a mix of local and regional authorities with different levels of expertise when dealing with CCIs. Focus is on economic development through CCIs. The level of political awareness on CCIs varies considerably within the consortium. Leverage EU funding for CCIs. Local “creative ecologies” across the consortium are extremely varied.

Benchmarking focus

• Local economic development (vs. social and cultural

development)

• Usable in non large urban centres and throughout Europe
• Adapted to the specifics of CCIs
• User-friendly tool for policy makers

Methodology

• Select a number of good practices (60-30-15) as practical examples of

support measures and looked at the use of indicators in local contexts (Literature review, peers’ recommendations).

• Classify policy support measures under different categories of support.
• Compile the benchmarking raster in relation to the previously identified

support measures and categories of policy support (Literature review, Questionnaire & interview).

• Test indicators with CREA.RE partners (Collection of qualitative and

quantitative inputs – locally defined methodologies).

• Provide “The 5 key steps to implement economic impact assessment of

local policies for CCIs”

Selected 15 good practices including 1 outside Europe (out

• f 30) based on preliminary assessment

1. Creative Tampere, Tampere (Finland) 2. Creative Factory, Rotterdam (Netherlands) 3. Zollverein World Heritage Site (Germany) 4. Cornwall and structural funds (UK) 5. Quartier de la Création, Nantes Métropole (France) 6. Aachen Founders’ Centre Culture Industries and Programme Kulturunternehmen! (Germany) 7. Abertay University (UK) 8. Musikpark, Mannheim (Germany) 9. Screen West Midlands (SWM) (UK)

• 10. Film ï Vast, Trollhattan, Västra Götaland

Region, (Sweden)

• 11. Creative Advantage Fund, West-

Midlands (UK)

• 12. Romagna Creative District (RCD) – Italy
• 13. Obidos Criativa (Portugal)
• 14. Tallin Creative Incubator, Tallin (Estonia)
• 15. Create Denver (USA)
• 16. Watershed Media Centre, Bristol (UK)
• 17. Design Connection Brainport, Eindhoven

(The Netherlands)

• 18. Tartu Centre for Creative Industries

(Estonia)

• 19. Krowji (Cornwall Creative Industries) – UK
• 20. South West Screen (SWS) (UK)
• 21. Source - Developing Rural Creativity (EU

project)

• 22. Cultuur & Ruimte Vouchers 2007-2013

(The Netherlands)

• 23. Wallimage, Wallonia (Belgium)
• 24. Creative Credits, Manchester (UK)
• 25. Programme Broedplaatsen, Amsterdam

(The Netherlands)

• 26. Microwave (UK)
• 27. Creative Berlin (Germany)
• 28. Design Forum Vorarlberg/IMPULSE

(Austria)

• 29. Creative Graz (Austria)
• 30. Puglia Creativa (Italy)

Classification of policy support measures (I)

The following classification is used to categorise support measures (SMs) and identify related indicators: 1.Infrastructure and clustering/networking 2.People, competences and entrepreneurship 3.Incubation – transversal to categories 1 and 2 4.Governance of CCIs’ policies – transversal/upstream

Policy Governance Incubation People, competences and entrepreneurship Infrastructure and clustering/networking Cultural and creative industries

Transversal and upstream category of support

Classification of policy support measures (II)

The benchmarking raster (I)

Governance

• N° of beneficiaries
• Ratio between

accomplished vs. planned deliverables or actions

• Business/management

plan

• Initial mapping of CCIs’

needs

• Stakeholders’

involvement

• Regular evaluations

Categories of support Benchmarking Raster Specific Indicators Common Indicators Key Overall Performance Indicators Foster Infrastructure & Networking People, Competences & Entrepreneurship Incubation (transversal)

Infrastructure Networking & Clustering Competences & Skills Access to Finance

• Institutional infrastructure for CCIs’ policies
• N°of new non‐CCIs policy documents having CCIs as a new priority
• Budget for CCI in economic development and innovation
• Share of public procurement used by CCIs
• N° and quality of

networks/clusters created.

• N° and quality of new B2B

partnerships/contracts developed

• Increase of investments into

CCIs

• Return On Investment for

investors

• N° of businesses able to move

from ‘incubation’ to ‘dis‐ incubation’

• Ratio “Surface available” vs

“Occupation of surface”

• Access to broadband lines/ICT

infrastructures and services

• N° of businesses established.
• N° of new products/services

developed/marketed

• Types of advice services

provided (on ICT, law, management, etc.)

• Increase in the number
• f jobs created in CCIs

in the city/region

• Growth in the turnover

and/or profitability of CCIs in the city/region.

• Growth in the number
• f creative businesses in

the city/region.

• 3 Key Overall Performance indicators
• 16 indicators for SMs (excluding ‘governance’):

 6 common indicators  5 specific indicators for ‘infrastructure and networking’ SMs  4 for ‘people, competences and entrepreneurship’ SMs  1 specific indicator related to the transversal support category for incubation schemes SMs

• 4 specific indicators for governance SMs

The benchmarking raster (II)

Purpose & selection criteria

• Monitor and assess CCIs’ policies and support measures

through selected indicators.

• Responding to 5 evaluation criteria (OECD, European

Commission).

Evaluation questions

• Is the measure relevant according to local CCIs’ needs and

policy priorities?

• Is it well managed?
• Are resources used efficiently?
• Are deliverables contributing to reach expected objectives?
• Are the ‘governance’ measures producing the expected

results?

• Is the measure providing any impact on its beneficiaries and

help to develop the local creative ecology?

• Does it help entrepreneurs develop their businesses in the

long term?

Evaluation criteria

• Relevance: the extent to which the objectives of the support measure

are consistent with beneficiaries’ requirement and local CCIs’ needs.

• Efficiency: how inputs (i.e. resources: funds, expertise, timeframe,

management model, appropriateness of infrastructures; etc.) are converted to results (outputs).

• Effectiveness: the extent to which the initiative’s objectives are

achieved, or are expected to be achieved, taking into account their relative importance.

• Impacts: global effects on the creative ecology.
• Sustainability: whether the benefits produced by the support measure

continue after the latter ends.

Evaluation of support measures under the category ‘Infrastructure and clustering/networking’: a practical example

Evalu atin g su p p o rt measu res u n d er th e su p p o rt categ o ry ‘In frastru ctu re an d clu sterin g /n etwo rkin g ’: a p ractical examp le (co mmo n p lu s sp ecific in d icato rs) Relevan ce Imp act an d Su stain ab ility Efficien cy Effectiven ess  In crease in th e n u mb er o f jo b s created in CCIs in th e city/reg io n  Gro wth in th e tu rn o ver an d /o r p ro fi tab ility o f CCIs in t h e city/reg io n  Gro wth in th e n u mb er o f creative b u sin esses in th e city/reg io n  Ratio b etween ‘Su rface availab le’ (e.g . sq u are meters) an d ‘Occu p atio n o f th e su rface b y b en eficiaries’  N÷an d q u ality

• f co llab o rative n etwo rks/Nu mb er an d typ e s o f clu sters cr e ated
• N÷ an d q

u ality o f n ew B2B p artn ersh ip s/co n tract s d ev elo p ed  N÷o f b u sin esse s e stab lish ed th an ks to th e in itiative  Ratio b etween acco mp lish ed vs. p lan n ed d eliverab les o r actio n s  Nu mb er o f b en eficiaries  Reg u lar evalu atio n s  Bu sin ess/man ag emen t p lan  In itial map p in g o f CCIs’ n eed s  Stakeh o ld ers’ in vo lvemen t  N° an d q u ality o f co lab o rative n etwo rks/Nu mb er an d typ es o f clu ster s cre ated .  Access to b ro ad b an d lin es/ICT in frastru ctu res an d services Evalu atio n criteria

The 5 key steps to implement economic impact assessment of local policies for CCIs”

1. Identify policy and evaluation goals 2. Assess evaluation readiness 3. Develop evaluation methodology and set up a targeted benchmarking raster 4. Implement the benchmarking raster 5. Disseminate the results

• Need to adapt indicators:

 Take into account the state of development of the sector: n.

• f start-ups? Increase % of turnover?

 and the specific objectives of the measures

• Dependence
• n

existing data and data collection tools/methods

• Look at the evolution since the start of the initiative

% Increase or Decrease = e.g. (( [2011 N. of beneficiaries] - [2008

• N. of beneficiaries] ) / [2008 N. of beneficiaries] ) * 100

In practice (I):

In practice (II):

Monitoring and evaluation throughout the life of the support measure:  Early recognition of issues to be changed/improved (Input and

• utput indicators)

 Measure and assess the overall effects of support measures at later stage/after the intervention (Outcome-impact indicators)

The raster will benefit creative ecologies

• Support the design of evidence-based policies
• Position the city/region as a specialised innovative centre
• Brand cities and regions to investors, companies and talents
• Stimulate and justify public and private investment in CCIs

(incl. from the EU Structural Funds: a result based approach is privileged in the EC’s proposal for the 2014-2020 Cohesion Policy).

• Help revisit local governance by supporting synergies

between different policy areas (culture, education, technology, urban planning, and support to SMEs)

• Support the design of Smart Specialisation Strategies (RIS3)

Nantes: a culture-led regeneration

Mons, European Capital of Culture 2015

• Culture investment for local

economic development

• Evaluation impact of the investment.
• Methodology to measure

sustainability of culture investment.

• Partnership with local university

for local capacity buidling.

• Data collection for evidence-based

policy.

Conclusions and next steps

• This raster is a first step towards the definition of

appropriate evaluation of CCIs’ policies at local level.

• Future research should explore:
• the development of indicators covering cultural participation, well being,

social cohesion, territorial attractiveness, tourism and branding.

• the raster’s contribution to the development of smart specialisation

strategies.

• The raster is a flexible instrument – it may be refined

according to local experiences.

Thanks for your attention!

Valentina Montalto 51 Rue du Trône B -1050 Brussels +32 2 289 26 00 vmontalto@keanet.eu www.keanet.eu Join us also on FB and LinkedIn groups: Creative Europe