Example of an Evaluation Tool: Ex-Post Cost-Effectiveness Analyses - - PowerPoint PPT Presentation

▶
example of an evaluation tool ex post cost effectiveness
SMART_READER_LITE
LIVE PREVIEW

Example of an Evaluation Tool: Ex-Post Cost-Effectiveness Analyses - - PowerPoint PPT Presentation

Sep 21, 2023 255 likes •448 views

ecologic.de Example of an Evaluation Tool: Ex-Post Cost-Effectiveness Analyses of Environmental Policies EEA expert workshop on ex-post policy effectiveness evaluations Copenhagen, 6 / 7 November 2006 Benjamin Grlach, Ecologic (with

slide-1
SLIDE 1

ecologic.de EEA expert workshop on ex-post policy effectiveness evaluations - Copenhagen, 6 / 7 Nov 06

Example of an Evaluation Tool: Ex-Post Cost-Effectiveness Analyses

  • f Environmental Policies

EEA expert workshop on ex-post policy effectiveness evaluations Copenhagen, 6 / 7 November 2006

Benjamin Görlach, Ecologic (with contributions by eftec and IVM)

slide-2
SLIDE 2

ecologic.de Datum ecologic.de EEA expert workshop on ex-post policy effectiveness evaluations - Copenhagen, 6 / 7 Nov 06 2

Contents

  • Background on the original research
  • What is cost-effectiveness, why is it relevant?
  • Ex-post cost-effectiveness requirements in

EU environmental legislation

  • Example: Directive on Renewable Energy
  • Existing guidelines and manuals for

assessing cost-effectiveness

  • Some lessons from the practice
  • Conclusions and way forward
slide-3
SLIDE 3

ecologic.de Datum ecologic.de EEA expert workshop on ex-post policy effectiveness evaluations - Copenhagen, 6 / 7 Nov 06 3

Outline of the project

  • Carried out for the EEA by Ecologic, eftec &

IVM, September 04 - April 05

  • Task: Survey of ex-post cost-effectiveness

analysis of European environmental policies

  • Legal requirements in EC law
  • Guidelines and manuals for ex-post CEA
  • Examples of applied ex-post CEA
  • Final reports and literature database at:

http://www.ecologic.de/projekte/3ea/panacea/

slide-4
SLIDE 4

ecologic.de Datum ecologic.de EEA expert workshop on ex-post policy effectiveness evaluations - Copenhagen, 6 / 7 Nov 06 4

Cost-effectiveness analysis explained

  • A CEA seeks to identify the activity, process
  • r intervention that minimises resource use

to achieve a given, desired result.

  • The cost-effectiveness of a policy option is

calculated as annualised costs divided by a quantified measure of the physical effect.

  • Interpretation of effectiveness depends on

policy target - can be pressures or impacts

  • Cost concepts: narrower or broader
slide-5
SLIDE 5

ecologic.de Datum ecologic.de EEA expert workshop on ex-post policy effectiveness evaluations - Copenhagen, 6 / 7 Nov 06 5

Cost-effectiveness analysis explained

  • Cost-effectiveness can be assessed both ex-

ante and ex-post

  • CEA is distinct from CBA:
  • CEA = achieve a given objective at least cost
  • CEA does not ask whether objective is worth

achieving (as a CBA would)

  • Drawback: CEA most applicable for single-

dimension objectives

  • With multiple side-effects / co-benefits /

spillovers, CEA is less suited

slide-6
SLIDE 6

ecologic.de Datum ecologic.de EEA expert workshop on ex-post policy effectiveness evaluations - Copenhagen, 6 / 7 Nov 06 6

C-E requirements in EU legislation

18 legislative items, 4 types of requirements:

  • Non-environmental legislation (with environmental

impact) requiring ex-post CEA: 4 items

  • Environmental legislation requiring ex-post CEA (or

evaluation with cost-effectiveness aspects): 4 items

  • Environmental legislation requiring ex-ante CEA (or

evaluation with cost-effectiveness aspects): 6 items

  • Environmental legislation requiring ex-post evaluation,

but not (necessarily) cost-effectiveness: 4 items

slide-7
SLIDE 7

ecologic.de Datum ecologic.de EEA expert workshop on ex-post policy effectiveness evaluations - Copenhagen, 6 / 7 Nov 06 7

C-E requirements in EU legislation II

Ex-post evaluation required Environmental legislation Cost-effectiveness considerations required

4 4

(4)*

6

*only most relevant for the environment

slide-8
SLIDE 8

ecologic.de Datum ecologic.de EEA expert workshop on ex-post policy effectiveness evaluations - Copenhagen, 6 / 7 Nov 06 8

Example: Evaluation of RES Directive

  • Article 4 of the Directive 2001/77/EC on

Electricity from Renewable Energy Sources:

“not later than 27 October 2005, the Commission shall present a well documented report ... [that] shall assess the success, including cost effectiveness, of the support systems [for] ... electricity from renewable energy sources [established in the Member States].”

  • Commission Communication & Impact

Assessment published in Dec 2006

  • Annex III: ex-post CEA for 1998 - 2004
slide-9
SLIDE 9

ecologic.de Datum ecologic.de EEA expert workshop on ex-post policy effectiveness evaluations - Copenhagen, 6 / 7 Nov 06 9

Evaluation of the RES Directive (II)

slide-10
SLIDE 10

ecologic.de Datum ecologic.de EEA expert workshop on ex-post policy effectiveness evaluations - Copenhagen, 6 / 7 Nov 06 10

Evaluation of the RES Directive (III)

slide-11
SLIDE 11

ecologic.de Datum ecologic.de EEA expert workshop on ex-post policy effectiveness evaluations - Copenhagen, 6 / 7 Nov 06 11

Evaluation of the RES Directive (IV)

  • Information presented in an accessible way
  • Cost ranges for generation cost and support

levels (= incentive for producers, cost for consumers)

  • Comparison of costs & effectiveness across

countries and across support instruments

  • No C/E ratios calculated, but graphical

comparison of cost & effectiveness

  • Verbal explanations, e.g. co-benefits
slide-12
SLIDE 12

ecologic.de Datum ecologic.de EEA expert workshop on ex-post policy effectiveness evaluations - Copenhagen, 6 / 7 Nov 06 12

Existing guidelines and manuals

  • Academic literature tends to focus on CBA:

CEA is mentioned in passing, little attention

  • Some tailored guidance on CEA for specific

policies, e.g. WFD - but: all ex-ante

  • Some general guidance on policy evaluation,

not necessarily focused on economic tools

  • Commission: DG Budget, DG Regions, SecGen

guidance on Impact Assessment

  • UK: Green Book
  • NL: Wegwijzer Evaluatieonderzoek
slide-13
SLIDE 13

ecologic.de Datum ecologic.de EEA expert workshop on ex-post policy effectiveness evaluations - Copenhagen, 6 / 7 Nov 06 13

Existing guidelines and manuals

  • Many useful elements in different

documents, together provide good insights

  • n ex-post CEA - but none that has it all.
  • General focus on ex-ante assessment
  • Work on ex-post evaluation does not address

cost-effectiveness in detail

  • Few documents specific on environmental policy
  • Little emphasis on real-life difficulties
  • Good examples found esp. in NL, UK, US
slide-14
SLIDE 14

ecologic.de Datum ecologic.de EEA expert workshop on ex-post policy effectiveness evaluations - Copenhagen, 6 / 7 Nov 06 14

Case studies of applied ex-post CEA

  • Non-exhaustive selection of (mainly)

European (mainly) ex-post CEA

  • 88 studies surveyed, 17 analysed in-depth
  • e.g. biodiversity management in Scotland
  • US evidence on CFC phase-out
  • CO2-based taxes in European countries
  • IMPOL studies on Large Combustion Plant Dir.,

Municipal Waste Incineration Dir., EMAS

  • Dutch Manure and Fertiliser Policy
slide-15
SLIDE 15

ecologic.de Datum ecologic.de EEA expert workshop on ex-post policy effectiveness evaluations - Copenhagen, 6 / 7 Nov 06 15

Lessons learnt from Case Studies

  • Diversity of topics covered
  • Diversity of methods and shortcuts applied:
  • Few studies have applied discounting
  • Importance & difficulty of defining a baseline
  • Sensitivity testing only in some cases
  • Marginal abatement cost taken as proxy for

compliance cost

  • Effects on national economy blended out
  • Results presented in qualitative form
slide-16
SLIDE 16

ecologic.de Datum ecologic.de EEA expert workshop on ex-post policy effectiveness evaluations - Copenhagen, 6 / 7 Nov 06 16

Summary CEA in environmental policy

  • Ex-post evaluation is not a new thing - but

assessment of the cost-effectiveness of environmental policy is fairly novel in EU

  • Some experiences in UK, NL
  • Little on EU level so far
  • Few cases where ex-post CEA is an

institutionalised part of the analysis

slide-17
SLIDE 17

ecologic.de Datum ecologic.de EEA expert workshop on ex-post policy effectiveness evaluations - Copenhagen, 6 / 7 Nov 06 17

Challenges and issues in ex-post CEA

  • Causality between policy measures and
  • bserved effects / blend out other influences
  • Data gathering - much more difficult ex-post

unless monitoring existed up front

  • Moving up to higher (spatial/temporal) scale
  • f analysis: dealing with greater uncertainty
  • Other issues:
  • Which cost types to consider (private / economic)
  • Effectiveness as intermediate or final goals

(pressures / impacts)

slide-18
SLIDE 18

ecologic.de Datum ecologic.de EEA expert workshop on ex-post policy effectiveness evaluations - Copenhagen, 6 / 7 Nov 06 18

Conclusions

  • Ex-post CEAs differ in scope, level of detail

and methodological rigour

  • No single “common approach” to ex-post

CEA across countries or policy issues

  • Ex-post evaluation requires clear targets

and good data - both difficult to find ex-post

  • Guidance required for “real-life solutions”:

methodological shortcuts to deal with time pressure, political desires, data gaps etc.

slide-19
SLIDE 19

ecologic.de EEA expert workshop on ex-post policy effectiveness evaluations - Copenhagen, 6 / 7 Nov 06

Thank you for your attention.

Benjamin Görlach

Ecologic, Pfalzburger Str. 43-44, D-10717 Berlin ! ! ! ! +49-30-86880-0, " " " " +49-30-86880-100

goerlach@ecologic.de, www.ecologic.de