Part 1 Part 1 I ntroduction Review of I ntroduction Review of I - PowerPoint PPT Presentation

Part 1 Part 1 I ntroduction Review of I ntroduction Review of I ntroduction, Review of I ntroduction, Review of Agenda, Agenda Revisions Agenda, Agenda Revisions 2

Part 2 Part 2 Review of ACT 129 and Other Review of ACT 129 and Other Review of ACT 129 and Other Review of ACT 129 and Other Historical Documents Historical Documents 3

Part 3 Part 3 Part 3 Part 3 Review of Objectives for the Review of Objectives for the Statewide Evaluation and Statewide Evaluation and Contents of the Audit Plan Contents of the Audit Plan Contents of the Audit Plan Contents of the Audit Plan 4

Act 129 Mandate Act 129 Mandate Act 129 Mandate Act 129 Mandate The Energy Efficiency and Conservation Program shall include: The Energy Efficiency and Conservation Program shall include: (2) An evaluation process, including a process to monitor (2) An evaluation process, including a process to monitor and verify data collection, quality assurance and and verify data collection, quality assurance and and verify data collection quality assurance and and verify data collection quality assurance and results of each plan and the program. results of each plan and the program. (3) An analysis of the cost and benefit of each plan (3) An analysis of the cost and benefit of each plan submitted under subsection (B) in accordance with a submitted under subsection (B) in accordance with a total resource cost test approved by the Commission total resource cost test approved by the Commission total resource cost test approved by the Commission. total resource cost test approved by the Commission. Source: House Bill No. 2200, page 51 5

Objectives of Statewide Objectives of Statewide Evaluation Evaluation l l i i • • Identify the cost-effectiveness of each plan in accordance with Identify the cost-effectiveness of each plan in accordance with the Commission adopted Total Resource Cost Test Manual. the Commission adopted Total Resource Cost Test Manual. • • Provide reasonable assurance that the claimed measures are Provide reasonable assurance that the claimed measures are being properly installed and utilized. being properly installed and utilized. • • Provide reasonable assurance that the installed measures are Provide reasonable assurance that the installed measures are obtaining the claimed energy savings or reductions in accordance obtaining the claimed energy savings or reductions in accordance with the Commission adopted Technical Reference Manual or with the Commission adopted Technical Reference Manual or metered savings. metered savings. • • Identify areas for improvement. Identify areas for improvement. • • Identify best practices that may be implemented on a Identify best practices that may be implemented on a y y p p y y p p going-forward basis. going-forward basis. 6

Requirements for Contents of the Requirements for Contents of the A dit Pl A dit Pl Audit Plan Audit Plan 1. 1. High-level summary of audit plan. High-level summary of audit plan. 2 2 2. 2. EE&C measures to be audited and the plan measure logic/theory. EE&C measures to be audited and the plan measure logic/theory. EE&C measures to be audited and the plan measure logic/theory EE&C measures to be audited and the plan measure logic/theory 3. 3. The type and format of documentation identifying EDC plan expenditures. The type and format of documentation identifying EDC plan expenditures. 4. 4. A description of the audit objectives, audit questions and audit rigor level for each EDC A description of the audit objectives, audit questions and audit rigor level for each EDC plan. plan. 5. 5. 5 5 A description of the methodologies procedures and data tracking systems to be used by A description of the methodologies, procedures and data tracking systems to be used by A description of the methodologies procedures and data tracking systems to be used by A description of the methodologies, procedures and data tracking systems to be used by the Contractor to verify the impact evaluations and project verifications for each plan, the Contractor to verify the impact evaluations and project verifications for each plan, including data gathering, sampling and analysis methods. including data gathering, sampling and analysis methods. 6. 6. Audit approaches to be used for each plan in accordance with the TRM or other approved Audit approaches to be used for each plan in accordance with the TRM or other approved EM&V plans EM&V plans. EM&V plans EM&V plans. 7. 7. Audit approaches to be used for each plan in accordance with approved custom measures. Audit approaches to be used for each plan in accordance with approved custom measures. 8. 8. Strategies to conduct audit studies coordinated on a statewide basis, such that results can Strategies to conduct audit studies coordinated on a statewide basis, such that results can be extrapolated back to individual EDC EE programs. be extrapolated back to individual EDC EE programs. 9 9. 9 9. Verification and due diligence procedures for site inspections and validating information Verification and due diligence procedures for site inspections and validating information Verification and due diligence procedures for site inspections and validating information Verification and due diligence procedures for site inspections and validating information provided by EDCs. provided by EDCs. 10. Data and information needed from EDCs and other sources. 10. Data and information needed from EDCs and other sources. 11. Budget, schedule and manpower plan for Contractor. 11. Budget, schedule and manpower plan for Contractor. Source: PA PUC RFP for Statewide Evaluator , page 31 7

Value of I nformation (VOI ) Value of I nformation (VOI ) • • What is the value of What is the value of information and data? information and data? • • What is the cost? What is it What is the cost? What is it worth? What is the return? worth? What is the return? • • VOI balance cost and rigor VOI balance cost and rigor • • Deterministic sampling Deterministic sampling • • Reduction of uncertainty Reduction of uncertainty • • Adjust sample size according to Adjust sample size according to the cost of changes compared to the cost of changes compared to the cost of changes compared to the cost of changes compared to changes in value changes in value • • Sampling plans guided by Sampling plans guided by e pectat o s e pectat o s expectations expectations • • Audits structured to test the Audits structured to test the validity of reported results – validity of reported results – reasonable scientific checks reasonable scientific checks 8

Value of I nformation (VOI ) Value of I nformation (VOI ) ( ( ) ) • • Actual reported impacts may Actual reported impacts may vary by program and EDC – vary by program and EDC – VOI considerations employed VOI considerations employed when evaluation efforts may when evaluation efforts may be large compared to risk, be large compared to risk, b b l l d d i k i k funding and impacts funding and impacts • • Frequent stratification Frequent stratification Frequent stratification Frequent stratification scheme – weight the sample scheme – weight the sample towards significant attributes towards significant attributes • • Greater focus on impacts Greater focus on impacts with greater uncertainty – with greater uncertainty – sensitivity analysis sensitivity analysis 9

Timetable for Development of Timetable for Development of A dit Pl A dit Pl Audit Plan Audit Plan • • Draft – November 1, 2009 Draft – November 1, 2009 , , • • Final – December 2, 2009 Final – December 2, 2009 10

Table of Contents of Audit Plan Table of Contents of Audit Plan Table of Contents of Audit Plan Table of Contents of Audit Plan Section 1 Section 1 Introduction and Purpose of the Audit Plan Introduction and Purpose of the Audit Plan Section 2 Section 2 Requirements for EDC Evaluation Requirements for EDC Evaluation Section 3 Section 3 Audit Plan Audit Plan Section 4 Section 4 EDC Reporting Guidelines EDC Reporting Guidelines Section 5 Section 5 EDC Data Requirements EDC Data Requirements Section 6 Section 6 Statewide Evaluator Reporting Activities Statewide Evaluator Reporting Activities Section 7 Section 7 Expected Deadlines Expected Deadlines p 11

Part 4 Part 4 Development of Overall Development of Overall Development of Overall Development of Overall Statewide Evaluation Plan Statewide Evaluation Plan 12

Topics to be Discussed Topics to be Discussed Topics to be Discussed Topics to be Discussed • Procedures for measurement and verification • Procedures for measurement and verification • Procedures for measurement and verification • Procedures for measurement and verification • Procedures for impact evaluation • Procedures for impact evaluation • Procedures for process evaluation • Procedures for process evaluation • Data reporting requirements • Data reporting requirements 13