Residential Evaluation Strategic Planning Feb 6, 2013 Agenda - - PowerPoint PPT Presentation

Feb 6, 2013

Residential Evaluation Strategic Planning

 Introduction to Objectives  Review of Evaluation work

• “What We’ve Done” Introduction of Researchable Questions

 Breakout Groups

• Brainstorming Sessions

 Large Group Discussion

• Prioritizing Ideas

 Next Steps and Voting

Agenda

2

Meeting Rules

3

Active participation is encouraged.

Please speak freely and bring any and all ideas to the table.

Raise Hand to speak in group

discussions.

No sidebar conversations. Silence cell phones. No interrupting others.

Meeting Objectives

Developing a Landscape

Flow of the day

 What We’ve Done

• An Introduction to work done in the research area, followed by

introduction to unanswered questions

 Breakout Groups are a brainstorming session

• Build off of Contractor’s presentation of unanswered questions
• Refine questions provided and develop other questions
• Speed Dating Round
• 30 Minutes with the Primary Group, 20 Minutes rotating

 Large Group Discussion

• Discuss final list and assess importance to researchable

questions.

Residential

Riley Hasting (Eversource Energy)

Savings by Sector

Residential 23% Low-Income 3% Commercial & Industrial 74%

Lifetime Electric Savings (MWh)

Residential 38% Low-Income 5% Commercial & Industrial 57%

Electric Benefits

Residential 50% Low-Income 12% Commercial & Industrial 38%

Lifetime Gas Savings (Therms)

Residential 53% Low-Income 17% Commercial & Industrial 30%

Gas Benefits

Electric by Initiative

RNC, 2% MF, 5% HES, 13% Behavior, 20% LI SF, 3% LI MF, 4% CS, 3% Lighting, 46% Products, 4%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Annual Electric Savings (MWh)

RNC, 3% MF, 6% HES, 14% Behavior, 3% LI SF, 4% LI MF, 5% CS, 5% Lighting, 54% Products, 4%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Lifetime Electric Savings (MWh)

RNC, 4% MF, 3% HES, 52% Behavior, 1% LI SF, 8% LI MF, 3% CS, 2% Lighting, 24% Products, 2%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Electric Benefits

Lighting/ Products CoolSmart Whole House Whole House CoolSmart Lighting/ Products Whole House CoolSmart Lighting/ Products

Gas by Initiative

LI SF, 8% LI MF, 12% Behavior, 3% RNC, 11% MF, 4% Home Energy Services, 34% Heating & Water Heating, 28%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Lifetime Gas Savings (Therms)

HEHE Whole House LI SF, 2% LI MF, 4% Behavior, 33% RNC, 3% MF, 2% Home Energy Services, 23% Heating & Water Heating, 21%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Annual Gas Savings (Therms)

Whole House HEHE LI SF, 10% LI MF, 14% Behavior, 1% RNC, 11% MF, 6% Home Energy Services, 35% Heating & Water Heating, 21%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Gas Benefits

HEHE Whole House

Residential Organization

 3 Evaluation Research Areas: Residential, Commercial & Industrial, Special &

Cross-Sector Studies

 Residential research areas will be divided into three categories today:

11

Residential Studies: This research area was divided by program delivery method. Whole House typically has a direct install delivery strategy. Products is generally an upstream delivery strategy. Heating and Cooling is typically is delivered through influencing contractor and distributor prevailing practices. We recognize that there can be overlap between all three areas. Residential Research

Whole House

• HES, New Con,

Multifamily (Cadmus)

Products

• Lighting and

Appliances (NMR)

Heating and Cooling

• CoolSmart and

HEHE (Navigant)

Massachusetts Residential Evaluation Planning Summit

February 6, 2015

Three Residential Elements

• 1. Whole House/Direct Install/New Construction

– Home Energy Services – Low Income (Single Family and Multifamily) – Multifamily – New Construction

2/24/2015 13

Three Residential Elements

• 2. HEHE and Cool Smart

– Boilers – Furnaces – Ductless Mini-split HPs – CAC/HPs

2/24/2015 14

Three Residential Elements

• 3. Lighting and Products

– Market Characterization – Saturation Trends – EISA Impact – Hard-to-Reach – Appliances and Other Products

2/24/2015 15

Home Energy Services

Research to Date

2/24/2015 17

Study Completion Date Key Data Collection Elements Program Delivery and HEAT Loan Assessment April 2015* Interviews with participants and contractor Realization Rate Assessment June 2013 Billing and engineering analysis, simulation modeling Pre-Weatherization Initiative April 2013 Interviews with participants and stakeholders; review of tracking data Packaged Measure Initiative June 2012 Review of tracking data Net-to-Gross June 2012 Interviews with participants and contractor, discrete choice modeling Impact Evaluation August 2012 Billing and engineering analysis, simulation modeling *In progress.

What did we learn?

• LV and HPCs overestimate per-measure gross savings.
• Cross-participation rates between HES, Cool Smart, and HEHE

were lower than anticipated.

• Participants install measures that capture, on average, 44% of

identified audit savings.

• HPCs trail behind LVs with respect to HEAT loan and cross-

participation and installing HES heating/water heating systems.

• The program causes insulation contractors to improve practices

at nonparticipating sites (spillover).

2/24/2015 18

Remaining Research Questions

• Can program design changes encourage deeper savings,

greater measure adoption, and cross-participation?

• Does the current realization rate adjustment to ex ante savings

remain appropriate?

• Does similar insulation SO occur as part of the MF program?
• Are the current NTG assumptions still appropriate?
• What opportunities exist for home automation savings?
• What about the current DI CFL NTG (73%) post-EISA?

2/24/2015 19

Low Income (SF and MF)

Research to Date

2/24/2015 21

Study Completion Date Key Data Collection Elements LIMF Impact Analysis June 2014* Billing analysis, metering common area lighting, and algorithm review Secondary Heating Assessment* June 2014* Billing analysis Lighting HOU and Secondary Heating May 2014 Metering Impact Evaluation August 2012 Billing and engineering analysis, simulation modeling Process Evaluation August 2012 Interviews with agencies, PAs, field staff, and participants Measure Assessment December 2011 Engineering analysis *In progress.

What did we learn?

• Significant weatherization savings (29% of SF heating usage);

greater than HES (263 vs. 149 therms).

• Opportunities exist for standardizing savings estimates across

CAPs and PAs as well as the QA/QC process.

• LI customers use their lights similarly to non-LI customers.
• Weatherization likely reduces LI customers’ need to use

supplementary electric space heaters.

• Minimizing the points of contact (for MF) may increase

participation.

• Completing a billing analysis requires information not

captured electronically.

2/24/2015 22

Remaining Research Questions

• Have measure savings estimates been appropriately

standardized?

• Does the program maximize lighting savings?
• Which LIMF measures require MF-specific savings or

assumptions?

2/24/2015 23

Multifamily

Research to Date

25

Study Completion Date Key Data Collection Elements

Multifamily Impact Evaluation June 2015* Billing analysis Multifamily Process Evaluation February 2015* Interviews with stakeholders and contractors; customer focus groups, in-depth interviews and phone surveys; literature review; on-site visits; and data review Multifamily Program Impact Analysis July 2012 Data review; savings algorithm review Multifamily Process Evaluation June 2012 Stakeholder interviews; literature review; participant and nonparticipant focus groups and phone surveys; and data review Multifamily Market Characterization & Potential Study May 2012 Stakeholder workshops and on-sites visits Residential New Construction 4-8 Story Multifamily Pilot April 2012 Stakeholder and participant interviews *In progress.

What did we learn?

• A single point of contact would improve the customer

participation process.

• Program tracking is insufficient for “linking” multifamily

properties across fuels and programs (e.g., C&I and Residential).

• The current audit process leaves room for missed and

lost opportunities.

2/24/2015 26

Remaining Research Questions

• Beyond expanding the MMI, what options are available for

creating a single point of contact?

• Is it feasible to create a link between C&I and Residential

multifamily customer data?

2/24/2015 27

CoolSmart and HEHE

Research to Date

2/24/2015 29

Study Completion Date Key Data Collection Elements Ductless Heat Pump Metering December 2015* Collected to date: Additional baseline information from participant interviews HEHE Impact Evaluation February 2015* High efficiency boiler/furnace metering, spot measurements of baseline equipment, billing data disaggregation Furnace Baseline Assessment December 2014 Review of market data, previous MA research Ductless Heat Pump Survey September 2014 Participant understanding of DMSHP installation, motivations, and perceptions of program involvement NTG Study June 2013 Surveys with participants, contractors, distributors *In progress

What did we learn (about furnaces)?

• Participant homes with furnaces consume less gas than other

homes–drives lower furnace savings than TRM.

• Standard efficiency furnaces perform slightly higher than

ratings.

• High-efficiency furnaces perform at their ratings.
• Program baseline is moving to 85% AFUE.
• Net-to-gross (for AFUE >= 95%) is 81%.

2/24/2015 30

What did we learn (about boilers)?

• High-efficiency boilers perform worse than their ratings; they

are not being controlled to maximize potential savings– an

• pportunity for higher savings.
• Standard efficiency boilers also perform worse than their

ratings due to stack losses–drives higher savings than TRM.

• Participant homes with combination boilers (combined

heating and small hot water tank) consume less gas than

• ther homes–drives lower savings than TRM.
• Most (~75%) participating standalone boilers were attached

to indirect hot water tanks–drives higher savings than TRM.

• Net-to-gross is 77%.

2/24/2015 31

Remaining Research Questions (Gas Heating Equipment)

• How can contractor installation practices be improved to capture

lost condensing boiler opportunities?

• How can programs influence people to change from non-condensing to

condensing equipment before the end of the useful life, and what’s the appropriate baseline for that sort of program design?

• Can smart thermostats improve the performance of condensing gas

boilers?

• What market effects are associated with the program’s market

intervention?

• What is the appropriate NTG to account for FR, SO, and other market

effects?

• What are the true incremental costs of equipment?
• What is the true lifetime of this equipment?

32

What did we learn about Ductless Heat Pumps?

• In 2013, DMSHPs primarily were installed in retrofit projects, either

to displace existing heating equipment or to provide a new cooling

• amenity. In 2014, units are going into WMECO’s new construction.
• DMSHPs are more often installed for cooling than heating.
• DMSHPs are not replacing existing heating systems, only

supplementing and offering an improved level of comfort.

• Participants who installed “cold climate” DMSHPs are generally

more positive about their heating performance.

• DMSHP usage is not always rational (i.e., some people should

displace oil heating and are not).

2/24/2015 33

Remaining Research Questions – Electric HVAC equipment

2/24/2015 34

• Do we understand DMSHPs?

– How are they being used? – What are they replacing? – What are the savings?

• Do contractors really understand what they are selling? Are they

correctly sized?

• What is the interaction of DMSHPs with other heating systems?
• Can smart thermostats be used to both optimize DMSHP usage

while increasing or maintaining occupant comfort?

• Similar to gas equipment, what are the following:

– Market effects that need to be considered in NTG? – True incremental costs of electric HVAC equipment? – True lifetimes of electric HVAC equipment?

Lighting Market Characterization

This slide contains Private and Proprietary Data of The Cadmus Group, Inc.

Research to Date

Study Completion Date Key Data Collection Elements Market Assessment February 2015* Consumer surveys, on-site visits, supplier interviews, shelf-stocking, market-adoption model Multistage Net-to-Gross March 2015** Supplier interviews, demand elasticity modeling, point-of-sale data analysis, on-site comparison Saturation Stagnation February 2015 On-site visits, point-of-sale data analysis, CA program manager interviews, literature review *Prior studies dating back to late 1990s. **Regular check-ins since mid 2000s.

2/24/2015 36

What did we learn?

• Self-reported purchases of CFLs and LEDs increased.
• LED market share small but increasing in MA, elsewhere.

– Most suppliers say the market is good or excellent.

• CFL market share stable in MA, decreasing elsewhere.
• Halogen market share stable in MA, increasing nationally.

– CA data suggest jump in 2013 due to full implementation of EISA; some programs dropping CFL incentives.

• Availability, diversity of LEDs on store shelves increasing.
• LED prices falling, but price still seen as a barrier.
• CFL prices increase in the absence of incentives.
• Consumers who use LEDs also use CFLs.

2/24/2015 37

Remaining Research Questions

• What will be the trends in prices, availability, market share,

use, and purchases of CFLs, LEDs, halogens?

• What are the characteristics of LED users?
• Does this change as more households adopt and how?
• What are the implications for program design?
• What role should lighting play in Residential Portfolio in the

near future?

• EISA, but perhaps also nearing transformation of standard CFLs?
• Will LEDs achieve ample, cost-effective savings? (NEEP thinks so.)

2/24/2015 38

Lighting Saturation Trends

This slide contains Private and Proprietary Data of The Cadmus Group, Inc.

Research to Date

Study Completion Date Key Data Collection Elements

Market Assessment February 2015* On-site visits in MA, GA, KS; NY study in progress; MA on-site panel visits Saturation Stagnation February 2015 On-site visits, point-of-sale data analysis, CA program manager interviews, literature review

*Prior studies dating back to late 1990s.

2/24/2015 40

What did we learn?

• Most households in MA use CFLs; over 1/5 use screw-in LEDs.
• CFL saturation up five percentage points from 2013 to 2014.

– Now 33%; had crept from 26% in 2009 to 28% in 2014.

• LED saturation at 3%, but rising steadily over time.
• Saturation not much higher in MA vs. KS but higher than GA.
• CFLs most common bulb type used to replace removed bulbs.

– Includes incandescents, CFLs, LEDs (but only five LEDs in sample).

• LEDs, incandescents equally likely to replace CFLs not replaced

by other CFLs.

2/24/2015 41

Remaining Research Questions

• Will CFL saturation continue to increase?
• How rapidly will LED penetration and saturation increase?
• What drives consumer decision making regarding bulb use in

specific applications?

• What are the continued trends in bulb replacement?
• EISA big player
• Types of bulbs
• Lumen ranges
• Wattage/Delta Watts
• How have saturation trends impacted energy use from residential

lighting relative to total residential energy usage?

2/24/2015 42

Impact of EISA

This slide contains Private and Proprietary Data of The Cadmus Group, Inc.

Research to Date

Study Completion Date Key Data Collection Elements Market Assessment February 2015* Consumer surveys, on-site visits, supplier interviews, shelf-stocking, market-adoption model Multistage Net-to-Gross March 2015** Supplier interviews, demand elasticity modeling, point-of-sale data analysis, on-site comparison Saturation Stagnation February 2015 On-site visits, point-of-sale data analysis, CA program manager interviews, literature review *Prior studies dating back to late 1990s. **Regular check-ins since mid 2000s.

2/24/2015 44

What did we learn?

• Suppliers strongly argue to maintain standard CFL incentives.

– Believe program incentives limit EISA-related backsliding. – Cite increased halogen sales and market share in CA in 2013. – Supported by POS data analysis for CA.

• Purchasing 2014/2015 data would allow further exploration.
• Increase in Fall 2013 in shelf space devoted to incandescents.

– Both participating and formerly participating retailers.

• Some consumers stockpile incandescents, not widespread.

2/24/2015 45

Remaining Research Questions

• How long should MA continue to support standard CFLs?
• Balance possible backsliding with cost-effectiveness, decreasing

delta watts/measure life (EISA 2020).

• How did CFL, LED, and halogen market share fare?
• Compared over time.
• Compared to CA and NY (natural experiment). What will be the

impact of EISA on socket saturation?

• Does EISA lead to changes in bulb replacement strategies?

2/24/2015 46

Hard-to-Reach Customers

This slide contains Private and Proprietary Data of The Cadmus Group, Inc.

Research to Date

Study Completion Date Key Data Collection Elements Market Assessment February 2015* Consumer surveys, on-site visits, supplier interviews Saturation Stagnation February 2015 On-site visits, point-of-sale data analysis, CA program manager interviews

2/24/2015 48

*Prior studies dating back to late 1990s.

What did we learn?

• Saturation rates among low-income households higher than
• ther households.

– Smaller homes = fewer sockets so easier to achieve high saturation. – Direct installation plays a role, but not the whole story.

• Suppliers generally agree with PA approach of focusing on

particular retail channels and customer base to reach HTR.

– Some argue HTR shop where non-HTR households shop (e.g., home improvement, mass merchandise). – On-site studies confirm that home improvement, mass merchandise the most common places low-income buy bulbs. – BUT low-income consumers do cite discount stores more frequently.

2/24/2015 49

Remaining Research Questions

• What are the lighting purchases, use habits, and saturations of

linguistic minorities, recent immigrants, and people with physical or developmental disabilities?

• How effective are current retail channel and store-customer-base

approaches to increasing efficient bulb use among HTR customers?

• Is a retail-based program an effective strategy for increasing

saturation among the most HTR, or should the PAs instead rely on direct-install programs?

• Which community partners could assist the PAs in identifying the

best ways to increase saturation among HTR customers?

2/24/2015 50

Appliances and Other Products

This slide contains Private and Proprietary Data of The Cadmus Group, Inc.

Research to Date

Study Completion Date Key Data Collection Elements Television Saturation April 2015 On-site visits to collect info on televisions, with focus on tube TVs; number, location, frequency of use

2/24/2015 52

Remaining Research Questions

• Which current products merit continued program support/

intervention?

• Should the PAs consider adding new technologies? If so,

what?

• What kind of support/intervention would work best for each

product or product type?

– Upstream? Rebates? Direct installation?

• What research is required to answer these questions?

2/24/2015 53

New Construction

Research to Date

2/24/2015 55

Study Completion Date Key Data Collection Elements Multifamily New Construction Baseline In Process On-site assessments Single-family New Construction Baseline and Code Compliance Study In Stage 3 Planning On-site assessments Compliance Assessment of Homes Built at the End of IECC 2006 and the Beginning of IECC 2009 (Conducted under Cross-Cutting) September 2014 Reanalysis of baseline

• n-site data

What did we learn?

• Difficulty of recruiting nonparticipating multifamily projects.
• Overall compliance for homes built at the end of 2006 IECC:

76% (overlap with C&S).

• Overall compliance for homes built at the beginning of 2009

IECC: 63% (overlap with C&S).

• PNNL checklist does not adequately account for energy

efficiency in its code compliance estimates (overlap with C&S).

2/24/2015 56

Remaining Research Questions

• Are our current baseline assumptions appropriate and

complete?

• What other MF measures should be considered by the

program?

• How can savings from the NC program be differentiated from

savings stemming from CCSI?

• How will compliance rates changes over time?
• How are heat pumps and DMSHPs incorporated into new

construction?

• How does the efficiency of stretch code homes compare to

that of non-stretch code homes?

2/24/2015 57

Other

Remaining Research Questions

• What drives the disparity between the Cross-Cutting Team’s

top-down assessment of residential savings and the program- level, bottom-up estimate?

• What have we learned from the Residential Profile Study and

how can we use the results to enhance programs?

• Do opportunities exist to improve demand reduction

estimates through targeted updates to the demand impact model?

2/24/2015 59

Breakout Session Intro

We want to hear from you…

 What questions need to be asked and answered?  What new methods development work is needed in

this research area?

 What are missing researchable questions?  Participation is key!

Residential Red Rover

Group 1: Whole House

• Facilitator:

Doug B (Cadmus)

Group 2: CoolSmart and HEHE

• Facilitator:

Justin S (Navigant)

Group 3: Lighting and Products

• Facilitator: Lisa

W-W (NMR)

62