Modelling sample data from smart-type meter electricity usage Susan - - PowerPoint PPT Presentation

Modelling sample data from smart-type meter electricity usage

Susan Williams NTTS Conference, 10-12 March 2015, Brussels

Introduction

• Big Data Project in the UK’s Office For National

Statistics

• Practical pilot on the potential of smart-type meter

data Pilot objectives:

• To investigate applications of big data sources within

Official Statistics

• To develop capability in methods and processing of

big datasets

Smart-type meter pilot: Background

• Smart meters: electronic devices taking high

frequency energy readings

• Gas and electric smart meters to be rolled out to all

households in UK by 2020 (UK policy)

• Readings (min 30 min frequency) to be wirelessly

transmitted to central body for purposes covered by

• legislation. Various levels of consumer opt-out built

in.

Privacy/Ethics

• High public concern over data security and access

could result in consumer rejection of smart metering

• Theoretically: without suitable protections smart

meter data might be used to identify individual households and real-time occupancy. Tracking behaviours over time may lead to the modelling of detailed profiles.

• Specific smart meter data access and privacy

legislation is in place in Great Britain

• ONS has approached UK privacy groups to gain

support for this research

Research Question

Benefit

• Validate Census returns or optimise Census

follow up Investigate the potential of smart-type meter electricity data (high frequency – 30 mins) to model likelihood of household occupancy patterns

What does typical household

• ccupancy look like?

0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 Mid- night 6am Noon 6pm Electricity consumption (kWh) Typical occupancy An unoccupied day

Research data

• Consumer behaviour trials of smart-type

meters conducted by the Commission for Energy Regulation in Ireland and held in the Irish Social Science Data Archive

• 4,225 domestic smart-type meters
• 30 minute frequency
• 18 month trial (14th July 2009 to 31 Dec 2010)
• Pre and post survey assigned

Approach

• Initial methods developed to automatically identify

household occupancy patterns

• Key variables of interest include: ratio of day time to

night time consumption, variance, average consumption, difference to usual consumption etc.

• Thresholds set for classification
• Manual checking (visual verification) – restricted to

sample of 10 meters

• Performance visualised by confusion matrices:

assessed using sensitivity and specificity measures

Confusion matrix

Method 1 (low variance over 24 hours) Examined by eye Days unoccupied Days occupied Days unoccupied 189 24 Days occupied 3 5144 Sensitivity (true positive rate) = 189/192 = 98 per cent Specificity (true negative rate) = 5144/5168 = 99.5 per cent

Continuing research

• Writing more efficient code and using big data tools –

now running methods on all meters, iterations to improve thresholds.

• Testing combinations of methods, longer term

vacants

• Machine learning algorithms: e.g. logistic regression,

cluster analysis

• Ultimately consider how to process circa 20 million

meters.

• New research data sourced from trials conducted in

Great Britain – comparison with Irish data ongoing

Summary

• ONS Big Data project has pilot project on the

potential use of smart meter data

• Huge privacy and ethical concerns
• Methods to identify household occupancy

patterns using smart-type meter trial data have given promising results

• Big data processing capability has increased

• A. Timed appliance in night and

activity during day

A more challenging profile

• B. Unoccupied
• C. Timed appliance in night ….but

..unoccupied?

A B C