Modelling sample data from smart-type meter electricity usage Susan - PowerPoint PPT Presentation

Modelling sample data from smart-type meter electricity usage Susan Williams NTTS Conference, 10-12 March 2015, Brussels

Introduction • Big Data Project in the UK’s Office For National Statistics • Practical pilot on the potential of smart-type meter data Pilot objectives : • To investigate applications of big data sources within Official Statistics • To develop capability in methods and processing of big datasets

Smart-type meter pilot: Background • Smart meters: electronic devices taking high frequency energy readings • Gas and electric smart meters to be rolled out to all households in UK by 2020 (UK policy) • Readings (min 30 min frequency) to be wirelessly transmitted to central body for purposes covered by legislation. Various levels of consumer opt-out built in.

Privacy/Ethics • High public concern over data security and access could result in consumer rejection of smart metering • Theoretically: without suitable protections smart meter data might be used to identify individual households and real-time occupancy. Tracking behaviours over time may lead to the modelling of detailed profiles. • Specific smart meter data access and privacy legislation is in place in Great Britain • ONS has approached UK privacy groups to gain support for this research

Research Question Investigate the potential of smart-type meter electricity data (high frequency – 30 mins) to model likelihood of household occupancy patterns Benefit • Validate Census returns or optimise Census follow up

What does typical household occupancy look like? 1.6 Electricity consumption (kWh) Typical occupancy 1.4 An unoccupied day 1.2 1.0 0.8 0.6 0.4 0.2 0.0 Mid- 6am Noon 6pm night

Research data • Consumer behaviour trials of smart-type meters conducted by the Commission for Energy Regulation in Ireland and held in the Irish Social Science Data Archive • 4,225 domestic smart-type meters • 30 minute frequency • 18 month trial (14 th July 2009 to 31 Dec 2010) • Pre and post survey assigned

Approach • Initial methods developed to automatically identify household occupancy patterns • Key variables of interest include: ratio of day time to night time consumption, variance, average consumption, difference to usual consumption etc. • Thresholds set for classification • Manual checking (visual verification) – restricted to sample of 10 meters • Performance visualised by confusion matrices: assessed using sensitivity and specificity measures

Confusion matrix Method 1 (low Examined by eye variance over 24 Days unoccupied Days occupied hours) Days unoccupied 189 24 Days occupied 3 5144 Sensitivity (true positive rate) = 189/192 = 98 per cent Specificity (true negative rate) = 5144/5168 = 99.5 per cent

Continuing research • Writing more efficient code and using big data tools – now running methods on all meters, iterations to improve thresholds. • Testing combinations of methods, longer term vacants • Machine learning algorithms: e.g. logistic regression, cluster analysis • Ultimately consider how to process circa 20 million meters. • New research data sourced from trials conducted in Great Britain – comparison with Irish data ongoing

Summary • ONS Big Data project has pilot project on the potential use of smart meter data • Huge privacy and ethical concerns • Methods to identify household occupancy patterns using smart-type meter trial data have given promising results • Big data processing capability has increased

A more challenging profile A. Timed appliance in night and A activity during day B. Unoccupied C. Timed appliance in night ….but ..unoccupied? C B