Continuous Deep Learning for Smarter Energy Igor Gotlibovych - PowerPoint PPT Presentation

Continuous Deep Learning for Smarter Energy Igor Gotlibovych Javier Asensio-Cubero Jaye Cribb Constantin Octopus

The Energy is Getting Smarter Traditional grid: Smart grid: - predictable generation - distributed renewable generation - static consumption - dynamic consumption behaviour - fixed tariffs - smart meters - timescale of years/months - agile tariffs - API integration - timescale of hours/minutes 2

A Brief History of Octopus Energy & AI founded in 2015 Machine Learning at Octopus: ● Apr 2017: 100% renewable sources ● Time Series Analysis & Forecasting: ● Feb 2018: UK’s first Agile tariff ● ○ household consumption Jul 2018: +100k IRESA customers ● response to agile tariffs ○ Nov 2018: acquired ML team from Usio ● ○ commercial sites Jan 2019: 500k customers ● PV generation ○ ~600k customers; ~50k smart meters ● ○ Customer communications: calls/emails 20+ developers ● NLP ● 6+ Data Scientists/ML Engineers ● Control theory/RL ● Optimizing PV/battery/EV utilization ○ Growing ETL/Deep Learning infrastructure ● 3

Smart tariffs and Behavioural Response Agile tariff response: early trial Agile tariff response analysis: regression + SHAP 4

Time Series Forecasting with Deep Learning “What will the half-hourly consumption be for customer #500001 at 2019-02-29 12:00?” prediction y features X 500001 2019 2 29 12 0 2.3 Why not iterative auto-regressive model? ● Deep learning is powerful enough to turn this: f(f(f(f(...f(X)...)))) [n times] into this: g(X, n) ○ 5

6

Feature Engineering - Datetime Keep continous features continuous ● Jan-31 12:00 0.084 0.5 Better still, use periodic encoding ● 7

Feature Engineering - IDs You have a categorical feature ● You decide to use one-hot encoding ● 500001 0 ... 0 1 0 0 Now you have 500k+ features! ● 8

What we really want to know What time do Do they have Do they work #500001 they come electric ... weekends? home? heating? m features 9

Embeddings: the MovieLens Rating Example 10

Putting it Together 11

What do Embeddings Learn? Portfolio Individual Some things we can’t predict: CTO decides to eat out 12

Model Life-cycle & Continuous Training time 13

Continuous deployment, training, and usage. Like cron on steroids: Flexible way to schedule tasks, rerun them, adds ● visibility… Written in python. ○ Rather a monolithic approach: code base sharing for the tasks. ○

Heterogeneous architecture

Final approach Dockerized workers ● Decoupling airflow of dependencies and scheduling. ○ Decoupling airflow from credentials management. ○ AWS autoscaling ● Turning on and off GPUs on demand. ○ More maintainable codebase. ● Based on command line scripts. ○ Highly testable and easy to isolate. ○ Airflow dags relegated to a glorified configuration file. ○

Lessons Learned Avoid iterative forecasting - use features instead ● Two ways to learn latent features ● Embeddings: learn features during backprop ○ Encoders: trained to extract features in forward pass (not covered) ○ Encapsulate feature generation inside your model ● simpler versioning and deployment ○ simplifies replacing hand-engineered features with end-to-end ML models ○ Build reusable ML code ● Grow your toolkit ○ Listen to ML Engineers when they tell you to write tests ○ Automate every part of the model lifecycle (Airflow, Kubernetes, ASGs, …) ● 17

Thanks! igor [at] octopus.energy javier.asensio [at] octopus.energy https://ig248.gitlab.io/ @igorgotlibovych Should Data Scientists maybe? always! write Unit Tests? Switch to Octopus Energy here: Switch to Octopus Energy here: https://share.octopus.energy/mist-rain-538 https://share.octopus.energy/sand-fawn-149 18