AI for a Responsible Power System Mathijs de Weerdt Associate - - PowerPoint PPT Presentation

AI for a Responsible Power System

Mathijs de Weerdt

Associate Professor in Algorithmics group, EEMCS Delft University of Technology February 21, 2019

My talk in a nutshell

• Artificial Intelligence can (should) contribute towards a more responsible society.
• Algorithmic innovations can tackle concrete AI challenges in the power system.

2 / 27

Current Allocation is Not Responsible

• Part of humanity is starving
• Part of humanity consumes too much
• Running out of some of the resources

Mainly caused by optimization of profit. Doughnut Economics by Kate Raworth [2017]

www.kateraworth.com/ 3 / 27

Current Allocation is Not Responsible

• Part of humanity is starving
• Part of humanity consumes too much
• Running out of some of the resources

Mainly caused by optimization of profit. We need more responsible allocations:

1 more fair across parties 2 better balance of optimal now versus

long-term effects Does this rise new algorithmic challenges? Doughnut Economics by Kate Raworth [2017]

www.kateraworth.com/ 3 / 27

Scientific gap

allocation decision support simple well understood complex AI & algorithmics

4 / 27

Scientific gap

allocation decision support with interaction simple well understood (behavioural) game theory complex AI & algorithmics algorithmic game theory Algorithmic game theory:

• Fair allocation under strict conditions [Bergemann and V¨

alim¨ aki, 2010, Parkes et al., 2010]

• For relevant settings: impossibility theorems [Satterthwaite, 1975].

4 / 27

Scientific gap

allocation decision support with interaction simple well understood (behavioural) game theory complex AI & algorithmics algorithmic game theory Algorithmic game theory:

• Fair allocation under strict conditions [Bergemann and V¨

alim¨ aki, 2010, Parkes et al., 2010]

• For relevant settings: impossibility theorems [Satterthwaite, 1975].

But these are situations we encounter —and deal with— in practice.

• Can algorithms and AI help to improve this current practice with respect to

efficiency, fairness, and longer term consequences. . . ?

• Let’s look at some more concrete computational challenges in the electricity grid.

4 / 27

Today’s Electricity System

Today’s Electricity System

• Electricity systems support the generation, transport and use of electrical energy.
• They are large and complex and provide for everyone.

Energy generated = energy consumed at all times

How it used to be. . .

• demand is predictable (at an aggregate level)
• which generators are used is decided one day in advance (unit commitment)
• minor corrections are made, based on frequency (primary control, secondary

control, etc.)

• a market with few actors (energy retailers)

6 / 27

The Energy Transition

The Energy Transition

Changes in the power system

• renewable energy is
• intermittent
• uncertain
• uncontrollable
• sometimes located in the distribution grid, and
• has virtually no marginal costs
• new loads such as heat pumps, airconditioning,

and electric vehicles are

• significantly larger than other household

demand, and

• more flexible (and therefore also less

predictable)

These new loads can also be part of the solution!

commons.wikimedia.org/wiki/File: Electric_Car_recharging.jpg 9 / 27

Consequences for the main stakeholders

Focus on (computational) challenges regarding

1 Wholesale market operators and system operators 2 Aggregators of flexible demand 3 Distribution network operators

10 / 27

Challenges in Wholesale Market Design

Challenges for Market Operators/Regulators and ISO/TSO

1 more accurate models for bidding and market clearing

• use finer granularity, power-based instead of energy-based (Philipsen et al., 2018)
• deal with intertemporal dependencies caused by flexible shiftable loads
• model stochastic information explicitly

but reasonable models are non-linear: interesting optimization problem

2 allow smaller, local producers and flexible loads (scalability)

11 / 27

Aggregators of flexible demand

New flexible loads can be used to match renewable generation, but

• consumers do not want to interact with the market, and
• markets do not want every consumer to interact.

Challenges for Aggregators (a new role!): demand-side management

1 design mechanism to interact with consumers with flexible demand 2 interact with both wholesale markets and distribution service/network operator 3 optimize use of (heterogeneous) flexible demand under uncertain prices and

uncertain consumer behavior

12 / 27

Challenges for DSOs

Aim to avoid unnecessary network reinforcement by demand side management to resolve congestion and voltage quality issues

Challenges for Distribution network system operators

1 (Close to) real-time coordination of generation, storage and flexible loads of

self-interested agents to stay within network capacity limitations:

• more agents than in traditional energy market
• interaction with wholesale markets
• communication may not be always reliable
• more complex power flow computations (losses and limitations more relevant in

distribution)

2 Long-term decision making under uncertainty

13 / 27

Challenges for DSOs

Aim to avoid unnecessary network reinforcement by demand side management to resolve congestion and voltage quality issues

Challenges for Distribution network system operators

1 (Close to) real-time coordination of generation, storage and flexible loads of

self-interested agents to stay within network capacity limitations:

• more agents than in traditional energy market
• interaction with wholesale markets
• communication may not be always reliable
• more complex power flow computations (losses and limitations more relevant in

distribution)

2 Long-term decision making under uncertainty

Some of these challenges we take up in our research.

13 / 27

Research on Responsible Multi-Party Optimization

Mission: to design (and understand fundamental properties of) planning and coordination algorithms for responsible optimization across organizational boundaries

Scientific challenges in responsible multi-party optimization

• efficiency (optimality) and scalability,

14 / 27

Research on Responsible Multi-Party Optimization

Mission: to design (and understand fundamental properties of) planning and coordination algorithms for responsible optimization across organizational boundaries

Scientific challenges in responsible multi-party optimization

• efficiency (optimality) and scalability,
• fairness, and
• accounting for both long- and short-term effects.

Example: Using Flexibility of Heat Pumps to Prevent Congestion (from the perspective

• f an aggregator working closely with network operator)

14 / 27

Heat Pumps to Prevent Congestion

with Frits de Nijs, Erwin Walraven, and Matthijs Spaan [de Nijs et al., 2015, 2017, 2018a,b, 2019]

De Teuge (near Zutphen)

• pilot sustainable district in 2003
• heatpumps for heating

But: at peak (cold) times, overload of electricity infrastructure

15 / 27

Potential Solutions

1 Reinforce network to cope with peak load 2 Optimal scheduling of demand 3 Re-allocation and online coordination 4 Pre-allocation and minimizing violations

16 / 27

• 2. Optimal Scheduling

Formulate as a mixed integer problem (MIP)

• decide when to turn on or off heat pump
• minimise discomfort (fair: squared distance to temperature set point)
• subject to physical characteristics and capacity constraint

MIP formulation minimize

[ act0 act1 ··· acth ] h

• t=1

cost(θt, θset

t

) (discomfort) subject to θt+1 = temperature(θt, actt, θout

t

)

n

• i=1

acti,t ≤ capacityt acti,t ∈ [off, on] ∀i, t

17 / 27

• 2. Optimal Scheduling

Formulate as a mixed integer problem (MIP)

• decide when to turn on or off heat pump
• minimise discomfort (fair: squared distance to temperature set point)
• subject to physical characteristics and capacity constraint

MIP formulation minimize

[ act0 act1 ··· acth ] h

• t=1

cost(θt, θset

t

) (discomfort) subject to θt+1 = temperature(θt, actt, θout

t

)

n

• i=1

acti,t ≤ capacityt acti,t ∈ [off, on] ∀i, t

This scales poorly (binary decision variables: houses × time slots). But that is not the only problem. . .

17 / 27

• 3. Re-allocation and online coordination

Not everything is known in advance (heat loss, available capacity), so adaptation may be required

Arbitrage with Best Response (BR)

1 Plan each thermostat individually, as if unconstrained. 2 Look at the expected plan utility to determine action 3 Determine resource costs per time slot 4 Re-plan including these costs

18 / 27

• 3. Re-allocation and online coordination

Not everything is known in advance (heat loss, available capacity), so adaptation may be required

Arbitrage with Best Response (BR)

1 Plan each thermostat individually, as if unconstrained. 2 Look at the expected plan utility to determine action 3 Determine resource costs per time slot 4 Re-plan including these costs

→ Iterative process → Inspired by Brown’s Fictitious Play Keep all past realizations to ensure convergence

18 / 27

• 3. Re-allocation and online coordination

Simulation of an extreme scenario

• 182 households
• almost no

capacity for heating 18–24 hours

50 100 150 6 12 18 24 30 36 42 48

Devices on (#) Time (h)

19 / 27

• 3. Re-allocation and online coordination

Simulation of an extreme scenario

• 182 households
• almost no

capacity for heating 18–24 hours Close to lower bound

• n optimal (relaxed

MIP)

50 100 150 6 12 18 24 30 36 42 48 19 20 21 22 6 12 18 24 30 36 42 48 0.00 0.25 0.50 0.75 1.00 6 12 18 24 30 36 42 48

Solver

No planning arbitrage−BR Relaxed MIP

Devices on (#)

• Avg. Temp. (◦ C)

Cumulative penalty Time (h)

19 / 27

• 3. Re-allocation and online coordination: runtime

1 2 3 4 5 6 10 15 20 25 30 35 40 45 10−2 10−1 100 101 102

Solver

arbitrage−BR Optimal MMDP Optimal MIP

Runtime (s) Agents (n) Horizon (h) Close to optimal, good scaling, but reliable communication is essential. . . Pre-allocate time-dependent limits per agent that limit probability of violations.

20 / 27

• 4. Pre-allocation and minimizing violations

Robust pre-allocation of available capacity (Wu and Dur- fee, 2010)

Frequency L

Satisfying Limits in Expectation (CMDP, Altman 1999)

Frequency L

Reduced Limits with Hoeffding’s inequality given α

Frequency L∗ L

Dynamic Relaxation of Reduced Limits by Simulation (De Nijs et al., 2017–2018)

Frequency L∗ ˜ L L

21 / 27

Simulations and Extensions

• Dynamic pre-allocation: efficient, tightly

bounded violation probability, and scalable

100 8 16 32 64

TCL, h = 24 Violations

0.005 0.050 0.500 1.000 10−2 10−1 100 101 102 103 4 8 16 32 64 128 256

• Num. Agents

% Ex. Value Runtime (s.)

Alg.

MILP LDD+GAPS CMDP Hoeffding (CMDP), α = 0.05 Dynamic (CMDP), α = 0.005 Dynamic (CG), α = 0.005 22 / 27

Simulations and Extensions

• Dynamic pre-allocation: efficient, tightly

bounded violation probability, and scalable

Extensions

• Re-run in case new information (and

communication) is available

• Include wholesale electricity prices to use

flexibility for balancing (within constraints)

• Include electric vehicles (and other flexible

loads)

100 8 16 32 64

TCL, h = 24 Violations

0.005 0.050 0.500 1.000 10−2 10−1 100 101 102 103 4 8 16 32 64 128 256

• Num. Agents

% Ex. Value Runtime (s.)

Alg.

MILP LDD+GAPS CMDP Hoeffding (CMDP), α = 0.05 Dynamic (CMDP), α = 0.005 Dynamic (CG), α = 0.005 22 / 27

Simulations and Extensions

• Dynamic pre-allocation: efficient, tightly

bounded violation probability, and scalable

Extensions

• Re-run in case new information (and

communication) is available

• Include wholesale electricity prices to use

flexibility for balancing (within constraints)

• Include electric vehicles (and other flexible

loads)

Toolbox available (soon): github.com/AlgTUDelft/ConstrainedPlanningToolbox PhD Defense and seminar: April 4, 2019

100 8 16 32 64

TCL, h = 24 Violations

0.005 0.050 0.500 1.000 10−2 10−1 100 101 102 103 4 8 16 32 64 128 256

• Num. Agents

% Ex. Value Runtime (s.)

Alg.

MILP LDD+GAPS CMDP Hoeffding (CMDP), α = 0.05 Dynamic (CMDP), α = 0.005 Dynamic (CG), α = 0.005 22 / 27

Our Other Contributions to the Smart Grid

• Scheduling of charging of electric vehicles:
• Computational complexity [de Weerdt et al., 2018]
• Stochastic optimization [van der Linden et al., 2018]
• En-route charging [de Weerdt et al., 2016]
• Market design:
• Auction design for DC grids [Piao et al., 2018, 2017]
• Design errors in existing markets [Philipsen et al., 2019]
• Combined: Online scheduling mechanism for flexible loads [Str¨
• hle et al., 2014]
• Long-term investments: Unbounded MDPs [Neustroev et al., 2019]

23 / 27

Conclusions

Concluding Observations

• AI and algorithms can contribute to a

responsible operation of common infrastructure

• More research required on fairness,

incentives in market design, and effects on the future!

Slides are available from www.alg.ewi.tudelft.nl/weerdt/

Thanks a lot to Matthijs, Frits, Erwin, Rens, Laurens, German, Natalia, Greg, Koos, Anna, Longjian, Neil, Gleb, Jedlix, Alliander, Eneco, and all my (other) students and collaborators!

24 / 27

References I

• D. Bergemann and J. V¨

alim¨

• aki. The dynamic pivot mechanism. Econometrica, 78(2):771–789, 2010.
• F. de Nijs, M. Spaan, and M. de Weerdt. Best-Response Planning of Thermostatically Controlled

Loads under Power Constraints. In Proceedings of the 29th AAAI Conference on Artificial Intelligence, pages 615–621, 2015.

• F. de Nijs, E. Walraven, M. T. Spaan, and M. M. de Weerdt. Bounding the Probability of Resource

Constraint Violations in Multi-Agent MDPs. Proc. AAAI 2017, 2017.

• F. de Nijs, M. Spaan, and M. M. de Weerdt. Preallocation and Planning under Stochastic Resource
• Constraints. In Proceedings of the 32th AAAI Conference on Artificial Intelligence. Association for

the Advancement of Artificial Intelligence (AAAI), Jan. 2018a.

• F. de Nijs, G. Theocharous, N. Vlassis, M. de Weerdt, and M. Spaan. Capacity-aware Sequential
• Recommendations. In Proceedings of the 17th International Conference on Autonomous Agents and

Multiagent Systems, pages 416–424. International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS), July 2018b.

• F. de Nijs, M. T. J. Spaan, and M. M. de Weerdt. Multi-agent Planning Under Uncertainty for

Capacity Management. In P. Palensky, M. Cvetkovi´ c, and T. Keviczky, editors, Intelligent Integrated Energy Systems, pages 197–213. Springer, Cham, 2019.

25 / 27

References II

• M. de Weerdt, M. Albert, V. Conitzer, and K. van der Linden. Complexity of Scheduling Charging in

the Smart Grid. In Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, pages 4736–4742, California, July 2018. International Joint Conferences on Artifical Intelligence (IJCAI).

• M. M. de Weerdt, S. Stein, E. H. Gerding, V. Robu, N. R. Jennings, and M. M. de Weerdt.

Intention-aware routing of electric vehicles. IEEE Transactions on Intelligent Transportation Systems, 17(5):1472–1482, 2016.

• G. Neustroev, M. M. de Weerdt, and R. A. Verzijlbergh. Discovery of Optimal Solution Horizons in

Non-Stationary Markov Decision Processes with Unbounded Rewards. In Proceedings of the International Conference on Planning and Scheduling (ICAPS’05), 2019.

• D. C. Parkes, R. Cavallo, F. Constantin, and S. Singh. Dynamic incentive mechanisms. AI Magazine,

31(4):79–94, 2010.

• R. Philipsen, G. Morales-Espana, M. de Weerdt, and L. de Vries. Trading power instead of energy in

day-ahead electricity markets. Applied Energy, 233-234:802–815, 2019.

• L. Piao, M. de Weerdt, and L. de Vries. Electricity market design requirements for DC distribution
• systems. In 2017 IEEE Second International Conference on DC Microgrids (ICDCM, pages 95–101.

IEEE, 2017.

26 / 27

References III

• L. Piao, L. de Vries, M. de Weerdt, and N. Yorke-Smith. Electricity Market for Direct Current

Distribution Systems: Exploring the Design Space. In Proceedings of the 15th International Conference on the European Energy Market, EEM’18, pages 1–5, United States, 2018. IEEE.

• K. Raworth. Doughnut Economics. Seven Ways to Think Like a 21st-Century Economist. 2017.
• M. A. Satterthwaite. Strategy-proofness and Arrow’s conditions: Existence and correspondence

theorems for voting procedures and social welfare functions. Journal of economic theory, 10(2): 187–217, 1975.

• P. Str¨
• hle, E. H. Gerding, M. M. de Weerdt, S. Stein, and V. Robu. Online mechanism design for

scheduling non-preemptive jobs under uncertain supply and demand. In Proceedings of the 2014 international conference on Autonomous agents and multi-agent systems, pages 437–444, 2014.

• K. van der Linden, M. de Weerdt, and G. Morales-Espana. Optimal non-zero Price Bids for EVs in

Energy and Reserves Markets using Stochastic Optimization. In Proceedings of the 15th International Conference on the European Energy Market, EEM 2018, pages 1–5, United States,

• 2018. IEEE.

27 / 27