for Balance Control in Smart Grids Francisco S. Melo, Alberto - - PowerPoint PPT Presentation

Decentralized Multiagent Planning for Balance Control in Smart Grids

Francisco S. Melo, Alberto Sardinha, Stefan Witwicki,

INEDC-ID / Instituto Superior Técnico, Porto Salvo, Portugal

Laura Ramirez-Elizondo, and Matthijs Spaan

Delft University of Technology, the Netherlands

Our position...

Multiagent Sequential Decision-Making (MSDM) Research Smartgrid Research

Sequential Decision Making

• Agents

– model their environment – receive observations – take actions that maximize their objective funtion

st st+1

• t+1
• t

at rt

current state

• bservation

reward action Agent Environment model actions

• bservations

(PO) MDP Model

st+2

• t+2

at+1

r t+1

Multiagent Sequential Decision Making

• Agents

– model their environment – receive observations – take actions that maximize their objective funtion

Agents Environment

(decentralized) actions and

• bservations

model

st st+1 r t current state

• bservations

team reward actions transition

• t

next state

Dec-POMDP Model

Smartgrid Agents

Transmission Grid Weather Prediction Substation A

Smart distribution agent

Substation B

Smart distribution agent Observations

Weather Supply level

Observations

Estimated consumption & production

Power Routing Actions Power Routing Actions

Neighborhood 1

Smart Meters

Neighborhood 2

Smart Meters

Energy Storage Facility

power flows

Planning with Decentralized POMDPs

• Policy maps individual observations to individual action
• Optimal joint policy optimizes objective function

(implemented as rewards associated with states and actions)

• Agents plan policies together, then execute them

separately

• Strengths

– policies account for uncertainty in state transitions – agents need not have a perfect view of the world state – principled notion of optimality

• Weakness: computational tractability

Recent Advances in MSDM

• Conventional Solutions Infeasible to apply
• Recent trends...

– Decomposition of Models and planning processes  Distributed (policy) computation [Guestrin & Gordon 2002, Nair et al 2003, Nair et al 2005,

Varakantham et al 2009, Witwicki & Durfee 2010]

– Heurstic Search [Szer et al 2005, Oliehoek et al 2010, Spaan et al 2011] – Exploiting weakly-coupled interaction structure

• Agent Locality

[Kim et al 2006, Varakantham et al 2009, Witwicki et al 2011, Oliehoek et al 2012]

• State Locality [Oliehoek et al 2008, Spaan & Melo 2008, Melo & Veloso 2011]
• Degree of Influence [Becker et al 2004, Witwicki & Durfee 2010, Witwicki 2011]
• Significant improvements in efficiency and scalability

Large-Scale Network of Agents

Potential Benefits for MSDM

• SmartGrid network contains intrinsic structure that recent

MSDM techniques should be able to exploit

– Sparsely-connected compontents – Limited interaction among agents

• Exciting real-world domain ripe with socially-relevant

problems

Potential Benefits for Smartgrid

• Decentralizing control

– eliminates single point of failure – well-aligned with future smartgrid paradigms

• widely-distributed power generation
• diversification of energy sources
• MSDM for smart control

– Control policies can be planned autonomously – Accounts for uncertainty of renewable energy sources – Agents consider both the short-term and long-term impacts of their control decisions – Can optimize a balance of a variety of different objectives (e.g., economic, safety, environmental impact) – Harness the many recent MSDM algorithmic advances

Thank You

• Questions?
• Suggestions?

Contact Info:

• witwicki@inesc-id.pt (Stefan Witwicki)
• jose.alberto.sardinha@ist.utl.pt (Alberto Sardinha)