Outline Special Topics in AI: Intelligent Agents and Multi-Agent - - PowerPoint PPT Presentation

23/10/2012 1

Special Topics in AI: Intelligent Agents and Multi-Agent Systems

Alessandro Farinelli

Distributed Constraint Optimization (Exact approaches, DPOP)

Outline

• Introduction

– DCOP for MAS – how to model problems in the DCOP framework

• Solution Techniques for DCOPs

– Exact algorithms (DCSP, DCOP)

• DPOP

– Heuristics and approximate algorithms (without/with quality guarantees)

• DSA, MGM, Max-Sum; k-optimality, bounded max-sum

Working together

Coordination problem: Choose agent’s individual actions so to maximise a system-wide objective Task allocation: individual actions: which fire to tackle system-wide objective: minimise total extinguish time solution: a joint action

Decentralised Coordination

• Decentralised coordination: Local decision with local

information

• Why Decentralised coordination ?

– In general no benefit for computation or solution quality – Robustness

• avoid single point of failure

– Scalability

• Not enough bandwidth to communicate/process all

information – Leads to problem decomposition

• Each agent cares only of local neighbours

23/10/2012 2

DCOPs for Decentralized Coordination

Why DCOPs for decentralized coordination ?

• Well defined problem

– Clear mathematical formulation that captures most important aspects – Many solution techniques

• Optimal: ABT, ADOPT, DPOP, ...
• Heuristics: DSA, MGM, Max-Sum, ...
• Solution techniques can handle large problems

– compared for example to sequential dec. Making (MDP, POMDP)

Incident Management

Reference Applications

Environment monitoring Cooperative Exploration Energy management

Modeling Problems as DCOP

• Surveillance
• Meeting Scheduling

Target Tracking

• Why decentralize

– Robustness to failure and message loss

23/10/2012 3

Target Tracking - DCOP

• Variables -> Cameras
• Domains -> Camera

actions

– look left, look right

• Constraints

– Overlapping cameras – Related to targets

• Diabolik, Eva
• Maximise sum of

constraints

L, R L, R L, R D E T1 T2 T3

Meeting Scheduling

• Why decentralize

– Privacy

Window 15:00 – 18:00 Duration 2h Window13:00 – 20:00 Duration 1h Better in [18:00 – 19:00]

Meeting Scheduling - DCOP

BS PS PL BL No overlap (Hard) Equals (Hard) Preference (Soft) 16:00 16:00 19:00 19:00 [15 – 18] [13 – 20] [13 – 20] BC [15 – 18] 16:00 [15 – 18]

Constraint Networks

a set of variables (e.g. meetings)

{ }

n

X X X ,...,

1

=

{ }

n

D D D ,...,

1

=

{ }

m

C C C ,...,

1

=

X Si ⊆

a set of discrete variable domains (e.g. time slots) a set of constraints (e.g., equality, non overlap, ) Scope of constraint

i

C

Hard constraints Soft constraint

23/10/2012 4

Graphical Representation

Hypergraph Dual Graph Primal Graph

Binary constraint networks

• Each constraint is

defined over two variables

• Every constraint

network can be mapped to a binary constraint network but

– Addition of variables/constraints – Add complexity

Constraint graph

• Link between two variables if they share at least one

constraint (i.e., primal graph)

– In general, constraint graph ≠ constraint network

Objectives for constraint networks

• Constraint Satisfaction Problem (CSP)

– Objective: find an assignment for all the variables in the network that satisfies all constraints

• Constraint Optimization Problems (COP)

– Objective: find an assignment for all the variables in the network that satisfies all constraints and optimizes a global

• bjective function

( )

• =
• i

i i X

X F X max arg

*

Global function: an aggregation (i.e., sum) of local functions

) (

i i X

F

23/10/2012 5

Distributed Constraint Reasoning

In a decentralized context:

• Agents control

Variables

• Agents communicate

to solve the problem

Benchmarking problems

• Motivations

– Analysis of complexity and optimality is not enough – Need to empirically evaluate algorithms on the same problem

• Graph coloring

– Simple to formalise very hard to solve – Well known parameters that influence complexity

• Number of nodes, number of colors, density (number of

link/number of nodes)

– Many versions of the problem

• CSP, MaxCSP, COP

Graph Coloring

• Network of nodes
• Nodes can take on various colors
• Adjacent nodes should not have the same color

– If it happens this is a conflict CSP Yes No

Graph Coloring - MaxCSP

• 4
• Optimization Problem
• Natural extension of CSP
• Minimise number of conflicts
• 1

23/10/2012 6

Weighted Graph Coloring - COP

• Optimization Problem
• Conflicts have a weight
• Maximise the sum of weights of violated constraints

COP

• 2
• 1
• 1
• 1
• 2
• 3
• 1
• 1
• 2
• 3
• 1
• 1
• 2
• 3

Distributed COP

• We focus on optimization
• DCOP = Constraint Network + Agents
• Where each agent:

– Controls a subset of the variables (typically just one) – Is only aware of constraints that involve the variables it controls – Communicate only with neighbours (constraint graph)

} ,..., {

1 k

A A A =

BS PS PL BL BC

Performance measures

• Solution quality

– Optimality not always achievable, – Optimality Guarantees

• Coordination Overdead

– Computation: computation effort (time complexity) – Communication: number and size of messages (network load)

• Desirable properties (hard to quantify)

– Robustness to failures, parallelism, flexibility, privacy maintenance, etc.

DCOP Solution techniques

• Exact approaches

– Guarantee optimal solution – Exponential coordination overhead – ADOPT, DPOP, OptAPO

• Heuristics

– Low coordination overhead – No guarantees on optimality – DSA, MGM, Max-Sum

• Approximate approaches

– Low coordination overhead – Optimality guarantees – Bounded max-sum, k-optimality

23/10/2012 7

Exact Approaches I

• ADOPT (Search based) [Modi et al 05]

– Distributed branch and bound – Partial order based on a DFS search (pseudotree) – Asynchronous (high parallelism, flexible) – Number of messages exponential in number of agents

• Small messages but exponentially many

Exact Approaches II

• DPOP (Inference) [Petcu and Faltings 07]

– Distributed Bucket Elimination – Partial order based on a DFS search (pseudotree) – Linear number of messages – Exponential message size (in width of DFS search tree) – DFS-tree width tipically much less than number of agents

• Few messages but exponentially large

Dynamic Programming Optimization Protocol

• 1. DFS-tree building (special case of Pseudo tree)

– Constraint graph DFS-Tree – Token passing

2. Utility propagation

– Compile information to compute optimal value – Util messages from leaves to root

• 3. Value Propagation

– Root chooses optimal value and propagate decision – Value messages from root to leaves

Pseudotrees: basic concepts

• Pseudotree arrangement of a graph G

– A rooted tree with the same node as G – Adjacent nodes in G falls in the same branch of the tree

• Nodes in different branches do not share direct coinstraints

– A DFS visit of a graph induces a Pseudotree

• Not every pseudotree can be obtained with a DFS visit

23/10/2012 8

Building a DFS tree

• Traverse the graph using a recursive procedure
• Each time we reach Xi from Xj we mark Xi as visited and

state that Xj is the father of Xi (and Xi is a children of Xj)

• When a node Xi has a visited neighbour that is not its

parent we state that Xj is a pseudo-parent of Xi (and Xi is a pseudochildren of Xj)

• Can be done with a distributed procedure:

– Each node need only to communicate with neighbours – Token passing to propagate information (e.g., visited nodes)

Building a DFS-tree: example

X1 X3 X2 X4 X1 X3 X2 X4

st = 0 et = 6 st = 1 et = 5 st = 2 et = 2 st = 4 et = 4 1 2 2 4 4 token movement st: first time node received the token et: last time node sent the token time++ = each time token moves

Pseudotrees and Separator

Separator Definition Basic Property Operative Definition Ci: children of node i

Util Propagation

Aim: build a value function so that root agent can make

• ptimal decision.

Dynamic programming: provide only key information

23/10/2012 9

Util Propagation: messages Message Computation

Functions tables (variable are all discrete) Aggregation join operator (relational algebra) Maximization projection (keeping most valuable tuples)

Join-sum operator Util Message propagation

23/10/2012 10

Value Propagation

Aim: inform all agents about decision from above so that they can choose best values for their variables

Value Computation

Can reuse stored tables for computing util messages

Value Propagation: messages DPOP analysis

• Synchronous algorithm
• Linear number of messages but exponentially large
• Messages (and computation) is exponential in

separator size

• Separator size graph induced width with DFS
• rdering

23/10/2012 11

Induced graph and Induced width

Given graph G = <V,E> Width of v = number of v’s ancestors Width of a graph = maximal width of nodes Given order o over vertices of a graph G: G* induced graph of G given o

– Process variables from last to first – When processing v connect all neighbours that precede v (ancestors)

Induced width of G (given o) = width of induced graph Induced width of G = min induced width over orderings Finding this is NP-hard

Separator size and induced width

Given DFS order o of a graph G: Induced width of G over o equals the size of largest separator given by o Intuition:

• Width of a node = number of induced ancestors
• connecting ancestors = propagating the children’s

separator in the separator computation

DFS tree and efficiency

• Depth first order is crucial for DPOP efficiency
• Finding optimal order is NP-hard

– Optimal minimize separator size

• Good heuristics:

– Maximum Connected Node (MCN) – Maximum Cardinality Set (MCS) for DFS

DFS tree Heuristics

23/10/2012 12

DFS tree Pseudotrees

• DPOP would work on any pseudotree arrangement
• f primal graph
• But DFS induces only a specific set of orderings:

– Not all pseudotres are DFS trees

• We might loose good orderings to keep computation

local

– Trade-off depends on applications

Summary

• DCOPs, general framework to address Multi-Agent coordination

– Many solution techniques for (relatively) large scale systems

• Complete approaches

– Suffers from exponential element (DCOPs are hard problems) – ADOPT:

• search based, asynchronous
• Small messages but exponentially many

– DPOP:

• Dynamic programming based, synchronous
• Few message but exponentially large
• Typically much more efficient than ADOPT

References

Constraint Network

• Constraint Processing, R. Dechter, Morgan Kaufmann

ADOPT

• [Modi et al., 2005] P. J. Modi, W. Shen, M. Tambe, and M.Yokoo. ADOPT:

Asynchronous distributed constraint optimization with quality guarantees. Artificial Intelligence Journal, (161):149-180, 2005. DPOP

• [Petcu 2007] A. Petcu. A Class of Algorithms for Distributed Constraint
• Optimization. PhD. Thesis No. 3942, Swiss Federal Institute of Technology

(EPFL), Lausanne (Switzerland), 2007. (Chapters 2, 3 and 4)