MODELS AND M MODELS AND METHODS ETHODS FOR FOR CYBER-PHY CYBER - - PowerPoint PPT Presentation

• C. G

. G. Cassand . Cassandras as

Division of Systems Engineering and Dept. of Electrical and Computer Engineering and Center for Information and Systems Engineering Boston University

Christos G. Cassandras

CODES Lab. - Boston University

DISCRETE DISCRETE EVENT AN EVENT AND HYBRID SY HYBRID SYSTEM STEM MODELS AND M MODELS AND METHODS ETHODS FOR FOR CYBER CYBER-PHY PHYSICAL SICAL SY SYSTEMS STEMS

1. SCALABILITY

• 2. DECENTRALIZATION
• 3. COMMUNICATION
• 4. NON-CONVEXITY
• 5. EXLOIT DATA

Christos G. Cassandras CODES Lab. - Boston University

CONTROL AND OPTIMIZATION – CHALLENGES Distributed Algorithms Global optimality, escape local optima Event-driven (asynchronous) Algorithms Data-Driven Algorithms

OUTLINE

Christos G. Cassandras CODES Lab. - Boston University

• MODELING:
• Discrete Event Systems (DES)
• Hybrid Systems (HS)
• CONTROL AND OPTIMIZATION:
• Event-Driven Distributed Algorithms
• Data-Driven + Event-driven Algorithms:

The IPA Calculus

• Global optimality, escaping local optima

STATES s1 s2 s3 s4 TIME t

TIME-DRIVEN SYSTEM

STATES TIME t

STATE SPACE:

X  

DYNAMICS:

 

 , x f x t 

EVENT-DRIVEN SYSTEM

STATE SPACE:

 

X s s s s 

1 2 3 4

, , ,

DYNAMICS:

 

e x f x , '

t2 e2 x(t) t3 t4 t5 e3 e4 e5 EVENTS x(t) t1 e1

Christos G. Cassandras CODES Lab. - Boston University

TIME-DRIVEN v EVENT-DRIVEN SYSTEMS

TIME-DRIVEN v EVENT-DRIVEN CONTROL

Christos G. Cassandras CODES Lab. - Boston University

REFERENCE

PLANT CONTROLLER

INPUT

• +

SENSOR

MEASURED OUTPUT OUTPUT ERROR REFERENCE

PLANT CONTROLLER

INPUT

• +

SENSOR

MEASURED OUTPUT OUTPUT ERROR

EVENT:

g(STATE) ≤ 0 EVENT-DRIVEN CONTROL: Act only when needed (or on TIMEOUT) - not based on a clock

REASONS FOR EVENT-DRIVEN MODELS, CONTROL, OPTIMIZATION

Christos G. Cassandras CODES Lab. - Boston University

• Many systems are naturally Discrete Event Systems (DES)

(e.g., Internet) → all state transitions are event-driven

• Most of the rest are Hybrid Systems (HS)

→ some state transitions are event-driven

• Many systems are distributed

→ components interact asynchronously (through events)

• Time-driven sampling inherently inefficient (“open loop” sampling)

REASONS FOR EVENT-DRIVEN MODELS, CONTROL, OPTIMIZATION

Christos G. Cassandras CODES Lab. - Boston University

• Many systems are stochastic

→ actions needed in response to random events

• Event-driven methods provide significant advantages in

computation and estimation quality

• System performance is often more sensitive to event-driven

components than to time-driven components

• Many systems are wirelessly networked → energy constrained

→ time-driven communication consumes significant energy UNNECESSARILY!

MODELING DES AN MODELING DES AND D HS: HS:

• Timed A

Timed Automa utomata ta

• Hybrid A

Hybrid Automa utomata ta

AUTOMATON: (E, X, G, f, x0) E : Event Set X : State Space

 

e e

1 2

, ,

 

x x

1 2

, ,

 

f x e x , ' 

G(x) : Set of feasible or enabled events at state x

x X

0 

x0 : Initial State,

f X E X :  

 

e x G

f : State Transition Function (undefined for events )

Christos G. Cassandras CODES Lab. - Boston University

AUTOMATON

Add a Clock Structure V to the automaton: (E, X, G, f, x0 , V) where: and vi is a Clock or Lifetime sequence:

• ne for each event i

 

V v  

i

i E :

 

 , ,

2 1 i i i

v v  v

Need an internal mechanism to determine NEXT EVENT e´ and hence NEXT STATE

 

x f x e ' , ' 

TIMED AUTOMATON

Christos G. Cassandras CODES Lab. - Boston University

 

x x

1 2

, ,

 

f x e x , ' ' 

v1 vN



NEXT EVENT

• CURRENT STATE
• CURRENT EVENT
• CURRENT EVENT TIME

Associate a CLOCK VALUE/RESIDUAL LIFETIME yi with each feasible event

 

i x G

xX with feasible event set G(x) e that caused transition into x t associated with e

HOW THE TIMED AUTOMATON WORKS...

Christos G. Cassandras CODES Lab. - Boston University

• NEXT/TRIGGERING EVENT e' :

  

i x i

y e

G 

 min arg '

• NEXT EVENT TIME t' :

  

t t y y y

i x i

' * min   



where: *

G

• NEXT STATE x' :

 

x f x e ' , ' 

HOW THE TIMED AUTOMATON WORKS...

Christos G. Cassandras CODES Lab. - Boston University

Determine new CLOCK VALUES for every event

 

i x G

 yi

         

i v

• therwise

e x x i v e i x i x i y y y

ij ij i i

event for lifetime new : where , , *                    G G G G

Christos G. Cassandras CODES Lab. - Boston University

HOW THE TIMED AUTOMATON WORKS...

 

x x

1 2

, ,

 

} { min arg ' , ' , '

) ( i x i

y e e x f x

G 

 

 

V x, , ' y g y 

v1 vN



   

 , 2 , 1 , ,   X d a E

 

f x e x e a x e d x , ,             1 1

   

  , , , , , : input Given

2 1 2 1 d d d a a a

v v v v   v v

       

a x d a x    all for , , G G

Christos G. Cassandras CODES Lab. - Boston University

TIMED AUTOMATON - AN EXAMPLE

1 a d 2 a d a d n a d n+1 a d

 

t0 x0 = 0

a

e1 = a

x0 = 1 t1

a d

e2 = a

x0 = 2 t2

a d

e3 = a

x0 = 3 t3

a d

x0 = 2

e4 = d

t4

Christos G. Cassandras CODES Lab. - Boston University

TIMED AUTOMATON - A STATE TRAJECTORY

• Same idea with the Clock Structure consisting of Stochastic Processes
• Associate with each event i a Lifetime Distribution based on

which vi is generated

Generalized Semi-Markov Process (GSMP)

In a simulator, vi is generated through a pseudorandom number generator

Christos G. Cassandras CODES Lab. - Boston University

STOCHASTIC TIMED AUTOMATON

HYBRID AUTOMATA

Christos G. Cassandras CODES Lab. - Boston University

HYBRID AUTOMATA

Q: set of discrete states (modes) X: set of continuous states (normally Rn) E: set of events U: set of admissible controls

) , , , , , , , , , , (

0 x

q guard Inv f U E X Q Gh   

f : vector field,  : discrete state transition function,

X U X Q f    : Q E X Q    : 

Inv: set defining an invariant condition (domain), guard: set defining a guard condition,  : reset function, q0: initial discrete state x0: initial continuous state

X Q Inv   X Q Q guard    X E X Q Q     : 

HYBRID AUTOMATA

Christos G. Cassandras CODES Lab. - Boston University

HYBRID AUTOMATA Key features:

Guard condition: Subset of X in which a transition from q to q' is enabled, defined through  Invariant condition: Subset of X to which x must belong in order (domain) to remain in q. If this condition no longer holds, a transition to some q' must occur, defined through  Transition MAY occur Transition MUST occur Reset condition: New value x' at q' when transition occurs from (x,q)

HYBRID AUTOMATA

Christos G. Cassandras CODES Lab. - Boston University

HYBRID AUTOMATA

Unreliable machine with timeouts

T        x 1 1      u x ] [ T  

a

2      x

b,

K x 

g

' '    x ' '    x

IDLE BUSY DOWN

x(t) : physical state of part in machine (t): clock a : START, b : STOP, g : REPAIR Invariant Guard Reset

HYBRID AUTOMATA

Christos G. Cassandras CODES Lab. - Boston University

HYBRID AUTOMATA

T        x 1 1      u x ] [ T  

a

2      x

b,

K x 

g

' '    x ' '    x

IDLE BUSY DOWN

Invariant Guard Reset     

• therwise

1 ) ; , ; ( a   e e x         

• therwise

1 , 2 ) ; , ; 1 ( b    e K x T e x     

• therwise

2 ) ; , ; 2 ( g   e e x

HYBRID AUTOMATA STOCHASTIC HYBRID AUTOMATA

www.mathworks.com/products/simevents/

Cassandras, Clune and Mostreman, 2006

SELECTED REFERENCES - MODELING

Christos G. Cassandras CODES Lab. - Boston University

Timed Automata and Timed Petri Nets – Alur, R., and D.L. Dill, “A Theory of Timed Automata,” Theoretical Computer Science, No. 126, pp. 183-235, 1994. – Cassandras, C.G, and S. Lafortune, “Introduction to Discrete Event Systems,” Springer, 2008. – Wang, J., Timed Petri Nets - Theory and Application, Kluwer Academic Pub- lishers, Boston, 1998. Hybrid Systems – Bemporad, A. and M. Morari, “Control of Systems Integrating Logic Dynamics and Constraints,” Automatica, Vol. 35, No. 3, pp.407-427, 1999. – Branicky, M.S., V.S. Borkar, and S.K. Mitter, “A Unified Framework for Hy- brid Control: Model and Optimal Control Theory,”IEEE Trans. on Automatic Control, Vol. 43, No. 1, pp. 31-45, 1998. – Cassandras, C.G., and J. Lygeros, “Stochastic Hybrid Systems,” Taylor and Francis, 2007. – Grossman, R., A. Nerode, A. Ravn, and H. Rischel, (Eds), Hybrid Systems, Springer, New York, 1993. – Hristu-Varsakelis, D. and W.S. Levine, Handbook of Networked and Embedded Control Systems, Birkhauser, Boston, 2005.

1. SCALABILITY

• 2. DECENTRALIZATION
• 3. COMMUNICATION
• 4. NON-CONVEXITY
• 5. EXLOIT DATA

Christos G. Cassandras CODES Lab. - Boston University

CONTROL AND OPTIMIZATION – CHALLENGES Distributed Algorithms Global optimality, escape local optima Event-driven (asynchronous) Algorithms Data-Driven Algorithms

WHEN CAN WE WHEN CAN WE DECENTRA DECENTRALIZE ? LIZE ?





 dx x R x P H ) ( ) , ( ) ( max s s

s

N i F si , , 1 ,     

• R(x): property of point x
• P(x, s): reward function
• Oj: obstacle (constraint)

x

i

a

a1

a2

a3

O1 O2

• si: agent state, i = 1,…, N

s=[s1, … , sN ]

Christos G. Cassandras CODES Lab. - Boston University

MULTI-AGENT OPTIMIZATION: PROBLEM 1

Ω GOAL: Find the best state vector s=[s1, … , sN ] so that agents achieve a maximal reward from interacting with the mission space

 





T t

dt dx x R t u x P J

) (

) ( ))) ( ( , ( max s

u

N i F t si , , 1 , ) (     

x

i

a

a1

a2

a3

O1 O2

Christos G. Cassandras CODES Lab. - Boston University

MULTI-AGENT OPTIMIZATION: PROBLEM 2

Ω GOAL: Find the best state trajectories si(t), 0 ≤ t ≤ T so that agents achieve a maximal reward from interacting with the mission space

N i t u s f s

i i i i

, , 1 ), , , (    

May also have dynamics





 dx x R x P H ) ( ) , ( ) ( max s s

s

N i F si , , 1 ,     

Christos G. Cassandras CODES Lab. - Boston University

WHEN CAN WE DECENTRALIZE A MULTI-AGENT PROBLEM 1?

 





  

N i i i

s x p x P

1

) , ( ˆ 1 1 ) , ( s

• Recall:

    

• therwise

) ( ) , ( ) , ( ˆ

i i i i i

s V x s x p s x p

 

i k a k s s b B

i k i k i i

        , , , 1 , 2 : 

• Define agent i

NEIGHBORHOOD:

i

si sk

) ( i s V x

sj

) (

j

s V x

Christos G. Cassandras CODES Lab. - Boston University

OBJECTIVE FUNCTION DECOMPOSITION

THEOREM: If P(x,s) = P(p1,…,pN) is a function of local reward functions pi, then H(s) can be expressed as: for any i = 1,…,N, where and ), ( ) ( ) (

2 1 i L i

s H H H   s s

• si =[s1,×

× × ,si-1,si+1,× × × ,sN]

] , , [

1 a i i

b b i L i

s s s    s

State of i and its neighbors only State of all agents except i

• Distributed gradient-based algorithm:
• Theorem implies

) ( ) (

1 i L i i

s H s H      s s ) (

1 1 k i L i k k i k i

s H s s    



s b

Christos G. Cassandras CODES Lab. - Boston University

OBJECTIVE FUNCTION DECOMPOSITION

• Theorem 1 often applies and is easy to check for the

“Problem 1” setting EXAMPLE: Coverage Control Problems

COVERAGE: PROBLEM FORMULATION

• Sensing attenuation:

pi(x, si) monotonically decreasing in di(x)  ||x - si||

• Data source at x emits signal with energy E
• N mobile sensors, each located at siR2
• Signal observed by sensor node i (at si )
• SENSING MODEL:

] ), ( | by Detected [ ) , (

i i i

s x A i P s x p  ( A(x) = data source emits at x )

Christos G. Cassandras CODES Lab. - Boston University



R(x) (Hz/ m2)

? ? ? ?? ? ? ? ?

• Joint detection prob. assuming sensor independence

( s = [s1,…,sN] : node locations)

 





  

N i i i

s x p x P

1

) , ( 1 1 ) , ( s

• OBJECTIVE: Determine locations s = [s1,…,sN] to

maximize total Detection Probability:

) , ( ) ( max dx x P x R





s

s

COVERAGE: PROBLEM FORMULATION

Christos G. Cassandras CODES Lab. - Boston University

Event sensing probability

Theorem 1 applies

CONTINUED

DISTRIBUTED COOPERATIVE SCHEME

• Set

  dx

x p x R s s H

N i i N

 

 

        

1 1

) ( 1 1 ) ( ) , , ( 

Christos G. Cassandras CODES Lab. - Boston University

• Maximize H(s1,…,sN) by forcing nodes to move using

gradient information:

 

dx x d x s x d x p x p x R s H

k k k k N k i i i k

 

  

       ) ( ) ( ) ( ) ( 1 ) (

, 1

k i k k i k i

s H s s    



b

1

Desired displacement = V·Dt

Cassandras and Li, EJC, 2005 Zhong and Cassandras, IEEE TAC, 2011

CONTINUED

DISTRIBUTED COOPERATIVE SCHEME … has to be autonomously evaluated by each node so as to determine how to move to next position:

CONTINUED

• Truncated pi(x)   replaced by node neighborhood i
• Discretize pi(x) using a local grid

Christos G. Cassandras CODES Lab. - Boston University

 

dx x d x s x d x p x p x R s H

k k k k N k i i i k

 

  

       ) ( ) ( ) ( ) ( 1 ) (

, 1

k i k k i k i

s H s s    



b

1

1. SCALABILITY

• 2. DECENTRALIZATION
• 3. COMMUNICATION
• 4. NON-CONVEXITY
• 5. EXLOIT DATA

Christos G. Cassandras CODES Lab. - Boston University

CONTROL AND OPTIMIZATION – CHALLENGES Distributed Algorithms Global optimality, escape local optima Event-driven (asynchronous) Algorithms Data-Driven Algorithms

EVENT EVENT-DRIVEN DRIVEN DISTRIB DISTRIBUTED UTED AL ALGORITHMS GORITHMS

DISTRIBUTED COOPERATIVE OPTIMIZATION

Christos G. Cassandras CODES Lab. - Boston University

i N s s

s t s s s H

N

each

• n

s constraint . . ) , , ( min

1 , ,

1





1 1

• n

s constraint . . ) , , ( min

1

s t s s s H

N s



N N s

s t s s s H

N

• n

s constraint . . ) , , ( min

1 

…

N system components

(processors, agents, vehicles, nodes),

• ne common objective:

DISTRIBUTED COOPERATIVE OPTIMIZATION

Christos G. Cassandras CODES Lab. - Boston University

i Controllable state si, i = 1,…,ni

)) ( ( ) ( ) 1 ( k d k s k s

i i i i

s a   

Step Size Update Direction, usually

)) ( ( )) ( ( k H k d

i i

s s  

i N s

s t s s s H

i

• n

s constraint . . ) , , ( min

1 

i requires knowledge of all s1,…,sN Inter-node communication

SYNCHRONIZED (TIME-DRIVEN) COOPERATION

Christos G. Cassandras CODES Lab. - Boston University

1 2 3

COMMUNICATE + UPDATE

Drawbacks:

• Excessive communication (critical in wireless settings!)
• Faster nodes have to wait for slower ones
• Clock synchronization infeasible
• Bandwidth limitations
• Security risks

ASYNCHRONOUS COOPERATION

Christos G. Cassandras CODES Lab. - Boston University

1 2 3

• Nodes not synchronized, delayed information used

Bertsekas and Tsitsiklis, 1997

Update frequency for each node is bounded + technical conditions 

)) ( ( ) ( ) 1 ( k d k s k s

i i i i

s a   

converges

ASYNCHRONOUS (EVENT-DRIVEN) COOPERATION

Christos G. Cassandras CODES Lab. - Boston University

2 3

UPDATE COMMUNICATE

• UPDATE at i :

locally determined, arbitrary (possibly periodic)

• COMMUNICATE from i : only when absolutely necessary

1

WHEN SHOULD A NODE COMMUNICATE?

Christos G. Cassandras CODES Lab. - Boston University

Node state at any time t : xi(t) Node state at tk : si(k)  si(k) = xi(tk)

j i

tk

) (k

j



Estimate examples: )) ( ( ) ( k x k s

j j i j

  Most recent value

 

)) ( ( ) ( )) ( ( ) ( k x d k t k x k s

j j j i j j k j j i j

 a     D    Linear prediction : node j state estimated by node i ) (k si

j

AT UPDATE TIME tk :

WHEN SHOULD A NODE COMMUNICATE?

Christos G. Cassandras CODES Lab. - Boston University

AT ANY TIME t :

• If node i knows how j estimates its state, then it can evaluate

) (t x j

i

• Node i uses
• its own true state, xi(t)
• the estimate that j uses,

) (t x j

i

… and evaluates an ERROR FUNCTION

 

) ( ), ( t x t x g

j i i

Error Function examples:

2 1

) ( ) ( , ) ( ) ( t x t x t x t x

j i i j i i

 

• : node i state estimated by node j

) (t x j

i

Christos G. Cassandras CODES Lab. - Boston University

i i j

) (t xi

i

WHEN SHOULD A NODE COMMUNICATE?

Node i communicates its state to node j only when it detects that its true state xi(t) deviates from j’ estimate of it so that ) (t x j

i

 

i j i i

t x t x g   ) ( ), (

 

) ( ), ( t x t x g

j i i

Compare ERROR FUNCTION to THRESHOLD i

 Event-Driven Control

THRESHOLD PROCESS

Christos G. Cassandras CODES Lab. - Boston University

     

• therwise

) 1 ( if ) ( ( ) ( k C k k d K k

i i i i i

 



s ) ( ( ) (

i i i

d K s



  



K

Update Direction, usually

)) ( ( )) ( ( k H k d

i i i i

s s  

Intuition: near convergence (small ), better estimates are needed

)) ( ( k d

i i s

Christos G. Cassandras CODES Lab. - Boston University

CONVERGENCE Asynchronous distributed state update process at each i:

)) ( ( ) ( ) 1 ( k d k s k s

i i i i

s     a

Estimates of other nodes, evaluated by node i

    

• therwise

) 1 ( update sends if ) ( ( ) ( k k k d K k

i i i i

 



s

Christos G. Cassandras CODES Lab. - Boston University

CONVERGENCE

CONTINUED

ASSUMPTION 1: There exists a positive integer B such that for all i = 1,…,N and k ≥ 0 at least one of the elements

• f the set {k−B+1, k−B+2,..., k} belongs to Ci.

INTERPRETATION: Each node updates its state at least once during a period in which B state update events take place (no time bound)

ASSUMPTION 2: The objective function H(s), satisfies: (a) H(s) ≥ 0, for all (b) H(·) continuously differentiable and Lipschitz continuous, i.e., there exists K1 such that for all

 

  

N i i m

n m

1

, s

m

  s ) ( H

m

  y x, y x y x     

1

) ( ) ( K H H

Christos G. Cassandras CODES Lab. - Boston University

CONVERGENCE

CONTINUED

ASSUMPTION 3: There exist positive constants K2, K3 such that for all i = 1,…,N and

i

C k  ) ( )) ( ( (b) / ) ( )) ( ( ) ( (a)

2 3 2

k d k H K K k d k H k d

i i i i i i i

      s s

NOTE: Very mild condition, immediately satisfied with K2 = K3 = 1 when we use the usual update direction given by

)) ( ( ) ( k H k d

i i i

s  

ASSUMPTION 4: There exists a positive constant K4 such that The ERROR FUNCTION satisfies

)) ( ) ( ( ) ( ) (

4

t x t x g K t x t x

j i i j i i

  

NOTE: Very mild condition, immediately satisfied with K4 = 1 when we use the common choice

) ( ) ( )) ( ) ( ( t x t x t x t x g

j i i j i i

  

Christos G. Cassandras CODES Lab. - Boston University

CONVERGENCE

CONTINUED

THEOREM: Under A1-A4, there exist positive constants α and Kδ such that

)) ( ( lim  

 

k H

k

s

Zhong and Cassandras, IEEE TAC, 2010

INTERPRETATION:

• Event-driven optimization achievable with

reduced communication requirements  energy savings

• No loss of performance

Christos G. Cassandras CODES Lab. - Boston University

CONVERGENCE

CONTINUED

THEOREM: Under A1-A4, there exist positive constants α and Kδ such that

)) ( ( lim  

 

k H

k

s

BYPRODUCT OF PROOF:

• btaining the largest possible Kδ and hence the

smallest possible number of communication events:

3 1 3 1 4

/ 2 2 ) 1 ( 1 K K K K m K B K              a a



State dim. ~ network dim. Comm. frequency

Christos G. Cassandras CODES Lab. - Boston University

COONVERGENCE WHEN DELAYS ARE PRESENT

Red curve: Black curve:

ij



t

 

k

i



 

j i i x

x g ~ ,

 

j i i x

x g ,

ij 3



ij 1



ij 2



ij 1



ij 2



ij 3



ij 4



ij 4



 

j i i x

x g ,

 

k

i

 t

ij 3



ij 2



ij 1



ij



Error function trajectory with NO DELAY

DELAY

Christos G. Cassandras CODES Lab. - Boston University

COONVERGENCE WHEN DELAYS ARE PRESENT

ASSUMPTION 5: There exists a non-negative integer D such that if a message is sent before tk-D from node i to node j, it will be received before tk.

INTERPRETATION: at most D state update events can occur between a node sending a message and all destination nodes receiving this message.

Add a boundedness assumption: THEOREM: Under A1-A5, there exist positive constants α and Kδ such that

)) ( ( lim  

 

k H

k

s

NOTE: The requirements on α and Kδ depend on D and they are tighter.

Zhong and Cassandras, IEEE TAC, 2010

SYNCHRONOUS v ASYNCHRONOUS OPTIMAL COVERAGE PERFORMANCE

Christos G. Cassandras CODES Lab. - Boston University

SYNCHRONOUS v ASYNCHRONOUS:

• No. of communication events

for a deployment problem with obstacles SYNCHRONOUS v ASYNCHRONOUS: Achieving optimality in a problem with obstacles Energy savings + Extended lifetime

OPTIMAL COVERAGE IN A MAZE

Christos G. Cassandras CODES Lab. - Boston University Zhong and Cassandras, 2008

http://www.bu.edu/codes/research/distributed-control/

DEMO: OPTIMAL DISTRIBUTED DEPLOYMENT WITH OBSTACLES – SIMULATED AND REAL

Christos G. Cassandras CODES Lab. - Boston University

IT IS HARD T IT IS HARD TO DECENTRALIZ DECENTRALIZE PROBLEM 2 …

MORE ON THAT LATER…

1. SCALABILITY

• 2. DECENTRALIZATION
• 3. COMMUNICATION
• 4. NON-CONVEXITY
• 5. EXLOIT DATA

Christos G. Cassandras CODES Lab. - Boston University

CONTROL AND OPTIMIZATION – CHALLENGES Distributed Algorithms Global optimality, escape local optima Event-driven (asynchronous) Algorithms Data-Driven Algorithms

DA DATA-DRIVEN + DRIVEN + EVENT EVENT-DRIVEN DRIVEN AL ALGORITHMS GORITHMS

DATA-DRIVEN STOCHASTIC OPTIMIZATION

CONTROL/DECISION (Parameterized by q) SYSTEM

PERFORMANCE

NOISE

Christos G. Cassandras CISE SE - CODES Lab. - Boston University

)] ( [ q L E )] ( [ max q

q

L E

• GOAL:

DIFFICULTIES: - E[L(q)] NOT available in closed form

• not easy to evaluate
• may not be a good estimate of

) (q L

• )

(q L

• )]

( [ q L E

• L(q)

GRADIENT ESTIMATOR

) (

1 n n n n

L q

• q

q

• x(t)

L(q) ∆ REAL-TIME DATA

• dt

t u t x c e E

t U t u

)) , ( ), , ( ( max

) , (

q q

• q

MDP:

• dt

t u t x c e E

t

)) , ( ), , ( ( max q q

• q

DATA-DRIVEN STOCHASTIC OPTIMIZATION IN DES: INFINITESIMAL PERTURBATION ANALYSIS (IPA)

CONTROL/DECISION (Parameterized by q) Discrete Event System (DES)

PERFORMANCE

L(q) IPA

NOISE ) (

1 n n n n

L q

• q

q

• Sample path

For many (but NOT all) DES:

• Unbiased estimators
• General distributions
• Simple on-line implementation

)] ( [ q L E

Christos G. Cassandras CISE SE - CODES Lab. - Boston University

x(t)

Model

x(t) L(q) ∆

REAL-TIME STOCHASTIC OPTIMIZATION: HYBRID SYSTEMS

CONTROL/DECISION (Parameterized by q) HYBRID SYSTEM

PERFORMANCE

L(q) IPA

) (

1 n n n n

L q

• q

q

• A general framework for an IPA theory in Hybrid Systems

Sample path Christos G. Cassandras CISE SE - CODES Lab. - Boston University

)] ( [ q L E

NOISE

x(t) L(q) ∆

Performance metric (objective function):

PERFORMANCE OPTIMIZATION AND IPA

• q

q

• q

q d d t x t x

k k

• ,

,

NOTATION:

Christos G. Cassandras CISE SE - CODES Lab. - Boston University

• T

x L E T x J ), , ( ; ), , ( ; q q q q

• IPA goal:
• q

q q d T x dJ ), , ( ;

q q

• q

q d dL

n n n n

) (

1

• q

q d dL

• Obtain unbiased estimates of

, normally

• Then:

THE IP THE IPA CAL A CALCUL CULUS US

IPA: THREE FUNDAMENTAL EQUATIONS

1. Continuity at events: Take d/dq : ) ( ) (

• k

k

x x

• k

k k k k k k

f f x x ' )] ( ) ( [ ) ( ' ) ( '

1

• If no continuity, use reset condition

q

• d

x q q d x

k

) , , , , ( ) ( '

• Christos G. Cassandras

CISE SE - CODES Lab. - Boston University

System dynamics over (k(q), k+1(q)]:

) , , ( t x f x

k

q

• q

q

• q

q

• k

k

t x t x , ,

NOTATION:

IPA: THREE FUNDAMENTAL EQUATIONS

Solve

• ver (k(q), k+1(q)]:

q

• )

( ) ( ' ) ( ) ( ' t f t x x t f dt t dx

k k

• t

k du x u f k du x u f

k v k k t k k

x dv e v f e t x

• q
• )

( ) ( ) (

) ( ) (

initial condition from 1 above

Christos G. Cassandras CISE SE - CODES Lab. - Boston University

• 2. Take d/dq of system dynamics
• ver (k(q), k+1(q)]:

) , , ( t x f x

k

q

• q
• )

( ) ( ' ) ( ) ( ' t f t x x t f dt t dx

k k

NOTE: If there are no events (pure time-driven system), IPA reduces to this equation

• 3. Get

depending on the event type:

IPA: THREE FUNDAMENTAL EQUATIONS

k

• Exogenous event: By definition,
• k
• )

), , ( (

• q
• q

k k x

g

• Endogenous event: occurs when
• )

( ) (

1 k k k k

x x g g f x g

• q
• Induced events:

) ( ) (

1

• k

k k k k

y t y

• Christos G. Cassandras

CISE SE - CODES Lab. - Boston University

Ignoring resets and induced events:

IPA: THREE FUNDAMENTAL EQUATIONS

k k k k k k k

f f x x ' )] ( ) ( [ ) ( ' ) ( '

1

• t

k du x u f k du x u f

k v k k t k k

x dv e v f e t x

• q
• )

( ' ) ( ) (

) ( ) (

• k
• )

( ) (

1 k k k k

x x g g f x g

• q
• r

1. 2. 3.

Christos G. Cassandras CISE SE - CODES Lab. - Boston University

2 1 3

) ( '

• k

x

• q

q

• q

q

• k

k

t x t x ,

Recall:

Cassandras et al, Europ. J. Control, 2010

IPA PROPERTIES

Back to performance metric:

• N

k k

k k

dt t x L L

1

) , , (

• q

q

• q

q q

• t

x L t x L

k k

, , , ,

NOTATION: Then:

• N

k k k k k k k k

k k

dt t x L L L d dL

1 1

1

) , , ( ) ( ) (

• q
• q

q What happens at event times What happens between event times

Christos G. Cassandras CISE SE - CODES Lab. - Boston University

IPA PROPERTY 1: ROBUSTNESS

THEOREM 1: If either 1,2 holds, then dL(q)/dq depends only on information available at event times k:

• 1. L(x,q,t) is independent of t over [k(q), k+1(q)] for all k
• 2. L(x,q,t) is only a function of x and for all t over [k(q), k+1(q)]:
• q

k k k

f dt d x f dt d x L dt d

IMPLICATION: - Performance sensitivities can be obtained from information limited to event times, which is easily observed

• No need to track system in between events !
• N

k k k k k k k k

k k

dt t x L L L d dL

1 1

1

) , , ( ) ( ) (

• q
• q

q

Christos G. Cassandras CISE SE - CODES Lab. - Boston University

IPA PROPERTY 1: ROBUSTNESS

EXAMPLE WHERE THEOREM 1 APPLIES (simple tracking problem):

Christos G. Cassandras CISE SE - CODES Lab. - Boston University

k k k k k k k

d du f a x f q q

• ,

N k t w u t x a x dt g t x E

k k k k k k T

, , 1 ) ( ) ( ) ( s.t. )] ( ) ( [ min

,

• q
• q

1

• x

L

NOTE: THEOREM 1 provides sufficient conditions only. IPA still depends on info. limited to event times if for “nice” functions uk(qk,t), e.g., bkqt N k t w t u t x a x

k k k k k k

, , 1 ) ( ) , ( ) (

• q

IPA PROPERTY 1: ROBUSTNESS

EVENTS

Evaluating requires full knowledge of w and f values (obvious)

) ; ( q t x

However, may be independent of w and f values (NOT obvious)

q q d t dx ) ; (

It often depends only on: - event times k

• possibly

) (

1

• k

f ) ; , , , ( q t w u x f x

• k

k+1

Christos G. Cassandras CISE SE - CODES Lab. - Boston University

IPA PROPERTY 2: DECOMPOSABILITY

THEOREM 2: Suppose an endogenous event occurs at k with switching function g(x,q). If , then is independent of fk−1. If, in addition, then IMPLICATION: Performance sensitivities are often reset to 0 sample path can be conveniently decomposed

) (

• k

k

f

• q

d dg ) (

• k

x ) (

• k

x

Christos G. Cassandras CISE SE - CODES Lab. - Boston University

IPA PROPERTY 3: SCALABILITY

IPA scales with the EVENT SET, not the STATE SPACE !

Christos G. Cassandras CISE SE - CODES Lab. - Boston University

As a complex system grows with the addition of more states, the number of EVENTS often remains unchanged or increases at a much lower rate. EXAMPLE: A queueing network may become very large, but the basic events used by IPA are still “arrival” and “departure” at different nodes. IPA estimators are EVENT-DRIVEN

IPA PROPERTIES

• No need for a detailed model (captured by fk) to describe state behavior

in between events

• This explains why simple abstractions of a complex stochastic system

can be adequate to perform sensitivity analysis and optimization, as long as event times are accurately observed and local system behavior at these event times can also be measured.

• This is true in abstractions of DES as HS since:

Common performance metrics (e.g., workload) satisfy THEOREM 1 In many cases:

Christos G. Cassandras CISE SE - CODES Lab. - Boston University

TOO CLOSE…

too much undesirable detail

TOO FAR…

model not detailed enough

WHAT IS THE RIGHT ABSTRACTION LEVEL ?

JUST RIGHT…

good model

CREDIT: W.B. Gong

Christos G. Cassandras CISE SE - CODES Lab. - Boston University

A SMAR A SMART CITY T CITY CPS APPLICA CPS APPLICATION: TION:

AD ADAPTIVE APTIVE TRAFFIC TRAFFIC LIGHT LIGHT CONTR CONTROL OL

TRAFFIC LIGHT CONTROL - BACKGROUND

Christos G. Cassandras CODES Lab. - Boston University

• Mixed Integer Linear Programming (MILP) [Dujardin et al, 2011]
• Extended Linear Complementarity Problem (ELCP) [DeSchutter, 1999]
• MDP and Reinforcement Learning [Yu et al., 2006]
• Game Theory [Alvarez et al., 2010]
• Evolutionary algorithms [Taale et al., 1998]
• Fuzzy Logic [Murat et al., 2005]
• Expert Systems [Findler and Stapp, 1992]
• Perturbation Analysis

A basic binary switching control (GREEN – RED) problem with a long history…

TRAFFIC LIGHT CONTROL - BACKGROUND

Christos G. Cassandras CODES Lab. - Boston University

• Perturbation Analysis [Panayiotou et al., 2005]

Use a Hybrid System Model: Stochastic Flow Model (SFM)

SFM DES

[Geng and Cassandras, 2012]

Single Intersection

Vehicle queue

Aggregate states into modes and keep only events causing mode transitions

SINGLE-INTERSECTION MODEL

Christos G. Cassandras CISE - CODES Lab. - Boston University

Traffic light control:

] , , , [

4 3 2 1

q q q q q

GREEN light cycle at queue n = 1,2,3,4 OBLECTIVE: Determine q to minimize total weighted vehicle queues

• 4

1

) , ( 1 ) ( min

n T n n T

dt t x w E T J q q

q

1 2 3 4

SINGLE-INTERSECTION MODEL

Christos G. Cassandras CISE - CODES Lab. - Boston University

• )

( 1 ) , ( 1 ) ( min

4 1

q q q

q T n T n n T

L E T dt t x w E T J

• IPA APPROACH:
• Observe events and event times, estimate

through

• Then,

q q d dJT

q q

• q

q d dL

n T n n n

) (

1

• q

q d dLT

HYBRID SYSTEM STATE DYNAMICS

Christos G. Cassandras CISE - CODES Lab. - Boston University

GREEN n n n

n

q

n

q

GREEN n ( ) ( )

n n n

z t if z t q

• 1

( ) ( ) ( )

n n n n n

if z t

• r z t

z t

• therwise

q q

• GREEN light “clock”

Control: GREEN light cycle

HYBRID SYSTEM STATE DYNAMICS

Christos G. Cassandras CISE - CODES Lab. - Boston University

( ) ( )

n n n

z t if z t q

• 1

( ) ( ) ( )

n n n n n

if z t

• r z t

z t

• therwise

q q

• 1

( ) ( ) ( )

n n n n n

if z t

• r z

t G t

• therwise

q q

• GREEN at queue n

Define: ( ) ( , ) ( ) ( ) ( ) ( ) ( ) ( )

n n n n n n n n

t if G z x t if x t and t t t t

• therwise
• q
• Queue content

Vehicle departure rate process Vehicle arrival rate process

[RESOURCE DYNAMICS] [USER DYNAMICS]

EVENTS IN THE TLC MODEL

Christos G. Cassandras CISE - CODES Lab. - Boston University

Event E Non-Empty-Period (NEP) ends endogenous Event R2G RED light switches to GREEN endogenous Event G2R GREEN light switches to RED endogenous Event S Non-Empty-Period (NEP) starts endogenous or exogenous

APPLY IPA EQUATIONS FOR q AND s VECTORS

Christos G. Cassandras CISE - CODES Lab. - Boston University

FOR EXAMPLE: Endogenous event with

) , ( ) ), , ( (

• t

x x g

n k k

q q

• q

) ( ) ( ) (

' , ' , k n k n k i n i k

x

• )

( ) ( ) ( ) ( ) ( ) ( ) (

' , ' , ' ,

• k

n k n k i n k n k n k i n k i n

x x x

• Perturbation in queue n

RESET to 0 when NEP ends

k

• )

( ) (

1 k k k k

x x g g f x g

• q
• k

k k k k k k

f f x x ' )] ( ) ( [ ) ( ' ) ( '

1

COST DERIVATIVE IN mth NEP

Christos G. Cassandras CISE - CODES Lab. - Boston University

• )

( ) ( ,

, ,

) , (

q

• q
• q

m n m n

dt t x L

n m n

NOTES: - Need only TIMERS, COUNTERS and state derivatives

• Scaleable in number of EVENTS – not states!

TYPICAL SIMULATION RESULTS

Christos G. Cassandras CISE - CODES Lab. - Boston University

9-fold cost reduction

Traffic pattern changes

Adaptivity

IT IS HARD T IT IS HARD TO DECENTRALIZ DECENTRALIZE PROBLEM 2 …

• T

t

dt dx x R t u x P J

) (

) ( ))) ( ( , ( max s

u

N i F t si , , 1 , ) (

• x

i

a

a1

a2

a3

O1 O2

Christos G. Cassandras CODES Lab. - Boston University

MULTI-AGENT OPTIMIZATION: PROBLEM 2

Ω GOAL: Find the best state trajectories si(t), 0 ≤ t ≤ T so that agents achieve a maximal reward from interacting with the mission space

N i t u s f s

i i i i

, , 1 ), , , (

• May also have dynamics

Christos G. Cassandras CODES Lab. - Boston University

PERSISTENT MONITORING PROBLEM

Need three model elements:

• 1. ENVIRONMENT MODEL
• 2. SENSING MODEL

(how agents interact with environment)

• 3. AGENT MODEL

N i t u s f s

i i i i

, , 1 ), , , (

• T

t

dt dx x R t u x P J

) (

) ( ))) ( ( , ( max s

u

GOAL: Find the best state trajectories si(t), 0 ≤ t ≤ T so that agents achieve a maximal reward from interacting with the mission space

Christos G. Cassandras CODES Lab. - Boston University

PERSISTENT MONITORING PROBLEM

Start with 1-dimensional mission space = [0,L] AGENT DYNAMICS:

1 ) ( ,

• t

u u s

j j j

• Analysis still holds for:

1 ) ( , ) (

• t

u bu s g s

j j j j j

Christos G. Cassandras CODES Lab. - Boston University

PERSISTENT MONITORING PROBLEM

s(t) x SENSING MODEL: p(x,s) Probability agent at s senses point x

Christos G. Cassandras CODES Lab. - Boston University

PERSISTENT MONITORING PROBLEM

s(t) x ENVIRONMENT MODEL: Associate to x Uncertainty Function R(x,t)

noise t s R f t R

x x

• )

, , ( ) (

• If x is a known “target”:
• therwise

)) ( , ( ) ( )) ( , ( ) ( , ) , ( if ) , ( t s x Bp x A t s x Bp x A t x R t x R

• Use:

Christos G. Cassandras CODES Lab. - Boston University

PERSISTENT MONITORING PROBLEM

…

Partition mission space = [0,L] into M intervals:

1 M

For each interval i = 1,…,M define Uncertainty Function Ri(t):

• therwise

)) ( ( )) ( ( , ) ( if ) (

i

t BP A t BP A t R t R

i i i i i

s s

• N

j j i i

s p P

1

) ( 1 1 ) (s

where Pi(s) = joint prob. i is sensed by agents located at s = [s1,…,sN]

) , ( ) (

j i j j i

s p s p

Christos G. Cassandras CODES Lab. - Boston University

PERSISTENT MONITORING (PM) WITH KNOWN TARGETS

s.t. ) ( 1 min

1 , ,

1

• T M

i i u u

dt t R T J

N

• L

b t s a t u u s

j j j j

• )

( , 1 ) ( ,

• therwise

)) ( ( )) ( ( , ) ( if ) (

i

t BP A t BP A t R t R

i i i i i

s s

Agent Network (time-varying) Agent-Target Interaction Network (time-varying)

Christos G. Cassandras CODES Lab. - Boston University

PERSISTENT MONITORING WITH KNOWN TARGETS

Hard to decentralize a controller that involves time-varying agent-environment interactions

Christos G. Cassandras CODES Lab. - Boston University

THREE TYPES OF NEIGHBORHOODS

(conventional)

𝐵5 𝐵2 𝐵3 𝐵1 𝐵4

𝑈2 𝑈

1

𝑈3 𝑈5 𝑈

4

Christos G. Cassandras CODES Lab. - Boston University

PM WITH KNOWN TARGETS – 1D CASE

• 1. Optimal Trajectories are bounded:

N j x t s x

M j

, , 1 ) (

* 1

• 2. Existence of finite dwell times at target on optimal trajectories:

Under certain conditions:

] , [ for ) ( and ) (

2 1 * *

t t t t u x t s

j k j

• 3. Under the constraint sj(t) < sj+1(t), on an optimal trajectory:

) ( ) (

1 t

s t s

j j

• We have shown that:

Zhou et al, IEEE CDC, 2016

Christos G. Cassandras CODES Lab. - Boston University

OPTIMAL CONTROL SOLUTION

Optimal trajectory is fully characterized by TWO parameter vectors:

• N

j

jS j j

, , 1 ,

1

• q

q q

• N

j w w w

jS j j

, , 1 ,

1

• Switching points

Waiting times at switching points, wjk ≥ 0

• K

k M i i

k k

dt t R T J

) , ( ) , ( 1

1

) ( 1 ) , (

w θ w θ

w θ

• k : kth event time

Under optimal control, this is a HYBRID SYSTEM

Christos G. Cassandras CODES Lab. - Boston University

HYBRID SYSTEM EVENTS

Type 1: switches in 𝑆𝑗(𝑢) Type 2: switches in agent sensing Type 3: switches in 𝑡

𝑘(𝑢)

Type 4: changes in neighbor sets

Christos G. Cassandras CODES Lab. - Boston University

HYBRID SYSTEM EVENTS: EXAMPLE

A simple example: 1 agent 1 target

𝑆𝑗 = 𝐵𝑗 − 𝐶𝑗𝑄𝑗(𝐭(𝑢))

𝑡𝑘 = −1

𝑆𝑗 = 𝐵𝑗 − 𝐶𝑗𝑄𝑗(𝐭(𝑢))

𝑡𝑘 = 1

𝑆𝑗 = 𝐵𝑗 − 𝐶𝑗𝑄𝑗(𝐭(𝑢))

𝑡𝑘 = 0 𝑆𝑗 = 0 𝑡𝑘 = −1 𝑆𝑗 = 0 𝑡𝑘 = 0 𝑆𝑗 = 0 𝑡𝑘 = 1

• therwise

)) ( ( )) ( ( , ) ( if ) (

i

t BP A t BP A t R t R

i i i i i

s s

• Event type 1

𝑆 ↓= 0 𝑆 ↑= 0

1 ) ( ,

• t

u u s

j j j

• Event type 2

𝑣 = −1 1

Christos G. Cassandras CODES Lab. - Boston University

IPA GRADIENTS

• M

i K k i

dt t R T J

k k

1 ) , ( ) , (

) ( 1 ) , (

1

w θ w θ

w θ

• Objective function gradient:

T i i i

t R t R t R

• w

θ ) ( ) ( ) (

) (t Ri

• where

is obtained using the IPA Calculus

k k k k k k k

f f x x ' )] ( ) ( [ ) ( ' ) ( '

1

• t

k du x u f k du x u f

k v k k t k k

x dv e v f e t x

• q
• )

( ' ) ( ) (

) ( ) (

• k
• )

( ) (

1 k k k k

x x g g f x g

• q
• r

1. 2. 3.

is updated on an EVENT-DRIVEN basis k : kth event time

) (t Ri

Christos G. Cassandras CODES Lab. - Boston University

AGENT AND TARGET EVENTS

AGENT Event Set ℰ𝐵 TARGET Event Set ℰ𝑈

Christos G. Cassandras CODES Lab. - Boston University

LOCAL EVENT SETS FOR AGENTS

ℰ𝑘

𝐵: Subset of ℰ𝐵 that contains only events related to agent j

ℰ𝑘

𝑈: Subset of ℰ𝑈 that contains only events related to agent j

Christos G. Cassandras CODES Lab. - Boston University

HOW CAN WE DECENTRALIZE ?

DECENTRALIZATION: Each agent should be able to evaluate …based only on LOCAL events (i.e., events it can observe)

• M

i K k i

dt t R T J

k k

1 ) , ( ) , (

) ( 1 ) , (

1

w θ w θ

w θ

• Can this gradient be evaluated by every agent j using ONLY

local events in ℰ𝑘 (t) ?

Christos G. Cassandras CODES Lab. - Boston University

“ALMOST DECENTRALIZATION”

THEOREM: Any centralized solution of the trajectory optimization problem can be recovered through In which each agent j optimizes its trajectory under the following conditions: 1. Initial trajectory parameters

• 2. The LOCAL information set
• 3. The subset of the GLOBAL information set

) , (

j j w

θ

Christos G. Cassandras CODES Lab. - Boston University

“ALMOST DECENTRALIZATION”: EXAMPLE

“Almost decentralized” solution, J* = 37.38 Fully decentralized solution (ignorinig non-local events), J* = 41.66 Red: true state of target 3 Blue: state of target 3 observed by agent 1 when in its neighborhood Green dots: instants when agent 1 receives non-local events

1. SCALABILITY

• 2. DECENTRALIZATION
• 3. COMMUNICATION
• 4. NON-CONVEXITY
• 5. EXLOIT DATA

Christos G. Cassandras CODES Lab. - Boston University

CONTROL AND OPTIMIZATION – CHALLENGES Distributed Algorithms Global optimality, escape local optima Event-driven (asynchronous) Algorithms Data-Driven Algorithms

Random

Christos G. Cassandras CODES Lab. - Boston University

LOCAL OPTIMUM EXAMPLE

) (

• 1

s i i

s H s

At a local optimum s1

Christos G. Cassandras CODES Lab. - Boston University

“BOOSTING FUNCTION” IDEA

Alter Hi(s) to

) ( ˆ

• 1

s i i

s H s

) ( ˆ s

i

H

NOTE: Hard to find the proper try altering directly

ˆ Hi(s)

i i

s H

• )

(s

di(x) = x - si

• i

ij i

j z i ij ij ij jx s V i x i i ix i

rdr s r w D n dx x d s x s x w s s H

2 ) ( 1

) ), ( ( sin ) sgn( + ) ( ) ( ) , ( ) (

• q

¶Hi(s) ¶siy = w1(x,si)(x - si)y di(x) dx

V (si )

ò

+ sgn(njy)cosqij Dij w2(r ij(r),si)rdr

zij

ò

jÎ G

i

å

w1(x,si) =- R(x)F i(x) dpi(x,si) ddi(x)

Christos G. Cassandras CODES Lab. - Boston University

PARTIAL DERIVATIVE STRUCTURE

Zhong and Cassandras, Proc. IFAC World Congress 2008

w2(x,si) =- R(x)F i(x)pi(x,si)

F i(x) = [1- ˆ pk(x,sk)]

kÎ Bi

Õ

joint probability that a point is not detected by any neighbor of i

x

w1(x,si)

x x

w1(x,si) w2(x,si)

) , ( ) , ( ) , ( ) , ( ˆ ) , ( ) , ( ) , ( ) , ( ˆ

2 2 2 2 1 1 1 1

s s s s x s x w x s x w x s x w x s x w

i i i i

• ¶Hi(s)

¶six = w1(x,si)(x - si)x di(x) dx

V (si )

ò

• +

sgn(njx)sinqij Dij w2(r ij(r),si)rdr

zij

ò

jÎ G

i

å

BOOSTING FUNCTION: Transform the derivative so its value is ≠ 0 and provides a “boost” towards more likely optimum

ˆ w1(x,si)= gi(w1(x,si)) ˆ w2(x,si)=hi(w2(x,si))

Focus on linear forms:

Christos G. Cassandras CODES Lab. - Boston University

BOOSTING FUNCTION APPROACH

• 1. P-boosting function
• 2. Neighbor-boosting function
• 3. F-boosting function

Christos G. Cassandras CODES Lab. - Boston University

THREE BOOSTING FUNCTIONS

Assign higher weights for low-coverage points

) , ( ) , ( ˆ ) , ( ) , ( ) , ( ˆ

2 2 1 1 i i i i

s x w s x w s x w x kP s x w

• s

¶ ˆ Hi(s) ¶six = ˆ w1(x,si)(x - si)x di(x) dx

V (si )

ò

• +

sgn(njx)sinqij Dij ˆ w2(r ij(r),si)rdr

zij

ò

jÎ G

i

å

b1(x,s) = 0 a 2(x,s) =1 b2(x,s) = 0 a1(x,s) =kP(x,s)- g

P(x,s) =1- P i=1

N [1- ˆ

pi(x,si)]

Boosted derivative:

Christos G. Cassandras CODES Lab. - Boston University

P-BOOSTING FUNCTION

a 2(x,s) =1 b2(x,s) = 0

b1(x,s) = d

jÎ Bi

å

x - sj

( )

kj si - x

g

a1(x,s) =1

• )

, ( ) , ( ˆ ) , ( ) , ( ˆ

2 2 1 1 i i i j j B j i i

s x w s x w x s k s x s x w s x w

i

• ¶ ˆ

Hi(s) ¶six = ˆ w1(x,si)(x - si)x di(x) dx

V (si )

ò

+ sgn(njx)sinqij Dij ˆ w2(r ij(r),si)rdr

zij

ò

jÎ G

i

å

Christos G. Cassandras CODES Lab. - Boston University

NEIGHBOR-BOOSTING FUNCTION Add repelling forces from agent’s neighbors Boosted derivative:

ˆ w1(x,si) = kF i(x)gw1(x,si) ˆ w2(x,si) = w2(x,si)

• i

ij i

j z i ij ij ij jx s V i x i i ix i

rdr s r w D n dx x d s x s x w s s H

2 ) ( 1

) ), ( ( ˆ sin ) sgn( + ) ( ) ( ) , ( ˆ ) ( ˆ

• q

b1(x,s) = 0 a 2(x,s) =1 b2(x,s) = 0

• F

i

B k k k i

s x p x )] , ( ˆ 1 [ ) (

a1(x,s) =kF i(x)g

Christos G. Cassandras CODES Lab. - Boston University

F-BOOSTING FUNCTION Boost weights for points poorly covered by agent’s neighbors Boosted derivative:

xmsun@bu.edu Xinmiao Sun CODES LAB 60

BOOSTING PROCESS EXAMPLE

Christos G. Cassandras CODES Lab. - Boston University

COVERAGE EXAMPLE: SIMULATED v REAL

Christos G. Cassandras CODES Lab. - Boston University

RELATED APPROACHES FOR GLOBAL OPTIMALITY Simulated Annealing:

• Random perturbations for escaping local optima
• Can reach global optimum but very slow

Multistarts, Stochastic Comparison Algorithm (SCA):

• Random initial points
• SCA can reach global optimum but very slow

Submodularity, greedy algorithms:

• If H(s) submodular, can obtain bounds

– sometimes very tight !

T k F k F T S T f k T f S f k S f

• ,

, any for ) ( }) { ( ) ( }) { (

f : 2N → R submodular if

SUMMARY, RESEARCH DIRECTIONS

Christos G. Cassandras CODES Lab. - Boston University

Small, cheap cooperating devices cannot handle complexity we need DISTRIBUTED control and optim. algorithms Cooperating agents operate autonomously (asynchronously) we need ASYNCHRONOUS (EVENT-DRIVEN) control/optimization schemes Too much communication kills node energy sources communicate ONLY when necessary we need EVENT-DRIVEN control/optimization schemes Networks grow large, sensing tasks grow large we need SCALABLE control and optim. algorithms