CHALLENGES CHALLENGES C. G . G. Cassand . Cassandras as - - PowerPoint PPT Presentation

• C. G

. G. Cassand . Cassandras as

Division of Systems Engineering Center for Information and Systems Engineering Boston University

Christos G. Cassandras

CODES Lab. - Boston University

CYBER CYBER-PHY PHYSICAL SY SICAL SYSTEMS: STEMS: MO MOTIV TIVATION TION AND AND CHALLENGES CHALLENGES

CYBER-PHYSICAL SYSTEMS

Christos G. Cassandras CISE SE - CODES Lab. - Boston University

INTERNET

CYBER PHYSICAL

Data collection: relatively easy… Control: a challenge…

THE “INTERNET OF THINGS”

Decision Making Data collection

Energy Management

Safety

Security

Control and Optimization Actions Information Processing Privacy SENSO NSOR NETWO TWORKS RKS BI BIG DAT ATA

“SMART CITY” AS A CYBER-PHYSICAL SYSTEM

Christos G. Cassandras CISE SE - CODES Lab. - Boston University

Decision Making Data collection Control and Optimization Actions Information Processing

“SMART CITY” AS A CYBER-PHYSICAL SYSTEM

Christos G. Cassandras CISE SE - CODES Lab. - Boston University

CYBER PHYSICAL CYBER x(t)

Model

PHYSICAL

Model t x(t)

) , , ( t u x f x  

COLLECTING DATA IS NOT “SMART”

• JUST A NECESSARY STEP TO

BEING “SMART” PROCESSING DATA TO MAKE GOOD DECISIONS IS “SMART”

INFO INFO ACTION

Christos G. Cassandras CISE SE - CODES Lab. - Boston University

WHAT IS REALLY “SMART” ?

MODELING: MODELING:

TIMED TIMED-DRIVEN DRIVEN vs vs EVENT EVENT-DRIVEN DRIVEN

STATES s1 s2 s3 s4 TIME t

TIME-DRIVEN SYSTEM

STATES TIME t

STATE SPACE:

X  

DYNAMICS:

 

 , x f x t 

EVENT-DRIVEN SYSTEM

STATE SPACE:

 

X s s s s 

1 2 3 4

, , ,

DYNAMICS:

 

e x f x , '

t2 e2 x(t) t3 t4 t5 e3 e4 e5 EVENTS x(t) t1 e1

Christos G. Cassandras CODES Lab. - Boston University

TIME-DRIVEN v EVENT-DRIVEN SYSTEMS

TIME-DRIVEN v EVENT-DRIVEN CONTROL

Christos G. Cassandras CODES Lab. - Boston University

REFERENCE

PLANT CONTROLLER

INPUT

• +

SENSOR

MEASURED OUTPUT OUTPUT ERROR REFERENCE

PLANT CONTROLLER

INPUT

• +

SENSOR

MEASURED OUTPUT OUTPUT ERROR

EVENT:

g(STATE) ≤ 0 EVENT-DRIVEN CONTROL: Act only when needed (or on TIMEOUT) - not based on a clock

SELECTED REFERENCES - EVENT-DRIVEN CONTROL

Christos G. Cassandras CODES Lab. - Boston University

• Astrom, K.J., and B. M. Bernhardsson, “Comparison of Riemann and Lebesgue sampling for

first order stochastic systems,” Proc. 41st Conf. Decision and Control, pp. 2011–2016, 2002.

• T. Shima, S. Rasmussen, and P. Chandler, “UAV Team Decision and Control using Efficient

Collaborative Estimation,” ASME J. of Dynamic Systems, Measurement, and Control, vol. 129,

• no. 5, pp. 609–619, 2007.
• Heemels, W. P. M. H., J. H. Sandee, and P. P. J. van den Bosch, “Analysis of event-driven

controllers for linear systems,” Intl. J. Control, 81, pp. 571–590, 2008.

• P. Tabuada, “Event-triggered real-time scheduling of stabilizing control tasks,” IEEE Trans.
• Autom. Control, vol. 52, pp. 1680–1685, 2007.
• J. H. Sandee, W. P. M. H. Heemels, S. B. F. Hulsenboom, and P. P. J. van den Bosch, “Analysis

and experimental validation of a sensor-based event-driven controller,” Proc. American Control Conf., pp. 2867–2874, 2007.

• J. Lunze and D. Lehmann, “A state-feedback approach to event-based control,” Automatica, 46,
• pp. 211–215, 2010.
• P. Wan and M. D. Lemmon, “Event triggered distributed optimization in sensor networks,”
• Proc. of 8th ACM/IEEE Intl. Conf. on Information Processing in Sensor Networks, 2009.
• Zhong, M., and Cassandras, C.G., “Asynchronous Distributed Optimization with Event-Driven

Communication”, IEEE Trans. on Automatic Control, AC-55, 12, pp. 2735-2750, 2010.

REASONS FOR EVENT-DRIVEN MODELS, CONTROL, OPTIMIZATION

Christos G. Cassandras CODES Lab. - Boston University

• Many systems are naturally Discrete Event Systems (DES)

(e.g., Internet) → all state transitions are event-driven

• Most of the rest are Hybrid Systems (HS)

→ some state transitions are event-driven

• Many systems are distributed

→ components interact asynchronously (through events)

• Many systems are wirelessly networked → energy constrained

→ time-driven communication consumes significant energy

REASONS FOR EVENT-DRIVEN MODELS, CONTROL, OPTIMIZATION

Christos G. Cassandras CODES Lab. - Boston University

• Many systems are stochastic

→ actions needed in response to random events

• Event-driven methods provide significant advantages in

computation and estimation quality

• Time-driven sampling inherently inefficient (“open loop” sampling)
• System performance is often more sensitive to event-driven

components than to time-driven components

SYNCHRONOUS v ASYNCHRONOUS BEHAVIOR

Christos G. Cassandras CODES Lab. - Boston University

Wasted clock ticks More wasted clock ticks Even more wasted clock ticks

…

INCREASING TIME GRANULARITY

Indistinguishable events

x + y x y x y

TIME

Time-driven (synchronous) implementation:

• Sum repeatedly evaluated unnecessarily
• When evaluation is actually needed, it is done at the wrong times !

TIME

t1 t2

SYNCHRONOUS v ASYNCHRONOUS COMPUTATION

Christos G. Cassandras CODES Lab. - Boston University

MUL MULTI TI-AGENT GENT NETW NETWORK SY ORK SYSTEMS STEMS

The multi-agent system framework consists of a team

• f autonomous agents cooperating to carry out

complex tasks within a given environment. Applications: – Monitoring (data sources/targets) – Search and rescue – Smart buildings – Intelligent transportation – Formation flight of Unmanned Aerial Vehicles COOPERATIVE MULTI-AGENT SYSTEMS

Christos G. Cassandras CODES Lab. - Boston University





 dx x R x P H ) ( ) , ( ) ( max s s

s

N i F si , , 1 ,     

• R(x): property of point x
• P(x, s): reward function
• Oj: obstacle (constraint)

x

i

a

a1

a2

a3

O1 O2

• si: agent state, i = 1,…, N

s=[s1, … , sN ]

Christos G. Cassandras CODES Lab. - Boston University

MULTI-AGENT OPTIMIZATION: PROBLEM 1

Ω GOAL: Find the best state vector s=[s1, … , sN ] so that agents achieve a maximal reward from interacting with the mission space

 





T t

dt dx x R t u x P J

) (

) ( ))) ( ( , ( max s

u

N i F t si , , 1 , ) (     

x

i

a

a1

a2

a3

O1 O2

Christos G. Cassandras CODES Lab. - Boston University

MULTI-AGENT OPTIMIZATION: PROBLEM 2

Ω GOAL: Find the best state trajectories si(t), 0 ≤ t ≤ T so that agents achieve a maximal reward from interacting with the mission space

N i t u s f s

i i i i

, , 1 ), , , (    

May also have dynamics

PR PROBLEMS OBLEMS THA THAT FIT THIS T FIT THIS FRAMEW FRAMEWORK ORK

Christos G. Cassandras CODES Lab. - Boston University

COVERAGE CONTROL: ACTIVE COOPERATION





 dx x R x P H ) ( ) , ( ) ( max s s

s

Event density: Prior estimate of event

• ccurrence frequency

Joint event detection probability:

 





  

N i i i

s x p x P

1

) , ( 1 1 ) , ( s

Event sensing probability



R(x) (Hz/ m2)

? ? ? ?? ? ? ? ? Deploy sensors to maximize “event” detection probability - unknown event locations

Christos G. Cassandras CODES Lab. - Boston University

COVERAGE CONTROL: VORONOI PARTITIONING





 

N i V i

i

dx x R s x f H

1

) ( ) ( ) ( max s

s

 

i j s x s x x V

j i i

       , :

quality sensing : ) (

i

s x f 





 dx x R x P H ) ( ) , ( ) ( max s s

s

frequency

• ccurrence

event : ) (x R







N i i i

s x p x P

1

) , ( ) , ( s

      

i i i i i

V x V x s x f s x p ) ( ) , (

Christos G. Cassandras CODES Lab. - Boston University

COVERAGE CONTROL:

ACTIVE COOPERATION vs PARTITIONING Voronoi patition; Optimal obj. function = 1346.5 Gradient-based cooperative algorithm; Optimal obj. function = 1388.1

Christos G. Cassandras CODES Lab. - Boston University

CONSENSUS





 

i

N j i j i

t s t s t s ) ( ) ( ) ( 

) (

1 t

s

Ω

) (

2 t

s ) (

3 t

s ) (

4 t

s

N

s s  

1





  dx x R x P H ) ( ) , ( ) ( max s s

s





 

N i i

s x x R

1

) ( ) ( 1

Only x that matter are agents





 

N i i

s P H

1

) , ( ) ( max s s

s

        

i i i j i j i

i j N j s s s s p

• therwise

, ) , (

2







i

N j i j i i

s s p s s P ) , ( 2 1 ) , (

Christos G. Cassandras CODES Lab. - Boston University

COVERAGE CONTROL v PERSISTENT MONITORING

COVERAGE CONTROL: Deploy sensors to maximize “event” detection probability – unknown event locations – event sources may be mobile – sensors may be mobile

Perceived event density (data sources) over given region (mission space)

5 10 2 4 6 8 10 10 20 30 40 50



R(x) (Hz/m2)

? ? ? ? ? ? ? ? ?

Christos G. Cassandras CODES Lab. - Boston University

COVERAGE CONTROL v PERSISTENT MONITORING

PERSISTENT MONITORING: – environment cannot be fully covered by stationary team of agents – all areas of mission space must be visited infinitely often – minimize some measure of overall uncertainty



? ? ? ? ? ? ? ? ?

• 2. Once a Data Source is detected, collect data from it,

track it if mobile ? ? ? ? ? ? ?

COVERAGE CONTROL + PERSISTENT MONITORING

Christos G. Cassandras CODES Lab. - Boston University

• 3. Continue to seek data sources while collecting data from

detected sources ? ? ? ? ? ? ? ? ?

• 1. Seek and detect “Data Sources”

(or “Targets”)

Christos G. Cassandras CODES Lab. - Boston University

REACTING TO EVENT DETECTION

Important to note: There is no external control causing this

• behavior. Algorithm

includes tracking functionality automatically

RELATED WORK

Christos G. Cassandras CODES Lab. - Boston University

Coverage control:

• J. Cortes, S. Martinez, T. Karatas, and F. Bullo, “Coverage control for mobile sensing

networks,” IEEE Trans. on Robotics and Automation, 2004.

• M. Zhong and C. G. Cassandras, “Distributed coverage control and data collection with mobile sensor

networks,” IEEE Trans. Autom. Control, 2011.

• W. Burgard, M. Moors, C. Stachniss, and F. E. Schneider, “Coordinated multi-robot exploration,” IEEE
• Trans. On Robotics, 2005.
• I. Rekleitis, V. Lee-Shue, A. New, and H. Choset, “Limited communication, multi-robot team based

coverage,” Proc. ICRA’04, 2004.

• S. L. Smith, M. Schwager, and D. Rus, “Persistent monitoring of changing environments using robots with

limited range sensing,” IEEE Trans. on Robotics, 2011.

• P. Hokayem, D. Stipanovic, and M. Spong, “On persistent coverage control,” Proc. 46th IEEE Conf.

Decision and Control, 2007.

• Y. Elmaliach, N. Agmon, and G. Kaminka, “Multi-robot area patrol under frequency constraints,” Proc.

ICRA’07, 2007.

• N. Nigam and I. Kroo, “Persistent surveillance using multiple unmanned air vehicles,” Proc. IEEE

Aerospace Conference, 2008.

• Y. Chen, K. Deng, and C. Belta, “Multi-agent persistent monitoring in stochastic environments with

temporal logic constraints,” Proc. 51stIEEE Conf. Decision and Control, 2012.

• C. G. Cassandras, X. Lin, and X. C. Ding, “An optimal control approach to the multi-agent persistent

monitoring problem,” IEEE Trans. Autom. Control, 2013.

Persistent monitoring/surveillance:

Christos G. Cassandras CODES Lab. - Boston University

PERSISTENT MONITORING PROBLEM

Need three elements:

• 1. ENVIRONMENT MODEL
• 2. SENSING MODEL

(how agents interact with environment)

• 3. AGENT MODEL

N i t u s f s

i i i i

, , 1 ), , , (    

 





T t

dt dx x R t u x P J

) (

) ( ))) ( ( , ( max s

u

GOAL: Find the best state trajectories si(t), 0 ≤ t ≤ T so that agents achieve a maximal reward from interacting with the mission space

Christos G. Cassandras CODES Lab. - Boston University

PERSISTENT MONITORING PROBLEM

Start with 1-dimensional mission space  = [0,L] AGENT DYNAMICS:

1 ) ( ,   t u u s

j j j



Analysis still holds for:

1 ) ( , ) (    t u bu s g s

j j j j j



Christos G. Cassandras CODES Lab. - Boston University

PERSISTENT MONITORING PROBLEM

s(t) x SENSING MODEL: p(x,s) Probability agent at s senses point x ENVIRONMENT MODEL: Associate to x Uncertainty Function R(x,t)

noise t s R f t R

x x

  ) , , ( ) ( 

If x is a known “target”:       

• therwise

)) ( , ( ) ( )) ( , ( ) ( , ) , ( if ) , ( t s x Bp x A t s x Bp x A t x R t x R  Use:

Christos G. Cassandras CODES Lab. - Boston University

PERSISTENT MONITORING PROBLEM

…

Partition mission space  = [0,L] into M intervals:

a1 aM

For each interval i = 1,…,M define Uncertainty Function Ri(t):       

• therwise

)) ( ( )) ( ( , ) ( if ) (

i

t BP A t BP A t R t R

i i i i i

s s 

 





  

N j j i i

s p P

1

) ( 1 1 ) (s

where Pi(s) = joint prob. i is sensed by agents located at s = [s1,…,sN]

) , ( ) (

j i j j i

s p s p a 

Christos G. Cassandras CODES Lab. - Boston University

OPTIMAL CONTROL PROBLEM

Determine u1(t),…,uN(t) such that

s.t. ) ( 1 min

1 , ,

1

 





T M i i u u

dt t R T J

N



L b t s a t u u s

j j j j

      ) ( , 1 ) ( ,        

• therwise

)) ( ( )) ( ( , ) ( if ) (

i

t BP A t BP A t R t R

i i i i i

s s  Uncertainty measure Agent dynamics Uncertainty dynamics            

j j j j j j j j

r s x r s x r s x s x p if if 1 ) , ( Sensing model

Christos G. Cassandras CODES Lab. - Boston University

PERSISTENT MONITORING IN 2D MISSION SPACE

Agents play a cooperative PACMAN game against “uncertainty” which continuously regenerates…

Dark brown: HIGH uncertainty White: NO uncertainty

JAVA multi-agent simulator designed to interactively test various

• controllers. Polygonal obstacles may be added to the environment.

http://people.bu.edu/cgc/gengyf/density/density.htm

Christos G. Cassandras CODES Lab. - Boston University

PERSISTENT MONITORING WITH KNOWN TARGETS

TRAFFIC TRAFFIC NETW NETWORK ORK CONTR CONTROL OL

The BU Bridge mess, Boston, MA (simulation using VISSIM)

… EVEN IF WE KNOW THE A THE ACHIEV CHIEVABLE ABLE OPTIMUM IN A OPTIMUM IN A TRAFF TRAFFIC IC NETW NETWORK ORK ??? ???

Christos G. Cassandras CISE SE - CODES Lab. - Boston University

WHY CAN’T WE IMPROVE TRAFFIC…

Because:

• Not enough controls (traffic lights, tolls, speed fines)

→ No chance to unleash the power of feedback!

• Not knowing other drivers’ behavior leads to poor decisions

(a simple game-theoretic fact)

→ Drivers seek individual (selfish) optimum,

not system-wide (social) optimum

PRICE OF ANARCHY (POA)

Christos G. Cassandras CISE SE - CODES Lab. - Boston University

GAME-CHANGING OPPORTUNITY: CONNECTED AUTOMATED VEHICLES (CAVs) NO TRAFFIC LIGHTS, NEVER STOP… FROM (SELFISH) “DRIVER OPTIMAL” TO (SOCIAL) “SYSTEM OPTIMAL” TRAFFIC CONTROL

THE “INTERNET OF CARS CARS”

Christos G. Cassandras CISE SE - CODES Lab. - Boston University

WHO NEEDS TRAFFIC LIGHTS?

With traffic lights With decentralized control of CAVs

One of the worst-designed double intersections ever… (BU Bridge – Commonwealth Ave, Boston)

KEY KEY TECHNICAL TECHNICAL CHALLENGES CHALLENGES

1. SCALABILITY

• 2. DECENTRALIZATION
• 3. COMMUNICATION
• 4. NON-CONVEXITY
• 5. EXLOIT DATA

Christos G. Cassandras CODES Lab. - Boston University

CONTROL AND OPTIMIZATION – CHALLENGES Distributed Algorithms Global optimality, escape local optima Event-driven (asynchronous) Algorithms Data-Driven Algorithms