EVENT-DRIVEN AND DATA-DRIVEN CONTROL AND OPTIMIZATION IN - - PowerPoint PPT Presentation
EVENT-DRIVEN AND DATA-DRIVEN CONTROL AND OPTIMIZATION IN CYBER-PHYSICAL SYSTEMS C. G. Cassandras Division of Systems Engineering Dept. of Electrical and Computer Engineering Center for Information and Systems Engineering Boston University
Division of Systems Engineering
Center for Information and Systems Engineering Boston University
https://christosgcassandras.org
Christos G. Cassandras CODES Lab. - Boston University
CODES Lab. - Boston University
REFERENCE
PLANT CONTROLLER
INPUT
SENSORS
MEASURED OUTPUT OUTPUT ERROR REFERENCE
PLANT CONTROLLER
INPUT
SENSORS
MEASURED OUTPUT OUTPUT ERROR
EVENT:
g(STATE) ≤ 0 EVENT-DRIVEN CONTROL: Act only when needed (or on TIMEOUT) - not based on a clock
CISE SE - CODES Lab. - Boston University
Data collection: relatively easy… Control: a challenge… TIME-DRIVEN EVENT-DRIVEN
CODES Lab. - Boston University
CODES Lab. - Boston University
(e.g., Internet) → all state transitions are event-driven
→ some state transitions are event-driven
→ components interact asynchronously (through events)
CODES Lab. - Boston University
→ actions needed in response to random events
computation and estimation quality
components than to time-driven components
→ time-driven communication consumes significant energy UNNECESSARILY!
TIME
Time-driven (synchronous) implementation:
TIME
t1 t2
CODES Lab. - Boston University
CONTROL, COMMUNICATION, ESTIMATION, OPTIMIZATION
CODES Lab. - Boston University
first order stochastic systems,” Proc. 41st Conf. Decision and Control, pp. 2011–2016, 2002.
Collaborative Estimation,” ASME J. of Dynamic Systems, Measurement, and Control, vol. 129,
controllers for linear systems,” Intl. J. Control, 81, pp. 571–590, 2008.
and experimental validation of a sensor-based event-driven controller,” Proc. American Control Conf., pp. 2867–2874, 2007.
Communication”, IEEE Trans. on Automatic Control, AC-55, 12, pp. 2735-2750, 2010.
CODES Lab. - Boston University
i N s s
N
1 , ,
1
1 1
s constraint . . ) , , ( min
1
s t s s s H
N s
N N s
s t s s s H
N
s constraint . . ) , , ( min
1
(processors, agents, vehicles, nodes),
CODES Lab. - Boston University
i Controllable state si, i = 1,…,ni
)) ( ( ) ( ) 1 ( k d k s k s
i i i i
s α + = +
Step Size Update Direction, usually
)) ( ( )) ( ( k H k d
i i
s s −∇ =
i N s
s t s s s H
i
s constraint . . ) , , ( min
1
i requires knowledge of all s1,…,sN Inter-node communication
CODES Lab. - Boston University
1 2 3
COMMUNICATE + UPDATE
CODES Lab. - Boston University
1 2 3
Bertsekas and Tsitsiklis, 1997
Update frequency for each node is bounded + technical conditions ⇒
)) ( ( ) ( ) 1 ( k d k s k s
i i i i
s α + = +
converges
CODES Lab. - Boston University
2 3
UPDATE COMMUNICATE
locally determined, arbitrary (possibly periodic)
1
CODES Lab. - Boston University
AT ANY TIME t :
) (t x j
i
) (t x j
i
… and evaluates an ERROR FUNCTION (
) ( ), ( t x t x g
j i i
Error Function examples:
2 1
) ( ) ( , ) ( ) ( t x t x t x t x
j i i j i i
− −
) (t x j
i
CODES Lab. - Boston University
i i j
) (t xi
δi
Node i communicates its state to node j only when it detects that its true state xi(t) deviates from j’ estimate of it so that ) (t x j
i
i j i i
t x t x g δ ≥ ) ( ), (
) ( ), ( t x t x g
j i i
Compare ERROR FUNCTION to THRESHOLD δi
CODES Lab. - Boston University
i i i i
Estimates of other nodes, evaluated by node i
THEOREM: Under certain conditions, there exist positive constants α and Kδ such that
)) ( ( lim = ∇
∞ →
k H
k
s
INTERPRETATION: Event-driven cooperation achievable with minimal communication requirements ⇒ energy savings
Zhong and Cassandras, IEEE TAC, 2010
− =
) 1 ( update sends if ) ( ( ) ( k k k d K k
i i i i
δ δ
δ
s
CODES Lab. - Boston University
Red curve: Black curve:
ij
τ
t
( )
k
i
δ
( )
j i i x
x g ~ ,
( )
j i i x
x g ,
ij 3
τ
ij 1
τ
ij 2
τ
ij 1
σ
ij 2
σ
ij 3
σ
ij 4
σ
ij 4
τ
( )
j i i x
x g ,
( )
k
i
δ t
ij 3
τ
ij 2
τ
ij 1
τ
ij
τ
Error function trajectory with NO DELAY
DELAY
CODES Lab. - Boston University
ASSUMPTION: There exists a non-negative integer D such that if a message is sent before tk-D from node i to node j, it will be received before tk.
INTERPRETATION: at most D state update events can occur between a node sending a message and all destination nodes receiving this message.
Add a boundedness assumption: THEOREM: Under certain conditions, there exist positive constants α and Kδ such that
)) ( ( lim = ∇
∞ →
k H
k
s
NOTE: The requirements on α and Kδ depend on D and they are tighter.
Zhong and Cassandras, IEEE TAC, 2010
CODES Lab. - Boston University
Ω
= dx x R x P H ) ( ) , ( ) ( max s s
s
N i F si , , 1 , = Ω ⊆ ∈
i
a
O1 O2
s=[s1, … , sN ]
CODES Lab. - Boston University
Ω GOAL: Find the best state vector s=[s1, … , sN ] so that agents achieve a maximal reward from interacting with the mission space
Ω
=
T t
dt dx x R t u x P J
) (
) ( ))) ( ( , ( max s
u
N i F t si , , 1 , ) ( = Ω ⊆ ∈
i
a
O1 O2
CODES Lab. - Boston University
Ω GOAL: Find the best state trajectories si(t), 0 ≤ t ≤ T so that agents achieve a maximal reward from interacting with the mission space
N i t u s f s
i i i i
, , 1 ), , , ( = =
May also have dynamics
CODES Lab. - Boston University
Ω
s
Event density: Prior estimate of event
Joint event detection probability:
=
− − =
N i i i
s x p x P
1
) , ( 1 1 ) , ( s
Event sensing probability
5 10 2 4 6 8 10 10 20 30 40 50Ω
R(x) (Hz/ m2)
? ? ? ? ? ? ? ? ? Deploy sensors to maximize “event” detection probability - unknown event locations
CODES Lab. - Boston University
∈
− =
i
N j i j i
t s t s t s ) ( ) ( ) (
) (
1 t
s
Ω
) (
2 t
s ) (
3 t
s ) (
4 t
s
N
s s = =
1
Ω
s
=
− =
N i i
s x x R
1
) ( ) ( 1
Only x that matter are agents
=
N i i
1
s
> ∈ − =
i i i j i j i
i j N j s s s s p
, ) , (
2
∈
=
i
N j i j i i
s s p s s P ) , ( 2 1 ) , (
CODES Lab. - Boston University
AGENT MODEL
N i t u s f s
i i i i
, , 1 ), , , ( = =
Ω
=
T t
dt dx x R t u x P J
) (
) ( ))) ( ( , ( max s
u
GOAL: Find the best state trajectories si(t), 0 ≤ t ≤ T so that agents achieve a maximal reward from interacting with the mission space ENVIRONMENT MODEL SENSING MODEL (how agents interact with environment)
CODES Lab. - Boston University
s(t) x ENVIRONMENT MODEL: Associate to x Uncertainty Function R(x,t)
− < = =
)) ( , ( ) ( )) ( , ( ) ( , ) , ( if ) , ( t s x Bp x A t s x Bp x A t x R t x R
CISE SE - CODES Lab. - Boston University
With traffic lights (non-Cooperative) No traffic lights: decentralized control of CAVs (Cooperative)
One of the worst-designed double intersections ever… (BU Bridge – Commonwealth Ave, Boston)
Zhang et al, Proc. of IEEE, 2018 Malikopoulos, Cassandras, Zhang et al, Automatica, 2018
CODES Lab. - Boston University
COVERAGE AND FORMATION CONTROL
Choset 2001, Leonard and Olshevsky 2013, Tron et al 2014, Egerstedt and Hu 2001 Cortes et al 2004 Zhong and Cassandras 2010, Sun and Cassandras 2016
SAMPLING AND TRACKING
Leonard and Zhang 2010, Ashley and Andersson 2016
PERSISTENT MONITORING
Smith et al, 2011, Michael et al, 2011, Lan and Schwager, 2014 Cassandras et al, 2013, Yu et al, 2017
CONSENSUS
Jadbabaie et al, 2003, Olfati-Saber and Murray, 2004, Ren and Beard 2005 Nedich et al, 2010
CODES Lab. - Boston University
CONTROL/DECISION (Parameterized by θ) SYSTEM
PERFORMANCE
NOISE
CISE SE - CODES Lab. - Boston University
)] ( [ θ L E
)] ( [ max θ
θ
L E
Θ ∈
GOAL:
DIFFICULTIES: - E[L(θ)] NOT available in closed form
L(θ) GRADIENT ESTIMATOR
) (
1 n n n n
L θ η θ θ ∇ + =
+
x(t) L(θ) ∆ REAL-TIME DATA
CONTROL/DECISION (Parameterized by θ) Discrete Event System (DES)
PERFORMANCE
L(θ) IPA
NOISE ) (
1 n n n n
L θ η θ θ ∇ + =
+
Sample path
For many (but NOT all) DES:
)] ( [ θ L E
CISE SE - CODES Lab. - Boston University
x(t)
Model
x(t) L(θ) ∆
Ho and Cao, 1991; Glasserman, 1991; Cassandras, 1993; Cassandras and Lafortune, 2008
CONTROL/DECISION (Parameterized by θ) HYBRID SYSTEM
PERFORMANCE
L(θ) IPA
) (
1 n n n n
L θ η θ θ ∇ + =
+
Sample path
CISE SE - CODES Lab. - Boston University
)] ( [ θ L E
NOISE
x(t) L(θ) ∆
) , , ( t x f x
k
=
Event at time τk(θ) Event at time τk+1(θ) kth discrete state (mode) θ : control parameter, (system design parameter, parameter of an input process,
Θ ∈ θ
θ
1. Continuity at events: Take d/dθ :
) ( ) (
− + = k k
x x τ τ
k k k k k k k
f f x x ' )] ( ) ( [ ) ( ' ) ( '
1
τ τ τ τ τ
+ − − − +
− + =
If no continuity, use reset condition ⇒
θ δ υ ρ τ d x q q d x
k
) , , , , ( ) ( ' ′ =
+
CISE SE - CODES Lab. - Boston University
System dynamics over (τk(θ), τk+1(θ)]:
) , , ( t x f x
k
θ = ( ) ( ) ( )
θ θ τ τ θ θ ∂ ∂ = ′ ∂ ∂ = ′
k k
t x t x , ,
NOTATION:
Solve
θ ∂ ∂ + ∂ ∂ = ) ( ) ( ' ) ( ) ( ' t f t x x t f dt t dx
k k
′ + ∫ ∂ ∂ ∫ = ′
+ ∂ ∂ − ∂ ∂ t k du x u f k du x u f
k v k k t k k
x dv e v f e t x
τ
τ θ
τ τ
) ( ) ( ) (
) ( ) (
initial condition from 1 above
CISE SE - CODES Lab. - Boston University
) , , ( t x f x
k
θ =
θ ∂ ∂ + ∂ ∂ = ) ( ) ( ' ) ( ) ( ' t f t x x t f dt t dx
k k
NOTE: If there are no events (pure time-driven system), IPA reduces to this equation
k
τ′
= ′
k
τ
) ), , ( ( = θ τ θ
k k x
g
′ ∂ ∂ + ∂ ∂ ∂ ∂ − = ′
− − −
) ( ) (
1 k k k k
x x g g f x g τ θ τ τ
) ( ) (
1 + −
′ ∂ ∂ − = ′
k k k k k
y t y τ τ τ
CISE SE - CODES Lab. - Boston University
Ignoring resets and induced events:
k k k k k k k
f f x x ' )] ( ) ( [ ) ( ' ) ( '
1
τ τ τ τ τ ⋅ − + =
+ − − − +
+ ∂ ∂ = ′
− − −
+ − ∂ ∂ − ∂ ∂
k k v k k k k k
k du x u f k du x u f k
x dv e v f e x
τ τ
τ θ τ
τ τ τ 1 1 1
) ( ' ) ( ) (
1 ) ( ) (
= ′
k
τ ′ ∂ ∂ + ∂ ∂ ∂ ∂ − = ′
− − −
) ( ) (
1 k k k k
x x g g f x g τ θ τ τ
1. 2. 3.
CISE SE - CODES Lab. - Boston University
2 1 3
) ( '
− k
x τ
( ) ( ) ( )
θ θ τ τ θ θ ∂ ∂ = ′ ∂ ∂ = ′
k k
t x t x ,
Recall:
Cassandras et al, Europ. J. Control, 2010
Back to performance metric:
( ) ∑ ∫
=
+
=
N k k
k k
dt t x L L
1
) , , (
τ τ
θ θ
( ) ( )
θ θ θ ∂ ∂ = ′ t x L t x L
k k
, , , ,
NOTATION: Then:
( ) ∑
= + +
′ + ⋅ ′ − ⋅ ′ =
+
N k k k k k k k k
k k
dt t x L L L d dL
1 1
1
) , , ( ) ( ) (
τ τ
θ τ τ τ τ θ θ
What happens at event times What happens between event times
CISE SE - CODES Lab. - Boston University
THEOREM 1: If either 1,2 holds, then dL(θ)/dθ depends only on information available at event times τk:
= ∂ ∂ = ∂ ∂ = ∂ ∂ θ
k k k
f dt d x f dt d x L dt d
IMPLICATION: - Performance sensitivities can be obtained from information limited to event times, which is easily observed
( ) ∑
= + +
′ + ⋅ ′ − ⋅ ′ =
+
N k k k k k k k k
k k
dt t x L L L d dL
1 1
1
) , , ( ) ( ) (
τ τ
θ τ τ τ τ θ θ
CISE SE - CODES Lab. - Boston University Yao and Cassandras, J. DEDS, 2011
EVENTS
Evaluating requires full knowledge of w and f values (obvious)
) ; ( θ t x
However, may be independent of w and f values (NOT obvious)
θ θ d t dx ) ; (
It often depends only on: - event times τk
) (
1 − + k
f τ ) ; , , , ( θ t w u x f x =
τk τk+1
CISE SE - CODES Lab. - Boston University
IPA estimators are EVENT-DRIVEN ⇒ IPA scales with the EVENT SET, not the STATE SPACE ! ⇒ no time discretization needed
CISE SE - CODES Lab. - Boston University
As a complex system grows with the addition of more states, the number of EVENTS often remains unchanged or increases at a much lower rate. EXAMPLE: A queueing network may become very large, but the basic events used by IPA are still “arrival” and “departure” at different nodes.
DECENTRALIZED SOLUTION = CENTRALIZED SOLUTION
(AGENTS ACTING USING ONLY LOCAL INFO.)
(no performance loss due to decentralization)
Agent Network (time-varying) Agent-Target Interaction Network (time-varying)
CODES Lab. - Boston University
Hard to decentralize in the presence of time-varying agent-environment interactions
CODES Lab. - Boston University
(conventional)
𝐵5 𝐵2 𝐵3 𝐵1 𝐵4
𝑈2 𝑈
1
𝑈3 𝑈5 𝑈
4
CODES Lab. - Boston University
segments defined by observable EVENTS e.g., agent enters target sensing range, agent leaves neighborhood
Infinitesimal Pertubation Analysis (IPA) calculus: Each agent evaluates its IPA derivative
THEOREM:: Each agent’s IPA derivative depends only on LOCAL events except for one global event
Zhou et al, IEEE TAC 2018
DECENTRALIZATION EVENT OBSERVABILITY
CODES Lab. - Boston University
When are these possible? How to design? Can this be done in a distributed manner? Is convergence guaranteed? How do Event-Driven algorithms perform compared to Time-Driven ones? Solve stochastic optimization problems robust to modeling assumptions