Optimal Decentralized Control of System with Partially Exchangeable - PowerPoint PPT Presentation

Optimal Decentralized Control of System with Partially Exchangeable Agents Aditya Mahajan McGill University Joint work with Jalal Arabneydi Allerton Conference on Communication, Control, and Computing 28 Sep, 2016

Decentralized control with exchangeable agents–(Arabneydi and Mahajan) 1 Optimal decentralized control: Applications Internet of Things

Decentralized control with exchangeable agents–(Arabneydi and Mahajan) 1 Optimal decentralized control: Applications Internet of Things Smart Grids

Decentralized control with exchangeable agents–(Arabneydi and Mahajan) 1 Optimal decentralized control: Applications Internet of Things Smart Grids Sensor Networks

Decentralized control with exchangeable agents–(Arabneydi and Mahajan) 1 Optimal decentralized control: Applications Internet of Things Smart Grids Sensor Networks Swarm Robotics

Decentralized control with exchangeable agents–(Arabneydi and Mahajan) 1 Optimal decentralized control: Applications and Theory Internet of Things Smart Grids Sensor Networks Swarm Robotics Salient features Multiple decision makers Access to difgerent information Cooperate towards a common objective

Decentralized control with exchangeable agents–(Arabneydi and Mahajan) 1 Optimal decentralized control: Applications and Theory Internet of Things Smart Grids Sensor Networks Swarm Robotics Salient features Multiple decision makers Access to difgerent information Cooperate towards a common objective

Decentralized control with exchangeable agents–(Arabneydi and Mahajan) 1 Optimal decentralized control: Applications and Theory Internet of Things Smart Grids Sensor Networks Swarm Robotics Salient features Multiple decision makers Access to difgerent information Cooperate towards a common objective

Decentralized control with exchangeable agents–(Arabneydi and Mahajan) 1 Optimal decentralized control: Applications and Theory Internet of Things Smart Grids Sensor Networks Swarm Robotics Salient features Multiple decision makers Access to difgerent information Cooperate towards a common objective

Decentralized control with exchangeable agents–(Arabneydi and Mahajan) 1 Optimal decentralized control: Applications and Theory Internet of Things Smart Grids Sensor Networks Swarm Robotics Salient features Multiple decision makers Access to difgerent information Cooperate towards a common objective Series of positive results in the last 10-15 years: funnel causality, quadratic invariance, common information approach, and others. Explicit solutions are rare and typically exist for systems with two or three agents.

Are there features that are present in the applications but are missing from the theory?

Decentralized control with exchangeable agents–(Arabneydi and Mahajan) 2 System with exchangeable agents Dynamics 𝐲 t+1 = f t (𝐲 t , 𝐯 t , 𝐱 t ) with per-step cost c t (𝐲 t , 𝐯 t ) .

Decentralized control with exchangeable agents–(Arabneydi and Mahajan) 2 System with exchangeable agents Pair of exchangeable agents Agents i and j are exchangeable if f t (σ ij 𝐲 t , σ ij 𝐯 t , σ ij 𝐱 t ) = σ ij (f t (𝐲 t , 𝐯 t , 𝐱 t )) c t (σ ij 𝐲 t , σ ij 𝐯 t ) = c t (𝐲 t , 𝐯 t ) . Dynamics 𝐲 t+1 = f t (𝐲 t , 𝐯 t , 𝐱 t ) with per-step cost c t (𝐲 t , 𝐯 t ) . 𝒴 i = 𝒴 j , 𝒱 i = 𝒱 j , 𝒳 i = 𝒳 j .

Decentralized control with exchangeable agents–(Arabneydi and Mahajan) 2 System with exchangeable agents Pair of exchangeable agents Agents i and j are exchangeable if f t (σ ij 𝐲 t , σ ij 𝐯 t , σ ij 𝐱 t ) = σ ij (f t (𝐲 t , 𝐯 t , 𝐱 t )) c t (σ ij 𝐲 t , σ ij 𝐯 t ) = c t (𝐲 t , 𝐯 t ) . Set of exchangeable agents A set of agents is exchangeable if every pairin that set is exchangeable Dynamics 𝐲 t+1 = f t (𝐲 t , 𝐯 t , 𝐱 t ) with per-step cost c t (𝐲 t , 𝐯 t ) . 𝒴 i = 𝒴 j , 𝒱 i = 𝒱 j , 𝒳 i = 𝒳 j .

Decentralized control with exchangeable agents–(Arabneydi and Mahajan) 2 System with exchangeable agents Pair of exchangeable agents Agents i and j are exchangeable if f t (σ ij 𝐲 t , σ ij 𝐯 t , σ ij 𝐱 t ) = σ ij (f t (𝐲 t , 𝐯 t , 𝐱 t )) c t (σ ij 𝐲 t , σ ij 𝐯 t ) = c t (𝐲 t , 𝐯 t ) . Set of exchangeable agents A set of agents is exchangeable if every pairin that set is exchangeable System with partially exchangeable agents . . . is a multi-agent system where the set of agents can be partitioned into disjoint sets of exchangeable agents. Dynamics 𝐲 t+1 = f t (𝐲 t , 𝐯 t , 𝐱 t ) with per-step cost c t (𝐲 t , 𝐯 t ) . 𝒴 i = 𝒴 j , 𝒱 i = 𝒱 j , 𝒳 i = 𝒳 j .

Decentralized control with exchangeable agents–(Arabneydi and Mahajan) 3 Notation N : number of heterogeneous agents K : number of subpopulations

Decentralized control with exchangeable agents–(Arabneydi and Mahajan) 3 Notation For agent i of sub- population k x i u i N : number of heterogeneous agents K : number of subpopulations x : state of agent i t ∈ ℝ d k u : control action of agent i t ∈ ℝ d k

Decentralized control with exchangeable agents–(Arabneydi and Mahajan) ¯ N : number of heterogeneous agents u i i∈𝒪 k 1 u k ¯ x i i∈𝒪 k 1 3 x k K : number of subpopulations : set of agents in sub-popln k 𝒪 k For sub-population k Notation For agent i of sub- population k u i x i x : state of agent i t ∈ ℝ d k u : control action of agent i t ∈ ℝ d k t = t : mean-fjeld of states |𝒪 k | ∑ t = t : mean-fjeld of actions |𝒪 k | ∑

Decentralized control with exchangeable agents–(Arabneydi and Mahajan) ¯ N : number of heterogeneous agents u K u 1 ¯ : global mean-fjeld of states x K 4 x 1 𝒧 = {1, . . . , K} : set of all sub-populations For the entire population Notation K : number of subpopulations 𝒪 = 𝒪 1 ∪ ⋅ ⋅ ⋅ ∪ 𝒪 K : set of all agents 𝐲 t = (x i t ) i∈𝒪 : global state of the system 𝐯 t = (u i t ) i∈𝒪 : joint actions of all agents t , . . . , ¯ 𝐲 t = vec (¯ t ) 𝐯 t = vec ( ¯ t , . . . , ¯ t ) : global mean-fjeld of actions

Decentralized control with exchangeable agents–(Arabneydi and Mahajan) 5 Linear quadratic system with partially exchangeable agents Dynamics Cost T ∑ 𝐲 t+1 = A t 𝐲 t + B t 𝐯 t + 𝐱 t t Q t 𝐲 t + 𝐯 ⊺ t R t 𝐯 t ] t=1 [𝐲 ⊺

Decentralized control with exchangeable agents–(Arabneydi and Mahajan) 5 Linear quadratic system with partially exchangeable agents Dynamics Cost T ∑ 𝐲 t+1 = A t 𝐲 t + B t 𝐯 t + 𝐱 t t Q t 𝐲 t + 𝐯 ⊺ t R t 𝐯 t ] t=1 [𝐲 ⊺

Decentralized control with exchangeable agents–(Arabneydi and Mahajan) 5 Linear quadratic system with partially exchangeable agents Dynamics Cost T ∑ Irrespective of the information structure such a system is equivalent to a mean-fjeld coupled system 𝐲 t+1 = A t 𝐲 t + B t 𝐯 t + 𝐱 t t Q t 𝐲 t + 𝐯 ⊺ t R t 𝐯 t ] t=1 [𝐲 ⊺

Decentralized control with exchangeable agents–(Arabneydi and Mahajan) |𝒪 k |[(x i T ∑ k∈𝒧 ∑ i∈𝒪 k 1 t ) 5 ⊺ Q k t +(u i t ) ⊺ R k 𝐲 ⊺ 𝐯 ⊺ Cost t x i Cost Linear quadratic system with partially exchangeable agents T Dynamics Irrespective of the information structure such a system is equivalent to a mean-fjeld coupled system Agent dynamics in ∑ sub-population k 𝐲 t+1 = A t 𝐲 t + B t 𝐯 t + 𝐱 t t Q t 𝐲 t + 𝐯 ⊺ t R t 𝐯 t ] t=1 [𝐲 ⊺ t ¯ t ¯ t+1 = A k t x i t + B k t u i t + D k 𝐲 t + E k 𝐯 t + w i t ]+ ¯ t ¯ 𝐲 t + ¯ t ¯ t x i t u i t P x t P u 𝐯 t ] t=1 [ ∑

Decentralized control with exchangeable agents–(Arabneydi and Mahajan) 6 There is a long history of mean-field approximations Mean-field approximation in statistical physics (Weiss 1907; Landau 1937)

Decentralized control with exchangeable agents–(Arabneydi and Mahajan) 6 There is a long history of mean-field approximations Mean-field approximation in statistical physics (Weiss 1907; Landau 1937) It is a well-known phenomenon in many branches of the exact and physical moving particles, is incomparably easier than that of the solar system, made up of 9 major bodies… This is, of course, due to the excellent possibility of applying the laws of statistics and probabilities in the fjrst case. — von Neumann and Morgenstern, Theory of Games and Economic Behavior (1944) § 2.4.2 sciences that very great numbers are often easier to handle than those of medium size. An almost exact theory of a gas, containing about 10 25 freely

Anonymous games Decentralized control with exchangeable agents–(Arabneydi and Mahajan) 6 There is a long history of mean-field approximations Mean-field approximation in statistical physics (Weiss 1907; Landau 1937) . . . Mean-field approximations in Game Theory Jovanovic Rosenthal 1988 Bergin Bernhardt 1995 Weintraub Benkard Van Roy 2008 . . .

Anonymous games Decentralized control with exchangeable agents–(Arabneydi and Mahajan) 6 There is a long history of mean-field approximations Mean-field approximation in statistical physics (Weiss 1907; Landau 1937) . . . Mean-field approximations in Game Theory Jovanovic Rosenthal 1988 Bergin Bernhardt 1995 Weintraub Benkard Van Roy 2008 . . . Mean-field approximations in Systems and Control (Mean-field games) Huang Caines Malhalmé 2003, . . . Larsy Lions 2006, . . . . . .