Measure and cost dependent properties of information strucutres Aditya Mahajan Serdar Yüksel Yale University Queen's University ACC 2010
2/17 Why are information structures useful?
2/17 Why are information structures useful? Info structures capture the design difficulties of decentralized control
2/17 Why are information structures useful? Info structures capture the design difficulties of decentralized control Classical info structures are centralized systems, hence easy to design Non-classical info structures are decentralized systems, hence hard to design
2/17 Why are information structures useful? Info structures capture the design difficulties of decentralized control Classical info structures are centralized systems, hence easy to design Non-classical info structures are decentralized systems, hence hard to design Is this really true? Can we have two systems with identical information structures that behave differently?
3/17 A controller with no memory Plant Controller Channel � � � State Equation: � = � �� , � , � � Observation Equation: � = ℎ �� , 𝑂 � Controller with no memory: � = � �� �
3/17 A controller with no memory info structure Non-classical Channel Controller Plant � � � State Equation: � = � �� , � , � � Observation Equation: � = ℎ �� , 𝑂 � Controller with no memory: � = � �� �
3/17 A controller with no memory info structure Non-classical Channel Controller Plant � � � State Equation: � = � �� , � , � � Observation Equation: � = ℎ �� , 𝑂 � Controller with no memory: � = � �� � The info structure does not depend on channel ℎ
3/17 Non-classical When the channel is noiseless, the system A controller with no memory info structure Channel Controller Plant is an MDP --- a centralized system � � � State Equation: � = � �� , � , � � Observation Equation: � = ℎ �� , 𝑂 � Controller with no memory: � = � �� � The info structure does not depend on channel ℎ
3/17 When the channel is noiseless, the system Plant Controller Channel Non-classical info structure A controller with no memory Two systems with identical info structures is an MDP --- a centralized system � � � State Equation: � = � �� , � , � � Observation Equation: � = ℎ �� , 𝑂 � Perfect observations ⇒ centralized Controller with no memory: � = � �� � Imperfect observations ⇒ decentralized The info structure does not depend on channel ℎ
4/17 What is missing? Information structures do not completely characterize the design difficulties of decentralized systems
4/17 What is missing? Information structures do not completely characterize the design difficulties of decentralized systems Information structures capture who knows what and when, but do not capture usefulness of available data
4/17 What is missing? Information structures do not completely characterize the design difficulties of decentralized systems We present a generalization of information that captures the usefulness of information. independence properties of the probability measure Information structures capture who knows what and when, but do not capture usefulness of available data structures, which we call � -generalization, This generalization depends on the coupling of the cost function and the
5/17 Contributions of the paper The solution technique for any info structure is also applicable to its Defined a � -generalization of an info structure � -generalization
5/17 Contributions of the paper The solution technique for any info structure is also applicable to its Implications: Follow a two step approach Define info structure in the usual manner (keeps analysis simple) Defined a � -generalization of an info structure � -generalization Define the � -generalization of an info structure We get the solution technique for � -generalized info structure for free!
5/17 Contributions of the paper The solution technique for any info structure is also applicable to its Implications: Follow a two step approach Define info structure in the usual manner (keeps analysis simple) Present coupled dynamic programs to find pbpo solution of quasiclassical info structures Works for non-linear systems Need to only solve parametric optimization problem Defined a � -generalization of an info structure � -generalization Define the � -generalization of an info structure We get the solution technique for � -generalized info structure for free!
6/17 Outline of the paper Model Information Structures Coupled dynamic programs for quasiclassical info structure Example � -generalization of info structures
7/17 The intrinsic model Originally proposed by Witsenhausen, 1971 and 1975
7/17 The intrinsic model Originally proposed by Witsenhausen, 1971 and 1975 Intrinsic event: 𝜕 taking values in a probability space
7/17 The intrinsic model Originally proposed by Witsenhausen, 1971 and 1975 Intrinsic event: 𝜕 taking values in a probability space 𝑂 agents
7/17 The intrinsic model Originally proposed by Witsenhausen, 1971 and 1975 Intrinsic event: 𝜕 taking values in a probability space 𝑂 agents Observations of agent � : � taking value in a measurable space � = � �𝜕, � � where � ⊂ [� − 1]
7/17 The intrinsic model Originally proposed by Witsenhausen, 1971 and 1975 Intrinsic event: 𝜕 taking values in a probability space 𝑂 agents Observations of agent � : � taking value in a measurable space � = � �𝜕, � � where � ⊂ [� − 1] Action of agent � : � taking values in a measurable space � = � �� �
7/17 Cost: Originally proposed by Witsenhausen, 1971 and 1975 The intrinsic model Intrinsic event: 𝜕 taking values in a probability space 𝑂 agents Observations of agent � : � taking value in a measurable space � = � �𝜕, � � where � ⊂ [� − 1] Action of agent � : � taking values in a measurable space � = � �� � Additive terms. Agents coupled by 𝑙 -th cost term: � ⊂ [𝑂] ∑ 𝜍 �𝜕, � �
7/17 Cost: Originally proposed by Witsenhausen, 1971 and 1975 The intrinsic model Intrinsic event: 𝜕 taking values in a probability space 𝑂 agents Observations of agent � : � taking value in a measurable space � = � �𝜕, � � where � ⊂ [� − 1] Action of agent � : � taking values in a measurable space � = � �� � Additive terms. Agents coupled by 𝑙 -th cost term: � ⊂ [𝑂] ∑ 𝜍 �𝜕, � � Objective: Choose �� , . . . , � � to minimize expected cost
8/17 Coupling through cost Agents are coupled in two ways: Coupling through dynamics Salient Features � * : set of agents that can influence the observations of agent � � ∈ � * ⇒ there exist � = � , � , . . . , � ℓ = � such that � ∈ � , 𝑗 = 1, . . . , ℓ � * : agents coupled to agent � through cost � * = ⋃ � 𝟚{� ∈ � }
9/17 Information Structures Information Structure Collection of information known to each agent
9/17 Information Structures Information Structure Collection of information known to each agent Classification of info structures Classical info structure Each agent knows the data available to all agents that act before it Quasiclassical info structure Each agent knows the data available to all agents that can influence its observation
9/17 Information Structures Information Structure Collection of information known to each agent Classification of info structures Classical info structure Each agent knows the data available to all agents that act before it Quasiclassical info structure Each agent knows the data available to all agents that can influence its observation Strictly classical info structures Each agent . . . data and control actions . . . Strictly quasiclassical info structure Each agent . . . data and control actions . . .
10/17 Expansion of info structures Classical expansion of info structure A new system obtained by � ↦ �� , � [] , � [] �
10/17 Expansion of info structures Classical expansion of info structure A new system obtained by Quasiclassical expansion of info structure A new system obtained by � ↦ �� , � [] , � [] � � ↦ �� , � * , � * �
11/17 The main idea (1) Dynamic programming works only for strictly classical info structure.
Recommend
More recommend
