Representing and reasoning about Bayesian coalitional games Oldooz - - PowerPoint PPT Presentation

Representing and reasoning about Bayesian coalitional games

Oldooz Dianat Supervisor: Prof. Mehmet Orgun Co-supervisor: Dr. Lee Flax

Macquarie University Department of Computing

June 2012

Oldooz Dianat (Department of computing) Bayesian coalitional game logic June 2012 1 / 28

Outline

1

Research gap and aims

2

Background

3

Methodology and research progress

4

Conclusion

Oldooz Dianat (Department of computing) Bayesian coalitional game logic June 2012 2 / 28

Research gap and aims

Outline

1

Research gap and aims

2

Background

3

Methodology and research progress

4

Conclusion

Oldooz Dianat (Department of computing) Bayesian coalitional game logic June 2012 3 / 28

Research gap and aims

Multi-agent systems

Agents Computer systems that are capable of independent, autonomous action in order to satisfy their design objectives. In multi-agent systems with self-interested agents; this is the case in most economic applications. The optimal action for one agent to take depends on the actions that other agents take. To determine the optimal action for the agent under specific circumstance, game theory can provide useful models.

Oldooz Dianat (Department of computing) Bayesian coalitional game logic June 2012 4 / 28

Research gap and aims

Game theory

Tries to predict or explain the behavior of agents under a sequence of interactions. Is the mathematical study of interaction among independent, self interested agents. Quantifies an agent’s degree of preference across a set of available alternatives Describes how these preferences change when an agent faces uncertainty about which alternative he will receive.

Oldooz Dianat (Department of computing) Bayesian coalitional game logic June 2012 5 / 28

Research gap and aims

Game theory and logic

Game theory does not consider the reasoning abilities of agents. Logical declarative languages These languages enable reasoning about the best strategy in games by considering other players rationality and reasoning abilities. These languages are used to represent game model explicitly.

Oldooz Dianat (Department of computing) Bayesian coalitional game logic June 2012 6 / 28

Research gap and aims

Research objectives

The specific objectives of this project include: Develop the syntax and semantics for a logic of Bayesian coalition games. Identify a suitable model checker. Develop a method for translating the specification language to the input language of a model checker. Verify the properties of the system by the model checker.

Oldooz Dianat (Department of computing) Bayesian coalitional game logic June 2012 7 / 28

Background

Outline

1

Research gap and aims

2

Background

3

Methodology and research progress

4

Conclusion

Oldooz Dianat (Department of computing) Bayesian coalitional game logic June 2012 8 / 28

Background

Classical coalitional games

Three or more players. Each player is individually rational and thus seeks to maximize individual gain. Players are free to negotiate binding and enforceable agreements about the formation of coalitions. Division of payoffs, which is the result from coordinated actions. Mathematically speaking a colitional game is a pair(N, v): N is a finite set of players,indexed by i v : 2N → R associates with each coalition S ⊆ N a real-valued payoff v(S) that the coalition,s members can distribute among themselves.

Oldooz Dianat (Department of computing) Bayesian coalitional game logic June 2012 9 / 28

Background

From classical models to Bayesian models

Coalitional Game assumptions The payoff to each coalition is given by a fixed, deterministic value. These values are common knowledge among all agents. They often fail to hold for real-world problems. Bayesian coalitional games are: Study cooperation under uncertainty Generalization of coalitional games to a Bayesian framework using the information partition model

Oldooz Dianat (Department of computing) Bayesian coalitional game logic June 2012 10 / 28

Background

Uncertainty in games and epistemic logic

Harsanyi analyzed uncertainty about the structure of games, specifically about the players’ payoff functions. He introduced the fundamental concept of a player’s type. It could be used to encode what the player believes the payoff functions to be, what the player believes other players believe the payoff functions to be, and so on, indefinitely. His formalism introduced the players hierarchies of beliefs over: strategies, and rationality. Interactive epistemology deals with the beliefs and the knowledge of players.

Oldooz Dianat (Department of computing) Bayesian coalitional game logic June 2012 11 / 28

Background

Epistemic logic

Epistemic logic is the logic of knowledge and belief Reasons about the knowledge of agents in a group. Agents may have different information and thus different epistemic alternatives at each possible state. The logical symbols used are : ¬, ∧ , ∨ , → , and ↔ . The conjunction (disjunction) of all sentences from a finite set Σ is abbreviated by Σ ( Σ), assuming commutativity. The language has knowledge operator Ki.

Oldooz Dianat (Department of computing) Bayesian coalitional game logic June 2012 12 / 28

Methodology and research progress

Outline

1

Research gap and aims

2

Background

3

Methodology and research progress

4

Conclusion

Oldooz Dianat (Department of computing) Bayesian coalitional game logic June 2012 13 / 28

Methodology and research progress

Epistemic logic by B.De Burin

We abstract from the specific actions and reason about what specific agents can achieve. B.De Burin introduced an epistemic language for representing the solution concept in normal games and extensive games. He proposed a formula that expresses the solution concept such as the Nash equilibrium. This formula simply says, if a player knows his opponent to be rational, and knows that his opponent knows the utility structure, these beliefs form a pure strategy Nash equilibrium. We refine the notation of rationality axiom, to adapt it for pure strategy in normal form games. Then we extend it for representing and reasoning about Bayesian games.

Oldooz Dianat (Department of computing) Bayesian coalitional game logic June 2012 14 / 28

Methodology and research progress

Propositions

The proposition letters im stand for the statement ’i plays his mth strategy im’. The proposition ui(1k1, ..., Nk1) = ri,1k1,...,Nk1 denotes that the utility for player i, when the strategy profile (1k1, ..., Nk1) is played, equals the number r. R is a set of countably many symbols such as r. The elements

• f R represent real numbers.

The proposition rati denotes the rationality of player i, in the sense that i is an expected utility maximizer.

Oldooz Dianat (Department of computing) Bayesian coalitional game logic June 2012 15 / 28

Methodology and research progress

Axioms

Any axiomatization for propositional logic Ki(ϕ → ψ) → (Kiϕ → Kiψ) Kiϕ → ¬Ki¬ϕ Kiϕ → KiKiϕ (Kiϕ ∧ Ki(ϕ → ψ)) → Kiψ Kiϕ → ϕ The inference rules are: If ⊢ ϕ → ψ and ⊢ ϕ then ⊢ ψ If ⊢ ϕ then Kiϕ

Oldooz Dianat (Department of computing) Bayesian coalitional game logic June 2012 16 / 28

Methodology and research progress

Rationality axiom and Nash equilibrium

Rationality axiom The proposition that captures the rationality of player i is called rati. The axiom RAT which is the formalism of utility maximization captures the rati as the following implication: rati ↔

• m

((Ki

• k,l

ui(k, l) = ri,k,l ∧ im) →

• k

ri,m,l ≥ ri,k,l (1) The above axiom states that player i aims at his utility maximizer, if he decides to play his mth strategy in a situation in which he has certain beliefs about utility (captured by the ri,k,l) then the mth strategy is better than any other, given his beliefs.

Oldooz Dianat (Department of computing) Bayesian coalitional game logic June 2012 17 / 28

Methodology and research progress

Assumptions for normal form games

There are some assumptions needed to use the epistemic logic for reasoning about Nash equilibrium of normal form games: All players know their own utility function, which is:

• i Ki
• k,l ui(k, l) = ri,k,l

All players are rational, which is:

i rati

All players know each player’s actual choice of an action, which is: K21m ∧ K12n Then the actual action profile played constitutes a Nash equilibrium. It means the solution concept for player 1 is

• k

r1,m,l ≥ r1,k,l (2) and for player 2 is

• l

r2,k,n ≥ r2,k,l (3) .

Oldooz Dianat (Department of computing) Bayesian coalitional game logic June 2012 18 / 28

Methodology and research progress

Syntax for Bayesian games

we define new propositions for utility and strategy. The notation θi,ni is a set of types for player i, which means: θi,ni = thetai,1, θi,2, ..., θi,n. It represents player i has n different types. The notation θi,ni,−i,n(−i) is a set of types for player i and a set of types for player −i, when their types conjunction is not null. The proposition letters imθi,ni stand for the statement ’i plays his mth strategy in his θi,ni type ’. The proposition ui(1k1, ..., Nk1, θi,ni, θ−i,n(−i)) = ri,1k1,...,Nk1,θi,ni ,θ−i,n(−i) denotes that the utility for player i with type θi,ni, when the strategy profile (1k1, ..., Nk1) is played and the opponent type is θ−i,n(−i), equals the number r.

Oldooz Dianat (Department of computing) Bayesian coalitional game logic June 2012 19 / 28

Methodology and research progress

Probabilistic expressions

We use the syntax introduced by B.D.Burin for probabilistic expressions. Pi(.) = . represents i’s probabilistic belief of player i’s type. Arbitrary finite sums of such expressions Pi(ϕ1).q1 + ... + Pi(ϕn).qn ≥ q are allowed as long as the are not mixed over players (as Pi(ϕ1).q1 + Pj(ϕ2).q2 ≥ q would be for i = j). In this work ϕ represents the type of players and q the payoffs. To allow for probabilistic reasoning the Kolmogorov axioms are essential. NonNeg: Pi(ϕ1) ≥ 0. True: Pi(⊤) = 1. False: Pi(⊥) = 0. Add: Pi(ϕ) = Pi(ϕ ∧ ψ) + Pi(ϕ ∧ ¬ψ). Dist: Pi(ϕ) = Pi(ψ) whenever ϕ ↔ ψ is a propositional tautology.

Oldooz Dianat (Department of computing) Bayesian coalitional game logic June 2012 20 / 28

Methodology and research progress

Relation between modal operators

In order to ensure that probabilistic and non-probabilistic beliefs are related in the right way, two additional axioms are useful. Cons: Kiϕ ↔ Pi(ϕ) = 1. KnProb: ϕ → Kiϕ for ϕ an i-probability sentence (the sentence starts with Pi or Boolean combinations thereof).

Oldooz Dianat (Department of computing) Bayesian coalitional game logic June 2012 21 / 28

Methodology and research progress

Rationality axiom form Bayesian games

rattypei ↔

• mΘi

((Ki

• k,l
• k,l

Pi(θi,ni,−i,n(−i))ui(k, l, θi,ni, θi,ni,−i,n(−i)) = ri,k,l,θi,ni ,θ−i,n(−i) ∧ imθi,ni ) →

• k
• m

pmθi,ni .ri,m,l,θi,ni ,θ−i,n(−i) ≥

• k

pkθi,ni ri,k,l,θi,ni ,θ−i,n(−i)) (4) The above axiom states that player i is a utility maximizer whenever, if he decides to play her mθith strategy in a situation in which he has probabilistic beliefs Pi(θi,ni,−i,n(−i)) about utility (captured by the ri,k,l,θi,ni ,θ−i,n(−i)) then the mθi,nith strategy is better than any other, given his beliefs.

Oldooz Dianat (Department of computing) Bayesian coalitional game logic June 2012 22 / 28

Methodology and research progress

Assumptions for Bayesian games

All players know their own utility function, which is:

• i Ki
• k,l ui(k, l, θi,ni, θ−i,ni) = ri,k,l,θi,ni ,θ−i,ni

All players know their own probabilistic belief about utility functions, which is:

i Pi(θi,ni,−i,n(−i))

All players are rational, which is:

i rattypei

All players know each player’s actual choice of an action, which is: K21mθ1,n1 ∧ K12nθ2,n2

Oldooz Dianat (Department of computing) Bayesian coalitional game logic June 2012 23 / 28

Methodology and research progress

Bayes-Nash equilibrium

The actual action profile played constitutes a Bayes-Nash equilibrium. It means the solution concept for player 1 is

• k
• m

pmθi,ni r1,m,l,θi,ni ,θ−i,n(−i) ≥

• k

pkθi,ni r1,k,l,θi,ni ,θ−i,n(−i) (5) and for player 2 is

• l
• n

pnθi,ni r2,k,n,θi,ni ,θ−i,n(−i) ≥

• l

pl θi,ni r2,k,l,θi,ni ,θ−i,n(−i) (6) .

Oldooz Dianat (Department of computing) Bayesian coalitional game logic June 2012 24 / 28

Conclusion

Outline

1

Research gap and aims

2

Background

3

Methodology and research progress

4

Conclusion

Oldooz Dianat (Department of computing) Bayesian coalitional game logic June 2012 25 / 28

Conclusion

Future work

To extend the language that can represent and reason about the cooperation in agents’ movement. To develop a model checker that supports the epistemic logic for Bayesian games. To to develop a model checker that supports the epistemic logic for Bayesian coalitional games.

Oldooz Dianat (Department of computing) Bayesian coalitional game logic June 2012 26 / 28

Conclusion

Application

Logic-based representation As query languages, for expressing properties ϕ of Bayesian coalitional games for multi-agent systems. For directly reasoning about Bayesian coalitional games via theorem proving. For expressing desirable properties ϕ of a Bayesian coalitional game that we want to synthesise.

Oldooz Dianat (Department of computing) Bayesian coalitional game logic June 2012 27 / 28

Conclusion

Conclusion

The conceptual study of Bayesian coalitional game-playing situations cannot be used to derive stable results as long as no appropriate formalism is available to model the situation. The main purpose of this paper was to show that a formal tool, namely epistemic logic for normal form games, can be used to represent and reason about Bayesian games. Some propositions and axioms are extended to model the uncertainty of players about payoffs in Bayesian games. We show that this language provides reasoning about the Bayes-Nash equilibrium in Bayesian games.

Oldooz Dianat (Department of computing) Bayesian coalitional game logic June 2012 28 / 28