Cooperation rather than contention (instead - Coopetition Paradigms in Competitive Wireless Networks) Mihaela van der Schaar (with Fangwen Fu, Yi Su) Electrical Engineering, UCLA Multimedia Communications and Systems Lab http://medianetlab.ee.ucla.edu/
Motivation Multi-user wireless networks • Coopetition – Competitive environments • Strategic users compete for available resources to maximize their utility • Cooperation is required to increase both network and users’ efficiency – Characterization of multi-user interaction in current literature
New dimensions of multi-user interactions? Heterogeneous users • different utility functions, which also change over time – various standards and architectures – ability to sense the environment and gather information – intelligence in determining actions and strategies (bounded rationality) – Dynamics • environment (channel, but also source and other competing users) – Differentiate among users’ non-collaborative behavior • strategic users – maximize their own utility – malicious users – bounded rationality – Consider impact of information decentralization • private information – information history – depends on the user’s observations/protocols – common knowledge – may differ – strategic message exchanges – Ability of users to learn • not single-agent, but multi-agent learning – Focus on equilibrium selection rather than characterization •
Possible new view for network design? Design network interactions as dynamic, stochastic games • played among strategic and heterogeneous agents The game is played with incomplete information and • heterogeneous, decentralized knowledge Users can learn about their environment and competing users • based on observations or explicitly exchanged information -> Foresighted (proactive) participation in the competition for resources rather than myopic adaptation Collaboration needs to be mutually beneficial, not imposed • Wireless User Sensing Information (existing techniques) vdSchaar – NSF Career 2004 { Incomplete } Learning Central Spectrum Negotiation messages Rules Moderator (CSM) Wireless Network Resource Negotiation or (Spectrum Access { Explicit/Implicit } { Fairness/Efficiency } Policy Maker Market) Actions Knowledge-driven Decision Making { Myopic, Foresighted, etc. }
Distributed stochastic games Numerous networking/computing games: - Networks: power control games, contention games, peer-to-peer games etc. Su, vdSchaar – 2007 Frequency-selective interference channels Goal: Optimal transmit PSD design that maximizes strategic users’ rates
Multi-user Power Control Games in Interference Channels Existing algorithms • – Non-cooperative solutions • Homogeneous users – Cooperative solutions • Central controllers required • Non-convex optimization • Nash Equilibrium – Best response in the competitive optimality sense Reached using iterative-waterfilling
Some Reflections… • Can we do better than Nash Eq. in decentralized environments, and how? • Stackelberg stage game – (Down, Left) : Nash equilibrium – (Up, Right) is better! – Heterogeneity in information availability • Nash: Myopic • Stackelberg: Foresighted
Illustrative Results myopic IW Algorithm IW Algorithm 8 8 6 6 P 1 (f) P 2 (f) 4 4 2 2 0 0 0 20 40 60 80 100 120 140 160 180 200 0 20 40 60 80 100 120 140 160 180 200 frequency bins frequency bins Algorithm 1 Algorithm 1 8 8 6 6 P 1 (f) P 2 (f) 4 4 2 2 0 0 0 20 40 60 80 100 120 140 160 180 200 0 20 40 60 80 100 120 140 160 180 200 frequency bins frequency bins User 1 User 2 foresighted Foresighted action: • – interference avoidance instead of water-filling Collaboration – leads to improved performance for both users! • Su, vdSchaar – 2007
Multi-user Power Control Games in Interference Channels Insights • – Value of information – Foresighted action is beneficial even if others are myopic (coopetition) How to acquire the • required information? – Estimation – Information exchange Fu, vd Schaar - 2007 – Multi-agent Learning Su, vdSchaar – 2007
Centralized general stochastic game model Numerous networking games: - Networks: 802.11 nets – polling based, Cellular nets, Cognitive radio nets - Existing work focuses - on one stage, static games and equilibrium characterization rather than on how to arrive at/select an equilibrium
How to play the stochastic game? , t t ∈ t t o ο i ⊂ O h Observation – part of the game’s history • i i [ ] t t t t t t = π o π × a b , ( ) : � O A B Policy i i i i • i i i i ( ) ( ) 0 t t t t t t π π π = π π = π π = π ( ,..., ,...) 1 ,..., , , − i i i M i i t t t t t t β π = π π ( ) arg max Q (( , ) | s , w ) Best response: • − i i i i -i π i How to solve this problem? Multi-agent learning! Fu, vdSchaar – 2006
Multi-agent learning - definition L as: We define a learning algorithm i ( ) ⎡ ⎤ = t t t t t t t π a b , s B , , B , B ⎢ ⎥ ⎣ ⎦ i i i i w s π − i − i Output of the multi-user interaction game : t ( t t t ) Ω = Game s a , , w Observation of SU i ( ) t t t t = Ω o O s , , b , i i i i where O is the observation function which depends on the current state, the current game output and the current internal action taken. Policy update: ( ) t +1 t t t π = F π , o I , − i i i i i F is the update function about the belief and policies t I − is the exchanged information with the other SUs i π s Beliefs about the other SUs’ states and the network resource state w : , policies − − i i ) ) ) t +1 ( t t t t +1 ( t t t t +1 ( t t t B = B , o I , B = F B , o I − , B = B , o I , F F , , π π π i − i w w w i i s s s i − i − i − i − i − i − i − i
Multi-agent learning - illustration t t π a ,b t s i i i i t o i Explicit information I − i exchange Solutions depend on the information availability: - Reinforcement learning (no explicit modeling of other users) - Fictitious Play (explicit modeling of other users – needs to know what actions opponents took, but not their strategies) - Regret Matching - Model-based Fu, vdSchaar – 2006 Shiang, vdSchaar – 2007
Multi-agent learning - illustration t t π a ,b t s i i i i t o i Explicit information I − i exchange Value of Learning T 1 ( ) o I , L ∑ i ) ( o I , π i − i L t = π ( ) T R ( ) V − i i i i i i T = t 1 How much to learn for a desired performance (utility)?
Proposed Goal * Next generation multi-user networks should explicitly consider strategic behavior of users, dynamics, heterogeneity, information availability and decentralized knowledge * Collaboration should be reached by mutual agreement rather than being imposed on users • Opens opportunities for new theoretical foundations and algorithm designs, new metrics needed • Performance improvements • Backwards compatibility to existing protocols should be respected
For more information on our research paper on Learning, Distributed Decision Making and Games in Networking and Computing Systems see our group’s website: http://medianetlab.ee.ucla.edu/
Recommend
More recommend
