Additively Coupled Sum Constrained Games • Definition – A multi ‐ user interaction in which A2: The utility function satisfies states cost in which is an increasing and strictly concave function. Both and are twice differentiable. 33
Additively Coupled Sum Constrained Games • Definition – A multi ‐ user interaction in which Structure of the utility: additive coupling between action and state A2: The utility function satisfies states cost in which is an increasing and strictly concave function. Both and are twice differentiable. 34
Additively Coupled Sum Constrained Games • Definition – A multi ‐ user interaction in which Structure of the utility: additive coupling between action and state A2: The utility function satisfies states diminishing return per invested action cost in which is an increasing and strictly concave function. Both and are twice differentiable. 35
Examples of ACSCG • Power control in interference channels 36
Examples of ACSCG • Power control in interference channels 37
Examples of ACSCG • Power control in interference channels 38
Examples of ACSCG (cont’d) • Delay minimization in Jackson networks m k r im j i k r ij k ψ k i r i 0 39
Examples of ACSCG (cont’d) • Delay minimization in Jackson networks m k r im j i k r ij k ψ k i r i 0 40
Examples of ACSCG (cont’d) • Delay minimization in Jackson networks m k r im j i k r ij k ψ k i r i 0 41
Nash equilibrium in ACSCG • Existence of pure NE – A subclass of concave games • When is the NE unique? When does best response converges to such a NE? – Existing literatures are not immediately applicable • Diagonal strict convexity condition [Rosen] • Use gradient play and stepsizes need to be carefully chosen • Super ‐ modular games [Topkis] • Action space is not a lattice • Sufficient conditions for specific and [Yu] 42
Best response dynamics • Best response iteration 43
Best response dynamics • Best response iteration in which is chosen such that • When does it converges? – By intuition, the weaker the mutual coupling is, the more likely it converges – How to measure and quantify this coupling strength? 44
Best response dynamics sum constraint additive coupling • Best response iteration in which is chosen such that state • When does it converges? A competition scenario in which every user aggressively uses up all his resources – By intuition, the weaker the mutual coupling is, the more likely it converges – How to measure and quantify this coupling strength? 45
Best response dynamics sum constraint additive coupling • Best response iteration in which is chosen such that state • When does it converges? – By intuition, the weaker the mutual coupling is, the more likely it converges – How to measure and quantify this coupling strength? 46
A measure of the mutual coupling Define represents the maximum impact that user m’s action can make over user n’s state 47
Convergence conditions Theorem 1 : If then best response dynamics converges linearly to a unique pure NE for any set of initial conditions. • The contraction factor is a measure of the overall coupling strength • If is affine, the condition in Theorem 1 is not impacted by ; otherwise it may depend on . 48
Convergence conditions Theorem 1 : If then best response dynamics converges linearly to a unique pure NE for any set of initial conditions. Contraction mapping • The contraction factor is a measure of the overall coupling strength • If is affine, the condition in Theorem 1 is not impacted by ; otherwise it may depend on . 49
Convergence conditions Theorem 1 : If then best response dynamics converges linearly to a unique pure NE for any set of initial conditions. Contraction mapping • The contraction factor is a measure of the overall coupling strength • If is affine, the condition in Theorem 1 is not impacted by ; otherwise it may depend on . is a constant for affine 50
Convergence conditions • If have the same sign, the condition in Theorem 1 can be relaxed to • This is true in many communication scenarios – Increasing power causes stronger interference – Increasing input rate congests the server 51
Convergence conditions • If have the same sign, the condition in Theorem 1 can be relaxed to • This is true in many communication scenarios – Increasing power causes stronger interference – Increasing input rate congests the server Strategic complements (or strategic substitutes) 52
A special class of For , define [Walrand] 53
A special class of For , define [Walrand] Define A measure of the similarity between users’ parameters 54
Convergence conditions Theorem 2 : If then best response dynamics converges linearly to a unique pure NE for any set of initial conditions. 55
Convergence conditions Theorem 2 : If then best response dynamics converges linearly to a unique pure NE for any set of initial conditions. Contraction mapping Theorem 1 Theorem 1 Theorem 2 Theorem 2 56
Conclusion so far… If Information is constrained and no message passing is available… Pareto boundary u 2 When will it c o nverge Concave games ? And ho w fast ? to a NE ACSCG Power control, Nash equilibrium Flow control u 1 57
Conclusion so far… If Information is constrained and no message passing is available… Pareto boundary u 2 When will it c o nverge Concave games ? And ho w fast ? to a NE Suffic ient c o nditio ns that ACSCG guarantee linear c o nvergenc e Power control, Nash equilibrium Flow control u 1 58
Power control as an ACSCG • Power control in interference channels 59
Performance comparison • Solutions without information exchange – Iterative water ‐ filling algorithm [Yu] k P n ∑ ∑ k k k k H H P P mn mn m m ≠ ≠ m m n n k k σ σ n n k k user n ’s spectrum • Solutions with information exchange ∑ ω max R k k k 60
Performance comparison • Solutions without information exchange – Iterative water ‐ filling algorithm [Yu] k P n OSB = Optimal Spectrum ∑ ∑ k k k k H H P P Balancing mn mn m m ≠ ≠ m m n n k k σ σ n n ASB = Autonomous k k user n ’s spectrum Spectrum Balancing • Solutions with information exchange ∑ ω max R k k k 61
Outline • Motivation and existing approaches • Informationally efficient multi ‐ user communication – Vector cases • Convergence conditions with decentralized information • Improve efficiency with decentralized information – Scalar cases • Achieve Pareto efficiency with decentralized information • Conclusions 62
How to model the mutual coupling • A reformulation of the coupling =× – State space S S ∈ n n N × → u : S A R – Utility function n n n → s : – State determination function A S − n n n → � s : A S – Belief function n n n – Conjectural Equilibrium (CE) : a configuration of ∗ ∗ ∗ ∗ ∗ � � = belief functions and joint action ( s , � , s ) a ( a , � , a ) 1 N 1 N satisfying ( ) ∗ ( ∗ ) ∗ ∗ ∗ ( ) � = s a s ( a ) and = � a arg max u s a , a − n n n n n n n n n ∈ a A n n 63
How to model the mutual coupling • A reformulation of the coupling it captures the =× – State space S S aggregate effect of ∈ n n N the other users’ actions × → u : S A R – Utility function n n n → s : – State determination function A S − n n n it models the aggregate effect → � s : A S – Belief function n n n of the other users’ actions – Conjectural Equilibrium (CE) : a configuration of ∗ ∗ ∗ ∗ ∗ � � = belief functions and joint action ( s , � , s ) a ( a , � , a ) 1 N 1 N satisfying ( ) ∗ ( ∗ ) ∗ ∗ ∗ ( ) � = s a s ( a ) and = � a arg max u s a , a − n n n n n n n n n ∈ a A n n 64
How to model the mutual coupling • A reformulation of the coupling it captures the =× – State space S S aggregate effect of ∈ n n N the other users’ actions × → u : S A R – Utility function n n n → s : – State determination function A S − n n n it models the aggregate effect → � s : A S – Belief function n n n of the other users’ actions – Conjectural Equilibrium (CE) : a configuration of ∗ ∗ ∗ ∗ ∗ � � = belief functions and joint action ( s , � , s ) a ( a , � , a ) 1 N 1 N satisfying ( ) ∗ ( ∗ ) ∗ ∗ ∗ ( ) � = s a s ( a ) and = � a arg max u s a , a − n n n n n n n n n ∈ a A n n each user behaves optimally beliefs are realized according to its expectation 65
CE in power control games [SuTSP’09] • One leader and multiple followers • State space k : the interference caused to user n in channel k I – n • Utility function ⎛ ⎞ k K P ⎟ ⎜ ∑ ⎟ n = ⎜ + R log 1 ⎟ ⎜ n 2 ⎟ ⎜ k k σ + ⎝ I ⎠ = n n k 1 • State determination function actual play = ∑ N k k k α I P n in i i = i ≠ n 1, • Belief function (linear form) conceived play � k k k k = β − γ I P 1 1 66
Why Linear belief? k k ∂ ∂ I I 1 1 is piece ‐ wise linear; , if the = ≠ 0, j k j k ∂ ∂ P P 1 1 number of frequency bins is sufficiently large. � Linear belief is sufficient to capture the interference coupling! 67
Why Linear belief? k k ∂ ∂ I I 1 1 is piece ‐ wise linear; , if the = ≠ 0, j k j k ∂ ∂ P P 1 1 number of frequency bins is sufficiently large. � Linear belief is sufficient to capture the interference coupling! f P 2 f f H P 12 1 f σ 2 f 68
Why Linear belief? k k ∂ ∂ I I 1 1 is piece ‐ wise linear; , if the = ≠ 0, j k j k ∂ ∂ P P 1 1 number of frequency bins is sufficiently large. � Linear belief is sufficient to capture the interference coupling! f f P P 2 2 f f f f H P H P 12 1 12 1 f f σ σ 2 2 f f 69
Why Linear belief? k k ∂ ∂ I I 1 1 is piece ‐ wise linear; , if the = ≠ 0, j k j k ∂ ∂ P P 1 1 number of frequency bins is sufficiently large. � Linear belief is sufficient to capture the interference coupling! f f f P P P 2 2 2 f f f f f f H P H P H P 12 1 12 1 12 1 f f f σ σ σ 2 2 2 f f f 70
Why Linear belief? k k ∂ ∂ I I 1 1 is piece ‐ wise linear; , if the = ≠ 0, j k j k ∂ ∂ P P 1 1 number of frequency bins is sufficiently large. � Linear belief is sufficient to capture the interference coupling! f f f P P P 2 2 2 f f f f f f H P H P H P 12 1 12 1 12 1 f f f σ σ σ 2 2 2 f f f 71
Main results • Stackelberg equilibrium ( ) ( ) * * – Strategy profile that satisfies a , NE a 1 1 ( ) ( ) ( ) * * ( ) ≥ ∀ ∈ A u a , NE a u a NE a , , a 1 1 1 1 1 1 1 1 • NE and SE are special CE SE R • 1 R 1 NE N R ∑ 1 k k k k • NE: β = α γ = P , 0 i 1 i = i 2 k k ∂ ∂ I I k k k k 1 1 β = − ⋅ γ = − SE: I P , . 1 1 k k ∂ ∂ P P β 1 1 • • Infinite set of CE Open sets of CE that contain • γ NE and SE may exist 72
Achieving the desired CE • Conjecture ‐ based rate maximization (CRM) leader followers solvable using dual method 73
Discussion about CRM • Essence of CRM – local approximation of the computation of SE • Advantages – the structure of the utility function is explored – only local information is required – it can be applied in the cases where N>2 – if it converges, the outcome is a CE 74
Simulation results 1 1 0.9 0.9 0.8 0.8 NE R 1 /R 1 0.7 0.7 NE R 2 /R 2 NE R 1 /R 1 0.6 0.6 NE R 2 /R 2 NE R 3 /R 3 0.5 0.5 0.4 0.4 0.3 0.3 0.2 0.2 0.1 0.1 0 0 0.5 1 1.5 2 2.5 3 0.8 1 1.2 1.4 1.6 1.8 2 Average rate improvements: ( ) 2 ∑ k α = ≠ 0.5, i j 2 ‐ user case: 24.4% for user 1; 33.6% for user 2 ij k ( ) 2 ∑ k 3 ‐ user case: 26.3% for user 1; 9.7% for user 2&3 α = ≠ 0.33, i j ij k 75
Conclusions so far… If Information is constrained and no message passing is allowed Pareto boundary u 2 Concave games H o w to impro ve an ineffic ient NE witho ut message passing ? ACSCG Power control Nash equilibrium u 1 76
Conclusions so far… If Information is constrained and no message passing is allowed Pareto boundary u 2 Concave games H o w to impro ve an ineffic ient NE witho ut message passing ? Overall effic ienc y may be impro ve d! ACSCG Build belief, learn, and adapt Power control Nash equilibrium u 1 77
Outline • Motivation and existing approaches • Informationally efficient multi ‐ user communication – Vector cases • Convergence conditions with decentralized information • Improve efficiency with decentralized information – Scalar cases • Achieve Pareto efficiency with decentralized information • Conclusions 78
Linearly coupled games • A non ‐ cooperative game model • Users’ states are linearly impacted by their competitor’s actions • Contributions – Characterize the structures of the utility functions – Explicitly compute Nash equilibrium and Pareto boundary – A conjectural equilibrium approach to achieve Pareto boundary without real ‐ time information exchange 79
Definition A multi ‐ user interaction is considered a linearly coupled game if the action set is convex and the utility function satisfies in which . In particular, the basic assumptions about include: States are linearly impacted by actions A1: is non ‐ negative; A2: is strictly linearly decreasing in ; is non ‐ increasing and linear in . 80
Definition (cont’d) Denote . A3: is an affine function, A4: Actions are linearly coupled at NE and PB 81
Two basic types • For the games satisfying A1 ‐ A4, the utility functions can take two types of form: – Type I [SuJSAC’10] • e.g. random access – Type II [SuTR’09] • e.g. rate control 82
Two basic types • For the games satisfying A1 ‐ A4, the utility functions can take two types of form: – Type I [SuJSAC’10] • e.g. random access – Type II [SuTR’09] • e.g. rate control 83
Type I games: wireless random access • Player set: Rx 2 Tx 1 – nodes in a single cell • Action set: Rx K – transmission probability Tx K • Payoff: Rx 1 Tx 2 – throughput • Key issues – stability, convergence, throughput, and fairness 84
Conjecture ‐ based Random Access • Individual conjectures actual play – state: conceived play – linear belief: • Two update mechanisms – Best response – Gradient play 85
Main results Protocol design: how to achieve efficient outcomes? • Existence of CE – all operating points in action space are CE • Stability and convergence – sufficient conditions • Throughput performance – the entire throughput region can be achieved with stable CE • Fairness issue – conjecture ‐ based approaches attain weighted fairness 86
How to select suitable a k ? • Adaptively alter a k when the network size changes • Adopt aggregated throughput or “idle interval” as the indicator of the system efficiency • Advantages – No need of a centralized solver – Throughput efficient with fairness guarantee – Stable equilibrium – Autonomously adapt to traffic fluctuation 87
Engineering interpretation • DCF vs. the best response update – re ‐ design the random access protocol 88
Engineering interpretation • DCF vs. the best response update – re ‐ design the random access protocol similar different 89
Engineering interpretation • DCF vs. the best response update – re ‐ design the random access protocol similar different CBRA makes use of 4-bit information, while DCF only uses 2 bits 90
Simulation results • Throughput • Stability and convergence Accumulative throughput (Mbps) 36 36 P-MAC 35 Best response 35.5 Gradient play 34 35 Accumulative throughput (Mbps) Optimal throughput 33 P-MAC 34.5 Conjecture-based algorithms 32 IEEE 802.11 DCF 34 31 33.5 30 33 29 32.5 28 32 27 31.5 26 25 31 5 10 15 20 25 30 35 40 45 50 0 100 200 300 400 500 600 Number of nodes DCF: low throughput; P ‐ MAC: instability due to the online estimation P ‐ MAC: needs to know the number of nodes 91
Conventional solutions in Type II games • Utility function • Nash equilibrium • Pareto boundary • Efficiency loss 92
Best response dynamics in Type II games Observed state Linear belief • At stage t , • Theorem 5 : A necessary and sufficient condition for the best response dynamics to converge is Determine the eigenvalues of the Jacobian matrix 93
Stability of the Pareto boundary • Theorem 6 : All the operating points on the Pareto boundary are globally convergent CE under the best response dynamics. The belief configurations lead to Pareto ‐ optimal operating points if and only if : the ratio between the immediate – performance degradation and the conjectured long ‐ term effect Theorem 5 and expressions of Pareto boundary and CE 94
Pricing vs. conjectural equilibrium • Pricing mechanism in communication networks [Kelly][Chiang] – Users repeatedly exchange coordination signals • Conjectural equilibrium for linearly coupled games – Coordination is implicitly implemented when the participating users initialize their belief parameters – Pareto ‐ optimality can be achieved solely based on local observations on the states – No message passing is needed during the convergence process – The key problem is how to design belief functions 95
Conclusions so far… Can we still ac hieve Pareto o ptimality ? u 2 Pareto boundary Global (exchanged) information Concave games If Information is constrained and no message passing is available… LCG Decentralized (insufficient) information Random Access, Nash equilibrium Rate control Decentralized (limited) information u 1 The optimal way of designing the beliefs and updating the actions based on conjectural equilibrium is addressed 96
Conclusions so far… Can we still ac hieve Pareto o ptimality ? u 2 Pareto boundary Global (exchanged) information Concave games If Information is constrained and Pareto o ptimality no message passing is available… c an be ac hieved! Conjectural equilibrium LCG Decentralized (insufficient) information Random Access, Nash equilibrium Rate control Decentralized (limited) information u 1 The optimal way of designing the beliefs and updating the actions based on conjectural equilibrium is addressed 97
Conclusions • We define new classes of games emerging in multi ‐ user communication networks and investigate the information and efficiency trade ‐ off – Provide sufficient convergence conditions to NE – Suggest a conjectural equilibrium based approach to improve efficiency – Quantify the performance improvement 98
References • J. Rosen, “Existence and uniqueness of equilibrium points for concave n ‐ person games,” Econometrica , vol. 33, no. 3, pp. 520 ‐ 534, Jul. 1965. • D. Monderer and L. S. Shapley, “Potential games,” Games Econ. Behav. , vol. 14, no. 1, pp. 124 ‐ 143, May 1996. • D. Topkis, Supermodularity and Complementarity . Princeton University Press, Princeton, 1998. • F. Kelly, A. K. Maulloo, and D. K. H. Tan, “Rate control in communication networks: shadow prices, proportional fairness and stability,” Journal of the Operational Research Society , vol. 49, pp. 237 ‐ 252, 1998. • M. Chiang, S. H. Low, A. R. Calderbank, and J. C. Doyle, “Layering as optimization decomposition: A mathematical theory of network architectures,” Proc. of the IEEE , vol. 95, no. 1, pp. 255 ‐ 312, January 2007. 99
References (cont’d) • W. Yu, G. Ginis, and J. Cioffi, “Distributed multiuser power control for digital subscriber lines,” IEEE J. Sel. Areas Commun. , vol. 20, no. 5, pp. 1105 ‐ 1115, June 2002. • J. Mo and J. Walrand, “Fair end ‐ to ‐ end window ‐ based congestion control,” IEEE Trans. on Networking , vol. 8, no. 5, pp. 556 ‐ 567, Oct. 2000. 100
Recommend
More recommend
