Motivation Geometric Preliminaries The Dynamics Asymptotic Analysis Well-posedness . . . . . . . . . . . . . . . . . . Inertial Game Dynamics R. Laraki § P. Mertikopoulos ∗ § CNRS – LAMSADE laboratory ∗ CNRS – LIG laboratory ADGO'13 – Playa Blanca, October 15, 2013 P. Mertikopoulos CNRS – Laboratoire d'Informatique de Grenoble Sunday, October 7, 2012
Motivation Geometric Preliminaries The Dynamics Asymptotic Analysis Well-posedness . . . . . . . . . . . . . . . . . . Motivation Main Idea: use second order tools to derive efficient learning algorithms in games. The second order exponential learning dynamics (Rida's talk) have many pleasant properties, but also various limitations: ▸ Cannot converge to interior equilibria (not a problem in many applications, desirable in others). ▸ Convex programming properties not clear – no damping mechanism. ▸ Lack of a bona fide "heavy ball with friction" interpretation. In this talk: use geometric ideas to derive a class of inertial (= admitting an energy function), second order dynamics for learning in games. P. Mertikopoulos CNRS – Laboratoire d'Informatique de Grenoble Sunday, October 7, 2012
Motivation Geometric Preliminaries The Dynamics Asymptotic Analysis Well-posedness . . . . . . . . . . . . . . . . . . Approach Breakdown The main steps of our approach will be as follows: 1. Endow the simplex with a Hessian Riemannian geometric structure. 2. Derive the equations of motion for a learner under the forcing of his unilateral gradient (taken w.r.t. the HR geometry on the simplex). 3. Derive an isometric embedding of the problem into an ambient Euclidean space. 4. Establish the well-posedness of the dynamics. 5. Use the system's energy function to derive the dynamics' asymptotic properties. P. Mertikopoulos CNRS – Laboratoire d'Informatique de Grenoble Sunday, October 7, 2012
Motivation Geometric Preliminaries The Dynamics Asymptotic Analysis Well-posedness . . . . . . . . . . . . . . . . . . Notation We will work with finite games G ≡ G ( N , A , u ) consisting of: ▸ A finite set of players: N = { , . . . , N } . ▸ The players' action sets A k = { α k , , α k , , . . . } , k ∈ N . ▸ The players' payoff functions u k ∶ A ≡ ∏ k A k → R , extended multilinearly to X ≡ ∏ k ∆ ( A k ) if players use mixed strategies x k ∈ X k ≡ ∆ ( A k ) . Note: indices will be suppressed when possible. Special case: if u k α ( x ) − u k β ( x ) = − [ V ( α ; x − k ) − V ( β ; x − k )] for some V ∶ X → R , the game is called a potential game. Equilibrium: we will say that q ∈ X is a Nash equilibrium of G if u k α ( q ) ≥ u k β ( q ) for all α ∈ supp ( q k ) , β ∈ A k , k ∈ N . P. Mertikopoulos CNRS – Laboratoire d'Informatique de Grenoble Sunday, October 7, 2012
The gradient of a scalar function with respect to is defined as: or, in components, where is the array of partial derivatives of . Fundamental property of the gradient: . More generally, the derivative of along a vector field on will be: Motivation Geometric Preliminaries The Dynamics Asymptotic Analysis Well-posedness . . . . . . . . . . . . . . . . . . Riemannian Metrics A Riemannian metric on an open set U ⊆ R m is a smoothly varying scalar product on U ( X , Y ) ≡ ⟨ X , Y ⟩ = ∑ j , k X j jk Y k , X , Y ∈ R m , where ≡ ( x ) is a smooth field of positive-definite matrices on U . P. Mertikopoulos CNRS – Laboratoire d'Informatique de Grenoble Sunday, October 7, 2012
Fundamental property of the gradient: . More generally, the derivative of along a vector field on will be: Motivation Geometric Preliminaries The Dynamics Asymptotic Analysis Well-posedness . . . . . . . . . . . . . . . . . . Riemannian Metrics A Riemannian metric on an open set U ⊆ R m is a smoothly varying scalar product on U ( X , Y ) ≡ ⟨ X , Y ⟩ = ∑ j , k X j jk Y k , X , Y ∈ R m , where ≡ ( x ) is a smooth field of positive-definite matrices on U . The gradient of a scalar function V ∶ U → R with respect to is defined as: grad V = − ( ∂ V ) ( grad V ) j = ∑ k − or, in components, jk ∂ k V , where ∂ V = ( ∂ j V ) n j = is the array of partial derivatives of V . P. Mertikopoulos CNRS – Laboratoire d'Informatique de Grenoble Sunday, October 7, 2012
Motivation Geometric Preliminaries The Dynamics Asymptotic Analysis Well-posedness . . . . . . . . . . . . . . . . . . Riemannian Metrics A Riemannian metric on an open set U ⊆ R m is a smoothly varying scalar product on U ( X , Y ) ≡ ⟨ X , Y ⟩ = ∑ j , k X j jk Y k , X , Y ∈ R m , where ≡ ( x ) is a smooth field of positive-definite matrices on U . The gradient of a scalar function V ∶ U → R with respect to is defined as: grad V = − ( ∂ V ) ( grad V ) j = ∑ k − or, in components, jk ∂ k V , where ∂ V = ( ∂ j V ) n j = is the array of partial derivatives of V . dt V ( x ( t )) = ⟨ grad V , ˙ x ⟩ . Fundamental property of the gradient: d More generally, the derivative of V along a vector field X on U will be: ∇ X f ≡ ⟨ d f ∣ X ⟩ = ⟨ grad f , X ⟩ . P. Mertikopoulos CNRS – Laboratoire d'Informatique de Grenoble Sunday, October 7, 2012
Motivation Geometric Preliminaries The Dynamics Asymptotic Analysis Well-posedness . . . . . . . . . . . . . . . . . . Parallel Transport How can we differentiate a vector field along another in a Riemannian setting? Definition Let X , Y be vector fields on U . A connection on U will be a map ( X , Y ) ↦ ∇ X Y s.t.: 1. ∇ f X + f X Y = f ∇ X Y + f ∇ X Y ∀ f , f ∈ C ∞ ( U ) . 2. ∇ X ( aY + bY ) = a ∇ X Y + b ∇ X Y for all a , b ∈ R . 3. ∇ X ( f Y ) = f ⋅ ∇ X Y + ∇ X f ⋅ Y for all f ∈ C ∞ ( U ) . In components: ( ∇ X Y ) k = ∑ i X i ∂ i Y k + ∑ i , j Γ k i j X i Y j , where Γ k i j are the connection's Christoffel symbols. P. Mertikopoulos CNRS – Laboratoire d'Informatique de Grenoble Sunday, October 7, 2012
Motivation Geometric Preliminaries The Dynamics Asymptotic Analysis Well-posedness . . . . . . . . . . . . . . . . . . Covariant Differentiation A Riemannian metric generates the so-called Levi-Civita connection with symbols k ℓ ( ∂ i ℓ j + ∂ j ℓ i − ∂ ℓ i j ) Γ k i j = ∑ ℓ − This leads to the notion of covariant differentiation along a curve x ( t ) of U : ( ∇ ˙ x X ) k ≡ ˙ X k + ∑ i , j Γ k i j X i ˙ x j If the field being differentiated is the velocity of x ( t ) , we obtain the acceleration of x ( t ) D x k x k + ∑ i , j Γ k Dt = ¨ i j ˙ x i ˙ x j . Definition A geodesic on U is a curve x ( t ) with zero acceleration: D x Dt = . P. Mertikopoulos CNRS – Laboratoire d'Informatique de Grenoble Sunday, October 7, 2012
Examples The Shahshahani metric: . The log-barrier metric: . The Euclidean metric (non-example): . Motivation Geometric Preliminaries The Dynamics Asymptotic Analysis Well-posedness . . . . . . . . . . . . . . . . . . Hessian Riemannian Metrics We will be interested in a specific class of Riemannian metrics on the positive orthant > of R m generated by a family of barrier functions. R m Definition Let θ ∶ [ , +∞ ) → R ∪ { +∞ } be a C ∞ function such that θ ( x ) < ∞ for all x > . 1. lim x → + θ ′ ( x ) = −∞ . 2. θ ′′ ( x ) > and θ ′′′ ( x ) < for all x > . 3. The Hessian Riemannian metric generated by θ on R n + > will be ( x ) = Hess ( ∑ k θ ( x k )) i j ( x ) = θ ′′ ( x i ) δ i j . or, in components, The function θ will be called the kernel of . P. Mertikopoulos CNRS – Laboratoire d'Informatique de Grenoble Sunday, October 7, 2012
Motivation Geometric Preliminaries The Dynamics Asymptotic Analysis Well-posedness . . . . . . . . . . . . . . . . . . Hessian Riemannian Metrics We will be interested in a specific class of Riemannian metrics on the positive orthant > of R m generated by a family of barrier functions. R m Definition Let θ ∶ [ , +∞ ) → R ∪ { +∞ } be a C ∞ function such that θ ( x ) < ∞ for all x > . 1. lim x → + θ ′ ( x ) = −∞ . 2. θ ′′ ( x ) > and θ ′′′ ( x ) < for all x > . 3. The Hessian Riemannian metric generated by θ on R n + > will be ( x ) = Hess ( ∑ k θ ( x k )) i j ( x ) = θ ′′ ( x i ) δ i j . or, in components, The function θ will be called the kernel of . Examples ▸ The Shahshahani metric: θ ( x ) = x log x � ⇒ i j ( x ) = δ i j / x j . ▸ The log-barrier metric: θ ( x ) = − log x � ⇒ i j ( x ) = δ i j / x j . ▸ The Euclidean metric (non-example): θ ( x ) = ⇒ i j ( x ) = δ i j . x � P. Mertikopoulos CNRS – Laboratoire d'Informatique de Grenoble Sunday, October 7, 2012
Recommend
More recommend
