Negative Momentum for Improved Game Dynamics Gauthier Gidel* , - - PowerPoint PPT Presentation

Negative Momentum for Improved Game Dynamics

Gauthier Gidel*, Reyhane Askari Hemmat*, Mohammad Pezeshki, Gabriel Huang, Remi Lepriol, Simon Lacoste-Julien, Ioannis Mitliagkas

*equal contribution

Simple Min-max smooth game:

Gradient dynamic: Gauthier Gidel, Workshop on learning and strategic behavior, August 22, 2018

Simple Min-max smooth game:

Gradient dynamic: Gauthier Gidel, Workshop on learning and strategic behavior, August 22, 2018

Simple Min-max smooth game:

Gradient dynamic: Gauthier Gidel, Workshop on learning and strategic behavior, August 22, 2018

Way to optimize bilinear games

Gauthier Gidel, Workshop on learning and strategic behavior, August 22, 2018

Gauthier Gidel, Workshop on learning and strategic behavior, August 22, 2018

Way to optimize bilinear games

Gauthier Gidel, Workshop on learning and strategic behavior, August 22, 2018

Way to optimize bilinear games

Gauthier Gidel, Workshop on learning and strategic behavior, August 22, 2018

Way to optimize bilinear games

This talk

> >

(Improvements) (Improvements) Gauthier Gidel, Workshop on learning and strategic behavior, August 22, 2018

Way to optimize bilinear games

General 2 player games:

Gauthier Gidel, Workshop on learning and strategic behavior, August 22, 2018 Two players aim to minimize their respective cost functions:

General 2 player games:

Gauthier Gidel, Workshop on learning and strategic behavior, August 22, 2018 Two players aim to minimize their respective cost functions: Examples:

• Simple class of zero-sum games: ( )

General 2 player games:

Gauthier Gidel, Workshop on learning and strategic behavior, August 22, 2018 Two players aim to minimize their respective cost functions: Examples:

• Simple class of zero-sum games: ( )
• Generative Adversarial Networks:

(non-saturating GAN from Goodfellow et al. 2014)

General 2 player games:

Gauthier Gidel, Workshop on learning and strategic behavior, August 22, 2018 Two players aim to minimize their respective cost functions: Dynamics of gradient based method depends on the gradient vector fields:

General 2 player games:

Gauthier Gidel, Workshop on learning and strategic behavior, August 22, 2018 Two players aim to minimize their respective cost functions: Dynamics of gradient based method depends on the gradient vector fields: And its associated Jacobian,

Fixed point dynamics

Gauthier Gidel, Workshop on learning and strategic behavior, August 22, 2018 Gradient method is defined as the repetition of the operator: Thus, the sequence computed is

Fixed point dynamics

Gauthier Gidel, Workshop on learning and strategic behavior, August 22, 2018 Gradient method is defined as the repetition of the operator: Thus, the sequence computed is We aim to converge to a Nash Equilibrium:

Tuning the step size

Gauthier Gidel, Workshop on learning and strategic behavior, August 22, 2018 Jacobian of our fixed point operator:

• To have fixed point we need to be definite positive.
• Thus, small enough step-size Eigenvalues in the unit disk.
• Want to find optimal step-size.

Fixed point dynamics

Gauthier Gidel, Workshop on learning and strategic behavior, August 22, 2018

• Local convergence.
• Stationary point may not be a Nash equilibrium. (See Adolphs et al. 2018)
• But any Nash equilibrium is an stationary point.
• In this talk: local results on stationary points.

Jacobian of our fixed point operator:

Tuning the step size

Gauthier Gidel, Workshop on learning and strategic behavior, August 22, 2018

Negative Momentum

Gauthier Gidel, Workshop on learning and strategic behavior, August 22, 2018 Fixed point operator requires a state augmentation : (because need previous iterates) Recall Polyak’s momentum :

Negative Momentum

Gauthier Gidel, Workshop on learning and strategic behavior, August 22, 2018

Negative Momentum

Gauthier Gidel, Workshop on learning and strategic behavior, August 22, 2018

• Fixed momentum.

(- 0.25)

• Step-size is not fixed.
• Helps when the eigenvalue

has large imaginary part.

What happens in practice ?

Gauthier Gidel, Workshop on learning and strategic behavior, August 22, 2018 Fashion MNIST:

What happen in practice ?

Gauthier Gidel, Workshop on learning and strategic behavior, August 22, 2018 CIFAR-10:

Negative Momentum

Gauthier Gidel, Workshop on learning and strategic behavior, August 22, 2018 To sum up:

• Negative momentum seems to improve the behaviour of the “bad” eigenvalues.
• If small enough seems to always help.
• It also allows larger step-size.

Thank you !

Gauthier Gidel, Workshop on learning and strategic behavior, August 22, 2018 If you are interested in that topic:

• NIPS Workshop : Smooth Games Optimization and Machine Learning

Co-organized with: Simon Lacoste-Julien · Ioannis Mitliagkas · Vasilis Syrgkanis · Eva Tardos · Leon Bottou · Sebastian Nowozin Soon : Call for contributions !!!