Faster Stochastic Alternating Direction Method of Multipliers for - PowerPoint PPT Presentation

Faster Stochastic Alternating Direction Method of Multipliers for Nonconvex Optimization Feihu Huang 1 , Songcan Chen 2,3 and Heng Huang 1,4* 1. Department of Electrical and Computer Engineering, University of Pittsburgh, USA 2. College of Computer Science & Technology, Nanjing University of Aeronautics & Astronautics 3. MIIT Key Laboratory of Pattern Analysis and Machine Intelligence 4. JD Finance America Corporation * heng.huang@pitt.edu ICML-2019 Long Beach, USA

Outline • Background • Faster Stochastic ADMM Methods • Convergence Analysis: Lower IFO Complexity • Experiments

Current Data Current data not only has large sample size, but also contains some complex structures.

Problem Statement • A finite/infinite-sum constraint problem encoding some complex encoding empirical or structures expected loss for big data

Contributions In the paper, our main contributions are summarized as follows:

IFO Complexity

Faster Stochastic ADMM (SPIDER-ADMM)

Convergence Analysis

Online SPIDER-ADMM

Experiments 1. Graph-Guided Binary Classification datasets # samples # features # classes a9a 32,561 123 2 w8a 64,700 300 2 ijcnn1 126,702 22 2 covtype.binary 581,012 54 2

Experiments

Experiments 2. Multi-Task Learning

Experiments datasets # samples # features # classes letter 15,000 16 26 sensorless 58,509 48 11 mnist 60,000 780 10 covtype 581,012 54 7

Thanks! Poster: Pacific ballroom #93