Improved Zeroth-Order Variance Reduced Algorithms and Analysis for - - PowerPoint PPT Presentation
Improved Zeroth-Order Variance Reduced Algorithms and Analysis for Nonconvex Optimization Kaiyi Ji 1 , Zhe Wang 1 , Yi Zhou 2 , Yingbin Liang 1 1 Ohio State University, 2 Duke University ICML 2019 K. Ji, Z. Wang, Y. Zhou, Y. Liang Zeroth-Order
1Ohio State University, 2Duke University
(The Ohio State University) Zeroth-Order Nonconvex Optimization ICML 2019 1 / 8
min
x∈Rd f (x) := 1
n
n
fi(x)
◮ fi(·): individual nonconvex loss function ◮ Gradient of fi(·) is unknown ◮ Only the function value of fi(·) is accessible ◮ Examples:
Generation of black-box adversarial samples Parameter optimization for black-box systems Action exploration in reinforcement learning
Generating black-box adversarial samples
(The Ohio State University) Zeroth-Order Nonconvex Optimization ICML 2019 2 / 8
min
x∈Rd f (x) := 1
n
n
fi(x)
◮ f (·) is bounded below, i.e., f ∗ = infx∈Rd f (x) > −∞ ◮ ∇fi(·) is L-smooth, i.e.,
∇fi(x) − ∇fi(y) ≤ Lx − y
◮ (Online case) ∇fi(·) has bounded variance, i.e., there exists σ > 0 s.t.
1 n
n
∇fi(x) − ∇f (x)2 ≤ σ2
(The Ohio State University) Zeroth-Order Nonconvex Optimization ICML 2019 3 / 8
gs = ˆ ∇randf (xs
0, us 0)
vs
t = 1
|B|
ˆ ∇randfi(xs
t; us t) − ˆ
∇randfi(xs
0; us 0)
gs,
∇randfi(xs
t, us t) = d β (fi(xs t + βus t) − fi(xs t))us t
t: smoothing vector; β: smoothing parameter
(The Ohio State University) Zeroth-Order Nonconvex Optimization ICML 2019 4 / 8
gs = ˆ ∇randf (xs
0, us 0)
vs
t = 1
|B|
ˆ ∇randfi(xs
t; us t) − ˆ
∇randfi(xs
0; us 0)
gs,
∇randfi(xs
t, us t) = d β (fi(xs t + βus t) − fi(xs t))us t
t: smoothing vector; β: smoothing parameter Algorithms Convergence rate # of function queries ZO-SGD O(
O(dǫ−2) ZO-SVRG O(d/T + 1/|B|) O(dǫ−2 + nǫ−1)
◮ Issue: ZO-SVRG has worse query complexity than ZO-SGD
(The Ohio State University) Zeroth-Order Nonconvex Optimization ICML 2019 4 / 8
gs = ˆ ∇coordfS(xk)
◮ As a comparison, ZO-SVRG uses ˆ
gs = ˆ ∇randf (xs
0, us 0)
vs
t = 1
|B|
ˆ ∇randfi(xs
t;
us
i,t
t
) − ˆ ∇randfi(xs
0;
us
i,t
)
gs,
Algorithms Convergence rate Function query complexity ZO-SGD O(
O(dǫ−2) ZO-SVRG O(d/T + 1/|B|) O(dǫ−2 + nǫ−1) ZO-SVRG-Coord-Rand O(1/T) O
(The Ohio State University) Zeroth-Order Nonconvex Optimization ICML 2019 5 / 8
gs = ˆ ∇coordfS(xk)
vs
t = 1
|B|
ˆ ∇coordfi(xs
t; us i,t) − ˆ
∇coordfi(xs
0; us i,t)
gs,
Algorithms Stepsize Convergence rate Function query complexity ZO-SVRG-Coord O( 1
d )
O( d
T )
O
ǫ + dn ǫ
O(1) O( 1
T )
O
ǫ5/3 , dn2/3 ǫ
(The Ohio State University) Zeroth-Order Nonconvex Optimization ICML 2019 6 / 8
◮ nonconvex nonsmooth optimization ◮ convex smooth optimization ◮ Polyak-
Lojasiewicz (PL) condition
450 900 1350 1800 # of iterations 8 10 12 14 16 18 20 Loss ZO-SGD ZO-SVRG-Ave SPIDER-SZO ZO-SVRG-Coord ZO-SVRG-Coord-Rand ZO-SPIDER-Coord 1 2 3 4 # of function queries 105 8 10 12 14 16 Loss ZO-SGD ZO-SVRG-Ave SPIDER-SZO ZO-SVRG-Coord ZO-SVRG-Coord-Rand ZO-SPIDER-Coord 200 400 600 800 1000 # of iterations 8 10 12 14 16 18 20 Loss ZO-SGD ZO-SVRG-Ave SPIDER-SZO ZO-SVRG-Coord ZO-SVRG-Coord-Rand ZO-SPIDER-Coord 1 2 3 4 5 # of function queries 105 7 10 13 16 17 Loss ZO-SGD ZO-SVRG-Ave SPIDER-SZO ZO-SVRG-Coord ZO-SVRG-Coord-Rand ZO-SPIDER-Coord
Generating black-box adversarial examples for DNNs
(The Ohio State University) Zeroth-Order Nonconvex Optimization ICML 2019 7 / 8
(The Ohio State University) Zeroth-Order Nonconvex Optimization ICML 2019 8 / 8