Theoretical Analysis of Adversarial Learning: A Minimax Approach - - PowerPoint PPT Presentation

Theoretical Analysis of Adversarial Learning: A Minimax Approach

Zhuozhuo Tu1, Jingwei Zhang2,1, Dacheng Tao1

1The University of Sydney 2The Hong Kong University of Science and Technology

NeurIPS 2019

• E(x,y)∼P [

x′∈N(x) l(h(x′), y)] ◮ lq ◮

• RP (h) = E(x,y)∼P [l(h(x), y)].

• RP (h) = E(x,y)∼P [l(h(x), y)].

RP (h, B) = E(x,y)∼P [

x′∈N(x)l(h(x′), y)],

N(x) = {x′ : x′ − x ∈ B}

• RP (h) = E(x,y)∼P [l(h(x), y)].

RP (h, B) = E(x,y)∼P [

x′∈N(x)l(h(x′), y)],

N(x) = {x′ : x′ − x ∈ B}

• Th : Z → Z

RP (h, B) = RP ′(h)

• Th : Z → Z

RP (h, B) = RP ′(h) P ′ P Wp(P, P ′) ≤ B

• Th : Z → Z

RP (h, B) = RP ′(h) P ′ P Wp(P, P ′) ≤ B RP (h, B) ≤ RB,1(P, h), ∀h ∈ H

• f ∈ F z ∈ Z

λf,z f(z′) − f(z) ≤ λf,zdZ(z, z′) z′ ∈ Z

• f ∈ F z ∈ Z

λf,z f(z′) − f(z) ≤ λf,zdZ(z, z′) z′ ∈ Z δ > 0 1 − δ f ∈ F

RP (f, B) ≤ 1 n n

i=1 f(zi) + λ+ f,PnB + 24C(F)

√n + 12√π √n ΛB · diam(Z) + M

• log( 1

δ)

2n ,

λ+

f,Pn := {λ : EPn(z′∈Z{f(z′) − λdZ(z, z′) − f(z)}) = 0}

• f ∈ F z ∈ Z

λf,z f(z′) − f(z) ≤ λf,zdZ(z, z′) z′ ∈ Z δ > 0 1 − δ f ∈ F

RP (f, B) ≤ 1 n n

i=1 f(zi) + λ+ f,PnB + 24C(F)

√n + 12√π √n ΛB · diam(Z) + M

• log( 1

δ)

2n ,

λ+

f,Pn := {λ : EPn(z′∈Z{f(z′) − λdZ(z, z′) − f(z)}) = 0}

• ◮

RP (f, B) ≤ 1 n n

i=1 f(zi) + λ+ f,PnB + 144

√nΛr √ d + 12√π √n ΛB · (2r + 1) + (1 + Λr)

• log( 1

δ)

2n ,

λ+

f,Pn ≤ i{2yiw · xi, ||w||2}

◮

RP (f, B) ≤ 1 n n

i=1 f(zi) + λ+ f,PnB + 288

γ√n L

i=1 ρisiBW

• L

i=1

bi si 1

2

2 + 12√π √n ΛB · (2B + 1) +

• log(1/δ)

2n ,

λ+

f,Pn ≤ j{2

γ L

i=1 ρi||Ai||σ, 1

γ (M(HA(xj), yj) + HA(xj) − HA(xj))}