AVERA RAGE GE SE SENSITIVIT SITIVITY OF OF GRA GRAPH PH ALGO - - PowerPoint PPT Presentation
AVERA RAGE GE SE SENSITIVIT SITIVITY OF OF GRA GRAPH PH ALGO GORITHM RITHMS Nithin in Varma rma Joint nt work k with Yuich ichi Yos oshid ida 1 Sensitivity of an Algorithm Measure of change in output as a function of change
Joint nt work k with Yuich ichi Yos
ida
1
2
3
4
5
𝑇′ ⊆ 𝑊 𝑇 ⊆ 𝑊 𝐻′ = 𝑊, 𝐹′ ; 𝐹′ ⊆ 𝐹 𝐻 = (𝑊, 𝐹)
6
𝑇′ ⊆ 𝑊 𝑇 ⊆ 𝑊 𝐻′ = 𝑊, 𝐹′ ; 𝐹′ ⊆ 𝐹 𝐻 = (𝑊, 𝐹)
7
𝑇′ ⊆ 𝑊 𝑇 ⊆ 𝑊 𝐻′ = 𝑊, 𝐹′ 𝐻 = (𝑊, 𝐹)
8
– Useful in cases where one has access only to 𝐻′
𝑇′ ⊆ 𝑊 𝑇 ⊆ 𝑊 𝐻′ = 𝑊, 𝐹′ 𝐻 = (𝑊, 𝐹)
9
10
11
Algorithm with low average sensitivity: stable le-on
average erage algorit
12
13
14
15
16
17
18
19
20
21
Probl blem: Given a graph 𝐻 on 𝑜 vertices and two vertices 𝑡, 𝑢, output the 𝑡-𝑢 shortest path Lower Bound: Consider a deterministic algorithm that outputs 𝑄 For any of the 𝑜/2 edges removed from 𝑄, the algorithm has to output 𝑅
22
23
– Generalization of 𝑀1 distance that penalizes ``significant differences" in probabilities on ``really different" solutions
24
– Generalization of 𝑀1 distance that penalizes ``significant differences" in probabilities on ``really different" solutions
25
26
Distribution 𝐸1
Distribution 𝐸2
27
Cost of moving prob. 𝑞 from 𝑇𝑗 to 𝑇
𝑘 is
𝑞 ⋅ Ham 𝑇𝑗, 𝑇
𝑘
Distribution 𝐸1
Distribution 𝐸2
28
Optimal cost of moving the probability mass from one distribution to the other Distribution 𝐸1
Distribution 𝐸2
Generalization to 𝑙-average sensitivity for the removal of 𝑙 random edges (without replacement)
29
Optimal cost of moving the probability mass from one distribution to the other Distribution 𝐸1
Distribution 𝐸2
30
– Edge Differential Privacy [Nissim Raskhodnikova Smith '07]
■ An algorithm 𝐵 on a graph 𝐻 is differentially private if for all 𝑓 ∈ 𝐹 the distributions 𝐵(𝐻) and 𝐵(𝐻 − 𝑓) are close to each other
31
– Edge Differential Privacy [Nissim Raskhodnikova Smith '07]
■ An algorithm 𝐵 on a graph 𝐻 is differentially private if for all 𝑓 ∈ 𝐹 the distributions 𝐵(𝐻) and 𝐵(𝐻 − 𝑓) are close to each other
– Much stricter notion than average sensitivity
32
– Edge Differential Privacy [Nissim Raskhodnikova Smith '07]
■ An algorithm 𝐵 on a graph 𝐻 is differentially private if for all 𝑓 ∈ 𝐹 the distributions 𝐵(𝐻) and 𝐵(𝐻 − 𝑓) are close to each other
– Much stricter notion than average sensitivity – Some of our algorithms inspired by differentially private algorithms
33
– Edge Differential Privacy [Nissim Raskhodnikova Smith '07]
■ An algorithm 𝐵 on a graph 𝐻 is differentially private if for all 𝑓 ∈ 𝐹 the distributions 𝐵(𝐻) and 𝐵(𝐻 − 𝑓) are close to each other
– Much stricter notion than average sensitivity – Some of our algorithms inspired by differentially private algorithms
34
– Edge Differential Privacy [Nissim Raskhodnikova Smith '07]
■ An algorithm 𝐵 on a graph 𝐻 is differentially private if for all 𝑓 ∈ 𝐹 the distributions 𝐵(𝐻) and 𝐵(𝐻 − 𝑓) are close to each other
– Much stricter notion than average sensitivity – Some of our algorithms inspired by differentially private algorithms
– A learner is stable if empirical loss does not change much by replacing any sample in the training data
35
– Edge Differential Privacy [Nissim Raskhodnikova Smith '07]
■ An algorithm 𝐵 on a graph 𝐻 is differentially private if for all 𝑓 ∈ 𝐹 the distributions 𝐵(𝐻) and 𝐵(𝐻 − 𝑓) are close to each other
– Much stricter notion than average sensitivity – Some of our algorithms inspired by differentially private algorithms
36
37
38
39
40
41
42
43
44
1 if 𝑓 ∈ 𝐵(𝐻) 0, otherwise 𝑓 ∈ 𝐹
45
1 if 𝑓 ∈ 𝐵(𝐻) 0, otherwise 𝑓 ∈ 𝐹
46
1 if 𝑓 ∈ 𝐵(𝐻) 0, otherwise 𝑓 ∈ 𝐹
47
1 if 𝑓 ∈ 𝐵𝜌(𝐻) 0, otherwise 𝑓 ∈ 𝐹
48
1 if 𝑤 is part of a solution to 𝑄
0, otherwise 𝑤 ∈ 𝑊
49
1 if 𝑤 is part of a solution to 𝑄
0, otherwise 𝑤 ∈ 𝑊
50
1 if 𝑤 is part of a solution to 𝑄
0, otherwise 𝑤 ∈ 𝑊
51
52
53
54
55
– Output a subset 𝑇 of vertices with minimum number of edges between 𝑇 and 𝑊 ∖ 𝑇
56
– Output a subset 𝑇 of vertices with minimum number of edges between 𝑇 and 𝑊 ∖ 𝑇
57
58
59
60
61
1 1+𝜗2
62
63
64
𝑃( 1 𝜗OPT)
65
𝑃( 1 𝜗OPT)
66
𝑃( 1 𝜗OPT)
67
𝑃( 1 𝜗OPT)
68
69
70
71
72
73
74
75
76
𝑃
1 𝜗OPT for the global minimum cut problem for all 𝜗 > 0.
77
log 𝑜 𝜗OPT);
78
log 𝑜 𝜗OPT);
79
log 𝑜 𝜗OPT);
80
log 𝑜 𝜗OPT);
81
log 𝑜 𝜗OPT);
Sa Samplin ling g from an approxim ximat ate e Gibb bbs s distribu tributio tion
82
log 𝑜 𝜗OPT);
alwar '10]
Sa Samplin ling g from an approxim ximat ate e Gibb bbs s distribu tributio tion
83
84
log 𝑜 𝜗OPT);
Sa Samplin ling g from Gibb bbs s distrib tributio tion
85
log 𝑜 𝜗OPT);
Sa Samplin ling g from Gibb bbs s distrib tributio tion
86
log 𝑜 𝜗OPT);
exp(−𝛽 ⋅ size 𝑇, 𝐻 )
𝑞 𝑇, 𝐻 :Probability that 𝐵 outputs cut 𝑇
Fix 𝑓 ∈ 𝐹. ■ For cuts 𝑇 such that 𝑓 crosses 𝑇, 𝑞 𝑇, 𝐻 − 𝑓 ≈ 𝑞 𝑇, 𝐻 ⋅ exp 𝛽 ■ Earth mover's distance between 𝐵(𝐻) and 𝐵 𝐻 − 𝑓
≈ 𝑜 ⋅
𝑇:𝑓 crosses 𝑇
𝑞 𝑇, 𝐻 − 𝑓 − 𝑞 𝑇, 𝐻 = 𝑜 ⋅ exp 𝛽 − 1 ⋅
𝑇:𝑓 crosses 𝑇
𝑞 𝑇, 𝐻
87
log 𝑜 𝜗OPT);
exp(−𝛽 ⋅ size 𝑇, 𝐻 )
𝑞 𝑇, 𝐻 :Probability that 𝐵 outputs cut 𝑇
Fix 𝑓 ∈ 𝐹. ■ For cuts 𝑇 such that 𝑓 crosses 𝑇, 𝑞 𝑇, 𝐻 − 𝑓 ≈ 𝑞 𝑇, 𝐻 ⋅ exp 𝛽 ■ Earth mover's distance between 𝐵(𝐻) and 𝐵 𝐻 − 𝑓
≈ 𝑜 ⋅
𝑇:𝑓 crosses 𝑇
𝑞 𝑇, 𝐻 − 𝑓 − 𝑞 𝑇, 𝐻 = 𝑜 ⋅ exp 𝛽 − 1 ⋅
𝑇:𝑓 crosses 𝑇
𝑞 𝑇, 𝐻
88
log 𝑜 𝜗OPT);
exp(−𝛽 ⋅ size 𝑇, 𝐻 )
𝑞 𝑇, 𝐻 :Probability that 𝐵 outputs cut 𝑇
Fix 𝑓 ∈ 𝐹. ■ For cuts 𝑇 such that 𝑓 crosses 𝑇, 𝑞 𝑇, 𝐻 − 𝑓 ≈ 𝑞 𝑇, 𝐻 ⋅ exp 𝛽 ■ Earth mover's distance between 𝐵(𝐻) and 𝐵 𝐻 − 𝑓
≈ 𝑜 ⋅
𝑇:𝑓 crosses 𝑇
𝑞 𝑇, 𝐻 − 𝑓 − 𝑞 𝑇, 𝐻 = 𝑜 ⋅ exp 𝛽 − 1 ⋅
𝑇:𝑓 crosses 𝑇
𝑞 𝑇, 𝐻
89
log 𝑜 𝜗OPT);
exp(−𝛽 ⋅ size 𝑇, 𝐻 )
𝑞 𝑇, 𝐻 :Probability that 𝐵 outputs cut 𝑇
Fix 𝑓 ∈ 𝐹. ■ For cuts 𝑇 such that 𝑓 crosses 𝑇, 𝑞 𝑇, 𝐻 − 𝑓 ≈ 𝑞 𝑇, 𝐻 ⋅ exp 𝛽 ■ Earth mover's distance between 𝐵(𝐻) and 𝐵 𝐻 − 𝑓
≈ 𝑜 ⋅
𝑇:𝑓 crosses 𝑇
𝑞 𝑇, 𝐻 − 𝑓 − 𝑞 𝑇, 𝐻 = 𝑜 ⋅ exp 𝛽 − 1 ⋅
𝑇:𝑓 crosses 𝑇
𝑞 𝑇, 𝐻
90
log 𝑜 𝜗OPT);
exp(−𝛽 ⋅ size 𝑇, 𝐻 )
𝑞 𝑇, 𝐻 :Probability that 𝐵 outputs cut 𝑇
Fix 𝑓 ∈ 𝐹. ■ For cuts 𝑇 such that 𝑓 crosses 𝑇, 𝑞 𝑇, 𝐻 − 𝑓 ≈ 𝑞 𝑇, 𝐻 ⋅ exp 𝛽 ■ Earth mover's distance between 𝐵(𝐻) and 𝐵 𝐻 − 𝑓
≈ 𝑜 ⋅
𝑇:𝑓 crosses 𝑇
𝑞 𝑇, 𝐻 − 𝑓 − 𝑞 𝑇, 𝐻 = 𝑜 ⋅ exp 𝛽 − 1 ⋅
𝑇:𝑓 crosses 𝑇
𝑞 𝑇, 𝐻
91
log 𝑜 𝜗OPT);
exp(−𝛽 ⋅ size 𝑇, 𝐻 )
■ Average sensitivity of 𝐵 is
≈ 𝑜 𝑛 ⋅ exp 𝛽 − 1 ⋅
𝑓
𝑇:𝑓 crosses 𝑇
𝑞 𝑇, 𝐻
■ Average sensitivity of 𝐵 is ≤
𝑜 𝑛 ⋅ exp 𝛽 − 1 ⋅(Expected
2𝑛 𝑜 , as min. cut size at most
average degree (More Detailed Analysis Overview)
92
log 𝑜 𝜗OPT);
exp(−𝛽 ⋅ size 𝑇, 𝐻 )
■ Average sensitivity of 𝐵 is
≈ 𝑜 𝑛 ⋅ exp 𝛽 − 1 ⋅
𝑓
𝑇:𝑓 crosses 𝑇
𝑞 𝑇, 𝐻
■ Average sensitivity of 𝐵 is ≤
𝑜 𝑛 ⋅ exp 𝛽 − 1 ⋅(Expected
2𝑛 𝑜 , as min. cut size at most
average degree (More Detailed Analysis Overview)
93
log 𝑜 𝜗OPT);
exp(−𝛽 ⋅ size 𝑇, 𝐻 )
■ Average sensitivity of 𝐵 is
≈ 𝑜 𝑛 ⋅ exp 𝛽 − 1 ⋅
𝑓
𝑇:𝑓 crosses 𝑇
𝑞 𝑇, 𝐻
■ Average sensitivity of 𝐵 is ≤
𝑜 𝑛 ⋅ exp 𝛽 − 1 ⋅(Expected
2𝑛 𝑜 , as min. cut size at most
average degree (More Detailed Analysis Overview)
94
log 𝑜 𝜗OPT);
exp(−𝛽 ⋅ size 𝑇, 𝐻 )
■ Average sensitivity of 𝐵 is
≈ 𝑜 𝑛 ⋅ exp 𝛽 − 1 ⋅
𝑓
𝑇:𝑓 crosses 𝑇
𝑞 𝑇, 𝐻
■ Average sensitivity of 𝐵 is ≤
𝑜 𝑛 ⋅ exp 𝛽 − 1 ⋅(Expected
2𝑛 𝑜 , as min. cut size at most
average degree (More Detailed Analysis Overview)
95
log 𝑜 𝜗OPT);
exp(−𝛽 ⋅ size 𝑇, 𝐻 )
96
𝑃
1 𝜗OPT for the global minimum cut problem for all 𝜗 > 0.
97
𝑃
1 𝜗OPT for the global minimum cut problem for all 𝜗 > 0.
98
99
100
101
– Sampling from Gibbs distribution (Global Mincut)
102
– Sampling from Gibbs distribution (Global Mincut) – Notion of average sensitivity for LPs and stable LP solvers (s-t Mincut)
103
– Sampling from Gibbs distribution (Global Mincut) – Notion of average sensitivity for LPs and stable LP solvers (s-t Mincut) – Reusing analyses of existing sublinear-time algorithms and dynamic algorithms (Maximum Matching & Min. Vertex Cover)
104
105
– Further applications of our techniques for design of stable algorithms
106
– Further applications of our techniques for design of stable algorithms
107
– Further applications of our techniques for design of stable algorithms
108
– Further applications of our techniques for design of stable algorithms
109
110
111
112
113
– Proof f Idea: T
than 2 + 𝜗 ⋅ OPT is 𝑝(1).
114
𝑎𝑓 𝑎 − 1
115
116
■ 𝑎 = σ𝑈⊆𝑊 exp(−𝛽 ⋅ size 𝑈, 𝐻 ) ■ 𝑞 𝑇, 𝐻 =
exp(−𝛽⋅size 𝑇,𝐻 ) 𝑎
■ 𝑞 𝑇, 𝐻 ⋅ 𝑎/𝑎𝑓 ≤ 𝑞 𝑇, 𝐻 − 𝑓 Claim aim: For 𝑓 ∈ 𝐹, we have dEM 𝐵 𝐻 , 𝐵 𝐻 − 𝑓 ≤ 𝑜 ⋅
𝑎𝑓 𝑎 − 1
Proof: T
𝑎 𝑎𝑓
𝑎𝑓 𝑎 − 1 .
𝑜 𝑛 ⋅ exp 𝛽 − 1 ⋅(Expected size of cut output by 𝐵)
117
𝑜 𝑛 ⋅ exp 𝛽 − 1 ⋅(Expected size of cut output by 𝐵)
118
𝑜 𝑛 ⋅ exp 𝛽 − 1 ⋅(Expected size of cut output by 𝐵)
2𝑛 𝑜 , as mincut size at most average degree
119
3 4 and 𝐵 𝐻 = 𝑇 ∖ {𝑤} w.p. 1 4
– 𝐵 𝐻 − 𝑓 = 𝑇 w.p.
1 4 and 𝐵 𝐻 − 𝑓 = 𝑇 ∖ {𝑤} w.p. 3 4
120
3 4 and 𝐶 𝐻 = 𝑇 ∖ {𝑤} w.p. 1 4
– 𝐶 𝐻 − 𝑓 = 𝑇 w.p.
1 4⋅2𝑜 , 𝐶 𝐻 − 𝑓 = 𝑇 ∖ {𝑤} w.p. 3 4 + 1 4⋅2𝑜 , and
𝐶 𝐻 − 𝑓 = 𝑈 w.p.
1 4⋅2𝑜
– TV distance ≤ 1 – Earth mover's distance = Ω(𝑜)
121