L ECTURE 3 Last time Properties of lists and functions. Testing if - - PowerPoint PPT Presentation

9/10/2020

Sublinear Algorithms

LECTURE 3

Last time

• Properties of lists and functions.
• Testing if a list is sorted/Lipschitz

and if a function is monotone.

Today

• Testing if a graph is connected.
• Estimating the number of connected

components.

• Estimating the weight of a MST

Sofya Raskhodnikova;Boston University

Graph Properties

Testing if a Graph is Connected [Goldreich Ron]

Input: a graph 𝐻 = (𝑊, 𝐹) on 𝑜 vertices

• in adjacency lists representation

(a list of neighbors for each vertex)

• maximum degree d, i.e., adjacency lists of length d with some empty entries

Query (𝑤, 𝑗), where 𝑤 ∈ 𝑊 and 𝑗 ∈ [𝑒]: entry 𝑗 of adjacency list of vertex 𝑤 Exact Answer: W(dn) time

• Approximate version:

Is the graph connected or ²-far from connected? dist 𝐻1, 𝐻2 =

# 𝑝𝑔 𝑓𝑜𝑢𝑗𝑠𝑓𝑡 𝑗𝑜 𝑏𝑒𝑘𝑏𝑑𝑓𝑜𝑑𝑧 𝑚𝑗𝑡𝑢𝑡 𝑝𝑜 𝑥ℎ𝑗𝑑ℎ 𝐻1 𝑏𝑜𝑒 𝐻2 𝑒𝑗𝑔𝑔𝑓𝑠 𝑒𝑜

Time: 𝑃

1 𝜁2𝑒

today + improvement on HW

No dependence on n!

3

Testing Connectedness: Algorithm

1. Repeat s=8/ed times: 2. pick a random vertex 𝑣 3. determine if connected component of 𝑣 is small: perform BFS from 𝑣, stopping after at most 4/ed new nodes 4. Reject if a small connected component was found, otherwise accept. Run time: O(𝑒/e2𝑒2)=O(1/e2𝑒) Analysis:

• Connected graphs are always accepted.
• Remains to show:

If a graph is ²-far from connected, it is rejected with probability ≥

2 3

4

Connectedness Tester(n, d, ε, query access to G)

Testing Connectedness: Analysis

• By Claim 2, at least e𝑒𝑜

4

nodes are in small connected components.

• By Witness lemma, it suffices to sample

2⋅4

e𝑒𝑜/𝑜 =

8

e𝑒 nodes to detect one from a small connected component.

5

Claim 1

If G is e-far from connected, it has ≥ e𝑒𝑜

2

connected components.

Claim 2

If G is e-far from connected, it has ≥ e𝑒𝑜

4

connected components

• f size at most 4/ed.

Testing Connectedness: Proof of Claim 1

We prove the contrapositive: If G has < e𝑒𝑜

2

connected components, one can make G connected by modifying < e fraction of its representation, i.e., < e𝑒𝑜 entries.

• If there are no degree restrictions, k components can be connected by

adding 𝑙-1 edges, each affecting 2 nodes. Here, 𝑙 < e𝑒𝑜

2 , so 2𝑙 − 2 < e𝑒𝑜 .

• What if adjacency lists of all vertices in a component are full,

i.e., all vertex degrees are d?

6

Claim 1

If G is e-far from connected, it has ≥ e𝑒𝑜

2

connected components.

Freeing up an Adjacency List Entry

What if adjacency lists of all vertices in a component are full, i.e., all vertex degrees are d?

• Consider an MST of this component.
• Let 𝑤 be a leaf of the MST.
• Disconnect 𝑤 from a node other than its parent in the MST.
• Two entries are changed while keeping the same number of components.

7

𝑤

Claim 1

If G is e-far from connected, it has ≥ e𝑒𝑜

2

connected components.

Freeing up an Adjacency List Entry

What if adjacency lists of all vertices in a component are full, i.e., all vertex degrees are d?

• Apply this to each component with <2 free spots in adjacency lists.
• Now we can connect all the components using the freed up spots while

ensuring that we never change more than 2 spots per component.

• Thus, k components can be connected by changing 2k spots.

Here, k < e𝑒𝑜

2 , so 2k < e𝑒𝑜 .

8

𝑤

Claim 1

If G is e-far from connected, it has ≥ e𝑒𝑜

2

connected components.

Testing Connectedness: Proof of Claim 2

• By Claim 1, there are at least e𝑒𝑜

2

connected components.

• Their average size is at most

𝑜

e𝑒𝑜/2 =

2

e𝑒.

• By an averaging argument (or Markov inequality), at least half of the

components are of size at most twice the average.

9

Claim 1

If G is e-far from connected, it has ≥ e𝑒𝑜

2

connected components.

Claim 2

If G is e-far from connected, it has ≥ e𝑒𝑜

4

connected components

• f size at most 4/ed.

Testing if a Graph is Connected [Goldreich Ron]

10

Input: a graph 𝐻 = (𝑊, 𝐹) on 𝑜 vertices

• in adjacency lists representation

(a list of neighbors for each vertex)

• maximum degree d

Connected or 𝜁-far from connected? 𝑃

1 𝜁2𝑒 time

(no dependence on 𝑜)

Randomized Approximation in sublinear time

A Simple Example

Randomized Approximation: a Toy Example

Input: a string 𝑥 ∈ 0,1 𝑜 Goal: Estimate the fraction of 1’s in 𝑥 (like in polls) It suffices to sample 𝑡 = 1 ⁄ 𝜁2 positions and output the average to get the fraction of 1’s ±𝜁 (i.e., additive error 𝜁) with probability ¸ 2/3 Yi = value of sample 𝑗. Then E[Y] =

1 𝑡 ⋅ ∑ 𝑡 𝑗=1

E[Yi] = (fraction of 1’s in 𝑥) Pr (sample mean) − fraction of 1′s in 𝑥 ≥ 𝜁 ≤ 2e−2𝑡𝜁2 = 2𝑓−2 < 1/3

12

Let Y1, … , Ys be independently distributed random variables in [0,1]. Let Y =

1 𝑡 ⋅ ∑ 𝑡 𝑗=1

Yi (called sample mean). Then Pr Y − E Y ≥ 𝜁 ≤ 2e−2𝑡𝜁2. 1 … 0 1

Hoeffding Bound

Apply Hoeffding Bound substitute 𝑡 = 1 ⁄ 𝜁2

Approximating # of Connected Components

[Chazelle Rubinfeld Trevisan]

Input: a graph 𝐻 = (𝑊, 𝐹) on n vertices

• in adjacency lists representation

(a list of neighbors for each vertex)

• maximum degree d

Exact Answer: W(dn) time Additive approximation: # of CC ±εn with probability ¸ 2/3 Time:

• Known: 𝑃

𝑒 𝜁2 log 1 𝜁 , W

𝑒 𝜁2

• Today: 𝑃

𝑒 𝜁3 . No dependence on n!

13

Partially based on slides by Ronitt Rubinfeld: http://stellar.mit.edu/S/course/6/fa10/6.896/courseMaterial/topics/topic3/lectureNotes/lecst11/lecst11.pdf

Breaks C up into contributions

• f different nodes

Approximating # of CCs: Main Idea

• Let 𝐷 = number of components
• For every vertex 𝑣, define

𝑜𝑣 = number of nodes in u’s component

– for each component A: ∑𝑣∈𝐵

1 𝑜𝑣 = 1

∑

𝑣∈𝑊 1 𝑜𝑣 = 𝐷

• Estimate this sum by estimating 𝑜𝑣’s for a few random nodes

– If 𝑣’s component is small, its size can be computed by BFS. – If 𝑣’s component is big, then 1/𝑜𝑣 is small, so it does not contribute much to the sum – Can stop BFS after a few steps Similar to property tester for connectedness [Goldreich Ron]

14

Approximating # of CCs: Algorithm

Estimating 𝑜𝑣 = the number of nodes in 𝑣’s component:

• Let estimate ො

𝑜𝑣 = min 𝑜𝑣,

2 𝜁

– When 𝑣’s component has · 2/e nodes , ො 𝑜𝑣 = 𝑜𝑣 – Else ො 𝑜𝑣 = 2/e, and so 0 < 1

ො 𝑜𝑣 − 1 𝑜𝑣 < 1 ො 𝑜𝑣 = 𝜁 2

• Corresponding estimate for C is መ

𝐷 = ∑𝑣∈𝑊

1 ො 𝑜𝑣. It is a good estimate:

መ 𝐷 − 𝐷 = ∑𝑣∈𝑊

1 ො 𝑜𝑣 − ∑𝑣∈𝑊 1 𝑜𝑣 ≤ ∑𝑣∈𝑊 1 ො 𝑜𝑣 − 1 𝑜𝑣 ≤ 𝜁𝑜 2

1. Repeat s=Θ(1/e2) times: 2. pick a random vertex 𝑣 3. compute ො 𝑜𝑣 via BFS from 𝑣, stopping after at most 2/e new nodes

4.

Return ሚ 𝐷 = (average of the values 1/ො 𝑜𝑣) ∙ 𝑜 Run time: O(d /e3)

15

ൢ 𝑏 𝑐 𝑑 1 ො 𝑜𝑣 − 1 𝑜𝑣 ≤ 𝜁 2 APPROX_#_CCs (n, d, ε, query access to G)

Approximating # of CCs: Analysis

Want to show: Pr ሚ 𝐷 − መ 𝐷 >

𝜁𝑜 2

≤

1 3

Let Yi = 1/ො 𝑜𝑣for the ith vertex 𝑣 in the sample

• Y =

1 𝑡 ⋅ ∑ 𝑡 𝑗=1

Yi =

ሚ 𝐷 𝑜

• E[Y] =

1 𝑡 ⋅ ∑ 𝑡 𝑗=1

E[Yi] = E[Y1] =

1 𝑜 ∑𝑣∈𝑊 1 ො 𝑜𝑣 = መ 𝐷 𝑜

Pr ሚ 𝐷 − መ 𝐷 >

𝜁𝑜 2

= Pr 𝑜𝑍 − 𝑜𝐹 𝑍 >

𝜁𝑜 2

= Pr Y − E Y >

𝜁 2 ≤ 2𝑓−𝜁2𝑡

2

• Need 𝑡 = Θ

1 𝜁2 samples to get probability ≤ 1 3

16

Let Y1, … , Ys be independently distributed random variables in [0,1]. Let Y =

1 𝑡 ⋅ ∑ 𝑡 𝑗=1

Yi (called sample mean). Then Pr Y − E Y ≥ 𝜁 ≤ 2e−2𝑡𝜁2.

Hoeffding Bound

Approximating # of CCs: Analysis

So far: መ 𝐷 − 𝐷 ≤

𝜁𝑜 2

Pr ሚ 𝐷 − መ 𝐷 >

𝜁𝑜 2

≤

1 3

• With probability ≥

2 3 ,

ሚ 𝐷 − 𝐷 ≤ ሚ 𝐷 − መ 𝐷 + መ 𝐷 − 𝐷 ≤ 𝜁𝑜 2 + 𝜁𝑜 2 ≤ 𝜁𝑜 Summary: The number of connected components in 𝑜-vetex graphs of degree at most 𝑒 can be estimated within ±𝜁𝑜 in time 𝑃

𝑒 𝜁3 .

17

Minimum spanning tree (MST)

• What is the cheapest way to connect all the dots?

Input: a weighted graph with n vertices and m edges

• Exact computation:

– Deterministic 𝑃(𝑛 ∙ inverse-Ackermann(𝑛)) time [Chazelle] – Randomized 𝑃(𝑛) time [Karger Klein Tarjan]

1 3 7 5 2 4

18

Partially based on slides by Ronitt Rubinfeld: http://stellar.mit.edu/S/course/6/fa10/6.896/courseMaterial/topics/topic3/lectureNotes/lecst11/lecst11.pdf

Approximating MST Weight in Sublinear Time

[Chazelle Rubinfeld Trevisan]

Input: a graph 𝐻 = (𝑊, 𝐹) on n vertices

• in adjacency lists representation
• maximum degree d and maximum allowed weight w
• weights in {1,2,…,w}

Output: (1+ ε)-approximation to MST weight, 𝑥𝑁𝑇𝑈 Time:

• Known: 𝑃

𝑒𝑥 𝜁3 log 𝑒𝑥 𝜁

, W

𝑒𝑥 𝜁2

• Today: 𝑃

𝑒𝑥4log 𝑥 𝜁3

19

No dependence on n!

Idea Behind Algorithm

• Characterize MST weight in terms of number of connected

components in certain subgraphs of G

• Already know that number of connected components can be

estimated quickly

20

• Recall Kruskal’s algorithm for computing MST exactly.

Suppose all weights are 1 or 2. Then MST weight

= (# weight-1 edges in MST) + 2 ⋅ (# weight-2 edges in MST)

= 𝑜 – 1 + (# of weight-2 edges in MST) = 𝑜 – 1 + (# of CCs induced by weight-1 edges) −1 weight 1 weight 2 connected components induced by weight-1 edges MST

MST and Connected Components: Warm-up

MST has 𝑜 − 1 edges By Kruskal

MST and Connected Components

In general: Let 𝐻𝑗 = subgraph of 𝐻 containing all edges of weight ≤ 𝑗 𝐷𝑗 = number of connected components in 𝐻𝑗 Then MST has 𝐷𝑗 − 1 edges of weight > 𝑗.

• Let 𝛾𝑗 be the number of edges of weight > 𝑗 in MST
• Each MST edge contributes 1 to 𝑥𝑁𝑇𝑈, each MST edge of weight >1

contributes 1 more, each MST edge of weight >2 contributes one more, … 𝑥𝑁𝑇𝑈 𝐻 = ෍

𝑗=0 𝑥−1

𝛾𝑗 = ෍

𝑗=0 𝑥−1

(𝐷𝑗 − 1) = −𝑥 + ෍

𝑗=0 𝑥−1

𝐷𝑗 = 𝑜 − 𝑥 + ෍

𝑗=1 𝑥−1

𝐷𝑗

22

𝑥𝑁𝑇𝑈 𝐻 = 𝑜 − 𝑥 + ෍

𝑗=1 𝑥−1

𝐷𝑗

Claim

Algorithm for Approximating 𝒙𝑵𝑻𝑼

1. For 𝑗 = 1 to 𝑥 − 1 do: 2. ሚ 𝐷𝑗 ←APPROX_#CCs(𝑜, 𝑒,

𝜁 w ; 𝐻𝑗).

3. Return ෥

𝑥𝑁𝑇𝑈 = 𝑜 − 𝑥 + ∑𝑗=1

𝑥−1 ሚ

𝐷𝑗 .

Analysis:

• Suppose all estimates of 𝐷𝑗’s are good: ሚ

𝐷𝑗 − 𝐷𝑗 ≤

𝜁 𝑥 𝑜.

Then ෥ 𝑥𝑁𝑇𝑈 − 𝑥𝑁𝑇𝑈 = | ∑𝑗=1

𝑥−1( ሚ

𝐷𝑗−𝐷𝑗)| ≤ ∑𝑗=1

𝑥−1 | ሚ

𝐷𝑗 − 𝐷𝑗| ≤ 𝑥 ⋅

𝜁 𝑥 𝑜 = 𝜁𝑜

• Pr[all 𝑥 − 1 estimates are good]≥ 2/3 𝑥−1
• Not good enough! Need error probability ≤

1 3𝑥 for each iteration

• Then, by Union Bound, Pr[error]≤ 𝑥 ⋅

1 3𝑥 = 1 3

• Can amplify success probability of any algorithm by repeating it and taking

the median answer.

• Can take more samples in APPROX_#CCs. What’s the resulting run time?

23

• Claim. 𝑥𝑁𝑇𝑈 𝐻 = 𝑜 − 𝑥 + ∑𝑗=1

𝑥−1 𝐷𝑗

APPROX_MSTweight (n, d, w, ε; G)

Multiplicative Approximation for 𝒙𝑵𝑻𝑼

For MST cost, additive approximation ⟹ multiplicative approximation

𝑥𝑁𝑇𝑈 ≥ 𝑜 − 1 ⟹ 𝑥𝑁𝑇𝑈 ≥ 𝑜/2 for 𝑜 ≥ 2

• 𝜁𝑜-additive approximation:

𝑥𝑁𝑇𝑈 − 𝜁𝑜 ≤ ෝ 𝑥𝑁𝑇𝑈 ≤ 𝑥𝑁𝑇𝑈 + 𝜁𝑜

• (1 ± 2𝜁)-multiplicative approximation:

𝑥𝑁𝑇𝑈 1 − 2𝜁 ≤ 𝑥𝑁𝑇𝑈 − 𝜁𝑜 ≤ ෝ 𝑥𝑁𝑇𝑈 ≤ 𝑥𝑁𝑇𝑈 + 𝜁𝑜 ≤ 𝑥𝑁𝑇𝑈 1 + 2𝜁

24