Advanced Algorithms (I) Shanghai Jiao Tong University Chihao Zhang - - PowerPoint PPT Presentation

Shanghai Jiao Tong University

Chihao Zhang

Advanced Algorithms (I)

March 2nd, 2020

Information

Every Monday, 10:00 am - 11:40 am Zoom @ 123363659 Instructor: 张驰豪 (chihao@sjtu.edu.cn) TA: 杨凤麟 (yangfl@sjtu.edu.cn) Office Hour: via Canvas or WeChat Group

References

Probability and Computing Randomized Algorithms The Probabilistic Method

• M. Mitzenmacher & E. Upfal
• R. Motwani & P

. Raghavan

• N. Alon & J. Spencer

Polynomial Identity Testing

Polynomial Identity Testing

Given two polynomials and

F(x) =

d

∏

i=1

(x − ai) G(x) =

d

∑

i=0

bixi

Polynomial Identity Testing

Given two polynomials and

F(x) =

d

∏

i=1

(x − ai) G(x) =

d

∑

i=0

bixi

Is F(x) ≡ G(x)?

Polynomial Identity Testing

Given two polynomials and

F(x) =

d

∏

i=1

(x − ai) G(x) =

d

∑

i=0

bixi

Is F(x) ≡ G(x)?

• Example. F(x) = (x − 1)(x − 2)(x + 3); G(x) = x3 − 7x + 6

One can expand and compare the coefficients…

F(x)

One can expand and compare the coefficients…

F(x)

It takes arithmetic operations.

O(d2)

One can expand and compare the coefficients…

F(x)

It takes arithmetic operations.

O(d2)

Can be improved to using FFT.

O(d log d)

One can expand and compare the coefficients…

F(x)

It takes arithmetic operations.

O(d2)

Can be improved to using FFT.

O(d log d)

If random coins are allowed…

One can expand and compare the coefficients…

F(x)

It takes arithmetic operations.

O(d2)

Can be improved to using FFT.

O(d log d)

If random coins are allowed…

• The problem can be solved much faster

One can expand and compare the coefficients…

F(x)

It takes arithmetic operations.

O(d2)

Can be improved to using FFT.

O(d log d)

If random coins are allowed…

• The problem can be solved much faster
• at the cost of making error.

Choosing a uniform number s ∈ {1,2,…,100d}

Choosing a uniform number s ∈ {1,2,…,100d} Test whether F(s) = G(s)

Choosing a uniform number s ∈ {1,2,…,100d} Test whether F(s) = G(s)

• If

, then it always holds that

F(x) ≡ G(x) F(s) = G(s)

Choosing a uniform number s ∈ {1,2,…,100d} Test whether F(s) = G(s)

• If

, then it always holds that

F(x) ≡ G(x) F(s) = G(s)

• If

, how likely is that ?

F(x) ≢ G(x) F(s) ≠ G(s)

Choosing a uniform number s ∈ {1,2,…,100d} Test whether F(s) = G(s)

• If

, then it always holds that

F(x) ≡ G(x) F(s) = G(s)

• If

, how likely is that ?

F(x) ≢ G(x) F(s) ≠ G(s)

• Theorem. (Fundamental Theorem of Algebra)

A polynomial of degree has at most roots in

d d ℂ

Our algorithm outputs wrong answer only when

Our algorithm outputs wrong answer only when

• ; and

F(x) ≢ G(x)

Our algorithm outputs wrong answer only when

• ; and

F(x) ≢ G(x)

• is a root of

s F(x) − G(x)

Our algorithm outputs wrong answer only when

• ; and

F(x) ≢ G(x)

• is a root of

s F(x) − G(x)

This happens with probability at most .

1 100

Our algorithm outputs wrong answer only when

• ; and

F(x) ≢ G(x)

• is a root of

s F(x) − G(x)

This happens with probability at most .

1 100

It only costs

• perations to compute

and .

O(d) F(s) G(s)

Our algorithm outputs wrong answer only when

• ; and

F(x) ≢ G(x)

• is a root of

s F(x) − G(x)

This happens with probability at most .

1 100

It only costs

• perations to compute

and .

O(d) F(s) G(s)

One can repeat the algorithm times:

t

• error reduces to

;

100−t

• cost increases to

.

O(td)

Multi-variable Polynomials

Multi-variable Polynomials

The idea applies to a more general setting.

Multi-variable Polynomials

The idea applies to a more general setting. Let for some field ,

F, G ∈ 𝔾(x1, …, xn) 𝔾

Multi-variable Polynomials

The idea applies to a more general setting. Let for some field ,

F, G ∈ 𝔾(x1, …, xn) 𝔾

Is F(x1, …, xn) ≡ G(x1, …, xn)?

• Theorem. (Schwartz-Zippel Theorem)

Let be a non-zero multivariate polynomial of degree at most . For any set , it holds that

Q ∈ 𝔾[x1, …, xn] d U ⊆ 𝔾 Pr

r1,…,rn∈RU[Q(r1, …, rn) = 0] ≤

d |U|

Proof of Schwartz-Zippel

Proof of Schwartz-Zippel

Induction on , case is Fundamental Theorem

• f Algebra.

n n = 1

Proof of Schwartz-Zippel

Induction on , case is Fundamental Theorem

• f Algebra.

n n = 1

Assuming it holds for smaller …

n

Proof of Schwartz-Zippel

Induction on , case is Fundamental Theorem

• f Algebra.

n n = 1

Assuming it holds for smaller …

n

Q(x1, …, xn) =

k

∑

i=0

xi

1 ⋅ Qi(x2, …, xn)

Proof of Schwartz-Zippel

Induction on , case is Fundamental Theorem

• f Algebra.

n n = 1

Assuming it holds for smaller …

n

Q(x1, …, xn) =

k

∑

i=0

xi

1 ⋅ Qi(x2, …, xn)

Pr [Q = 0] ≤ Pr [Qk = 0] + Pr [Q = 0|Qk ≠ 0] ≤ d − k |U| + k |U|

Our randomized algorithm generalizes to the multi- variable setting by Schwartz-Zippel theorem.

Our randomized algorithm generalizes to the multi- variable setting by Schwartz-Zippel theorem. If the polynomials are given in product form, one scan of and is sufficient to evaluate them.

F G

Our randomized algorithm generalizes to the multi- variable setting by Schwartz-Zippel theorem. If the polynomials are given in product form, one scan of and is sufficient to evaluate them.

F G

Linear time algorithm with at most error!

1 %

Our randomized algorithm generalizes to the multi- variable setting by Schwartz-Zippel theorem. If the polynomials are given in product form, one scan of and is sufficient to evaluate them.

F G

Linear time algorithm with at most error!

1 %

It is a wide open problem in the complexity theory that whether this can be done in deterministic polynomial time.

Some Complexity Theory

Some Complexity Theory

Problems solvable in deterministic polynomial-time:

P

Some Complexity Theory

Problems solvable in deterministic polynomial-time:

P

Problems solvable in randomized polynomial-time: BPP

Some Complexity Theory

Problems solvable in deterministic polynomial-time:

P

Problems solvable in randomized polynomial-time: BPP

Is ?

BPP = P

Min-Cut in a Graph

Min-Cut in a Graph

A cut in a graph is a set of edges whose removal disconnects .

G = (V, E) C ⊆ E G

Min-Cut in a Graph

A cut in a graph is a set of edges whose removal disconnects .

G = (V, E) C ⊆ E G

How to find the minimum cut?

Min-Cut in a Graph

A cut in a graph is a set of edges whose removal disconnects .

G = (V, E) C ⊆ E G

How to find the minimum cut? It can be solved using max-flow techniques

Min-Cut in a Graph

A cut in a graph is a set of edges whose removal disconnects .

G = (V, E) C ⊆ E G

How to find the minimum cut? It can be solved using max-flow techniques With the fastest max-flow algorithm, it takes time.

O(n × mn)

Karger’s Min-Cut Algorithm

Karger’s Min-Cut Algorithm

David Karger

Karger’s Min-Cut Algorithm

David Karger

Using random bits, Karger found a much simpler algorithm

Karger’s Min-Cut Algorithm

David Karger

Using random bits, Karger found a much simpler algorithm The only operation required is edge contraction

Edge Contraction

Edge Contraction

1 2 3

Edge Contraction

1 2 3 1 2 3

Edge Contraction

1 2 3 1 2 3

Edge Contraction

1 2 3 1 2 3 1/2 3

Edge Contraction

1 2 3 1 2 3 1/2 3

Edge Contraction

1 2 3 1 2 3 1/2 3

• no self-loop

Edge Contraction

1 2 3 1 2 3 1/2 3

• parallel edges may exist
• no self-loop

The Algorithm

The Algorithm

Karger’s Min-cut Algorithm

• 1. Randomly choose an edge and contract it

until only two vertices remains.

• 2. Output remaining edges.

Analysis

Analysis

The algorithm contracts pair of vertices in total.

n − 2

Analysis

The algorithm contracts pair of vertices in total.

n − 2

Fix an minimum cut , we bound the probability that it survives.

C

Analysis

The algorithm contracts pair of vertices in total.

n − 2

Fix an minimum cut , we bound the probability that it survives.

C

Assume the removal of separates and .

C S ⊆ V ¯ S = V∖S

Analysis

The algorithm contracts pair of vertices in total.

n − 2

Fix an minimum cut , we bound the probability that it survives.

C

Assume the removal of separates and .

C S ⊆ V ¯ S = V∖S

All contractions happen within or .

S ¯ S

For , let be the event that “ -th contraction avoids ”

i = 1,…, n − 2 Ai i C

For , let be the event that “ -th contraction avoids ”

i = 1,…, n − 2 Ai i C

We need to bound

For , let be the event that “ -th contraction avoids ”

i = 1,…, n − 2 Ai i C

We need to bound Pr [

n−2

⋂

i=1

Ai] =

n−2

∏

i=1

Pr Ai

i−1

⋂

j=1

Aj

For , let be the event that “ -th contraction avoids ”

i = 1,…, n − 2 Ai i C

We need to bound Pr [

n−2

⋂

i=1

Ai] =

n−2

∏

i=1

Pr Ai

i−1

⋂

j=1

Aj We assume |C| = k

In -th contraction,

i

In -th contraction,

i

• the graph contains

vertices;

• each vertex is of degree at least .

n − i + 1 k

In -th contraction,

i

• the graph contains

vertices;

• each vertex is of degree at least .

n − i + 1 k

Therefore, conditional on that still survives,

C

In -th contraction,

i

• the graph contains

vertices;

• each vertex is of degree at least .

n − i + 1 k

Therefore, conditional on that still survives,

C

Pr Ai

i−1

⋂

j=1

Aj ≥ 1 − 2k k(n − i + 1) = n − i − 1 n − i + 1

Therefore

Therefore Pr [

n−2

⋂

i=1

Ai] =

n−2

∏

i=1

Pr Ai

i−1

⋂

j=1

Aj ≥

n−2

∏

i=1

n − i − 1 n − i + 1 = n − 2 n ⋅ n − 3 n − 1 ⋯ 1 3 = 2 n(n − 1)

So if we repeat the algorithm times, the minimum cut survives with probability at least

50n2

So if we repeat the algorithm times, the minimum cut survives with probability at least

50n2

1 − (1 − 2 n(n − 1))

50n2

≥ 1 − e−100

So if we repeat the algorithm times, the minimum cut survives with probability at least

50n2

1 − (1 − 2 n(n − 1))

50n2

≥ 1 − e−100 How about the time cost?

So if we repeat the algorithm times, the minimum cut survives with probability at least

50n2

1 − (1 − 2 n(n − 1))

50n2

≥ 1 − e−100 How about the time cost? If we store the graph in a adjacency matrix, one needs to contract an edge…

O(n)

Karger-Stein’s Trick

Karger-Stein’s Trick

Recall Pr [

n−2

⋂

i=1

Ai] = n − 2 n ⋅ n − 3 n − 1 ⋯ 1 3

Karger-Stein’s Trick

Recall Pr [

n−2

⋂

i=1

Ai] = n − 2 n ⋅ n − 3 n − 1 ⋯ 1 3 The more we contracts, the easier gets hit

C

Karger-Stein’s Trick

Recall Pr [

n−2

⋂

i=1

Ai] = n − 2 n ⋅ n − 3 n − 1 ⋯ 1 3 The more we contracts, the easier gets hit

C

Idea: Make a copy before it becomes too bad!

Karger-Stein’s Trick

Recall Pr [

n−2

⋂

i=1

Ai] = n − 2 n ⋅ n − 3 n − 1 ⋯ 1 3 The more we contracts, the easier gets hit

C

Idea: Make a copy before it becomes too bad! The success probability can be improved to Ω (

1 log n )