[PPT] - Polynomial-Time Problems Lecture 3: The polynomial method Part I: PowerPoint Presentation

SLIDE 1

Complexity Theory of Polynomial-Time Problems

Lecture 3: The polynomial method Part I: Orthogonal Vectors

Sebastian Krinninger

SLIDE 2

Organization of lecture

No lecture on 26.05. (State holiday)
2nd exercise sheet: Next week
Tutorials:
New slot: Friday, 12:15 - 14:00, U12 E1.1, biweekly
Fr, 13.05. (tomorrow)
Fr, 03.06.
Etc.

SLIDE 3

The polynomial method

Recently developed technique in Algorithm Design
Current fastest algorithms for
All-Pairs Shortest Paths [Williams 14]
Orthogonal Vectors [Abboud/Williams/Yu 15]
Hamming Nearest Neighbors [Alman/Williams 15]
Two main tools

1. Razborov-Smolensky from Circuit Complexity 2. Fast rectangular matrix multiplication

SLIDE 4

Reminder: Orthogonal Vectors Problem

Input: Two sets 𝐵, 𝐶 ⊆ 0,1 𝑒 of 𝑒-dimensional 0/1-vectors of size 𝑜 Output: Is there a pair 𝑏 ∈ 𝐵, 𝑐 ∈ 𝐶 s.t. 𝑏 and 𝑐 are orthogonal? ∃𝑏 ∈ 𝐵, 𝑐 ∈ 𝐶: 〈𝑏, 𝑐〉 = 0 𝑙=1

𝑒

𝑏 𝑙 ⋅ 𝑐 𝑙 = 0 ∀1 ≤ 𝑙 ≤ 𝑒: (𝑏 𝑙 = 0) ∨ (𝑐 𝑙 = 0) Trivial algorithms:

𝑃 𝑜2𝑒
𝑃(2𝑒𝑜)

Interesting Regime: 𝑒 = 𝑑 log 𝑜

SLIDE 5

Today’s result

Reminder: State of the art: In this lecture: 𝑜2−1/𝑃(log 𝑒) Algorithm is randomized and correct with high probability, i.e., probability ≥ 1 − 1/𝑜

Conjecture: There is no algorithm for the orthogonal vectors problem with running time 𝑃(𝑜2−𝜗poly(𝑒)) for any 𝜗 > 0. Theorem: There is an algorithm for the orthogonal vectors problem with running time 𝑜2−1/𝑃(log(𝑒/ log 𝑜)).

SLIDE 6

Overview

1. Reduce problem to many subproblems of very small size
2. Precompute small circuits for solving subproblems
3. Evaluate circuits with probabilistic polynomials of low degree
4. Evaluate polynomials using fast rectangular matrix multiplication

SLIDE 7

Overview

1. Reduce problem to many subproblems of very small size
2. Precompute small circuits for solving subproblems
3. Evaluate circuits with probabilistic polynomials of low degree
4. Evaluate polynomials using fast rectangular matrix multiplication

SLIDE 8

Dividing into smaller subproblems

1. Divide 𝐵 and 𝐶 into 𝑟 = ⌈

𝑜 𝑡⌉ subsets of size ≤ 𝑡:

𝐵1, … , 𝐵𝑟 and 𝐶1, … , 𝐶𝑟

2. Construct a polynomial 𝑄(𝑏1 1 , … 𝑏1 𝑒 , … , 𝑏𝑡 1 … , 𝑏𝑡 𝑒 , 𝑐1 1 , … 𝑐1 𝑒 , … , 𝑐𝑡 1 … , 𝑐𝑡 1 )

𝑄 𝐵𝑗, 𝐶

𝑘 = 1 if and only if 𝐵𝑗, 𝐶 𝑘 contains orthogonal pair

…only with high probability

3. For every pair of subsets 𝐵𝑗, 𝐶

𝑘: evaluate 𝑄 on 𝐵𝑗, 𝐶 𝑘

…simultaneously! → 𝑃(

𝑜2 𝑡2 polylog(𝑜))

4. Return “yes” if some 𝐵𝑗, 𝐶

𝑘 contains orthogonal pair, “no” otherwise

We set 𝑡 = 2𝜗 log 𝑜/ log 𝑒 = 𝑜𝜗/ log 𝑒 for some sufficiently small 𝜗

SLIDE 9

Questions

1. How to construct suitable polynomial 𝑄?
2. How to evaluate 𝑄 fast on all pairs of inputs?

SLIDE 10

Overview

1. Reduce problem to many subproblems of very small size
2. Precompute small circuits for solving subproblems
3. Evaluate circuits with probabilistic polynomials of low degree
4. Evaluate polynomials using fast rectangular matrix multiplication

SLIDE 11

Boolean circuits

Boolean circuit

Directed acyclic graph
Sources: input bits
Sink: output bit
Inner nodes: Boolean operations
AND: ∧
OR: ∨
Arbitrary “fan-in”

SLIDE 12

Circuit for checking orthogonality of vectors

Are 𝑦 and 𝑧 orthogonal? 𝑦 and 𝑧 orthogonal iff ¬∃𝑗: 𝑦 𝑗 = 1 ∧ 𝑧 𝑗 = 1 Output bit 𝑨 = 1 iff 𝑦 and 𝑧 orthogonal AND of ORs with

2𝑒 negated inputs
1 output

∨

¬𝑦[1] ¬𝑧[1]

∨

¬𝑦[2] ¬𝑧[2]

∨

¬𝑦[𝑒] ¬𝑧[𝑒] … …

∧

𝑨

SLIDE 13

Circuit for finding orthogonal pair

Is there an orthogonal pair? Check orthogonality for every pair 𝑦1, 𝑧1 orthogonal OR 𝑦1, 𝑧2 orthogonal OR … OR 𝑦𝑡, 𝑧𝑡 orthogonal? OR of ANDs of ORs 2𝑒𝑡2 negated inputs

∨ ∧

… … … …

∨ ∨ ∧ ∨ ∨ ∧ ∨ ∨

… … … … … … ¬𝑦1[1] ¬𝑧1[1] 𝑨 ¬𝑦1[1] ¬𝑧2[1] ¬𝑦𝑡[𝑒] ¬𝑧𝑡[𝑒] … … …

SLIDE 14

Overview

1. Reduce problem to many subproblems of very small size
2. Precompute small circuits for solving subproblems
3. Evaluate circuits with probabilistic polynomials of low degree
4. Evaluate polynomials using fast rectangular matrix multiplication

SLIDE 15

From circuits to polynomials

Obtain polynomial over 𝐺2 outputting 1 if and only if circuit outputs 1
𝐺2: Field of {0, 1} with operations ⊕ and ⋅
⊕ is XOR-operation:
0 ⊕ 0 = 0

1 ⊕ 0 = 1 0 ⊕ 1 = 1 1 ⊕ 1 = 0

XOR of multiple variables:

𝑦1 ⊕ 𝑦2 ⊕ ⋯ ⊕ 𝑦𝑙 = 1 if and only if odd number of 𝑦𝑗’s is 1

Expanded polynomials:
𝑏 ⋅ 𝑐 ⊕ 𝑑 ⋅ 𝑏 ⊕ 𝑐 ⊕ 𝑒 = 𝑏𝑑 ⊕ 𝑏𝑐𝑑 ⊕ 𝑏𝑐𝑒 ⊕ 𝑏𝑑𝑒
XOR of monomials
Goal: Few monomials allows fast evaluation later

SLIDE 16

Representing circuit by polynomial

Straightforward approach:

AND:

𝑏 ∧ 𝑐 ⇒ 𝑏 ⋅ 𝑐

Negation: ¬𝑏 ⇒ 1 ⊕ 𝑏

(addition=subtraction in 𝐺2)

OR:

𝑏 ∨ 𝑐 ⇒ ¬(¬𝑏 ∧ ¬𝑐) (DeMorgan’s Law) Example: Bottom-level of circuit ¬𝑏 ∨ ¬𝑐 ⇒ 1 ⊕ 𝑏 ⋅ 𝑐

SLIDE 17

Expanding a multiplication (distributive law)

𝑦1,1 ⊕ ⋯ ⊕ 𝑦1,𝑙 ⋅ 𝑦2,1 ⊕ ⋯ ⊕ 𝑦2,𝑙 ⋅ … ⋅ 𝑦𝑢,1 ⊕ ⋯ ⊕ 𝑦𝑢,𝑙 = 𝑦1,1 ⋅ 𝑦2,1 ⋅ … ⋅ 𝑦𝑢,1 ⊕ ⋯ ⊕ 𝑦1,𝑙 ⋅ 𝑦2,𝑙 ⋅ … ⋅ 𝑦𝑢,𝑙

⊕ ∧ ⊕ ⊕

𝑦𝑢,𝑙 𝑦1,1 … 𝑦1,𝑙 … … AND of 𝑢 XORs Each XOR has 𝑙 inputs 𝑦𝑢,1 … …

∧ ⊕ ∧ ∧

𝑦𝑢,𝑙 𝑦1,1 … 𝑦𝑢,1 … … XOR of 𝑙𝑢 ANDs Each AND has 𝑢 inputs 𝑦1,𝑙 … … 𝑙 choices 𝑙 choices 𝑙 choices 𝑙𝑢 monomials Running time: 𝑃(𝑙𝑢𝑢 ⋅ deg) where deg is degree after expansion (maximum size of any monomial); here deg ≤ 𝑢

SLIDE 18

Razborov/Smolensky trick [Raz87] [Smo87]

Naïve representation of OR DeMorgan: 𝑃𝑆 𝑦1, … , 𝑦𝑙 = 1 ⊕

𝑗=1 𝑙

1 ⊕ 𝑦𝑗 After expansion: 2𝑙 monomials Probabilistic representation of OR Parameter 𝑢 Fewer monomials, correct whp ⊕ ∨ ⊕ ⊕

𝑦𝑙 𝑦1 … 𝑦2 … … OR of 𝑢 XORs Add each edge With probability 1

2

∨

𝑦1 𝑦2 𝑦𝑙 … …

SLIDE 19

Correct representation with high probability

Case 1: 𝑦1 ∨ ⋯ ∨ 𝑦𝑙 = 0

Easy case: each XOR outputs 0, top OR outputs 0

Case 2: 𝑦1 ∨ ⋯ ∨ 𝑦𝑙 = 1

Let 𝑌 be set of inputs with 𝑦𝑗 = 1 For each XOR:

If XOR has odd number of links to 𝑌: XOR outputs 1

(good event: top OR outputs 1)

If XOR has even number of links to 𝑌: XOR outputs

0 (bad event!) Probability that XOR has even number of links to 𝑌: = 1/2 because last element of X “decides” whether number of links is even or odd (each with prob. 1/2)

Probabilistic representation of OR

⊕ ∨ ⊕ ⊕

𝑦𝑙 𝑦1 … 𝑦2 … … OR of 𝑢 XORs Add each edge With probability 1

2

⇒ Probability that all XORs output 0: =

1 2 𝑢

=

1 2𝑢

⇒ Probability that OR outputs 1: = 1 −

1 2𝑢

SLIDE 20

Bounding number of monomials

Formal definition of polynomial For 𝑗 = 1 … 𝑢, 𝑘 = 1 … 𝑙:

With probability

1 2: Set 𝑠 𝑗,𝑘 = 1

Otherwise:

Set 𝑠

𝑗,𝑘 = 0

Polynomial: 𝑃𝑆𝑢 𝑦1, … , 𝑦𝑙 = 1 ⊕

𝑗=1 𝑢

1 ⊕⊕𝑘=1

𝑙

𝑠

𝑗,𝑘 ⋅ 𝑦𝑗

After expansion: 𝑙 + 1 𝑢 monomials Probabilistic representation of OR ⊕ ∨ ⊕ ⊕

𝑦𝑙 𝑦1 … 𝑦2 … … OR of 𝑢 XORs Add each edge With probability 1

2

SLIDE 21

Formal definition

Example: OR-gate represented by 𝑃𝑆𝑢 𝑦1, … , 𝑦𝑙 = 1 ⊕ 𝑗=1

𝑢

1 ⊕⊕𝑘=1

𝑙

𝑠

𝑗,𝑘 ⋅ 𝑦𝑗

with error 𝜀 = 1 −

1 2𝑢

Definition: Let 𝐷 be a Boolean circuit with 𝑙 input gates and let 𝐸 be a finite distribution of polynomials on 𝑙 variables over a ring 𝑆 containing 0 and 1(∗). The distribution 𝐸 is a probabilistic polynomial over 𝑆 representing 𝐷 with error 𝜀 if for all 𝑦1, … , 𝑦𝑙 ∈ 0,1 𝑙: Pr

𝑞∼𝐸 𝑞 𝑦1, … , 𝑦𝑙 = 𝐷 𝑦1, … , 𝑦𝑙

> 1 − 𝜀.

(*) In our case, 𝑆 is the field 𝐺2

SLIDE 22

Representing OV circuit I

Are 𝑦 and 𝑧 orthogonal? Bottom OR: ¬𝑦 𝑙 ∨ ¬𝑧 𝑙 ⇒ 1 ⊕ 𝑦[𝑙] ⋅ 𝑧[𝑙] Middle AND:

1. DeMorgan
2. Razborov/Smolensky

with 𝑢1 = 3 log 𝑡 Number of monomials: 𝑒 + 1 𝑢1

∨

¬𝑦[1] ¬𝑧[1]

∨

¬𝑦[2] ¬𝑧[2]

∨

¬𝑦[𝑒] ¬𝑧[𝑒] … …

∧

𝑨

SLIDE 23

Representing OV circuit II

Is there an orthogonal pair?

∨ ∧

… … … …

∨ ∨ ∧ ∨ ∨ ∧ ∨ ∨

… … … … … … ¬𝑦1[1] ¬𝑧1[1] 𝑨 ¬𝑦1[1] ¬𝑧2[1] ¬𝑦𝑡[𝑒] ¬𝑧𝑡[𝑒] … … …

Middle ANDs: XOR of 𝑡2 polynomials, each with 𝑒 + 1 𝑢1 monomials ⇒ 𝑡2 𝑒 + 1 𝑢1 monomials Top OR: Raz/Smol with 𝑢2 = 2 ⇒ 𝑡2 𝑒 + 1 𝑢1 𝑢2 monomials = 𝑡4 𝑒 + 1 6 log 𝑡

SLIDE 24

Analysis of error

We apply Razborov/Smolensky

𝑡2 times with 𝑢1 = 3 log 𝑡
1 time with 𝑢2 = 2

Union Bound: Pr 𝑌 ∪ 𝑍 ≤ Pr 𝑌 + Pr 𝑍 Probability of error: ≤

𝑡2 2𝑢1 + 1 2𝑢2 = 𝑡2 𝑡3 + 1 4 = 1 𝑡 + 1 4 ≤ 1 3 (1/𝑡 is small enough for instances with sufficiently large 𝑜)

SLIDE 25

Plugging in the right values

𝑡 = 2𝜗 log 𝑜/ log 𝑒 𝜗 = 1/160 #monomials: 𝑛 ≤ 𝑡4 𝑒 + 1 6 log 𝑡 ≤ 𝑡4 𝑒 + 1 6𝜗 log 𝑜/ log 𝑒 ≤ 4𝜗 log 𝑜 + 12𝜗 log 𝑜 = 0.1 log 𝑜

⇒ 𝑛 ≤ 𝑜0.1

SLIDE 26

Running time for expanding polynomial

We explicitly have to expand our polynomial into XOR of monomials! Running time dominated by applications of distributive law 1st expansion (repeated 𝑡2 times):

Degree after expansion: 𝑃(𝑢1)
Total time: 𝑃(𝑡2 𝑒 + 1 𝑢1𝑢1

2)

2nd expansion :

Degree after expansion: 𝑃(𝑢1𝑢2)
Running time: 𝑃

𝑡2 𝑒 + 1 𝑢1 𝑢2 𝑢1

2𝑢2 2

≤ 𝑃 𝑜0.1 𝑢1

2𝑢2 2 ≤ 𝑃 𝑜0.1 log2 𝑜

⇒ Total time: 𝑃 𝑜0.1 log2 𝑜 (negligible)

SLIDE 27

Summary for probabilistic polynomial

We can construct polynomial 𝑄 over 𝐺2 with 2𝑡𝑒 inputs such that, given two sets 𝐵′, 𝐶′ ⊆ 0,1 𝑒 of 𝑒-dimensional 0/1-vectors of size 𝑡, with probability >

2 3: 𝑄 𝐵′, 𝐶′ = 1 iff 𝐵′ and 𝐶′ have orthogonal pair.

SLIDE 28

Overview

1. Reduce problem to many subproblems of very small size
2. Precompute small circuits for solving subproblems
3. Evaluate circuits with probabilistic polynomials of low degree
4. Evaluate polynomials using fast rectangular matrix multiplication

SLIDE 29

Fast matrix multiplication

Goal: Compute 𝐷 = 𝐵 × 𝐶 where 𝐵 and 𝐶 are 𝑜 × 𝑜 matrices
Naïve algorithm: 𝑃 𝑜3
Strassen’s algorithm: 𝑃 𝑜2.807
Current fastest: 𝑃 𝑜2.373
Best we can hope for: 𝑃 𝑜2

SLIDE 30

Rectangular matrix multiplication

Lemma: There is an algorithm for multiplying an 𝑂 × 𝑂0.17 matrix with an 𝑂0.17 × 𝑂 matrix in time 𝑃(𝑂2 log2 𝑜). 𝑂0.1 𝑂 Also works for finite fields such as 𝐺2! × =

SLIDE 31

Fast evaluation of polynomial

Given: Polynomial 𝑄(𝑦[1], … , 𝑦[𝐿], 𝑧[1], … , 𝑧[𝐿]) over 𝐺2
With at most 𝑂0.1 monomials
Two sets of inputs:

𝑌 = 𝑦1, … , 𝑦𝑂 ⊆ 0,1 𝐿, 𝑍 = 𝑧1, … , 𝑧𝑂 ⊆ 0,1 𝐿

Evaluate 𝑄 on all pairs 𝑦𝑗, 𝑧𝑘 ∈ 𝑌 × 𝑍 simultaneously

in time 𝑃(𝑂2polylog 𝑜 )

SLIDE 32

Reduction to matrix multiplication

𝑦1 ⋮ 𝑦𝑂 × 𝑂0.1 monomials 𝑧1 … 𝑧𝑂 𝑂0.1 monomials

Entry 𝑗, 𝑘 : Restriction of 𝑘-th monomial to input 𝑦𝑗 Entry 𝑗, 𝑘 : Restriction of 𝑗-th monomial to input 𝑧𝑘

SLIDE 33

Evaluating OV-polynomial on all subgroups

1. Divide 𝐵 and 𝐶 into 𝑟 = ⌈

𝑜 𝑡⌉ subsets of size ≤ 𝑡:

𝐵1, … , 𝐵𝑟 and 𝐶1, … , 𝐶𝑟

2. Construct a polynomial 𝑄(𝑏1 1 , … 𝑏1 𝑒 , … , 𝑏𝑟 1 … , 𝑏𝑟 𝑒 ,

𝑐1 1 , … 𝑐1 𝑒 , … , 𝑐𝑟 1 … , 𝑐𝑟 1 )

𝑄 𝐵𝑗, 𝐶

𝑘 = 1 if and only if 𝐵𝑗, 𝐶 𝑘 contains orthogonal pair

3. For every pair of subsets 𝐵𝑗, 𝐶

𝑘: evaluate 𝑄 on 𝐵𝑗, 𝐶 𝑘

𝑄 has ≤ 𝑜0.1 monomials Simultaneous evaluation in time 𝑃(𝑜2/𝑡2 polylog 𝑜 ) ≤ 𝑜2−1/𝑃(log 𝑒)

𝑡 = 2𝜗 log 𝑜/ log 𝑒 = 𝑜𝜗/ log 𝑒 for 𝜗 = 1/160

SLIDE 34

Remarks

Correctness with high probability

Polynomial is only correct with probability ≥ 2/3
Amplify the success probability by repeating with 10 log 𝑜 independent

polynomials and taking majority value

⇒ Chernoff Bound

Faster algorithm:

𝑜2−1/𝑃(log(𝑒/ log 𝑜))
Just needs better estimate for number of monomials and slightly different

choice of 𝑡