On formal complexity measures Pavel Pudl ak Mathematical - - PowerPoint PPT Presentation

On formal complexity measures

Pavel Pudl´ ak

Mathematical Institute, Academy of Sciences, Prague and Charles University, Prague

Workshop on Randomness and Enumeration November 24-28, 2008 La Suiza Andina, Curacaut´ ın, Chile 1

Overview

1st lecture: boolean formulas, NC 1 formal complexity measures examples convexity rank as a formal complexity measure 2nd lecture nlog n lower bound on monotone formulas linear upper bound for general formulas

2

Literature (main):

• M. Karchmer, E. Kushilevitz, N. Nisan, Fractional covers and communication

complexity, 1995 A.A. Razborov, Applications of Matrix Methods to the Theory of Lower Bounds in Computational Complexity 1990 A.A. Razborov, On submodular complexity measures, 1992

• P. Hrubeˇ

s, S. Jukna, A. Kulikov, P. Pudl´ ak and P. Savick´ y, On convex complexity measures (in preparation)

3

NC1 = “the class of problems with fast parallel algorithms” = computable by log-depth circuits = computable by polynomial size formulas NC1 ⊆ P

Problem

NC1 = P?, NC1 = NP? etc.

4

NC1 = “the class of problems with fast parallel algorithms” = computable by log-depth circuits = computable by polynomial size formulas NC1 ⊆ P

Problem

NC1 = P?, NC1 = NP? etc.

Problem

Find explicitly defined boolean functions that cannot be computed by polynomial size formulas. By counting argument: there exist functions that can only be computed by exponential size formulas.

4

formula size complexity

basis of connectives ∨, ∧, ¬ variables x1, . . . , xn size of a formula = number of occurrences of variables

• Example. (x1 ∧ ¬x2) ∨ (¬x1 ∧ x2) has size 4.

Definition

Let f : {0, 1}n → {0, 1}. The formula size complexity of f is L(f ) := the minimal s such that f can be computed by a formula of size s

• Example. Let f (x1, x2) := x1 ⊕ x2, then L(f ) = 4.

To prove NC1 = P it suffices to prove super polynomial lower bounds on the formula size complexity of a function in P.

5

functions, rectangles, measures

Let f : {0, 1}n → {0, 1}. The associated rectangle of f is Sf := f −1(0) × f −1(1) , M ⊆ Sf is a monochromatic rectangle, if for some ǫ ∈ {0, 1} and 1 ≤ i ≤ n, for all (x, y) ∈ M, xi = ǫ and yi = 1 − ǫ There are 2n maximal monochromatic rectangles Mi,ǫ = {(x, y) ∈ Sf ; xi = ǫ and yi = 1 − ǫ}.

6

Let R be the set of all subrectangles of Sf . A rectangle function is a real-valued function µ : R → R. A rectangle measure is a rectangle function µ : R → R satisfying the following condition: (i) Normalization: µ(M) ≤ 1 for every monochromatic rectangle M ∈ R (ii) Subadditivity: µ(R) ≤ µ(R1) + µ(R2), for every rectangle R ∈ R and for every its partition into disjoint union of rectangles R1, R2 ∈ R.

7

Let R be the set of all subrectangles of Sf . A rectangle function is a real-valued function µ : R → R. A rectangle measure is a rectangle function µ : R → R satisfying the following condition: (i) Normalization: µ(M) ≤ 1 for every monochromatic rectangle M ∈ R (ii) Subadditivity: µ(R) ≤ µ(R1) + µ(R2), for every rectangle R ∈ R and for every its partition into disjoint union of rectangles R1, R2 ∈ R.

Theorem

If µ is a rectangle measure, then µ(Sf ) ≤ L(f )

7

Proof: Let φ be a formula computing f . To every subformula ψ of φ, assign a rectangle A × B such that x ∈ A → ψ(x) = 0 and y ∈ B → ψ(y) = 1 (1) as follows.

1

assign Sf to φ;

2

if ψ ↔ A × B and ψ = ψ1 ∨ ψ2, then split vertically into A × B1 and A × B2 while preserving (1);

3

if ψ ↔ A × B and ψ = ψ1 ∧ ψ2, then split horizontally into A1 × B and A2 × B while preserving (1); Then we have the rectangles assigned to xi and ¬xi are monochromatic; if ψ ↔ R, then µ(R) ≤ the size of ψ (proof by induction on the size of ψ) Hence µ(Sf ) ≤ size of φ, whence µ(Sf ) ≤ L(f ). QED

8

Let λ(A × B) := minimal size of ψ satisfying (1). Then

1

λ is a measure;

2

λ(Sf ) = L(f );

3

λ(R) = maxµ µ(R). Thus in principle we can prove exponential lower bounds. So to prove NC1 = P we need only to find a suitable measure!

9

why are we not able to prove exponential lower bound?

1

simple measures give only small bounds

2

measures that could prove exponential lower bounds are difficult to estimate

10

Example 1 [Khrapchenko/Paterson] Let f (x1, . . . , xn) = x1 ⊕ · · · ⊕ xn. Define w(R) = |{(x, y) ∈ R; d(x, y) = 1}| and µ(R) := w(R)2 |R| . Then µ is a measure and µ(Sf ) = n2. Whence L(f ) ≥ n2. Proof. Normalization: Suppose A × B is monochromatic, w.l.o.g. |A| ≤ |B|. Then for every x ∈ A, there exists at most one y with d(x, y) = 1. Hence w(A × B) ≤ |A|. Whence µ(A × B) ≤

|A|2 |A|·|B| ≤ |A|2 |A|2 = 1.

Subadditivity follows from the inequality (w1+w2)2

r1+r2

≤ w 2

1

r1 + w 2

2

r2 , for r1, r2 > 0.

w(Sf ) = n2n−1, |Sf | = 2n−1 · 2n−1.

• Remark. L(x1 ⊕ · · · ⊕ xn) ≤ 9

8n2; if n is a power of 2, then ≤ n2.

11

Example 2 [Rychkov] Let n be a power of 2, let H be the words of the Hamming code of length n. Let g(x) = 1 iff x ∈ H. Define w(R) = |{(x, y) ∈ R; d(x, y) = 2}| and µ(R) := w(R)2 n|R| . Then µ is a measure and µ(Sg) = Ω(n2). Whence L(g) = Ω(n2). This is not known to be optimal! We only know L(g) = O(n2 log n).

12

(iii) Additivity: µ(R) = µ(R1) + µ(R2), for every rectangle R ∈ R and for every its partition into disjoint union of rectangles R1, R2 ∈ R. µ is additive iff ∃ α : Sf → R such that ∀R ∈ R µ(R) =

• (x,y)∈R

α(x, y). Think of α as a matrix.

Lemma

Let F : R → R be a convex function, let w be an additive measure. Then µ, defined by µ(R) = |R| · F w(R) |R|

• ,

is subadditive. If F(x) = xr, for some r ≥ 1, and µ is normalized, we call µ a polynomial measure. Khrapchenko measure is a polynomial measure with F(x) = x2 and α(x, y) = 1 if d(x, y) = 1, and = 0 otherwise.

13

Theorem

Let µ be a polynomial measure defined by µ(R) = w(R)r |R|r−1 , where w is a nonnegative additive measure. Then

1

for 1 ≤ r ≤ 2, µ(Sf ) ≤ (2n)r;

2

for r > 2, µ(Sf ) = O(n). Thus always µ(Sf ) = O(n2). Proof of 1. From normalization we get w(M) ≤ |M|

r−1 r .

Since w is nonnegative and additive and Sf is the union of the 2n maximal monochromatic rectangles, we have w(Sf ) ≤

• w(Mi,ǫ) ≤ 2n|Sf |

r−1 r .

Hence

w(Sf ) |Sf |

r−1 r

≤ 2n. Raising it to the r we get the theorem.

14

• Remark. Using an additive measure with some negative values one can obtain an

n2 lower bound.

• Example. [Karchmer, Kushilevitz, Nisan] Let f be the parity function. Define

α(x, y) =

• 1/2n−1

if d(x, y) = 1 −1/2n(2n − 1)

• therwise.

The corresponding additive measure gives the lower bound n2.

15

• Remark. Using an additive measure with some negative values one can obtain an

n2 lower bound.

• Example. [Karchmer, Kushilevitz, Nisan] Let f be the parity function. Define

α(x, y) =

• 1/2n−1

if d(x, y) = 1 −1/2n(2n − 1)

• therwise.

The corresponding additive measure gives the lower bound n2. What if the measure is defined using F(x) = x2 log x (for x ≥ 1)?

15

strong subadditivity and convexity

(iv) Strong subadditivity: if µ(R) ≤ m

i=1 µ(Ri), for every rectangle R and every

its partition into disjoint union of rectangles Ri ⊆ R. The partition number of a rectangle R is defined by κ(R) = min{m; R can be decomposed into m disjoint monochromatic rectangles}.

1

κ is a strongly subadditive measure;

2

µ ≤ κ for every strongly subadditive measure µ;

3

for almost all f , κ(Sf ) ≥ 2Ω(√n).

• Problem. What is maxf κ(Sf )?

16

A fractional partition of R is R1, . . . , Rm ∈ R and r1, . . . , rm ∈ (0, 1], such that χR =

m

• i=1

riχRi where χR, χRi denote the characteristic functions of the rectangles.

Definition

A measure µ is convex, if for every R ∈ R and every fractional partition R1, . . . , Rm ∈ R, r1, . . . , rm ∈ (0, 1] of R µ(R) ≤

m

• i=1

riµ(Ri) .

17

A fractional partition of R is R1, . . . , Rm ∈ R and r1, . . . , rm ∈ (0, 1], such that χR =

m

• i=1

riχRi where χR, χRi denote the characteristic functions of the rectangles.

Definition

A measure µ is convex, if for every R ∈ R and every fractional partition R1, . . . , Rm ∈ R, r1, . . . , rm ∈ (0, 1] of R µ(R) ≤

m

• i=1

riµ(Ri) .

Theorem

It is not possible to prove more than quadratic lower bounds using convex measures. More precisely, if µ is a convex measure, then µ(R) ≤ 9

8n2 for all R ∈ Rect.

17

The size of a fractional partition χR = m

i=1 riχRi is defined to be m i=1 ri.

The fractional partition number of a rectangle R is defined by κ∗(R) = min{r; R has a fractional partition of size r consisting of monochromatic rectangles}

1

κ∗ is a convex measure;

2

µ ≤ κ∗ for every convex measure µ.

Lemma (KKN95)

For every rectangle R, κ∗(R) ≤ ( 9

8 + o(1))n2 (in particular for Sf , for any f ).

• Proof. Idea — fractional partition by parities.

18

Let R be rectangle. Let P{i1,...,ik} := Sxi1⊕···⊕xik (the rectangle of the parity function xi1 ⊕ · · · ⊕ xik). The (nonempty) rectangles R ∩ PI, I ⊆ {1, . . . , n}, with weights 2−n+1 form a fractional partition of R, because for x, y ∈ {0, 1}n, x = y, pair (x, y) is in exactly 2n−1 rectangles PI. Split each (nonempty) rectangle R ∩ PI into at most 9

8n2 monochromatic

• rectangles. Thus we obtain a fractional covering of size

≤ 2n · 9 8n2 · 2−n+1 = 9 4n2. Hence κ∗(R) ≤ 9

• 4n2. More precise computation gives κ∗(R) ≤ ( 9

8 + o(1))n2.

19

Let R be rectangle. Let P{i1,...,ik} := Sxi1⊕···⊕xik (the rectangle of the parity function xi1 ⊕ · · · ⊕ xik). The (nonempty) rectangles R ∩ PI, I ⊆ {1, . . . , n}, with weights 2−n+1 form a fractional partition of R, because for x, y ∈ {0, 1}n, x = y, pair (x, y) is in exactly 2n−1 rectangles PI. Split each (nonempty) rectangle R ∩ PI into at most 9

8n2 monochromatic

• rectangles. Thus we obtain a fractional covering of size

≤ 2n · 9 8n2 · 2−n+1 = 9 4n2. Hence κ∗(R) ≤ 9

• 4n2. More precise computation gives κ∗(R) ≤ ( 9

8 + o(1))n2.

• Problem. We know n2 ≤ max κ∗(R) ≤ ( 9

8 + o(1))n2. What is the maximum

exactly?

19

Examples

How to construct a measure?

1

find a subadditive rectangle function ν,

2

normalize by dividing by max{ν(m); M monochromatic}.

20

Examples

How to construct a measure?

1

find a subadditive rectangle function ν,

2

normalize by dividing by max{ν(m); M monochromatic}.

Lemma

Suppose νi, i ∈ I, are convex rectangle functions. Define ν(R) := supi∈I νi(R). Then ν is convex. The spectral norm of a matrix A is A2 = max

• u=

u=1 |

uTA v| = max

• u,

v=0

| uTA v|

• u

v . Notation: Let A be a matrix on Sf . For a subrectangle R ⊆ Sf , we denote by AR the submatrix of A determined by R.

20

sumPI (this was skipped during presentation)

Laplante, Lee and Szegedy (2006) defined sumPI(f ) := max

A=0

A2 maxM AM2 , where A denotes matrices defined on Sf and M denotes monochromatic subrectangles of Sf . They proved sumPI(f ) is computable in polynomial time from the truth table of f ; sumPI(f )2 ≤ κ(Sf ); sumPI(f )2 = O(n2).

21

sumPI (this was skipped during presentation)

Laplante, Lee and Szegedy (2006) defined sumPI(f ) := max

A=0

A2 maxM AM2 , where A denotes matrices defined on Sf and M denotes monochromatic subrectangles of Sf . They proved sumPI(f ) is computable in polynomial time from the truth table of f ; sumPI(f )2 ≤ κ(Sf ); sumPI(f )2 = O(n2).

Theorem

sumPI(f )2 ≤ κ∗(Sf )

21

• Proof. Let A = 0. Let

u, v be such that A2 = |

uT A v|

• u

v . W.l.o.g. assume all

coordinates of u and v are nonzero. Define µA(R) := 1 c · | uTAR v|2

• uR12

vR22 , where R = R1 × R2 and c = max

M

| uTAM v|2

• uM12

vM22 . Notice that | uTAR v| and uR12 vR22 are additive rectangle functions. Thus µA has the form w(R)2 s(R) , for two additive measures, with s > 0. By a generalization of our lemma, such a measure is convex, hence bounded by κ∗. Since sumPI(f ) ≤ supA=0 µA(Sf ), it is also bounded by κ∗. QED

22

The rank measure

Let A = 0 be a matrix on Sf . Define, for R ∈ R, ρA(R) := rankAR maxM rankM . Then ρA is a measure. Moreover, it is strongly additive.

• Proof. It follows from

rank(

• i

Ai) ≤

• i

rankAi.

• Problem. Is ρA convex?

Theorem [Razborov] ρA = O(n).

23

Exercise. Let A = 0 be a matrix on Sf . Define, for R ∈ R, σA(R) :=

• (x,y),(x′,y ′)∈R ax,yax,y ′ax′,yax′,y ′

maxM

• (x,y),(x′,y ′)∈M ax,yax,y ′ax′,yax′,y ′

where we do not exclude x = x′ or y = y ′. Prove:

• 1. σA is a measure;
• 2. derive n2 lower bound on the parity function using this measure.
• Problem. Is σA strongly additive? convex?

24

(from here on it was done on the white board) Monotone formulas

f : {0, 1} → {0, 1} is monotone, if for every x ≤ y (componentwise), f (x) ≤ f (y) Monotone basis of connectives {∧, ∨}. Monotone formula complexity of Lm(f ) is the size of the smallest monotone formula computing f .

1

• ne can construct explicitly monotone functions whose formula size

complexity is exponential;

2

thus one gets separation results for corresponding monotone complexity classes, in particular, mNC1 = mP;

3

lower bounds on monotone circuit size complexity can be used to derive lower bounds on proof complexity (of propositional formulas in the Resolution proof system). I will show that it is possible to obtain a superpolynomial lower bound on formula size using a rectangle measure based on the rank.

25

Construction

• 1. There are constructions of bipartite graphs E ⊆ U × V , |U| = |V | = n/2,

U ∩ V = ∅ such that for all X, Y ⊆ U disjoint and of size ≤ k there exists a v ∈ V such that ∀u ∈ X (u, v) ∈ E and ∀u ∈ Y (u, v) ∈ E Furthermore k ≈ log n. (Paley graphs, etc.)

• 2. Let f be a monotone function whose

minterms include all sets X such that |X ∩ U| = k and X ∩ V are all common neighbors of elements of X ∩ U; maxterms include all sets X such that |X ∩ U| ≤ k and X ∩ V are all common neighbors of elements of X ∩ U in the complement of E.

26

For the lower bound on f , since the minterms and the maxterms are uniquely determined by X ∩ Y , it suffices to consider the rectangle indexed by subsets of U

• f size ≤ k respectively k.

Define matrix AX,Y = 0 if X ∩ Y = ∅, and = 1 otherwise. M ⊆ Sf is a monotone monochromatic rectangle, if for some 1 ≤ i ≤ n, for all (x, y) ∈ M, xi = 0 and yi = 1. For monotone formulas we only need monotone monochromatic rectangles. This enables us to prove larger lower bounds than in the case of the basis with negation. Claim 1. If M is a monotone monochromatic rectangle, then rank AM ≤ 1. Proof - easy. Claim 2. A has full rank. Proof - not difficult (see A. Gal, P. Pudlak: Monotone complexity and the rank of matrices, Information Processing Letters 87 (2003))

27

The monotone lower bound Lmon(f ) ≥ rank A maxM rank AM = rank A = nΩ(log n)

28

Next I will present Razborov’s result that shows that for the nonmonotone basis rank measures are useless—they can prove at most linear lower bounds.

Theorem (Razborov 1992)

For every boolean function f of n variables and every matrix defined on Sf rank A maxM rank AM ≤ 4(n + 1). To prove this theorem it is better to work with measures defined on all boolean functions of n variables. Say that µ is a (formal complexity) measure if

1

µ(xi), µ(¬xi) ≤ 1;

2

µ(f ∧ g), µ(f ∨ g) ≤ µ(f ) + µ(g).

29

A rank measure ρ is defined by a matrix A defined on {0, 1}n × {0, 1}n by ρ(g) = rank ASg maxM rank AM . (If we want to prove a lower bound on the complexity of f we can assume w.l.o.g. that A is 0 outside of Sf .)

Theorem (Razborov)

1

Rank measures are submodular, i.e., they satisfy µ(f ∧ g) + µ(f ∨ g) ≤ µ(f ) + µ(g).

2

If µ is submodular, then µ(f ) = 4(n + 1).

30

For proving 2, Razborov shows that E(µ(g)) ≤ n + 1 for random g, which he proves by induction on n. Then he uses the formula f = (g ∧ (g ⊕ f ⊕ 1)) ∨ ((g ⊕ 1) ∧ (g ⊕ f )) for f the given function and g random function. Observe that if g is random, then f is expressed using four random functions. Thus, by subadditivity µ(f ) ≤ 4 E(µ(g)) = 4(n + 1).

31

Open problems

Prove more than a quadratic lower bound using an explicitly defined

• measure. (One can prove an n3−o(1) lower bound on an explicitly defined

function, but the proof uses a different method.) Are rank measures convex? Is it possible to prove a superpolynomial lower bound on monotone formula size using a convex measure?

32