Polynomial Identity Testing If A , B , C are three matrices of size n - - PowerPoint PPT Presentation

Randomized Computation

Nabil Mustafa Computability and Complexity

Polynomial Identity Testing

If A,B,C are three matrices of size n x n and we want to find

• ut whether AB = C or not, then this can be solved in O(n3)
• r O(n2.37) by multiplying A and B first.

If we introduce a new matrix X of size n x 1 then we can solve this problem in O(n2) by using A(BX) = CX equation. If AB = C ⇒ Prob[A(BX) = CX] = 1 If AB = C ⇒ Prob[A(BX) = CX]≤ k, where k could have any value over the interval [0, 1] In other words we can say that for fixed values of X and C we can have different values of AB AB = 1 1 1 1

• X =

1 1

• CX =

2 2

• AB =

2 2

• X =

1 1

• CX =

2 2

• Nabil Mustafa

Polynomial Identity Testing

Similar problem can happen with polynomials as well Let A(x), B(x) and C(x) be polynomials (over reals) of degrees m, n and mn respectively. Question: Is A(x) · B(x) = C(x) Idea: reduce problem to checking zero’s of a polynomial A(x) · B(x) = C(x) ⇐ ⇒ D(x) = A(x) · B(x) − C(x) ≡ 0 D(x) has degree at most mn So if D(x) is not identically 0, it can have at most mn roots Pick a random real r out of 2mn reals. If D(r) = 0, return that A(x) · B(x) = C(x) Otherwise A(x) · B(x) = C(x) Since at most mn of these can be roots of D(·), with probability 1/2, D(r) = 0 if A(x) · B(x) = C(x)

Probabilistic Turing Machines (PTM )

So far, considered the following types of machines for input x: TM : only one computation that either accepts or rejects NTM : there are many sequences of computation, and x accepted if there exists a sequence of computations that accept ATM : many sequences of computations, and x accepted based on the type of nodes (∃ or ∀) along its path OTM : a TM with access to an oracle Last kind of machine: Probabilistic Turing Machine Very similar to a TM , except at each step, M could toss a coin to get a random bit, and decide which transition rule to follow based on the outcome of the coin toss A PTM takes input x and a random bit string r Prob[M(x, r) = z]: probability M outputs z. A polynomial PTM halts after a polynomial (in the length of the input) number of steps

Failure Probability

PTM Can Make Mistakes A major aspect of randomized algorithms or probabilistic machines is that they may fail to produce the desired output with some specified failure probability. One-sided error The machine may err only in one direction i.e. either on ‘yes’ instances or on ‘no’ instances. Either false positive or false negative, not both. Two-sided error The machine may err in both directions i.e. it may result a ‘yes’ instance as a ‘no’ instance and vice versa. Both false positive and false negative can occur. Zero-sided error The algorithm never provides a wrong answer. But sometimes it returns ‘don’t know’

Randomized Complexity Classes

RP (Randomized Polynomial time) One-sided error coRP (complement of RP ) One-sided error BPP (Bounded-error Probabilistic Polynomial time) Two-sided error ZPP (Zero-error Probabilistic Polynomial time) Zero error

Randomized Polynomial time

Definition: RP The complexity class RP is the class of all languages L for which there exists a polynomial PTM M such that x ∈ L = ⇒ Prob[M(x) = 1] ≥ 1 2 x / ∈ L = ⇒ Prob[M(x) = 0] = 1 Definition: coRP The complexity class coRP is the class of all languages L for which there exists a polynomial PTM M such that x ∈ L = ⇒ Prob[M(x, r) = 1] = 1 x / ∈ L = ⇒ Prob[M(x, r) = 0] ≥ 1 2

RP is in NP

Theorem RP ⊆ NP Proof Let L be a language in RP . Let M be a polynomial probabilistic Turing Machine that decides L. If x ∈ L, then there exists a sequence of coin tosses y such that M accepts x with y as the random string. So we can consider y as a certificate instead of a random

• string. y can be verified in polynomial time by the same

machine. If x / ∈ L, then Prob[M(x) = 1] = 0. So there is no certificate. M rejects x. So L ∈ NP

Boosting for RP

The constant 1/2 in the definition of RP can be replaced by any constant k, 0 < k < 1 Since the error is one-sided, we can repeat the algorithm t times independently: We reject the string x if any of the t runs reject, and accept x

• therwise

Clearly, if x / ∈ L, all t runs will return a 0 If x ∈ L, if any of the t runs returns a 1, we return the correct answer If x ∈ L, if all t runs returns 0, we make mistake Pr[algorithm makes a mistake t times] ≤ 1

Thus, we can make the error exponentially small by polynomial number of repetitions.

Examples of RP and coRP

Example of RP Primes, the set of all prime numbers, is in RP . This was shown by Adelman and Huang (1992), who gave a primality testing algorithm that always rejected composite numbers, and accepted primes with probability at least 1/2 Example of coRP Primes, the set of all prime numbers, is in coRP also. Miller and Rabin gave a primality test that always accepts prime numbers, but rejects composites with probability at least 1/2

Bounded-error Probabilistic Polynomial time

Definition: BPP The complexity class BPP is the class of all languages L for which there exists a polynomial PTM M such that x ∈ L = ⇒ Prob[M(x) = 1] ≥ 2 3 x / ∈ L = ⇒ Prob[M(x) = 1] < 1 3 M answers correctly with probability 2/3 on any input x regardless if x ∈ L or x / ∈ L Example of BPP Primes, the set of all prime numbers, is in BPP . BPP is closed under complement. BPP = coBPP

Boosting for BPP

2/3 is arbitrary and can be improved as follows: Repeat the algorithm t times, say it returns Xi at the i-th run Take majority answer, i.e., if ≥ t/2 times M returns 1, return 1, otherwise return 0 The Chernoff Bound Suppose X1, . . . , Xt are t independent random variables with values in {0, 1} and for every i, Pr[Xi = 1] = p. Then Pr[1 t

• i=1

Xi − p > ǫ] < e−ǫ2.

Pr[1 t

• i=1

Xi − p > −ǫ] < e−ǫ2.

Boosting for BPP

x ∈ L: M outputs correctly if t

The algorithm makes a mistake when t

Pr[ Algorithm outputs the wrong answer on x] = Pr[t

= Pr[1

t

= Pr[1

t

≤ e−

= e−ct, where c is some constant ≤ 1

So by running the algorithm O(n) times, reduce probability exponentially

RP ⊆ BPP

RP x ∈ L ⇒ Prob[M(x) = 1] ≥ 1

x / ∈ L ⇒ Prob[M(x) = 1] = 0 BPP x ∈ L ⇒ Prob[M(x) = 1] ≥ 2

x / ∈ L ⇒ Prob[M(x) = 1] < 1

Can boost RP probability by running twice: x ∈ L ⇒ Prob[M(x) = 1] ≥ 3 4 x / ∈ L ⇒ Prob[M(x) = 1] = 0 Clearly the above definition satisfies BPP

A Theorem on ZPP

Theorem ZPP = RP ∩ coRP This Theorem has two parts: ZPP ⊆ RP ∩ coRP RP ∩ coRP ⊆ ZPP ZPP ⊆ RP ∩ coRP has two parts ZPP ⊆ RP ZPP ⊆ coRP Here we prove ZPP ⊆ coRP Let L ∈ ZPP . Then there is a machine M that, for all x, either correctly determines whether or not x ∈ L or outputs “don’t know”. It does the latter with probability at most 1/2.

A Theorem on ZPP

Let M′ be a Turing Machine that on input x, returns 0 if M(x) =“don’t know” and otherwise returns M(x). If x ∈ L, then Prob[M(x) = 1 or M(x) =“don’t know”] = 1. So Prob[M′(x) = 1] = 1. If x / ∈ L, then Prob[M(x) = 0] ≥ 1/2. So Prob[M′(x) = 0] ≥ 1/2 as well. So L ∈ coRP, and ZPP ⊆ coRP

A Theorem on BPP

Theorem BPP ⊆ Σ2 Let L ∈ BPP . Then ∃ a randomized TM M such that: x ∈ L: Prob [M(x, y) = 1] ≥ 1 − 2−|x| x / ∈ L: Prob [M(x, y) = 1] ≤ 2−|x| where y is a m-bit random string. To prove L ∈ Σ2, show there exists TM N: x ∈ L ⇐ ⇒ ∃ u ∀ v N(x, u, v) = 1 m is the number of random bits M uses on |x| = n. Note that m is polynomial in n. y1, . . . , y2m denote the 2m possible random strings Input to M is a m-bit string picked uniformly from all the yi’s

The Construction

From now on, lets fix the input x where |x| = n. Then: x ∈ L: Prob [M(x) = 1] ≥ 1 − 2−n x / ∈ L: Prob [M(x) = 1] ≤ 2−n This is equivalent to the following: x ∈ L: ≥ (1 − 2−n) · 2m strings yi s.t. M(x, yi) = 1 x / ∈ L: ≤ (2−n) · 2m strings yi s.t. M(x, yi) = 1 To prove L ∈ Σ2, have to find a way to distinguish the two cases using a ‘∃’ and a ‘∀’ quantifier. Basic idea: Find a way to formulate, with polynomial-sized certificates u and v, the fact that x ∈ L if and only if there exist lots (≥ (1 − 2−n) · 2m) of m-bit strings yi such that for all of them, N returns 1.

The Construction

Consider 2m strings yi’s and take 2m permutations p1,...,p2m are 2m permutations of the yi’s Claim: Each permutation pi can be coded by a m-bit string For a fixed p, and any yi, consider m-bit string yi ⊕ p For contradiction, assume T = yi ⊕ p = yj ⊕ p A ⊕ B = C = ⇒ A = B ⊕ C. So ⊕ is a 1-to-1 function Hence yi = yj = T ⊕ p contradiction since each yi distinct The value in each cell: Aij = M(x, Pi ⊕ yj)

Some Properties of these Permutations

Because each row is just a permutation of yi’s, we have If x ∈ L, every row has ≥ (1 − 2−n) · 2m 1’s If x / ∈ L, every row has ≤ (2−n) · 2m 1’s No string yi is mapped to the same string yj twice Suppose p1 maps yi → yj and p2 also maps yi → yj Then yj = yi ⊕ p1 and yj = yi ⊕ p2 But then, p1 = yj ⊕ yi and p2 = yj ⊕ yi Hence p1 = p2, contradiction since all pi distinct strings If x ∈ L, every column has ≥ (1 − 2−n) · 2m 1’s If x / ∈ L, every column has ≤ (2−n) · 2m 1’s

Case: x ∈ L

Claim 1 If x ∈ L, there exist m columns such that none of the rows have all 0’s in these m columns Proof We know each row has at most 2m−n 0’s. For a fixed row r, there are 2m−n

• ways to choose m columns

such that each one of them contains a 0 for that row Therefore, for each row r, there are at most 2m−n

• ways to

choose m columns that are ‘bad’ for that row Therefore, there are at most 2m−n

• · 2m ways to choose m

columns that are ‘bad’ for some row This is an over-count since same set of m columns can be ‘bad’ for more than one row The number of ways to choose m columns: 2m

• Nabil Mustafa

Case: x ∈ L

Pigeon-hole Principle: If the number of ways to choose m columns that are ‘bad’ for some row is less than the total number of ways to choose m columns, then there must exist m columns that are ‘good’ for each row First note that ( e2m−n

)m · 2m ≤ (2m

Using the following well-known inequality, (n k )k ≤ n k

• ≤ (e · n

k )k we get that ( 2m

2m

• and

2m−n

• ≤ (e2m−n

)m Therefore, we have 2m−n

• · 2m <

2m

• Hence there exists a way to select m columns such that none of

the rows have all 0’s.

Case: x / ∈ L

Claim 2 If x / ∈ L, no matter how we choose our m columns, at least one of the rows will have all 0’s. Proof We know each column has at most 2m−n 1’s. So each column picked can only ‘cover’ at most 2m−n rows The total number of rows covered with m columns: 2m−n · m The total number of rows is 2m. As m < 2n, 2m−n · m < 2m Therefore, there will always be some rows having all 0’s, for any choice of m columns

Putting Everything Together

Claim 1: If x ∈ L, we will be able to choose m strings, y1, . . . , ym, such that for all m-bit permutation strings p, the machine M will

• utput 1 for at least one of M(x, yi ⊕ p), where 1 ≤ i ≤ m

Claim 2: If x / ∈ L, whichever y1, . . . , ym we choose, for at least one particular p, the machine M will output 0 for all of M(x, yi ⊕ p), where 1 ≤ i ≤ m From Claims 1 and 2 we conclude x ∈ L if and only if ∃y1, . . . , ym ∀p (M(x, y1 ⊕ z) ∨ . . . ∨ M(x, ym ⊕ p)) = 1 Therefore, L ∈ Σ2, and so BPP ⊆ Σ2.