Chebyshev Polynomials, Approximate Degree, and Their Applications - PowerPoint PPT Presentation

Chebyshev Polynomials, Approximate Degree, and Their Applications Justin Thaler 1 Georgetown University

Boolean Functions Boolean function f : {− 1 , 1 } n → {− 1 , 1 } � if x = ( − 1) n − 1 ( TRUE ) AND n ( x ) = 1 ( FALSE ) otherwise

Approximate Degree A real polynomial p ǫ -approximates f if ∀ x ∈ {− 1 , 1 } n | p ( x ) − f ( x ) | < ǫ � deg ǫ ( f ) = minimum degree needed to ǫ -approximate f � deg( f ) := deg 1 / 3 ( f ) is the approximate degree of f

Threshold Degree Definition Let f : {− 1 , 1 } n → {− 1 , 1 } be a Boolean function. A polynomial p sign-represents f if sgn( p ( x )) = f ( x ) for all x ∈ {− 1 , 1 } n . Definition The threshold degree of f is min deg( p ) , where the minimum is over all sign-representations of f . An equivalent definition of threshold degree is lim ǫ → 1 � deg ǫ ( f ) .

Why Care About Approximate and Threshold Degree? Upper bounds on � deg ǫ ( f ) and deg ± ( f ) yield efficient learning algorithms. ǫ ≈ 1 / 3 : Agnostic Learning [KKMS05] ǫ ≈ 1 − 2 − n δ : Attribute-Efficient Learning [KS04, STT12] ǫ → 1 (i.e., deg ± ( f ) upper bounds): PAC learning [KS01]

Why Care About Approximate and Threshold Degree? Upper bounds on � deg ǫ ( f ) and deg ± ( f ) yield efficient learning algorithms. ǫ ≈ 1 / 3 : Agnostic Learning [KKMS05] ǫ ≈ 1 − 2 − n δ : Attribute-Efficient Learning [KS04, STT12] ǫ → 1 (i.e., deg ± ( f ) upper bounds): PAC learning [KS01] Upper bounds on � deg 1 / 3 ( f ) also imply fast algorithms for differentially private data release [TUV12, CTUW14].

Why Care About Approximate and Threshold Degree? Lower bounds on � deg ǫ ( f ) yield lower bounds on: Quantum query complexity [BBCMW98, AS01, Amb03, KSW04] Communication complexity [She08, SZ08, CA08, LS08, She12] Lower bounds hold for a communication problem related to f . Technique is called the Pattern Matrix Method [She08]. Circuit complexity [MP69, Bei93, Bei94, She08] Oracle Separations [Bei94, BCHTV16]

Why Care About Approximate and Threshold Degree? Lower bounds on � deg ǫ ( f ) yield lower bounds on: Quantum query complexity [BBCMW98, AS01, Amb03, KSW04] Communication complexity [She08, SZ08, CA08, LS08, She12] Lower bounds hold for a communication problem related to f . Technique is called the Pattern Matrix Method [She08]. Circuit complexity [MP69, Bei93, Bei94, She08] Oracle Separations [Bei94, BCHTV16] Lower bounds on � deg( f ) also yield efficient secret-sharing schemes [BIVW16]

Details of Communication Applications Lower bounds on � deg ǫ ( f ) and deg ± ( f ) yield communication lower bounds (often in a black-box manner) [Sherstov 2008] ǫ ≈ 1 / 3 : BQP cc lower bounds. ǫ ≈ 1 − 2 − n δ : PP cc lower bounds ǫ → 1 (i.e., deg ± ( f ) lower bounds): UPP cc lower bounds.

Example 1: The Approximate Degree of AND n

Example: What is the Approximate Degree of AND n ? deg(AND n ) = Θ( √ n ) . � Upper bound: Use Chebyshev Polynomials . Markov’s Inequality: Let G ( t ) be a univariate polynomial s.t. deg( G ) ≤ d and sup t ∈ [ − 1 , 1] | G ( t ) | ≤ 1 . Then | G ′ ( t ) | ≤ d 2 . sup t ∈ [ − 1 , 1] Chebyshev polynomials are the extremal case.

Example: What is the Approximate Degree of AND n ? deg(AND n ) = O ( √ n ) . � After shifting a scaling, can turn degree O ( √ n ) Chebyshev polynomial into a univariate polynomial Q ( t ) that looks like: !"#$%&'()*+*&',* Define n -variate polynomial p via p ( x ) = Q ( � n i =1 x i /n ) . ∀ x ∈ {− 1 , 1 } n . Then | p ( x ) − AND n ( x ) | ≤ 1 / 3

Example: What is the Approximate Degree of AND n ? deg(AND n ) = Ω( √ n ) . [NS92] � Lower bound: Use symmetrization . ∀ x ∈ {− 1 , 1 } n . Suppose | p ( x ) − AND n ( x ) | ≤ 1 / 3 There is a way to turn p into a univariate polynomial p sym that looks like this: !"#$%&'()*+*&',* Claim 1: deg( p sym ) ≤ deg( p ) . ⇒ deg( p sym ) = Ω( n 1 / 2 ) . Claim 2: Markov’s inequality =

Example 2: The Threshold Degree of the Minsky-Papert DNF

The Minsky-Papert DNF The Minsky-Papert DNF is MP ( x ) := OR n 1 / 3 ◦ AND n 2 / 3 .

The Minsky-Papert DNF Claim: deg ± ( MP ) = ˜ Θ( n 1 / 3 ) . The Ω( n 1 / 3 ) lower bound was proved by Minsky and Papert in 1969 via a symmetrization argument. More generally, deg ± (OR t ◦ AND b ) ≥ Ω(min( t, b 1 / 2 )) .

The Minsky-Papert DNF Claim: deg ± ( MP ) = ˜ Θ( n 1 / 3 ) . The Ω( n 1 / 3 ) lower bound was proved by Minsky and Papert in 1969 via a symmetrization argument. More generally, deg ± (OR t ◦ AND b ) ≥ Ω(min( t, b 1 / 2 )) . We will prove the matching upper bound: deg ± (OR t ◦ AND b ) ≤ ˜ O (min( t, b 1 / 2 )) . First, we’ll construct a sign-representation of degree O (( b log t ) 1 / 2 ) using Chebyshev approximations to AND b . Then we’ll construct a sign-representation of degree ˜ O ( t ) using rational approximations to AND b .

A Sign-Representation for OR t ◦ AND b of degree ˜ O ( b 1 / 2 ) Let p 1 be a (Chebyshev-derived) polynomial of degree � √ b · log t � approximating AND b to error 1 O 8 t . Let p = 1 2 · (1 − p 1 ) . 2 − � t Then 1 i =1 p ( x i ) sign-represents OR t ◦ AND b .

A Sign-Representation for OR t ◦ AND b of degree ˜ O ( b 1 / 2 ) Let p 1 be a (Chebyshev-derived) polynomial of degree � √ b · log t � approximating AND b to error 1 O 8 t . Let p = 1 2 · (1 − p 1 ) . 2 − � t Then 1 i =1 p ( x i ) sign-represents OR t ◦ AND b . If AND b ( x i ) = FALSE for all i , then t � 1 p ( x i ) ≥ 1 2 − t · 1 2 − 8 t ≥ 3 / 8 . i =1 If AND b ( x i ) = TRUE for even one i , then t � 1 p ( x i ) ≤ 1 2 − 7 / 8 + ( t − 1) · 1 2 − 8 t ≤ − 1 / 4 . i =1

A Sign-Representation for OR t ◦ AND b of degree ˜ O ( t ) Fact: there exist p 1 , q 1 of degree O (log b · log t ) such that � � � � � AND b ( x ) − p 1 ( x ) � ≤ 1 � � 8 t for all x ∈ {− 1 , 1 } b . q 1 ( x ) � � Let p ( x ) 1 − p 1 ( x ) q ( x ) = 1 2 · . q 1 ( x )

A Sign-Representation for OR t ◦ AND b of degree ˜ O ( t ) Fact: there exist p 1 , q 1 of degree O (log b · log t ) such that � � � � � AND b ( x ) − p 1 ( x ) � ≤ 1 � � 8 t for all x ∈ {− 1 , 1 } b . q 1 ( x ) � � Let p ( x ) 1 − p 1 ( x ) q ( x ) = 1 2 · . q 1 ( x ) Claim: The following polynomial sign-represents OR t ◦ AND b .     t � � �  1 q 2 ( x i )  −  p ( x i ) · q ( x i ) · q 2 ( x i ′ )  . r ( x ) := 2 · 1 ≤ i ≤ t i =1 1 ≤ i ≤ t,i ′ � = i

A Sign-Representation for OR t ◦ AND b of degree ˜ O ( t ) Fact: there exist p 1 , q 1 of degree O (log b · log t ) such that � � � � � AND b ( x ) − p 1 ( x ) � ≤ 1 � � 8 t for all x ∈ {− 1 , 1 } b . q 1 ( x ) � � Let p ( x ) 1 − p 1 ( x ) q ( x ) = 1 2 · . q 1 ( x ) Claim: The following polynomial sign-represents OR t ◦ AND b .     t � � �  1 q 2 ( x i )  −  p ( x i ) · q ( x i ) · q 2 ( x i ′ )  . r ( x ) := 2 · 1 ≤ i ≤ t i =1 1 ≤ i ≤ t,i ′ � = i 2 − � t p ( x i ) Proof: sgn(OR t ◦ AND b ( x )) = 1 q ( x i ) = i =1 2 − � t p ( x i ) · q ( x i ) r ( x ) 1 = i =1 q 2 ( x i ) . The denominator of the i =1 q 2 ( x i ) � t RHS is non-negative, so throw it away w/o changing the sign.

Recent Progress on Lower Bounds: Beyond Symmetrization

Beyond Symmetrization Symmetrization is “lossy”: in turning an n -variate poly p into a univariate poly p sym , we throw away information about p . Challenge problem: What is � deg(OR-AND n ) ?

History of the OR-AND Tree Upper bounds � deg(OR-AND n ) = O ( n 1 / 2 ) [HMW03] Lower bounds Ω( n 1 / 4 ) [NS92] Ω( n 1 / 4 √ log n ) [Shi01] Ω( n 1 / 3 ) [Amb03] [Aar08] Reposed Question Ω( n 3 / 8 ) [She09] Ω( n 1 / 2 ) [BT13] Ω( n 1 / 2 ) , independently [She13]

Linear Programming Formulation of Approximate Degree What is best error achievable by any degree d approximation of f ? Primal LP (Linear in ǫ and coefficients of p ): min p,ǫ ǫ for all x ∈ {− 1 , 1 } n s.t. | p ( x ) − f ( x ) | ≤ ǫ deg p ≤ d Dual LP: � max ψ ψ ( x ) f ( x ) x ∈{− 1 , 1 } n � s.t. | ψ ( x ) | = 1 x ∈{− 1 , 1 } n � ψ ( x ) q ( x ) = 0 whenever deg q ≤ d x ∈{− 1 , 1 } n

Dual Characterization of Approximate Degree Theorem: deg ǫ ( f ) > d iff there exists a “dual polynomial” ψ : {− 1 , 1 } n → R with � (1) ψ ( x ) f ( x ) > ǫ “high correlation with f ” x ∈{− 1 , 1 } n � (2) | ψ ( x ) | = 1 “ L 1 -norm 1” x ∈{− 1 , 1 } n � (3) ψ ( x ) q ( x ) = 0 , when deg q ≤ d “pure high degree d ” x ∈{− 1 , 1 } n A lossless technique. Strong duality implies any approximate degree lower bound can be witnessed by dual polynomial.

Goal: Construct an explicit dual polynomial ψ OR-AND for OR-AND

Constructing a Dual Polynomial By [NS92], there are dual polynomials ψ OUT for � deg (OR n 1 / 2 ) = Ω( n 1 / 4 ) and ψ IN for � deg (AND n 1 / 2 ) = Ω( n 1 / 4 ) Both [She13] and [BT13] combine ψ OUT and ψ IN to obtain a dual polynomial ψ OR-AND for OR-AND . The combining method was proposed in independent earlier work by [Lee09] and [She09].