A conjecture regarding optimality of the dictator function under - PowerPoint PPT Presentation

A conjecture regarding optimality of the dictator function under Hellinger distance Chandra Nair The Chinese University of Hong Kong 2017 ITA Workshop, UCSD February 13, 2017

Collaborators (co-authors) Andrej Bogdanov Venkat Anantharam CUHK U.C. Berkeley Thathachar Jayram Amit Chakrabarti IBM Alamaden Dartmouth Thanks: Simon’s institute

Introduction Introduction Starting point : the following conjecture 1 by Kumar (’12) X : uniform on {− 1 , +1 } n Y : obtained from X via the standard noise-operator , i.e. flip each bit (independently) with probability 1 − ρ 2 . Conjecture-MI The dictator function f d ( X ) = X 1 maximizes the mutual information I ( f ( X ); Y ) among all boolean functions f ( X ) . 1 Thomas A Courtade and Gowtham R Kumar. “Which Boolean functions maximize mutual information on noisy inputs?” In: IEEE Transactions on Information Theory 60.8 (2014), pp. 4515–4525. chandra@ie.cuhk.edu.hk Z-interference 13-Feb-2017 3 / 15

Introduction Introduction Starting point : the following conjecture 1 by Kumar (’12) X : uniform on {− 1 , +1 } n Y : obtained from X via the standard noise-operator , i.e. flip each bit (independently) with probability 1 − ρ 2 . Conjecture-MI The dictator function f d ( X ) = X 1 maximizes the mutual information I ( f ( X ); Y ) among all boolean functions f ( X ) . Alternate view Let Φ JS ( x ) := 1 − H b ( x ) = JS [( x, 1 − x ) , (1 − x, x )] . Given f ( X ) : {− 1 , +1 } n �→ {− 1 , +1 } , let Z f ( Y ) := 1 − ( T ρ f )( Y ) where 2 ( T ρ f )( Y ) = E( f ( X ) | Y ) . Conjecture-MI (restatement) The dictator function f d ( X ) = X 1 maximizes the Φ JS -entropy, E(Φ JS ( Z f ( Y ))) − Φ JS (E( Z f ( Y ))) , among all boolean functions f ( X ) . 1 Courtade and Kumar, “Which Boolean functions maximize mutual information on noisy inputs?” chandra@ie.cuhk.edu.hk Z-interference 13-Feb-2017 3 / 15

Introduction Main Conjecture Idea : Replace Φ JS ( x ) by other convex functions. Consider squared Hellinger distance between ( x, 1 − x ) and (1 − x, x ) � Φ H 2 ( x ) := 1 − 2 x (1 − x ) . As before, given f ( X ) : {− 1 , +1 } n �→ {− 1 , +1 } , let Z f ( Y ) := 1 − ( T ρ f )( Y ) . 2 Conjecture-SH The dictator function f d ( X ) = X 1 maximizes the Φ H 2 -entropy, E(Φ H 2 ( Z f ( Y ))) − Φ H 2 (E( Z f ( Y ))) , among all boolean functions f ( X ) . Equivalently �� � 1 − E( f ) 2 − E � � 1 − ( T ρ f ) 2 ( Y ) 1 − ρ 2 . ≤ 1 − chandra@ie.cuhk.edu.hk Z-interference 13-Feb-2017 4 / 15

Introduction Main Conjecture Idea : Replace Φ JS ( x ) by other convex functions. Consider squared Hellinger distance between ( x, 1 − x ) and (1 − x, x ) � Φ H 2 ( x ) := 1 − 2 x (1 − x ) . As before, given f ( X ) : {− 1 , +1 } n �→ {− 1 , +1 } , let Z f ( Y ) := 1 − ( T ρ f )( Y ) . 2 Conjecture-SH The dictator function f d ( X ) = X 1 maximizes the Φ H 2 -entropy, E(Φ H 2 ( Z f ( Y ))) − Φ H 2 (E( Z f ( Y ))) , among all boolean functions f ( X ) . Equivalently �� � 1 − E( f ) 2 − E � � 1 − ( T ρ f ) 2 ( Y ) 1 − ρ 2 . ≤ 1 − 1 Why is this interesting? (or, why should one care about this Φ ?) 2 Evidence to the veracity of the conjecture 3 Weaker forms chandra@ie.cuhk.edu.hk Z-interference 13-Feb-2017 4 / 15

Introduction About the Hellinger conjecture In short, two lemmas: 1 Conjecture-SH implies Conjecture-MI 2 Conjecture-SH is extremal chandra@ie.cuhk.edu.hk Z-interference 13-Feb-2017 5 / 15

Introduction About the Hellinger conjecture In short, two lemmas: 1 Conjecture-SH implies Conjecture-MI 2 Conjecture-SH is extremal Proposition Conjecture-SH implies Conjecture-MI Lemma √ � � 1 − 1 − x 2 The function H b is non-negative, increasing, and convex in x for x ∈ [0 , 1] . 2 √ Let Ψ( x ) = 1 − 1 − x 2 . Observe that 2 � 1 − x � � �� �� 1 − x 2 H = H Ψ . 2 Conjecture-MI can be expressed as � � �� ��� � �� �� � �� �� Ψ 1 − E( f ) 2 − E Ψ 1 − ( T ρ f ) 2 ( Y ) ≤ H (Ψ(1)) − H Ψ 1 − ρ 2 H H . chandra@ie.cuhk.edu.hk Z-interference 13-Feb-2017 5 / 15

Introduction Idea of proof By the convexity of H (Ψ( x )) (lemma) suffices to show � � �� ��� � �� �� � �� �� H Ψ 1 − E( f ) 2 − H Ψ E 1 − ( T ρ f ) 2 ( Y ) ≤ H (Ψ(1)) − H Ψ 1 − ρ 2 . However Conjecture-SH implies �� � 1 − E( f ) 2 − E � � 1 − ( T ρ f ) 2 ( Y ) ≤ 1 − 1 − ρ 2 . Apply weak-majorization inequality: in particular use convexity, non-negativeness, and increasing property of H (Ψ( x )) . chandra@ie.cuhk.edu.hk Z-interference 13-Feb-2017 6 / 15

Introduction Extremality of Hellinger Conjecture Conjecture-SH states �� � 1 − E( f ) 2 − E � � 1 − ( T ρ f ) 2 ( Y ) ≤ 1 − 1 − ρ 2 . Take lim ρ → 1 (clean channel). If Conjecture-SH is true then (for balanced Boolean functions) �� � E Sen f ( X ) ≥ 1 . Sen f ( x ) : sensitivity at x , number of neighbors with opposite value of f ( x ) . chandra@ie.cuhk.edu.hk Z-interference 13-Feb-2017 7 / 15

Introduction Extremality of Hellinger Conjecture Conjecture-SH states �� � � 1 − E( f ) 2 − E � 1 − ( T ρ f ) 2 ( Y ) ≤ 1 − 1 − ρ 2 . Take lim ρ → 1 (clean channel). If Conjecture-SH is true then (for balanced Boolean functions) �� � E Sen f ( X ) ≥ 1 . Sen f ( x ) : sensitivity at x , number of neighbors with opposite value of f ( x ) . Similar limit for Conjecture-MI would be equivalent to (for balanced Boolean functions) E ( Sen f ( X )) ≥ 1 . This is known to be true (Poincare’s inequality, Pareseval’s theorem, Harper’s isoperimetric inequality) chandra@ie.cuhk.edu.hk Z-interference 13-Feb-2017 7 / 15

Introduction Extremality of Hellinger Conjecture Conjecture-SH states �� � 1 − E( f ) 2 − E � � 1 − ( T ρ f ) 2 ( Y ) ≤ 1 − 1 − ρ 2 . Take lim ρ → 1 (clean channel). If Conjecture-SH is true then (for balanced Boolean functions) �� � E Sen f ( X ) ≥ 1 . Sen f ( x ) : sensitivity at x , number of neighbors with opposite value of f ( x ) . On the other hand, best lower bound for balanced functions � �� � 2 E Sen f ( X ) ≥ (Bobkov ’98) . π Therefore even in this limit, the conjecture would imply something new. chandra@ie.cuhk.edu.hk Z-interference 13-Feb-2017 7 / 15

Introduction Extremality of Hellinger Conjecture Conjecture-SH states �� � 1 − E( f ) 2 − E � � 1 − ( T ρ f ) 2 ( Y ) ≤ 1 − 1 − ρ 2 . Take lim ρ → 1 (clean channel). If Conjecture-SH is true then (for balanced Boolean functions) �� � E Sen f ( X ) ≥ 1 . Sen f ( x ) : sensitivity at x , number of neighbors with opposite value of f ( x ) . Lemma For any α < 1 2 , let maj ( Y ) denote the majority function (assume that n is odd). Then there exists large enough n such that Sen α < E ( Sen α � � E maj ( Y ) dic ( Y )) = 1 . chandra@ie.cuhk.edu.hk Z-interference 13-Feb-2017 7 / 15

Introduction Evidence to the veracity of Conjecture-SH �� � 1 − E( f ) 2 − E � � 1 − ( T ρ f ) 2 ( Y ) ≤ 1 − 1 − ρ 2 . 1 verified numerically until n = 8 2 Conjecture-SH is true if 1 − ρ 2 + 1 − E( f ) 2 ≤ 1 � � chandra@ie.cuhk.edu.hk Z-interference 13-Feb-2017 8 / 15

Introduction Evidence to the veracity of Conjecture-SH �� � 1 − E( f ) 2 − E � � 1 − ( T ρ f ) 2 ( Y ) ≤ 1 − 1 − ρ 2 . 1 verified numerically until n = 8 2 Conjecture-SH is true if 1 − ρ 2 + 1 − E( f ) 2 ≤ 1 + (1 − ρ 2 )(1 − E( f ) 2 ) � � On numerical verification Issue: Number of Boolean functions is 2 2 n Lemma : For any convex Φ , there is a doubly monotone boolean function that maximizes the Φ -entropy, E(Φ( Z f )) − Φ(E( Z f )) , where maximization is over all boolean functions. While the number of doubly monotone boolean functions also grows doubly exponentially, still amenable till n = 8 (or a bit more). chandra@ie.cuhk.edu.hk Z-interference 13-Feb-2017 8 / 15

Introduction On lemma Doubly-monotone : A boolean function is said to be doubly-monotone if it is monotone, and for any 1 ≤ i < j ≤ n , f ( S ∪ { i, j } ) ≥ f ( S ∪ { i } ) ≥ f ( S ∪ { j } ) ≥ f ( S ) , ∀ S ⊆ [1 : n ] . Proof: Follows from majorization and Karamata’s inequality. Similar argument also present in (Courtade-Kumar ’14) chandra@ie.cuhk.edu.hk Z-interference 13-Feb-2017 9 / 15

Introduction 2nd evidence: Proof in the parameter regime �� � 1 − (1 − λ ) E( f ) 2 − E 1 − λρ 2 − (1 − λ ) g 2 ( y ) � G ( λ ) := , where g ( y ) = ( T ρ f )( y ) . Want to show G (1) ≥ G (0) . � � E( f ) 2 g 2 ( y ) − ρ 2 G ′ ( λ ) = 1 − (1 − λ ) E( f ) 2 − E 1 − λρ 2 − (1 − λ ) g 2 ( y ) � � 2 2 chandra@ie.cuhk.edu.hk Z-interference 13-Feb-2017 10 / 15