Greedy-Merge Degrading has Optimal Power-Law Assaf Kartowsky and Ido Tal Technion, Israel June 28, 2017 Assaf Kartowsky and Ido Tal Greedy-Merge Degrading has Optimal Power-Law June 28, 2017 1 / 17
Introduction Motivation Motivation X Y W W : X → Y , P X , |Y| is very large Common problem in – Digital receiver design – Polar code construction ⇒ Quantize Y to L letters Assaf Kartowsky and Ido Tal Greedy-Merge Degrading has Optimal Power-Law June 28, 2017 2 / 17
Introduction Motivation Motivation X Y Z W Φ Q Q : X → Z , |Z| = L ∆ I � I ( X ; Y ) − I ( X ; Z ) ≥ 0 Question Given |X| , what is ∆ I ∗ � min Q ∆ I = O (?) in terms of L ? Assaf Kartowsky and Ido Tal Greedy-Merge Degrading has Optimal Power-Law June 28, 2017 3 / 17
Introduction Previous Results Previous Results Binary input, |X| = 2 O ( L − 1 . 5 log L ) Pedarsani et al. 2011 Finite |X| (constant) O ( L − 1 / ( |X|− 1) ) Gulcu, Ye, and Barg 2016 Ω( L − 2 / ( |X|− 1) ) Tal 2015 Related work Kurkoski and Yagi 2014 Nazer, Ordentlich, and Polyanskiy 2017 Assaf Kartowsky and Ido Tal Greedy-Merge Degrading has Optimal Power-Law June 28, 2017 4 / 17
Main Result Main Result Theorem � |Y| > 2 |X| ∆ I ∗ = O ( L − 2 |X|− 1 ) = ⇒ L ≥ 2 |X| In particular, 2 |X|− 1 π |X| ( |X| − 1) 2 |X| ∆ I ∗ ≤ 2 · L − |X|− 1 � � �� � 2 1 + |X|− 1 1 Γ 2( |X|− 1) − 1 2 1 + 2 This bound is: Attained by “greedy-merge” algorithm Tight in power-law sense Assaf Kartowsky and Ido Tal Greedy-Merge Degrading has Optimal Power-Law June 28, 2017 5 / 17
Proof Outline Main Ideas Proof - Main Ideas Greedy-merge algorithm Simple upper bounds on ∆ I “Sphere-packing” Assaf Kartowsky and Ido Tal Greedy-Merge Degrading has Optimal Power-Law June 28, 2017 6 / 17
Proof Outline Notation Notation Channel, input and output probabilities: W ( y | x ) � P ( Y = y | X = x ) π x � P ( X = x ) W ( x | y ) � P ( X = x | Y = y ) π y � P ( Y = y ) Mutual information: � � I ( W , P X ) � I ( X ; Y ) = η ( π x ) − π y η ( W ( x | y )) x ∈X x ∈X , y ∈Y � − p log p p > 0 η ( p ) � 0 p = 0 Loss in mutual information: ∆ I = I ( W , P X ) − I ( Q , P X ) Assaf Kartowsky and Ido Tal Greedy-Merge Degrading has Optimal Power-Law June 28, 2017 7 / 17
Proof Outline Greedy-Merge Algorithm Merging a Pair of Letters For y a , y b ∈ Y define: y a α x � W ( x | y a ) α � ( α x ) x ∈X π a � π y a y ab β x � W ( x | y b ) β � ( β x ) x ∈X π b � π y b y b Merging y a , y b to y ab : W ( x | y ab ) = π a α x + π b β x π y ab = π a + π b π a + π b Loss by a single merger: � π a α x + π b β x � ∆ I x � ( π a + π b ) η − π a η ( α x ) − π b η ( β x ) π a + π b � ∆ I = ∆ I x x ∈X Assaf Kartowsky and Ido Tal Greedy-Merge Degrading has Optimal Power-Law June 28, 2017 8 / 17
Proof Outline Greedy-Merge Algorithm Greedy-Merge Algorithm Algorithm: – Merge y a , y b that minimize ∆ I – Repeat |Y| − L times If min ∆ I = O ( |Y| − |X| +1 |X|− 1 ) ⇒ proof is finished New Goal Prove existence of y a , y b ∈ Y s.t. ∆ I = O ( |Y| − |X| +1 |X|− 1 ) Assaf Kartowsky and Ido Tal Greedy-Merge Degrading has Optimal Power-Law June 28, 2017 9 / 17
Proof Outline Simple upper bounds on ∆ I Simple upper bounds on ∆ I ∆ I is complicated Upper bound ∆ I : ∆ I x ≤ ( π a + π b ) ( α x − β x ) 2 ∆ I x ≤ ( π a + π b ) | α x − β x | min( α x ,β x ) � � | α x − β x | , ( α x − β x ) 2 ⇒ ∆ I ≤ ( π a + π b ) |X| · max x ∈X min min( α x , β x ) � �� � � d ( α , β ) Limit search to: � � |Y small | ≥ |Y| 2 Y small � y ∈ Y : π y ≤ |Y| 2 y a , y b ∈Y small ∆ I ≤ 4 |X| ⇒ min |Y| · y a , y b ∈Y small d ( α , β ) min Assaf Kartowsky and Ido Tal Greedy-Merge Degrading has Optimal Power-Law June 28, 2017 10 / 17
Proof Outline Sphere-Packing Sphere-Packing Essentials y 4 y 6 y 2 1 2 r y 7 y 3 y 5 y 1 A metric d : M × M → R + 0 r / 2-radius volumed spheres � α , r � � ζ ∈ M : d ( α , ζ ) ≤ r � � B 2 2 Find r = r critical > 0 s.t.: � � α , r �� � � � B Vol = Vol whole space 2 α ∈ S ⇒ d ( α , β ) ≤ r for some α , β ∈ M Assaf Kartowsky and Ido Tal Greedy-Merge Degrading has Optimal Power-Law June 28, 2017 11 / 17
Proof Outline Sphere-Packing Sphere-Packing Reasoning � � � � α , r β , r � = ∅ ⇒ d ( α , ζ ) , d ( β , ζ ) ≤ r B ∩ B 2 2 2 Triangle inequality: ⇒ d ( α , β ) ≤ d ( α , ζ ) + d ( ζ , β ) ≤ r B ( α i , r 2 ) d ≤ r α 3 α 2 α 5 α 1 α 4 Assaf Kartowsky and Ido Tal Greedy-Merge Degrading has Optimal Power-Law June 28, 2017 12 / 17
Proof Outline Sphere-Packing Sphere-Packing Reasoning d is a semimetric Find Q ′ ( · , r ) s.t.: Q ′ ( α , r ) ∩ Q ′ ( β , r ) � = ∅ ⇒ d ( α , β ) ≤ r Q ′ ( α i , r ) r ≤ α 3 d α 2 α 5 α 1 α 4 Assaf Kartowsky and Ido Tal Greedy-Merge Degrading has Optimal Power-Law June 28, 2017 13 / 17
Proof Outline Sphere-Packing Towards a “Sphere” Our d ( · , · ) is a semimetric B ( α , r ) is a box s 1 s 0 + s 1 = 1 B ( α , r ) ω ( α 1 , r ) α 1 ω ( α 1 , r ) s 0 α 0 ω ( α 0 , r ) ω ( α 0 , r ) �� � r 2 ω ( α x , r ) � max ( √ α x r , r ) 4 + α x r − r ω ( α x , r ) � max 2 , r � x ∈X α x = 1 ⇒ dimension reduction is preferable Assaf Kartowsky and Ido Tal Greedy-Merge Degrading has Optimal Power-Law June 28, 2017 14 / 17
Proof Outline Sphere-Packing Towards a “Sphere” s 0 + s 1 = 1 s 1 B ( α , r ) ω ( α 1 , r ) Q ′ ( α , r ) B K ( α , r ) α 1 C ( α , r ) ω ( α 1 , r ) s 0 ω ( α 0 , r ) α 0 ω ( α 0 , r ) � ζ ∈ R |X| : � � B K ( α , r ) � B ( α , r ) ∩ x ∈X ζ x = 1 - complicated C ( α , r ) ⊆ B K ( α , r ) - a box in R |X|− 1 : C ( α , r ) ∩ C ( β , r ) � = ∅ d ( α , β ) ≤ r � Q ′ ( α , r ) ⊆ R |X|− 1 - suitable Assaf Kartowsky and Ido Tal Greedy-Merge Degrading has Optimal Power-Law June 28, 2017 15 / 17
Proof Outline Sphere-Packing Weighted “Sphere”-Packing Variable volume “spheres” ϕ 1 2 √ s 1 Q ′ ( α , r ) s 1 ∼√ α 1 r α 1 |X| − 1 dimensional density: 1 � ϕ ( ζ ′ ) � 2 √ ζ x x ∈X ′ Volume → Weight Assaf Kartowsky and Ido Tal Greedy-Merge Degrading has Optimal Power-Law June 28, 2017 16 / 17
Proof Outline Sphere-Packing Conclusion and Further Results Conclusion 2 ∆ I ∗ = O ( L − |X|− 1 ) Tight in power-law Attained by “greedy-merge” algorithm Further results (full paper) For the upgrading setting: 2 ∆ I ∗ = Ω( L − |X|− 1 ), same sequence of channels 2 ∆ I ∗ = O ( L − |X|− 1 ) for |X| = 2 Optimal upgrading algorithm for |X| = 2 Assaf Kartowsky and Ido Tal Greedy-Merge Degrading has Optimal Power-Law June 28, 2017 17 / 17
Recommend
More recommend
