Covers Open Problem: The Capacity of the Relay Channel Ayfer Ozg - PowerPoint PPT Presentation

Cover’s Open Problem: “The Capacity of the Relay Channel” Ayfer ¨ Ozg¨ ur Stanford University Advanced Networks Colloquia Series University of Maryland, March 2017 Joint work with Xiugang Wu and Leighton Pate Barnes. Ayfer ¨ Ozg¨ ur (Stanford) “The Capacity of the Relay Channel” March’17 1 / 26

Father of the Information Age Claude Shannon (1916-2001) Ayfer ¨ Ozg¨ ur (Stanford) “The Capacity of the Relay Channel” March’17 2 / 26

The Bell System Technical Journal Vol. XXVII J Illy, 1948 No.3 A Mathematical Theory of Communication By c. E. SHANNON IXTRODUCTION T HE recent development of various methods of modulation such as reM and PPM which exchange bandwidth for signal-to-noise ratio has in- tensified the interest in a general theory of communication. A basis for such a theory is contained in the important papers of Nyquist! and Hartley" on this subject. In the present paper we will extend the theory to include a number of new factors, in particular the effect of noise in the channel, and the savings possible due to the sta tistiral structure of the original message and due to the nature of the final destination of the information. The fundamental problem of communication is that of reproducing at one point either exactly or approximately a message selected at another point. Frequently the messages have meaning; that is they refer to or are correlated according to some system with certain physical or conceptual entities. These semantic aspects of communication are irrelevant to the engineering problem. The significant aspect is that the actual message is Ayfer ¨ Ozg¨ ur (Stanford) one selected from a set of possible messages. “The Capacity of the Relay Channel” The system must be designed March’17 3 / 26 to operate for each possible selection, not just the one which will actually be chosen since this is unknown at the time of design. If the number of messages in the set is finite then this number or any monotonic function of this number can be regarded as a measure of the in- formation produced when one message is chosen from the set, all choices being equally likely. As was pointed out by Hartley the most natural choice is the logarithmic function. Although this definition must be gen- eralized considerably when we consider the influence of the statistics of the message and when we have a continuous range of messages, we will in all cases use an essentially logarithmic measure. The logarithmic measure is more convenient for various reasons: 1. It is practically more useful. Parameters of engineering importance 1 Nyquist, H., "Certain Factors Affecting Telegraph Speed," Belt System Tectmical J OUT- nal, April 1924, p, 324; "Certain Topics in Telegraph Transmission Theory," A. I. E. E. TI aIlS., v. 47, April 1928, p. 617. 2 Hartley. R. V. L.. "Transmission oi Information.' Belt System Technical Journal, July 1928, p. . 'US. 379

“A method is developed for representing any communication system geometrically...” Ayfer ¨ Ozg¨ ur (Stanford) “The Capacity of the Relay Channel” March’17 4 / 26

AWGN Channel P Transmitter Receiver Capacity � � 1 + P C = log N Ayfer ¨ Ozg¨ ur (Stanford) “The Capacity of the Relay Channel” March’17 5 / 26

Converse: Sphere Packing √ nN X n (1) p √ n ( P + N ) nN 0 X n (2) √ nN X n (3) noise sphere Y sphere � �� �� � � � Sphere n ( P + N ) n 2 log 2 π e ( P + N ) = 2 � . # of X n ≤ n 2 log ( 1+ P N ) = 2 � √ � �� n 2 log 2 π eN 2 � � � Sphere nN � Ayfer ¨ Ozg¨ ur (Stanford) “The Capacity of the Relay Channel” March’17 6 / 26

Achievability: Geometric Random Coding | Lens | Pr( ∃ false X n ) ≤ × 2 nR � √ � �� � � � Sphere nP � n 2 log 2 π e PN = 2 . P + N × 2 nR n 2 log 2 π eP 2 = 2 − n 2 log ( 1+ P N ) × 2 nR Ayfer ¨ Ozg¨ ur (Stanford) “The Capacity of the Relay Channel” March’17 7 / 26

The story goes... Ayfer ¨ Ozg¨ ur (Stanford) “The Capacity of the Relay Channel” March’17 8 / 26

Cover’s Open Problem Ayfer ¨ Ozg¨ ur (Stanford) “The Capacity of the Relay Channel” March’17 9 / 26

Gaussian case Z n Relay W 1 ∼ N (0 , N ) I n = f ( Z n ) C 0 W 2 ∼ N (0 , N ) Source Destination X n Y n � � C ( ∞ ) = 1 1 + 2 P 2 log N Achievability: C ∗ 0 = ∞ . Cutset-bound (Cover and El Gamal’79): � � � � 0 ≥ 1 1 + 2 P − 1 1 + P C ∗ 2 log 2 log . N N Potentially, C ∗ 0 → 0 as P / N → 0. Ayfer ¨ Ozg¨ ur (Stanford) “The Capacity of the Relay Channel” March’17 10 / 26

Main Result Z n Relay W 1 ∼ N (0 , N ) I n = f ( Z n ) C 0 W 2 ∼ N (0 , N ) Source Destination Y n X n Theorem C ∗ 0 = ∞ Ayfer ¨ Ozg¨ ur (Stanford) “The Capacity of the Relay Channel” March’17 11 / 26

Upper Bound on the Capacity SNR = 15 dB 3.1 Cut-set bound C-F Old bound 3 New bound 2.9 2.8 2.7 2.6 2.5 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 C 0 (bit/channel use) Ayfer ¨ Ozg¨ ur (Stanford) “The Capacity of the Relay Channel” March’17 12 / 26

Cutset Bound Z n Relay W 1 ∼ N (0 , N ) I n = f ( Z n ) C 0 W 2 ∼ N (0 , N ) Source Destination X n Y n nR ≤ I ( X n ; Y n , I n ) + n ǫ n = I ( X n ; Y n ) + I ( X n ; I n | Y n ) + n ǫ n = I ( X n ; Y n ) + H ( I n | Y n ) − H ( I n | Y n , X n ) + n ǫ n � �� � � �� � ≤ nC 0 ≥ 0 ≤ n ( I ( X ; Y ) + C 0 + ǫ n ) Ayfer ¨ Ozg¨ ur (Stanford) “The Capacity of the Relay Channel” March’17 13 / 26

Cutset Bound Z n Relay W 1 ∼ N (0 , N ) I n = f ( Z n ) C 0 W 2 ∼ N (0 , N ) Source Destination X n Y n nR ≤ I ( X n ; Y n , I n ) + n ǫ n = I ( X n ; Y n ) + I ( X n ; I n | Y n ) + n ǫ n = I ( X n ; Y n ) + H ( I n | Y n ) − H ( I n | X n ) + n ǫ n � �� � � �� � ≤ nC 0 ≥ 0 ≤ n ( I ( X ; Y ) + C 0 + ǫ n ) Ayfer ¨ Ozg¨ ur (Stanford) “The Capacity of the Relay Channel” March’17 13 / 26

Z n √ Y n nN X n Typical set of Z n /Y n ⊆ I n -th bin Ayfer ¨ Ozg¨ ur (Stanford) “The Capacity of the Relay Channel” March’17 14 / 26

Z n √ Y n nN X n Typical set of Z n /Y n ⊆ I n -th bin If H ( I n | X n ) = 0, Ayfer ¨ Ozg¨ ur (Stanford) “The Capacity of the Relay Channel” March’17 14 / 26

Z n √ Y n nN X n Typical set of Z n /Y n ⊆ I n -th bin If H ( I n | X n ) = 0, then H ( I n | Y n ) = 0. Ayfer ¨ Ozg¨ ur (Stanford) “The Capacity of the Relay Channel” March’17 14 / 26

Z n Relay W 1 ∼ N (0 , N ) I n = f ( Z n ) C 0 W 2 ∼ N (0 , N ) Source Destination X n Y n R ≤ I ( X n ; Y n ) + H ( I n | Y n ) − H ( I n | X n ) + n ǫ n � �� � � �� � ≤ ? = − n log sin θ n Goal: I n = f ( Z n ) − Z n − X n − Y n � �� � H ( I n | X n )= − n log sin θ n � �� � H ( I n | Y n ) ≤ ? Ayfer ¨ Ozg¨ ur (Stanford) “The Capacity of the Relay Channel” March’17 15 / 26

H ( I n | X n ) � = 0 Typical set of Z n √ nN X n Multiple bins # of bins =? P (each bin) =? Ayfer ¨ Ozg¨ ur (Stanford) “The Capacity of the Relay Channel” March’17 16 / 26

From n - to nB - Dimensional Space X , Y , Z , I : B -length i.i.d. from { ( X n ( b ) , Y n ( b ) , Z n ( b ) , I n ( b )) } B b =1 . If H ( I n | X n ) = − n log sin θ n , then for any typical ( x , i ) p ( i | x ) . = 2 nB log sin θ n , P ( Z ∈ A ( i ) | x ) . = 2 nB log sin θ n i th bin A x ( i ) Pr . = 2 nB log sin θ n | A x ( i ) | . 2 log 2 π eN sin 2 θ n ) = 2 nB ( 1 √ nBN x Typical set of Z / Y Ayfer ¨ Ozg¨ ur (Stanford) “The Capacity of the Relay Channel” March’17 17 / 26

Isoperimetric Inequalities Isoperimetric Inequality in the Plane (Steiner 1838) Among all closed curves in the plane with a given enclosed area, the circle has the smallest perimeter. Ayfer ¨ Ozg¨ ur (Stanford) “The Capacity of the Relay Channel” March’17 18 / 26

Isoperimetric Inequalities Isoperimetric Inequality on the Sphere (Levy 1919) Among all sets on the sphere with a given volume, the spherical cap has the smallest boundary, or the smallest volume of ω -neighborhood for any ω > 0. Ayfer ¨ Ozg¨ ur (Stanford) “The Capacity of the Relay Channel” March’17 19 / 26

Blowing-up Lemma | A x ( i ) | . 2 log 2 π eN sin 2 θ n ) = 2 nB ( 1 Isoperimetric Inequality on the Sphere + Measure Concentration: P ( Z ∈ blow-up of A x ( i ) | x ) ≈ 1 . ⇓ P ( Y ∈ blow-up of A x ( i ) | x ) ≈ 1 . Ayfer ¨ Ozg¨ ur (Stanford) “The Capacity of the Relay Channel” March’17 20 / 26

Geometry of Typical Sets n -dimensional space: nB -dimensional space: Almost all ( X n , Y n , Z n , I n ) Almost all ( x , y , i ) Z n → I n z → i √ nBN √ y Y n nN x X n π 2 − θ n Information Inequality: (Wu and Ozgur, 2015) H ( I n | Y n ) ≤ n (2 log sin θ n + √ 2 log sin θ n ln 2 log e ) . Ayfer ¨ Ozg¨ ur (Stanford) “The Capacity of the Relay Channel” March’17 21 / 26

A new approach A x ( i ) A x ( i ) Y Y Control the intersection of a sphere drawn around a randomly chosen Y and A x ( i ). Ayfer ¨ Ozg¨ ur (Stanford) “The Capacity of the Relay Channel” March’17 22 / 26