From Feedback Capacity to Tight Achievable Rates without Feedback for AGN Channels with Stable and Unstable Autoregressive Noise Christos Kourtellaris, Charalambos D. Charalambous and Sergey Loyka ISIT 2020
1 Introduction: Objectives & Methodology 2 Problem Formulation 3 Achievable rates without feedback 4 Application Examples 5 Summary
1 Introduction: Objectives & Methodology 2 Problem Formulation 3 Achievable rates without feedback 4 Application Examples 5 Summary
Introduction: Objectives & Methodology Problem Formulation Achievable rates without feedback Application Examples Summary Objectives Additive Gaussian Noise (AGN) channels for stable and unstable autoregressive (AR) noise. Objectives: Achievable rates which emerge directly from the time-domain analysis of feedback capacity, and optimal channel input processes. Expressions of tight lower bounds on nofeedback capacity, which are computed based on finite memory channel input processes, that induce asymptotic stationarity: Markov channel input process. Independent and Identically Distributed (IID) channel input processes. C. Kourtellaris, C. D. Charalambous and S. Loyka — From Feedback Capacity to Tight Achievable Rates without Feedback for AGN Channels with Stable and Unstable Autoregressive Noise 4/27
Introduction: Objectives & Methodology Problem Formulation Achievable rates without feedback Application Examples Summary Methodology Methodology 1 : Time domain analysis and methods. Define feedback capacity and optimal channel input process. Employ the results of feedback capacity to derive achievable rates without feedback. The achievable rates without feedback provide lower bounds on no-feedback capacity. Application examples where the achievable rates are computed explicitly. 1 C. K. Kourtellaris and C. D. Charalambous, ”Information Structures for Feedback Capacity of Channels With Memory and Transmission Cost: Stochastic Optimal Control and Variational Equalities,” in IEEE Transactions on Information Theory, vol. 64, no. 7, pp. 4962-4992, July 2018. C. Kourtellaris, C. D. Charalambous and S. Loyka — From Feedback Capacity to Tight Achievable Rates without Feedback for AGN Channels with Stable and Unstable Autoregressive Noise 5/27
1 Introduction: Objectives & Methodology 2 Problem Formulation 3 Achievable rates without feedback 4 Application Examples 5 Summary
Introduction: Objectives & Methodology Problem Formulation Achievable rates without feedback Application Examples Summary Problem formulation I: Channel Model � t =1 ( X t ) 2 � Channel model: Y t = X t + V t , t = 1 ,..., n , 1 ∑ n n E s 0 ≤ κ X n � { X 1 , X 2 ,..., X n } is the sequence of channel input random variables (RVs). Y n � { Y 1 , Y 2 ,..., Y n } is the sequence of channel output RVs. V n � { V 1 , V 2 ,..., V n } is the sequence of nonstationary, jointly Gaussian distributed RVs, for fixed initial state, with distribution P V n | S 0 ( dv n | s 0 ). The autoregressive (AR) noise with memory M, is defined by M S t � ( V t V t − 1 ... V t − M +1 ) T , t = 1 ,..., n , ∑ V t = c t , j V t − j + W t = C t S t − 1 + W t , (2.1) j =1 where S 0 is Gaussian independent of W t ∈ N (0 , K W t ) , K W t > 0 , t = 1 ,..., n , C t is nonrandom. C. Kourtellaris, C. D. Charalambous and S. Loyka — From Feedback Capacity to Tight Achievable Rates without Feedback for AGN Channels with Stable and Unstable Autoregressive Noise 7/27
Introduction: Objectives & Methodology Problem Formulation Achievable rates without feedback Application Examples Summary Problem formulation II: State Space Representation A state space representation of V t is 2 S t = A t S t − 1 + B t W t , S 0 = s 0 , t = 1 ,..., n , (2.2) V t = D t S t , Y t = X t + V t = X t + H t S t − 1 + N t W t , (2.3) △ △ H t = D t A t , N t = D t B t (2.4) where ( A t , B t , D t ) are easily determined. 2 T. Kailauth, A. Sayed, and B. Hassibi, Linear Estimation, Prentice Hall, 2000. C. Kourtellaris, C. D. Charalambous and S. Loyka — From Feedback Capacity to Tight Achievable Rates without Feedback for AGN Channels with Stable and Unstable Autoregressive Noise 8/27
Introduction: Objectives & Methodology Problem Formulation Achievable rates without feedback Application Examples Summary Problem formulation III: Codes with & without Feedback Let W : Ω → M ( n ) △ = { 1 , 2 ,..., ⌈ M n ⌉} be the set of uniformly distributed messages, which are independent of ( V n , S 0 ). A time-varying feedback code for the AGN channel is denoted by C fb Z + , and consists of codewords of block length n , defined by � � n ( X t ) 2 � � X 1 = e 1 ( W , S 0 ) ,..., X n = e n ( W , S 0 , X n − 1 , Y n − 1 ) : 1 n E e E [0 , n ] ( κ ) � ∑ ≤ κ (2.5) . s 0 i =1 A time-varying code without feedback for the AGN Channel, denoted by C nfb Z + , is the restriction of the feedback code C fb Z + to the subset E nfb [0 , n ] ( κ ) ⊂ E [0 , n ] ( κ ), defined by � � n ( X t ) 2 � � ( W , S 0 , X n − 1 ) : 1 n E e nfb E nfb [0 , n ] ( κ ) � X 1 = e nfb ( W , S 0 ) ,..., X n = e nfb ∑ ≤ κ (2.6) . 1 n s 0 i =1 C. Kourtellaris, C. D. Charalambous and S. Loyka — From Feedback Capacity to Tight Achievable Rates without Feedback for AGN Channels with Stable and Unstable Autoregressive Noise 9/27
Introduction: Objectives & Methodology Problem Formulation Achievable rates without feedback Application Examples Summary Problem formulation IV: An Important Observation For the AGN channel with AR noise and feedback code, we consider S 0 = s 0 is known to encoder. Hence, V t − 1 uniquely defines S t − 1 and vice-versa, for t = 1 ,..., n . Consequently, P X t | X t − 1 , Y t − 1 , S 0 = P X t | V t − 1 , Y t − 1 , S 0 , by Y t = X t + V t = P X t | S t − 1 , Y t − 1 , S 0 , t = 1 ,..., n where the last is due to V t − 1 uniquely defines S t − 1 (since it is assumed that S 0 = s 0 is known to the encoder). C. Kourtellaris, C. D. Charalambous and S. Loyka — From Feedback Capacity to Tight Achievable Rates without Feedback for AGN Channels with Stable and Unstable Autoregressive Noise 10/27
1 Introduction: Objectives & Methodology 2 Problem Formulation 3 Achievable rates without feedback 4 Application Examples 5 Summary
Introduction: Objectives & Methodology Problem Formulation Achievable rates without feedback Application Examples Summary Results on Feedback Capacity I Define the error covariance by �� �� � T � � � Y t � � �� �� � T � � △ △ S t − � S t − � � X t − � X t − � K t = E s 0 S t S t S t = E s 0 S t = E s 0 X t X t , , � � � Y t � � � T , △ � △ � = E s 0 = t = 1 ,..., n . X t X t X t X t X t − 1 ... X t − M +1 , (a) Any achievable rate R ( s 0 ) for feedback code C fb Z + satisfies 1 △ R ( s 0 ) ≤ C ( κ , s 0 ) = lim n C n ( κ , s 0 ) (3.7) n − → ∞ where C n ( κ , s 0 ) is the FTFI capacity given by △ H ( Y n | s 0 ) − H ( V n | s 0 ) C n ( κ , s 0 ) = sup (3.8) P [0 , n ] ( κ ) H ( ·| s 0 ) denotes differential entropy, C. Kourtellaris, C. D. Charalambous and S. Loyka — From Feedback Capacity to Tight Achievable Rates without Feedback for AGN Channels with Stable and Unstable Autoregressive Noise 12/27
Introduction: Objectives & Methodology Problem Formulation Achievable rates without feedback Application Examples Summary Results on Feedback Capacity II The channel input and output ( X n , Y n ) is jointly Gaussian represented by � � S t − 1 − � X t = Λ t S t − 1 + Z t , S 0 = s 0 , t = 1 ,..., n , ( S 0 , X t − 1 , V t − 1 , Y t − 1 ) , Z t ∈ N (0 , K Z t ) indep. of � � S t − 1 − � Y t = Λ t + Z t + V t , S t − 1 � � � Y t − 1 � � �� � △ � S t − 1 − � = Y t − E s 0 = Λ t + H t + N t W t + Z t , I t Y t S t − 1 � � � � � � T � � I t I T I 1 I T K Y t | Y t − 1 , s 0 = E s 0 = Λ t + H t K t − 1 Λ t + H t + K Z t + K W t K Y 1 | s 0 = E s 0 t 1 I n is an orthogonal innovations process, Λ t ∈ R 1 × M is nonrandom, and K Z t ≥ 0. Moreover, C n ( κ , s 0 ) is given by n K Y t | Y t − 1 , s 0 1 ∑ C n ( s 0 , κ ) = sup log (3.9) 2 K W t P [0 , n ] t =1 C. Kourtellaris, C. D. Charalambous and S. Loyka — From Feedback Capacity to Tight Achievable Rates without Feedback for AGN Channels with Stable and Unstable Autoregressive Noise 13/27
Introduction: Objectives & Methodology Problem Formulation Achievable rates without feedback Application Examples Summary Results on Feedback Capacity III � S t and K t are solutions of the generalized Kalman-filter and generalized difference Riccati equation (DRE): � � T B t K W t N T t + A t K t − 1 Λ t + H t S t = A t � � S t − 1 + � T I t , t = 1 ,..., n , � � � N t K W t N T t + K Z t + Λ t + H t K t − 1 Λ t + H t � � T B t K W t N T t + A t K t − 1 Λ t + H t K t = A t K t − 1 A T t + B t K W t B T t − � � � � T N t K W t N T t + K Z t + Λ t + H t K t − 1 Λ t + H t � �� T � B t K W t N T t + A t K t − 1 Λ t + H t t = 1 ,..., n (3.10) . , with initial conditions � S 0 = s 0 and K 0 = 0. C. Kourtellaris, C. D. Charalambous and S. Loyka — From Feedback Capacity to Tight Achievable Rates without Feedback for AGN Channels with Stable and Unstable Autoregressive Noise 14/27
Recommend
More recommend
