Integer-Forcing Source Coding Or Ordentlich Joint work with Uri - PowerPoint PPT Presentation

Integer-Forcing Source Coding Or Ordentlich Joint work with Uri Erez June 30th, 2014 ISIT, Honolulu, HI, USA Or Ordentlich and Uri Erez Integer-Forcing Source Coding

Motivation 1 - Universal Quantization R x 1 E 1 � x 1 (ˆ x 1 , d ) � ∼ N ( 0 , K xx ) D x 2 (ˆ x 2 , d ) R x 2 E 2 Or Ordentlich and Uri Erez Integer-Forcing Source Coding

Motivation 1 - Universal Quantization R x 1 E 1 � x 1 (ˆ x 1 , d ) � ∼ N ( 0 , K xx ) D x 2 (ˆ x 2 , d ) R x 2 E 2 Goal: Simple, identical, universal, non-cooperating quantizers E 1 , E 2 Simple decoder D that can depend on K xx � � I + 1 Good performance for all K xx with the same log det d K xx Or Ordentlich and Uri Erez Integer-Forcing Source Coding

Motivation 1 - Universal Quantization R x 1 E 1 � x 1 (ˆ x 1 , d ) � ∼ N ( 0 , K xx ) D x 2 (ˆ x 2 , d ) R x 2 E 2 Goal: Simple, identical, universal, non-cooperating quantizers E 1 , E 2 Simple decoder D that can depend on K xx � � I + 1 Good performance for all K xx with the same log det d K xx Extreme cases: � 1 � � a � � b � 0 0 b K 1 , K 2 , and K 3 xx = xx = xx = 0 1 0 0 b b Or Ordentlich and Uri Erez Integer-Forcing Source Coding

Motivation 1 - Universal Quantization R x 1 E 1 � x 1 (ˆ x 1 , d ) � ∼ N ( 0 , K xx ) D P x 2 (ˆ x 2 , d ) R x 2 E 2 Goal: Simple, identical, universal, non-cooperating quantizers E 1 , E 2 Simple decoder D that can depend on K xx � � I + 1 Good performance for all K xx with the same log det d K xx Extreme cases: � 1 � � a � � b � 0 0 b K 1 , K 2 , and K 3 xx = xx = xx = 0 1 0 0 b b Willing to apply a universal linear transformation before quantization Or Ordentlich and Uri Erez Integer-Forcing Source Coding

Motivation 2 -Distributed Lossy Compression R 1 x 1 E 1 (ˆ x 1 , d 1 ) . . . . . D . (ˆ x K , d K ) R K x K E K Or Ordentlich and Uri Erez Integer-Forcing Source Coding

Motivation 2 -Distributed Lossy Compression R 1 x 1 E 1 (ˆ x 1 , d 1 ) . . . . . D . (ˆ x K , d K ) R K x K E K Fundamental limits understood in some cases Inner and outer bounds known Or Ordentlich and Uri Erez Integer-Forcing Source Coding

Motivation 2 -Distributed Lossy Compression R 1 x 1 E 1 (ˆ x 1 , d 1 ) . . . . . D . (ˆ x K , d K ) R K x K E K Fundamental limits understood in some cases Inner and outer bounds known Some applications require Extremely simple encoders/decoder Extremely short delay Or Ordentlich and Uri Erez Integer-Forcing Source Coding

Motivation 2 -Distributed Lossy Compression R x 1 E 1 (ˆ x 1 , d ) .  x 1  . . . . .  ∼ N ( 0 , K xx ) D .   . .  x K (ˆ x K , d ) R x K E K We restrict attention to: Gaussian sources x ∼ N ( 0 , K xx ) One-shot compression - block length is 1 Symmetric rates R 1 = · · · = R K = R Symmetric distortions d 1 = · · · = d K = d x k ) 2 ≤ d MSE distortion measure: E ( x k − ˆ Or Ordentlich and Uri Erez Integer-Forcing Source Coding

Goal and Means Goal Simple encoders: uniform scalar quantizers Decoupled decoding Performance close to best known inner bounds (Berger-Tung) Or Ordentlich and Uri Erez Integer-Forcing Source Coding

Goal and Means Goal Simple encoders: uniform scalar quantizers Decoupled decoding Performance close to best known inner bounds (Berger-Tung) Binning: Well understood for large blocklengths, less for short blocks Requires sophisticated joint decoding techniques Or Ordentlich and Uri Erez Integer-Forcing Source Coding

Goal and Means Goal Simple encoders: uniform scalar quantizers Decoupled decoding Performance close to best known inner bounds (Berger-Tung) Binning: Well understood for large blocklengths, less for short blocks Requires sophisticated joint decoding techniques Scalar Modulo A simple 1-D binning operation Allows for efficient decoding using integer-forcing Or Ordentlich and Uri Erez Integer-Forcing Source Coding

Integer-Forcing Source Coding: Overview Basic Idea: Rather than solving the problem R x 1 E 1 ˆ x 1 . . . . . D . x K ˆ R x K E K Or Ordentlich and Uri Erez Integer-Forcing Source Coding

Integer-Forcing Source Coding: Overview First solve R x 1 E 1 � K � m =1 a 1 m x m . . . . . D . � K � m =1 a Km x m R x K E K and then invert equations to get ˆ x 1 , . . . , ˆ x K Or Ordentlich and Uri Erez Integer-Forcing Source Coding

Integer-Forcing Source Coding: Overview First solve R x 1 E 1 � K � m =1 a 1 m x m . . . . . D . � K � m =1 a Km x m R x K E K and then invert equations to get ˆ x 1 , . . . , ˆ x K Problem reduces to simultaneous distributed compression of K linear combinations Can be efficiently solved with small rates for certain choices of coefficients Equation coefficients can be chosen to optimize performance Or Ordentlich and Uri Erez Integer-Forcing Source Coding

Distributed Compression of Integer Linear Combination R x 1 E 1 . . . � a T x D R x K E K Or Ordentlich and Uri Erez Integer-Forcing Source Coding

Distributed Compression of Integer Linear Combination Scalar Quantization x i Q ( · ) ˜ x i x i ˜ x i √ 0 12 d High resolution/dithered quantization: x i = x i + u i ˜ �� �� √ √ 12 d 12 d where u i ∼ Uniform − , , u i = x i 2 2 | x i − x i ) 2 = d E (˜ Or Ordentlich and Uri Erez Integer-Forcing Source Coding

Distributed Compression of Integer Linear Combination Modulo Scalar Quantization x ∗ x i Q ( · ) ˜ mod∆ i x i ˜ x i √ x ∗ 12 d ˜ i − 3∆ − 2∆ − ∆ 0 ∆ 2∆ 3∆ ∆ = 2 R √ 12 d = ⇒ Compression rate is R High resolution/dithered quantization: x ∗ i = [ x i + u i ] ∗ ˜ Or Ordentlich and Uri Erez Integer-Forcing Source Coding

Distributed Compression of Integer Linear Combination Encoders x ∗ Each encoder is a modulo scalar quantizer with rate R : produces ˜ k Or Ordentlich and Uri Erez Integer-Forcing Source Coding

Distributed Compression of Integer Linear Combination Encoders x ∗ Each encoder is a modulo scalar quantizer with rate R : produces ˜ k Simple modulo property For any set of integers a 1 , . . . , a K and real numbers ˜ x 1 , . . . , ˜ x K � K � ∗ � K � ∗ � � x ∗ a k ˜ x k = a k ˜ k k =1 k =1 Or Ordentlich and Uri Erez Integer-Forcing Source Coding

Distributed Compression of Integer Linear Combination Encoders x ∗ Each encoder is a modulo scalar quantizer with rate R : produces ˜ k Simple modulo property For any set of integers a 1 , . . . , a K and real numbers ˜ x 1 , . . . , ˜ x K � K � ∗ � K � ∗ � � x ∗ a k ˜ x k = a k ˜ k k =1 k =1 Decoder x ∗ x ∗ Gets: ˜ 1 , . . . , ˜ K Outputs: � K � ∗ � K � ∗ � � ∗ � � � x ∗ a T ( x + u ) a T x = a k ˜ = a k ˜ x k = k k =1 k =1 Or Ordentlich and Uri Erez Integer-Forcing Source Coding

Compression of Integer Linear Combination - P e � � ∗ � a T ( x + u ) a T x = � � � − ∆ 2 , ∆ a T x + a T u if a T ( x + u ) ∈ � 2 a T x = error otherwise ∆ P e is small if Var ( a T ( x + u ) ) is large � ∆ grows exponentially with R Or Ordentlich and Uri Erez Integer-Forcing Source Coding

Compression of Integer Linear Combination - P e � � ∗ � a T ( x + u ) a T x = � � � − ∆ 2 , ∆ a T x + a T u if a T ( x + u ) ∈ � 2 a T x = error otherwise ∆ P e is small if Var ( a T ( x + u ) ) is large � ∆ grows exponentially with R � �� � � � a T ( Kxx + d I ) a R − 1 − 3 2 2 log d P e ≤ 2 exp 22 Or Ordentlich and Uri Erez Integer-Forcing Source Coding

Compression of Integer Linear Combination - P e � � ∗ � a T ( x + u ) a T x = � � � − ∆ 2 , ∆ a T x + a T u if a T ( x + u ) ∈ � 2 a T x = error otherwise ∆ P e is small if Var ( a T ( x + u ) ) is large � ∆ grows exponentially with R � �� � � � a T ( Kxx + d I ) a R − 1 − 3 2 2 log d P e ≤ 2 exp 22 � � a T ( x + u ) For a with small Var we can take small R Or Ordentlich and Uri Erez Integer-Forcing Source Coding

Integer-Forcing Source Coding R x 1 E 1 � K � m =1 a 1 m x m . . . . . D . � K � m =1 a Km x m R x K E K Need to estimate K linearly independent integer linear combinations If all combinations estimated without error, can compute x = A − 1 � Ax = A − 1 ( Ax + Au ) = x + u ˆ Or Ordentlich and Uri Erez Integer-Forcing Source Coding

Integer-Forcing Source Coding R x 1 E 1 � K � m =1 a 1 m x m . . . . . D . � K � m =1 a Km x m R x K E K Need to estimate K linearly independent integer linear combinations If all combinations estimated without error, can compute x = A − 1 � Ax = A − 1 ( Ax + Au ) = x + u ˆ  ��   max m =1 ,..., K a T  � � m ( Kxx + d I ) a m R − 1  − 3 2 2 log d P e ≤ 2 K exp 22  Or Ordentlich and Uri Erez Integer-Forcing Source Coding

Integer-Forcing Source Coding - Performance Let � � � � R IF ( A , d ) � 1 I + 1 m =1 ,..., K a T 2 log max d K xx a m m Or Ordentlich and Uri Erez Integer-Forcing Source Coding