Near-capacity joint source and channel coding of symbols from an - PowerPoint PPT Presentation

Near-capacity joint source and channel coding of symbols from an infinite set Robert G. Maunder, Wenbo Zhang, Tao Wang, Lajos Hanzo Presented by Rob Maunder Electronics and Computer Science, University of Southampton, SO17 1BJ, UK. Email:rm@ecs.soton.ac.uk http://users.ecs.soton.ac.uk/rm/

Part 1 - Background Robert G. Maunder, Wenbo Zhang, Lajos Hanzo

Outline Outline ❏ Symbol values from an infinite set ❏ Elias Gamma (EG) code ❏ EG-CC SSCC benchmarker ❏ Capacity loss analysis ❏ Conclusions * Separate Source and Channel Coding (SSCC) * Convolutional Code (CC)

Symbol values from an infinite set Background Finite symbol set Infinite symbol set e.g. { a, b, c, . . . , z } e.g. N 1 = { 1 , 2 , 3 , . . . , ∞} Separate Source and • Huffman code • Unary code Channel Coding (SSCC) • Shannon-Fano code • Elias Gamma code Joint Source and Channel • Variable Length Error ? Coding (JSCC) Correction (VLEC) code When decoding symbol values selected from an infinite set: • existing SSCC schemes have significant capacity loss; • existing JSCC schemes have infinite complexity.

Symbol values from an infinite set Symbol values from an infinite set 10 0 Zeta Zeta distribution P ( x ) = x − s H.264 10 − 1 ζ ( s ) , ζ ( s ) = � x ∈ N 1 x − s , 10 − 2 s > 1 , 10 − 3 p 1 = 1 /ζ ( s ) . P ( x ) p 1 10 − 4 Symbol entropy 10 − 5 H X = � x ∈ N 1 P ( x ) · log 2 (1 /P ( x )) . 10 − 6 10 − 7 1 10 100 1000 x Here, p 1 ∈ { 0 . 1 , 0 . 2 , 0 . 3 , 0 . 4 , 0 . 5 , 0 . 6 , 0 . 7 , 0 . 8 , 0 . 9 }

Elias gamma code P ( x i ) EG( x i ) x i p 1 = 0 . 7 p 1 = 0 . 8 p 1 = 0 . 9 1 0.7000 0.8000 0.9000 1 2 0.1414 0.1158 0.0717 010 3 0.0555 0.0374 0.0163 011 Elias Gamma code 4 0.0286 0.0168 0.0057 00100 5 0.0171 0.0090 0.0025 00101 Average codeword length l = � 6 0.0112 0.0054 0.0013 00110 x ∈ N 1 P ( x )(2 ⌊ log 2 ( x ) ⌋ + 1) . 7 0.0079 0.0035 0.0007 00111 8 0.0058 0.0024 0.0004 0001000 9 0.0044 0.0017 0.0003 0001001 10 0.0034 0.0013 0.0002 0001010 Table 1: The first ten codewords of Elias Gamma (EG) code.

Elias gamma code Elias Gamma coding rate 1 0.8 Coding rate 0.6 R = H X l . R 0.4 Region above curve represents 0.2 residual redundancy EG R 0 0.5 0.6 0.7 0.8 0.9 1 p 1 Coding rate of EG code for zeta distribution.

EG-CC SSCC benchmarker EG-CC SSCC benchmarker EG CC IrURC QPSK x y z π 1 π 2 encoder encoder encoder modulator y a ˜ z e ˜ π 1 ˆ EG CC IrURC QPSK x π − 1 2 decoder decoder decoder demodulator z a ˆ ˜ y π − 1 1 ➯ EG code is identical to k = 0 Exponential-Golomb code. ➯ Convolutional Code (CC) with n = 2 encoded bits, 4 states and recursive gen- erator polynomial. ➯ Irregular Unity Rate Code (IrURC), whose components have n = 1 encoded bit, 2, 4 or 8 states and recursive generator polynomial. ➯ Quaternary Phase Shift Keying (QPSK) with Gray mapping and puncturing.

Capacity loss analysis Significant EG-CC capacity loss 1 A is the area beneath the EXtrinsic Information Transfer (EXIT) curve 0.8 of the EG-CC decoder. 0.6 n is the number of encoded bits Rn or An produced by the CC encoder. 0.4 The region above the An curve represents residual redundancy exploited for error correction. 0.2 The region between the curves EG-CC An EG-CC Rn represents un-exploited residual 0 0.5 0.6 0.7 0.8 0.9 1 redundancy, giving capacity loss. p 1 Rn and An of EG-CC scheme, for zeta distribution.

Conclusions Conclusions ❏ All previous JSCC schemes have infinite complexity when decoding symbols selected from infinite sets. ❏ SSCC benchmarker suffers from significant capacity loss.

Part 2 - Unary Error Correction Codes Wenbo Zhang, Robert G. Maunder, Lajos Hanzo

Outline Outline ❏ Proposed JSCC scheme using UEC code ❏ Near-capacity analysis ❏ Error ratio performance ❏ Conclusions * Joint Source and Channel Coding (JSCC) * Unary Error Correction (UEC) Code

Proposed JSCC scheme using UEC code Proposed JSCC UEC scheme UEC encoder Unary Trellis IrURC QPSK x y z π 1 π 2 encoder encoder encoder modulator UEC decoder z e ˜ π 1 y p ˆ Unary ˜ Trellis IrURC QPSK x π − 1 2 decoder decoder decoder demodulator z a ˜ π − 1 1 ➯ Replace EG code with a unary code. ➯ Replace CC code with a novel trellis code, having n = 2 encoded bits and r states.

Proposed JSCC scheme using UEC code x i P ( x i ) Unary( x i ) EG( x i ) p 1 = 0 . 7 p 1 = 0 . 8 p 1 = 0 . 9 1 0.7000 0.8000 0.9000 0 1 Unary 2 0.1414 0.1158 0.0717 10 010 code 3 0.0555 0.0374 0.0163 110 011 4 0.0286 0.0168 0.0057 1110 00100 Average code- word length 5 0.0171 0.0090 0.0025 11110 00101 l = � x ∈ N 1 P ( x ) x 6 0.0112 0.0054 0.0013 111110 00110 7 0.0079 0.0035 0.0007 1111110 00111 8 0.0058 0.0024 0.0004 11111110 0001000 l becomes infinite 9 0.0044 0.0017 0.0003 111111110 0001001 for p 1 < 0 . 608 10 0.0034 0.0013 0.0002 1111111110 0001010 Table 2: The first ten codewords of unary and Elias Gamma (EG) codes.

Proposed JSCC scheme using UEC code Trellis code m j − 1 y j / z j m j 1 / 00 5 5 0 / 11 Here, the trellis has r = 6 states. 1 / 00 3 3 Encoding begins in state m 0 = 1 . 0 / 11 1 / 10 e.g. for symbols x = [4 , 1 , 2 , 1 , 3 , 1 , 1 , 1 , 2 , 2] , 1 1 0 / 01 ➯ y = [111001001100001010] . 0 / 10 ➯ m = [1 , 3 , 5 , 5 , 2 , 1 , 3 , 2 , 1 , 3 , 5 , 2 , 1 , 2 , 1 , 3 , 2 , 4 , 1] . 2 2 1 / 01 ➯ z = [100000111010111010001110011001110100] . 0 / 00 4 4 Each transition occurs with a different prob- 1 / 11 ability, which is exploited during soft-in soft- 0 / 00 out decoding. 6 6 1 / 11

Near-capacity analysis Vanishing UEC capacity loss 10 0 1 Zeta H.264 10 − 1 0.8 r 10 − 2 0.6 10 − 3 An − Rn Rn or An p 1 10 − 4 0.4 10 − 5 UEC An 0.2 10 − 6 UEC Rn EG-CC An EG-CC Rn 10 − 7 0 1 10 100 1000 0.5 0.6 0.7 0.8 0.9 1 r/ 2 p 1 Rn and An of EG-CC scheme and UEC scheme Capacity loss in UEC scheme, for zeta distribution having having r ∈ { 2 , 4 , 6 , 30 } states, for zeta distribution. p 1 ∈ { 0 . 7 , 0 . 8 , 0 . 9 } .

Error ratio performance Symbol Error Ratio (SER) Performance (a) p 1 = 0 . 7 (b) p 1 = 0 . 8 (c) p 1 = 0 . 9 10 0 10 0 10 0 Capacity bound Capacity bound Capacity bound 10 − 1 10 − 1 10 − 1 SER 2.6 dB 2.7 dB SER 1.8 dB 1 dB SER 1.6 dB 1.3dB 10 − 2 10 − 2 10 − 2 UEC UEC UEC 10 − 3 10 − 3 10 − 3 EG-CC EG-CC EG-CC r = 4 r = 4 r = 4 r = 6 r = 6 r = 6 r = 8 r = 8 r = 8 r = 10 r = 10 r = 10 10 − 4 10 − 4 10 − 4 1 2 3 4 5 6 7 8 0.5 1 1.5 2 2.5 3 3.5 4 -0.5 0 0.5 1 1.5 2 2.5 3 E b /N 0 [dB] E b /N 0 [dB] E b /N 0 [dB] SER performance of EG-CC and schemes, for zeta distribution having p 1 ∈ { 0 . 7 , 0 . 8 , 0 . 9 } . Uncorrelated narrowband Rayleigh fading channel with QPSK modulation. 10 4 symbols per frame and up to 10 4 Add-Compare-Select (ACS) operations per symbol.

Conclusions Conclusions ❏ SSCC benchmarker suffers from significant capacity loss. ❏ Proposed JSCC UEC scheme has only moderate complexity and its capacity loss asymptotically approaches zero as the number states r increases. ❏ As much as 1.3 dB gain within 1.6 dB of capacity bound, without any increase in transmission energy, duration, bandwidth or decoding complexity. ❏ However, the proposed UEC has an infinite average codeword length for zeta distributed source symbols having p 1 < 0 . 608 , as well as poor SER performance for p 1 = 0 . 7 .

Part 3 - Elias Gamma Error Correction Codes Tao Wang, Wenbo Zhang, Robert G. Maunder, Lajos Hanzo

Outline Outline ❏ Proposed JSCC scheme using EGEC code ❏ Near-capacity analysis ❏ Error ratio performance ❏ Conclusions * Joint Source and Channel Coding (JSCC) * Elias Gamma Error Correction (EGEC) Code

Proposed JSCC scheme using EGEC code Unary( d i ) EG( d i ) x i Unary( x i ) FLC( t i ) t i d i Elias Gamma code 1 1 1 1 1 0 revisited 2 01 010 2 01 0 0 3 001 011 2 01 1 1 An Elias Gamma (EG) codeword 4 0001 00100 3 001 00 0 EG( d i ) can be thought of as a 5 00001 00101 3 001 01 1 concatenation of codewords from 6 000001 00110 3 001 10 2 a unary code and a Fixed Length 7 0000001 00111 3 001 11 3 Code (FLC). 8 00000001 0001000 4 0001 000 0 x i = ⌊ log 2 ( d i ) ⌋ + 1 9 000000001 0001001 4 0001 001 1 t i = d i − 2 ⌊ log 2 ( d i ) ⌋ d i = 2 x i − 1 + t i 10 0000000001 0001010 4 0001 010 2 Table 3: The first ten codewords of various source codes.

Proposed JSCC scheme using EGEC code Proposed JSCC EGEC scheme EGEC encoder EGEC(UEC) encoder d x Unary y Trellis z URC π 1 π 2 S encoder encoder encoder QPSK EGEC(FLC-CC) encoder modulator t FLC u v CC w URC π 3 π 4 π 5 encoder encoder encoder EGEC decoder EGEC(UEC) decoder z e ˜ π 1 ˆ ˆ d x Unary ˆ Trellis URC y S − 1 π − 1 2 decoder decoder decoder z a ˜ π − 1 1 QPSK EGEC(FLC-CC) decoder demodulator u e v a w e ˜ ˜ ˜ π 3 π 4 ˆ t FLC CC URC π − 1 5 decoder decoder decoder u a v e w a ˜ ˜ ˜ π − 1 π − 1 3 4 ➯ π 2 and π 5 can use different puncturing rates, to achieve Unequal Error Protection (UEP). ➯ The FLC decoder only engages in iterative decoding for symbols satisfying ˆ x i ≤ x max .