Multiple Description Coding with Many Channels Raman Venkataramani - - PowerPoint PPT Presentation

Multiple Description Coding with Many Channels

Raman Venkataramani Harvard University, Cambridge, MA raman@deas.harvard.edu DIMACS workshop on network information theory Joint work with Gerhard Kramer (Bell Labs) and Vivek Goyal (Digital Fountain)

DIMACS 2003

Packet Networks

Output E D Decoder Encoder J1 J2 J3 Packet loss Packets Source

Model: Packets are either lost completely or received error-free.

1

DIMACS 2003

How do we deal with packet loses?

• Request a retransmission.

– Good for loss-less transmission. – Not feasible for real time data such as voice and video. Alternate Approach:

• Reconstruct using available packets.

– Requires adding redundancy to packets (coding). – Advantage: Graceful degradation of output quality when packet losses increase. The second approach is called Multiple Description Coding.

2

DIMACS 2003

Example: Coding with 3 Descriptions

Descriptions (packets) ˆ XN XN E D IID Source Decoder Encoder J1 J2 J3 Packet loss

• The source is IID vector XN (length N).
• The encoder produces L “descriptions” J1,. . . , JL of XN.
• The decoder produces an output ˆ

XN from the available descriptions.

3

DIMACS 2003

XN

23

XN

123

E D1 D2 D3 Encoder Source Channel Decoder D13 D12 D23 D123

Central decoder Side decoder Side decoder Side decoder

XN | {z } J3 J2 J1 XN

1

XN

12

XN

2

XN

13

XN

3 4

DIMACS 2003

Review of Rate-Distortion Theory

1 NH(J) ≤ R

ˆ XN Rate R Distortion D achievable

1 Nd(XN, ˆ

XN) ≤ D J not achievable Source Encoder Decoder E D XN

Theorem 1. [Shannon] The RD region is the convex set R ≥ R(D) where R(D) = min

ˆ X

I(X; ˆ X) s.t. Ed(X, ˆ X) ≤ D minimized over all ˆ X jointly distributed with X. Gaussian Source X ∼ N(0, 1): R(D) = 1

2 log( 1 D)

5

DIMACS 2003

Multiple Description (MD) coding

Source: Length N vector XN of i.i.d. random variables. Encoder: XN → {J1, . . . , JL} which are the L “descriptions” of XN at rates R1,. . . RL per source symbol. Descriptions: Jl = fl(XN), H(Jl) ≤ NRl, l = 1, . . . L. Decoder: Consists of 2L − 1 decoders: one for each non-empty subset of the available descriptions. Decoder Outputs: XN

S = gS({Jl : l ∈ S}) where S ⊆ {1, . . . , L}, S = ∅.

In the last example (L = 3): S = {1}, {2}, {3}, {1, 2}, {1, 3}, {2, 3}, or {1, 2, 3}

6

DIMACS 2003

Problem Statement

• Problem: What is the Rate-Distortion (RD) region?
• The rates (L parameters) are

R1, . . . , RL.

• The distortions ((2L − 1) parameters) are

DS = 1 N Ed(XN, XN

S ),

S ⊆ {1, . . . , L}, S = ∅

• The RD region is (L + 2L − 1)-dimensional.
• Remark: For L = 1, it is Shannon’s RD region.

7

DIMACS 2003

L = 2 case

The RD region is the set of possible rates and distortions as N → ∞: R1 = 1

NH(J1)

D1 = 1

NEd(XN, XN 1 )

R2 = 1

NH(J1)

D2 = 1

NEd(XN, XN 2 )

D12 = 1

NEd(XN, XN 12)

XN

2

D12 D2 Source Encoder Decoders Outputs J1 J2 XN

1

XN

12

XN E D1

8

DIMACS 2003

Review of Past Research

El Gamal and Cover (1982) found an achievable region for L = 2: R1 ≥ I(X; X1) R2 ≥ I(X; X2) R1 + R2 ≥ I(X; X1X2X12) + I(X1; X2) DS ≥ Ed(X, XS), S = 1, 2, 12 where X1, X2, X12 are any r.v’s jointly distributed with the source X.

such that DS ≥ Ed(X, XS). R1 achievable R2 Rate region for fixed X1, X2, X12

Remark 1: The convex hull of this region is achievable by time-sharing. Remark 2: Gives the RD region for the Gaussian source.

9

DIMACS 2003

• Ozarow (1980) computed an outer bound on the RD region for L = 2

for the Gaussian source. The bound meets the inner bound by El Gamal and Cover.

• Zhang and Berger (1987) provided a stronger achievable result than

El Gamal and Cover for L = 2. For the binary symmetric source with Hamming distortion measure, their result provides a strict improvement.

• Wolf, Wyner and Ziv (1980), Witsenhausen and Wyner (1981),

Zhang and Berger (1983) provided some results for the binary symmetric source.

10

DIMACS 2003

An Achievable Region for L > 2

Theorem 2. The RD region contains the rates and distortions satisfying

• l∈S

Rl ≥ (|S| − 1)I(X; X∅) − H(XU : U ∈ 2S|X) +

• T ⊆S

H(XT |XU : U ∈ 2T − T ) DS ≥ EdS(X, XS) for every ∅ = S ⊆ L = {1, . . . , L} and some joint distribution between

• utputs {XS} and the source X.

Remark: This result generalizes the results of El Gamal and Cover, and

• f Zhang and Berger.

11

DIMACS 2003

Gaussian Source: Outer Bound on the RD Region

• Gaussian source: X ∼ N(0, 1).
• Squared-error distortion: d(x, y) = |x − y|2.
• An outer bound on the RD region:

Theorem 3. The RD region is contained in exp

• −2
• k∈K

Rk

• ≤

min

{Km}M

m=1

inf

λ≥0

• DK

M

m=1(DKm + λ)

(DK + λ)(1 + λ)M−1

• ,

∀K ∈ 2L minimized over all partitions {Km} of K.

12

DIMACS 2003

Special Case: L Channels and L + 1 Decoders

XN

0 = XN 12...L

XN E D1 D2 D0 DL

Source Decoders Encoder

Outputs: XN

1 , XN 2 ,. . . , XN L

Rates: R1, R2,. . . , RL Distortions: D1, D2,. . . , DL and XN

0 =XN 12...L

and D0 XN

1

XN

2

XN

L

Keep only side and central decoders. Ignore all other decoders.

13

DIMACS 2003

Inner and Outer Bounds on the RD region

• Inner Bound: Computable from our achievable region (Theorem 2).
• Outer Bound: Compuatable from Theorem 3 for the Gaussian source
• Tightness of Bounds: The inner and outer bounds meet for over some

range of rates and distortions for the Gaussian source.

14

DIMACS 2003

Example: 3-Channel 4-Decoder Problem

Take L = 3, D1 = D2 = D3 = 1/2 and D0 = 1/16. Outer Bound: Rl ≥ 0.5, l = 1, 2, 3 R1 + R2 + R3 ≥ 2.1755 Achievable Rates: Rl ≥ 0.5, l = 1, 2, 3 R1 + R2 + R3 = 2.1755 Rl + Rm ≥ 1.1258, l < m

and D0 = 1/16 R2 R1 Rate region for D1 = D2 = D3 = 1/2 R3 Outer bound Achievable Provably

Remark: Excess rate = 2.1755 − R(D0) = 0.1755 bits.

15

DIMACS 2003

0.5 1 1.5 0.5 1 1.5 0.5 0.6 0.7 0.8 0.9 1 1.1 1.2 1.3 R2 bits/symbol R1 bits/symbol R3 bits/symbol

Blue Region: Inner and outer bounds meet on a hexagon. Green Region: Inner bound (does not meet outer bound).

16

DIMACS 2003

Another Example: L = 3, D1 = D2 = 1/2, D3 = 3/4, and D0 = 1/16:

0.5 1 1.5 0.5 1 1.5 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 R2 bits/symbol R1 bits/symbol R3 bits/symbol

The RD Region

17

DIMACS 2003

Gaussian Sources are Successively Refinable

Chains:

Gaussian X R1 R2 R3 X1 X2 X123 . . .

R1 ≥ R(D1) R1 + R2 ≥ R(D12) R1 + R2 + R3 ≥ R(D123) Trees:

. . . X Gaussian R1 R4 R5 R2 R3 X15 X1 X2 X124 X123

R1 ≥ R(D1) R1 + R2 ≥ R(D12) R1 + R2 + R3 ≥ R(D123) R1 + R2 + R4 ≥ R(D124) R1 + R5 ≥ R(D15)

18

DIMACS 2003

Summary

• Multiple Description Coding:

– Motivation: coding for packet networks.

• Results:

– An achievable region – An outer bound for the Gaussian source.

• Still unsolved:

– The Gaussian problem for L ≥ 3 – Non-Gaussian sources

19