Lossy coding of a time-limited piece of a band-limited white - - PowerPoint PPT Presentation

Lossy coding of a time-limited piece of a band-limited white Gaussian source

Youssef Jaffal, Ibrahim Abou-Faycal

American University of Beirut Department of Electrical and Computer Engineering Email: yaj03@mail.aub.edu 2020 IEEE International Symposium on Information Theory 21-26 June 2020 – Los Angeles, California, USA

Youssef Jaffal, Ibrahim Abou-Faycal (American University of Beirut Department of Electrical and Computer Engine ISIT 2020 1 / 23

Motivation

1

Motivation

2

Prolate Spheroidal Wave Functions (PSWFs)

3

System Model

4

Bounds Lower Bound Upper Bound

5

Asymptotic analysis as T → ∞

6

Numerical Results

7

Summary

Youssef Jaffal, Ibrahim Abou-Faycal (American University of Beirut Department of Electrical and Computer Engine ISIT 2020 2 / 23

Motivation

Rate Distortion Function of Band-limited Sources

For a real W-Hz band-limited “white” Gaussian source: RShannon = W log2 Q D

• bits per seconds

Q: the average power of the source D: the average mean-squared error distortion level How to derive this equation? 1- Transform the problem to a discrete one using the Shannon-Nyquist sampling theorem 2- Apply the rate distortion function for the discrete-time stationary Gaussian source However: This requires compression at once of an asymptotically infinite duration piece from the source

Youssef Jaffal, Ibrahim Abou-Faycal (American University of Beirut Department of Electrical and Computer Engine ISIT 2020 3 / 23

Motivation

Practical Setup

Independently encode adjacent T-seconds short-duration pieces

. . . . . . . . . . . . . . . . . . . . . . . . . . . .

✲ ✻

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

X0(t) X1(t) X−1(t) X(t)

3T 2

− 3T

2

− T

2 T 2

Then:

1

Sampling the short duration sub-functions is lossy:

→ Solution: make use of Prolate Spheroidal Wave Functions (PSWFs)

2

Deriving non-asymptotic results for the equivalent discrete-time problem:

→ Solution: Derive bounds in the finite-blocklength regime

Youssef Jaffal, Ibrahim Abou-Faycal (American University of Beirut Department of Electrical and Computer Engine ISIT 2020 4 / 23

Prolate Spheroidal Wave Functions (PSWFs)

1

Motivation

2

Prolate Spheroidal Wave Functions (PSWFs)

3

System Model

4

Bounds Lower Bound Upper Bound

5

Asymptotic analysis as T → ∞

6

Numerical Results

7

Summary

Youssef Jaffal, Ibrahim Abou-Faycal (American University of Beirut Department of Electrical and Computer Engine ISIT 2020 5 / 23

Prolate Spheroidal Wave Functions (PSWFs)

Properties of PSWFs

Define c = 2WT > 0. The PSWFs {ϕc,i(t)}i∈N satisfy: 1-

∞

−∞

ϕc,i(t)ϕc,l(t)dt = δil, ∀i, l ∈ N

2-

T/2

−T/2

ϕc,i(t)ϕc,l(t)dt = λc,iδil, ∀i, l ∈ N

3- The normalized time-limited PSWFs

• Dϕc,i(t)

√

λc,i

• i∈N

form a CON set

The Eigenvalues {λc,i} satisfy:

∞

• i=0

λc,i = 2WT = c For any γ > 0, lim

c→∞ λc,c(1+γ) = 0

lim

c→∞ λc,c(1−γ) = 1.

84 88 92 96 100 104 108 112 116

i

10 -10 10 -5 10 0

100,i

1-

100,i

Youssef Jaffal, Ibrahim Abou-Faycal (American University of Beirut Department of Electrical and Computer Engine ISIT 2020 6 / 23

System Model

1

Motivation

2

Prolate Spheroidal Wave Functions (PSWFs)

3

System Model

4

Bounds Lower Bound Upper Bound

5

Asymptotic analysis as T → ∞

6

Numerical Results

7

Summary

Youssef Jaffal, Ibrahim Abou-Faycal (American University of Beirut Department of Electrical and Computer Engine ISIT 2020 7 / 23

System Model

System Model

X(t) is a 0-mean W-Hz band-limited Gaussian process with a flat PSD SX(f) = σ2 f ∈ [−W, W]

• therwise

,

. . . . . . . . . . . . . . . . . . . . . . . . . . . .

✲ ✻

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

X0(t) X1(t) X−1(t) X(t)

3T 2

− 3T

2

− T

2 T 2

X(t) is encoded by independently coding {Xk(t)}k∈Z. Focus on coding X0(t) = X(t) t ∈

• − T

2 , T 2

• therwise

.

Youssef Jaffal, Ibrahim Abou-Faycal (American University of Beirut Department of Electrical and Computer Engine ISIT 2020 8 / 23

System Model

System Model

Let {Ym(t)}2R

m=1 be the representation signals whenever X0(t) is

represented by R bits The distortion measure is the mean squared error:

To encode X0(t), select the nearest representation signal Y ˙

m(t)

The distortion is d (X0(t) − Y ˙

m(t)) = 1

T

• T

2

− T

2

|X0(t) − Y ˙

m(t)|2 dt

Distortion level d that must be guaranteed with probability (1 − ǫ), P [d (X0(t) − Y ˙

m(t)) > d] ≤ ǫ

The problem is to find the minimum possible rate R∗, for a given fixed d and ǫ

Youssef Jaffal, Ibrahim Abou-Faycal (American University of Beirut Department of Electrical and Computer Engine ISIT 2020 9 / 23

System Model

Equivalent Discrete-Time problem

By the properties of the PSWFs, X0(t) =

• i∈N

Xi Dϕc,i(t)

• λc,i

, where {Xi}i∈N are independent Gaussian Random Variables (RV) with mean zero and variances λc,iσ2 respectively. The use of PSWFs is necessary to get independent RV Similarly, Ym(t) =

• i∈N

Ym,i Dϕc,i(t)

• λc,i

Equivalent discrete-time problem:

Source coding of a discrete source that generates independent random variables X = {Xi}i∈N each Gaussian Xi ∼ N(0, λc,iσ2) The representation signals are Ym = {Ym,i}i∈N for m = 1, · · · , 2R The distortion is equal to d(X, Y ˙

m) = 1

T

• i∈N

|Xi − Y ˙

m,i|2 .

Youssef Jaffal, Ibrahim Abou-Faycal (American University of Beirut Department of Electrical and Computer Engine ISIT 2020 10 / 23

Bounds

1

Motivation

2

Prolate Spheroidal Wave Functions (PSWFs)

3

System Model

4

Bounds Lower Bound Upper Bound

5

Asymptotic analysis as T → ∞

6

Numerical Results

7

Summary

Youssef Jaffal, Ibrahim Abou-Faycal (American University of Beirut Department of Electrical and Computer Engine ISIT 2020 11 / 23

Bounds

Geometrical Interpretation

To satisfy the distortion requirement, with probability of at least (1 − ǫ), for a random vector X there must be at least one representation signal Ym such that

• i∈N

|Xi − Ym,i|2 ≤ dT Consider that each Ym covers a sphere B with radius √ dT. Therefore, {Ym}2R

m=1 and their corresponding spheres should

cover a set of a probability –under pX(·)– that is larger than (1 − ǫ) The smallest-volume set E such that the generated vector X falls inside it with probability (1 − ǫ) is an –infinite dimensional– ellipsoid centered at the origin with radius ri = a

• λc,iσ in the i-th

dimension (a is a constant determined by ǫ)

Youssef Jaffal, Ibrahim Abou-Faycal (American University of Beirut Department of Electrical and Computer Engine ISIT 2020 12 / 23

Bounds Lower Bound

Converse – Lower Bound

For every positive integer k, let ak be the positive solution to P

• Zk < a2

k

• = (1 − ǫ),

where Zk is chi-square distributed with k degrees of freedom Theorem (4) Under the guarantee that the distortion does not exceed d with probability (1 − ǫ), coding a T-seconds piece from a W-Hz band-limited white Gaussian process of spectrum σ2 requires more than RL = max

k≥1

1 2

k−1

• i=0

log2 λc,ia2

kσ2

dT

• bits.

Proof:

1

Consider the subset Xk = {Xi}k−1

i=0 2

Derive the result through volume division

3

Maximize over k

Youssef Jaffal, Ibrahim Abou-Faycal (American University of Beirut Department of Electrical and Computer Engine ISIT 2020 13 / 23

Bounds Upper Bound

Achievability – Upper Bound

• We divide X into two sub-vectors Xk = {Xi}k−1

i=0 and ˆ

Xk = {Xi}∞

i=k

• The first sub-vector is encoded and the second one is “pure” distortion

Theorem (5) It is possible to encode a T-seconds portion of a W-Hz band-limited white Gaussian process with flat PSD using RU = min

k

log2M   ˆ akσ

• dT − ˆ

b2

kσ2

, k   bits, with the guarantee that the distortion does not exceed d with probability (1 − ǫ), where ˆ ak and ˆ bk are defined as P k−1

• i=0

X2

i < ˆ

a2

kσ2

• = P

∞

• i=k

X2

i < ˆ

b2

kσ2

• =

√ 1 − ǫ, where the {Xi}’s are independent ∼ N(0, λc,iσ2), and where M is given below.

Youssef Jaffal, Ibrahim Abou-Faycal (American University of Beirut Department of Electrical and Computer Engine ISIT 2020 14 / 23

Bounds Upper Bound

Achievability – Upper Bound

M(r, n) =                  e (n ln n + n ln ln n + 5n) rn n ≤ r n (n ln n + n ln ln n + 5n) rn

n ln n ≤ r < n 7(4/7) ln 7 4

√ 2π

n√n

• (n−1) ln rn+(n−1) ln ln n+ 1

2 ln n+ln π √ 2n √πn−2

• r (1−

2 ln n )

• 1−

2 √πn

• ln2 n

rn 2 < r <

n ln n

√ 2π

√n

• (n−1) ln rn+(n−1) ln ln n+ 1

2 ln n+ln π √ 2n √πn−2

• r (1−

2 ln n )

• 1−

2 √πn

• rn

1 < r ≤ 2

Proof:

1

Divide X into two sub-vectors Xk = {Xi}k−1

i=0 and ˆ

Xk = {Xi}∞

i=k 2

Only Xk is encoded into bits

3

Take into account the distortion ˆ b2

kσ2 associated with ignoring ˆ

Xk

4

Consider the k-dimensional sphere Sk centered at the origin with a radius ˆ akσ that guarantees that Xk falls inside Sk with probability √1 − ǫ

5

Being independent, P

• {Xk ∈ Sk} ∩ {∞

i=k X2 i < ˆ

b2

kσ2}

• = (1 − ǫ)

Youssef Jaffal, Ibrahim Abou-Faycal (American University of Beirut Department of Electrical and Computer Engine ISIT 2020 15 / 23

Asymptotic analysis as T → ∞

1

Motivation

2

Prolate Spheroidal Wave Functions (PSWFs)

3

System Model

4

Bounds Lower Bound Upper Bound

5

Asymptotic analysis as T → ∞

6

Numerical Results

7

Summary

Youssef Jaffal, Ibrahim Abou-Faycal (American University of Beirut Department of Electrical and Computer Engine ISIT 2020 16 / 23

Asymptotic analysis as T → ∞

Asymptotic analysis as T → ∞

Theorem (6) As T goes to infinity, the rate per second converges to the Shannon rate W log2

• 2W σ2

d

• bits/sec.

Proof: For all k ∈ N,

1 2T

k−1

i=0 log2

• λc,ia2

kσ2

dT

• ≤ R∗

T ≤ 1 T log2M

• ˆ

akσ

√

dT −ˆ b2

kσ2 , k

• By the Berry-Esseen theorem:

For some |β| ≤ 12

√ 2 √ k , a2 k = k +

√ 2k Q−1 (ǫ + β) ≤ k + √ 2k Q−1 (2ǫ) , k ≫ 1 ˆ a2

k = k−1 i=0 λc,i +

• 2 k−1

i=0 λ2 c,iQ−1

1 − √1 − ǫ

• + O(1) and

ˆ b2

k = ∞ i=k λc,i +

• 2 ∞

i=k λ2 c,iQ−1

1 − √1 − ǫ

• + O

∞

i=k λ3 c,i

∞

i=k λ2 c,i

• For the Lower bound: Since limc→∞ λc,c(1−γ) = 1, choose k = c(1 − γ)

For the upper bound: Since limc→∞ λc,c(1+γ) = 0, choose k = c(1 + γ) Then for any γ > 0

(1 − γ)W

• log2

2W(1 − γ)σ2 d

• + log2
• 1 +
• 1

2c(1 − γ)

• ≤ R∗

T ≤ (1 + γ)W log2 2Wσ2 d

• Youssef Jaffal, Ibrahim Abou-Faycal

(American University of Beirut Department of Electrical and Computer Engine ISIT 2020 17 / 23

Numerical Results

1

Motivation

2

Prolate Spheroidal Wave Functions (PSWFs)

3

System Model

4

Bounds Lower Bound Upper Bound

5

Asymptotic analysis as T → ∞

6

Numerical Results

7

Summary

Youssef Jaffal, Ibrahim Abou-Faycal (American University of Beirut Department of Electrical and Computer Engine ISIT 2020 18 / 23

Numerical Results

Numerical Results

We compare the derived bounds with Shannon’s expression using the ratio R T × RShannon = R 0.5 c log2

• 2W σ2

d

• Sample results:

100 200 300 400 500 600 700 800 900 1000

c

1 1.05 1.1 1.15

R/RShannon

Upper Bound Lower Bound

Bounds vs. c for σ = 1, W = 1000Hz, d =

1 10002W, ǫ = 10−2 Youssef Jaffal, Ibrahim Abou-Faycal (American University of Beirut Department of Electrical and Computer Engine ISIT 2020 19 / 23

Numerical Results

Numerical Results

Bounds vs.c for σ = 1, W = 1000Hz, d =

1 42W

and ǫ = 10−2

100 200 300 400 500 600 700 800 900 1000

c

1 1.1 1.2 1.3 1.4 1.5

R/RShannon

Upper Bound Lower Bound

Bounds vs.c for σ = 1, W = 1000Hz, d =

1 42W

and ǫ = 10−4.

100 200 300 400 500 600 700 800 900 1000

c

1 1.1 1.2 1.3 1.4 1.5 1.6

R/RShannon

Upper Bound Lower Bound

Youssef Jaffal, Ibrahim Abou-Faycal (American University of Beirut Department of Electrical and Computer Engine ISIT 2020 20 / 23

Summary

1

Motivation

2

Prolate Spheroidal Wave Functions (PSWFs)

3

System Model

4

Bounds Lower Bound Upper Bound

5

Asymptotic analysis as T → ∞

6

Numerical Results

7

Summary

Youssef Jaffal, Ibrahim Abou-Faycal (American University of Beirut Department of Electrical and Computer Engine ISIT 2020 21 / 23

Summary

Conclusions

We studied the rate distortion limits when independently coding T-seconds time-limited signals from a W-Hz band-limited white Gaussian source We derived upper and lower bounds and computed them As the time frequency index c = 2WT increases,

the relative gap between the bounds decreases the bounds are shown to converge to Shannon’s rate-distortion fct

While there are alternative ways to represent the time-limited piece by a discrete-time vector, the use of PSWFs is necessary to get independent random variables If the PSD of the source is not flat, then the PSWFs do not provide independent discrete-time symbols

Youssef Jaffal, Ibrahim Abou-Faycal (American University of Beirut Department of Electrical and Computer Engine ISIT 2020 22 / 23

Summary

Acknowledgment

The authors would like to acknowledge the American University of Beirut (AUB) and the National Council for Scientific Research of Lebanon (CNRS-L) for granting a doctoral fellowship to Youssef Jaffal

Youssef Jaffal, Ibrahim Abou-Faycal (American University of Beirut Department of Electrical and Computer Engine ISIT 2020 23 / 23