Asymptotically Scale-invariant Multi-resolution Quantization Cheuk - - PowerPoint PPT Presentation

Asymptotically Scale-invariant Multi-resolution Quantization

Cheuk Ting Li

• Dept. of Information Engineering, Chinese University of Hong Kong

Email: ctli@ie.cuhk.edu.hk 2020 IEEE International Symposium on Information Theory

Multi-resolution Quantizer (MRQ)

4 12 8 2 4 12 6 2 10 8 4 12 5 1 10 8 3 7 2 6 Q1 Q2 Q3

Studied by, e.g. Equitz and Cover [1991], Brunk and Farvardin [1996], Jafarkhani et al. [1997], Wu and Dumitrescu [2002], Dumitrescu and Wu [2004], Effros and Dugatkin [2004] A sequence of quantizers Coarser quantization can be obtained from finer quantization by discarding information Finer quantization can be obtained from coarser quantization by adding additional information (successive refinement)

Multi-resolution Quantizer (MRQ)

4 12 8 2 4 12 6 2 10 8 4 12 5 1 10 8 3 7 2 6 Q1 Q2 Q3

Studied by, e.g. Equitz and Cover [1991], Brunk and Farvardin [1996], Jafarkhani et al. [1997], Wu and Dumitrescu [2002], Dumitrescu and Wu [2004], Effros and Dugatkin [2004] A sequence of quantizers Coarser quantization can be obtained from finer quantization by discarding information Finer quantization can be obtained from coarser quantization by adding additional information (successive refinement)

Multi-resolution Quantizer (MRQ)

4 12 8 2 4 12 6 2 10 8 4 12 5 1 10 8 3 7 2 6 Q1 Q2 Q3

Studied by, e.g. Equitz and Cover [1991], Brunk and Farvardin [1996], Jafarkhani et al. [1997], Wu and Dumitrescu [2002], Dumitrescu and Wu [2004], Effros and Dugatkin [2004] A sequence of quantizers Coarser quantization can be obtained from finer quantization by discarding information Finer quantization can be obtained from coarser quantization by adding additional information (successive refinement)

Multi-resolution Quantizer (MRQ)

4 12 8 2 4 12 6 2 10 8 4 12 5 1 10 8 3 7 2 6 Q1 Q2 Q3

Studied by, e.g. Equitz and Cover [1991], Brunk and Farvardin [1996], Jafarkhani et al. [1997], Wu and Dumitrescu [2002], Dumitrescu and Wu [2004], Effros and Dugatkin [2004] A sequence of quantizers Coarser quantization can be obtained from finer quantization by discarding information Finer quantization can be obtained from coarser quantization by adding additional information (successive refinement)

Multi-resolution Quantizer (MRQ)

0.0 0.2 0.4 0.6 0.8 1.0 10

2

10

1

100 0.0 0.2 0.4 0.6 0.8 1.0 10

2

10

1

100

A quantizer is a function Q : R → R where the range Q(R) is finite/countable A MRQ is a set of quantizers {Qs}s with parameter s > 0 (≈ step size), such that output of coarser quantizer can be deduced from

• utput of finer quantizer, without knowledge of the original data:

Qs2(Qs1(x)) = Qs2(x) for s2 ≥ s1 > 0

Multi-resolution Quantizer (MRQ)

0.0 0.2 0.4 0.6 0.8 1.0 10

2

10

1

100 0.0 0.2 0.4 0.6 0.8 1.0 10

2

10

1

100

A quantizer is a function Q : R → R where the range Q(R) is finite/countable A MRQ is a set of quantizers {Qs}s with parameter s > 0 (≈ step size), such that output of coarser quantizer can be deduced from

• utput of finer quantizer, without knowledge of the original data:

Qs2(Qs1(x)) = Qs2(x) for s2 ≥ s1 > 0

Multi-resolution Quantizer (MRQ)

4 12 6 2 10 8 4 12 5 1 10 8 3 7 2 6 Q1 Q2 4 12 8 2 Q3 2.5 4 12 5 1 10 8 3 7 2 6 Q1

Consider a piece of data being sent and recompressed multiple times between multiple users Let the data be x ∈ R, compressed by the first user to Qs1(x), then by the second user to Qs2(Qs1(x)), ... If Qs is MRQ, final output y =Qsn(· · ·Qs1(x) · · · ) is as good as the worst quantizer Qmaxj sj If Qs is not MRQ, final output can be far from x

Multi-resolution Quantizer (MRQ)

4 12 6 2 10 8 4 12 5 1 10 8 3 7 2 6 Q1 Q2 4 12 8 2 Q3 2.5 4 12 5 1 10 8 3 7 2 6 Q1

Consider a piece of data being sent and recompressed multiple times between multiple users Let the data be x ∈ R, compressed by the first user to Qs1(x), then by the second user to Qs2(Qs1(x)), ... If Qs is MRQ, final output y =Qsn(· · ·Qs1(x) · · · ) is as good as the worst quantizer Qmaxj sj If Qs is not MRQ, final output can be far from x

Multi-resolution Quantizer (MRQ)

4 12 6 2 10 8 4 12 5 1 10 8 3 7 2 6 Q1 Q2 4 12 8 2 Q3 2.5 4 12 5 1 10 8 3 7 2 6 Q1

Consider a piece of data being sent and recompressed multiple times between multiple users Let the data be x ∈ R, compressed by the first user to Qs1(x), then by the second user to Qs2(Qs1(x)), ... If Qs is MRQ, final output y =Qsn(· · ·Qs1(x) · · · ) is as good as the worst quantizer Qmaxj sj If Qs is not MRQ, final output can be far from x

Multi-resolution Quantizer (MRQ)

12 5 11 10 4 12 5 1 10 8 3 7 2 6 Q1 Q2 4 12 8 2 Q3 2.5 4 12 5 1 10 8 3 7 2 6 Q1

Consider a piece of data being sent and recompressed multiple times between multiple users Let the data be x ∈ R, compressed by the first user to Qs1(x), then by the second user to Qs2(Qs1(x)), ... If Qs is MRQ, final output y =Qsn(· · ·Qs1(x) · · · ) is as good as the worst quantizer Qmaxj sj If Qs is not MRQ, final output can be far from x

Binary Multi-resolution Quantizer (BMRQ)

0.0 0.2 0.4 0.6 0.8 1.0 10

2

10

1

100

Q❜✐♥

s

(x) = 2⌊log2 s⌋(⌊2−⌊log2 s⌋x⌋ + 1/2) Uniform quantizers with step sizes that are powers of 2

Q❜✐♥

s

changes abruptly rather than smoothly with s

If we require step size ≤ 3.9, then we have to choose step size 2, and use almost twice as many quantization cells as needed Not “scale-invariant”

Binary Multi-resolution Quantizer (BMRQ)

0.0 0.2 0.4 0.6 0.8 1.0 10

2

10

1

100

Q❜✐♥

s

(x) = 2⌊log2 s⌋(⌊2−⌊log2 s⌋x⌋ + 1/2) Uniform quantizers with step sizes that are powers of 2

Q❜✐♥

s

changes abruptly rather than smoothly with s

If we require step size ≤ 3.9, then we have to choose step size 2, and use almost twice as many quantization cells as needed Not “scale-invariant”

Binary Multi-resolution Quantizer (BMRQ)

0.0 0.2 0.4 0.6 0.8 1.0 10

2

10

1

100

Q❜✐♥

s

(x) = 2⌊log2 s⌋(⌊2−⌊log2 s⌋x⌋ + 1/2) Uniform quantizers with step sizes that are powers of 2

Q❜✐♥

s

changes abruptly rather than smoothly with s

If we require step size ≤ 3.9, then we have to choose step size 2, and use almost twice as many quantization cells as needed Not “scale-invariant”

Binary Multi-resolution Quantizer (BMRQ)

0.0 0.2 0.4 0.6 0.8 1.0 10

2

10

1

100

Q❜✐♥

s

(x) = 2⌊log2 s⌋(⌊2−⌊log2 s⌋x⌋ + 1/2) Uniform quantizers with step sizes that are powers of 2

Q❜✐♥

s

changes abruptly rather than smoothly with s

If we require step size ≤ 3.9, then we have to choose step size 2, and use almost twice as many quantization cells as needed Not “scale-invariant”

Dithered Binary Multi-resolution Quantizer (DBMRQ)

0.0 0.2 0.4 0.6 0.8 1.0 10

2

10

1

100

Q❞✐

s (x) :=

     2⌊log2 s⌋+1(⌊2−⌊log2 s⌋−1x⌋+1/2) ✐❢ ❢r❛❝(φ⌊2−⌊log2 s⌋−1x⌋) < 2−2⌊log2 s⌋+1/s 2⌊log2 s⌋(⌊2−⌊log2 s⌋x⌋ + 1/2) ♦t❤❡r✇✐s❡, Same as BMRQ when s = 2k (uniform quantizer w/ step 2k) When s decreases from 2k to 2k−1, more and more cells with size 2k are bisected into cells with size 2k−1 Not “scale-invariant”

Dithered Binary Multi-resolution Quantizer (DBMRQ)

0.0 0.2 0.4 0.6 0.8 1.0 10

2

10

1

100

Q❞✐

s (x) :=

     2⌊log2 s⌋+1(⌊2−⌊log2 s⌋−1x⌋+1/2) ✐❢ ❢r❛❝(φ⌊2−⌊log2 s⌋−1x⌋) < 2−2⌊log2 s⌋+1/s 2⌊log2 s⌋(⌊2−⌊log2 s⌋x⌋ + 1/2) ♦t❤❡r✇✐s❡, Same as BMRQ when s = 2k (uniform quantizer w/ step 2k) When s decreases from 2k to 2k−1, more and more cells with size 2k are bisected into cells with size 2k−1 Not “scale-invariant”

Dithered Binary Multi-resolution Quantizer (DBMRQ)

0.0 0.2 0.4 0.6 0.8 1.0 10

2

10

1

100

Q❞✐

s (x) :=

     2⌊log2 s⌋+1(⌊2−⌊log2 s⌋−1x⌋+1/2) ✐❢ ❢r❛❝(φ⌊2−⌊log2 s⌋−1x⌋) < 2−2⌊log2 s⌋+1/s 2⌊log2 s⌋(⌊2−⌊log2 s⌋x⌋ + 1/2) ♦t❤❡r✇✐s❡, Same as BMRQ when s = 2k (uniform quantizer w/ step 2k) When s decreases from 2k to 2k−1, more and more cells with size 2k are bisected into cells with size 2k−1 Not “scale-invariant”

Dithered Binary Multi-resolution Quantizer (DBMRQ)

0.0 0.2 0.4 0.6 0.8 1.0 10

2

10

1

100

Q❞✐

s (x) :=

     2⌊log2 s⌋+1(⌊2−⌊log2 s⌋−1x⌋+1/2) ✐❢ ❢r❛❝(φ⌊2−⌊log2 s⌋−1x⌋) < 2−2⌊log2 s⌋+1/s 2⌊log2 s⌋(⌊2−⌊log2 s⌋x⌋ + 1/2) ♦t❤❡r✇✐s❡, Same as BMRQ when s = 2k (uniform quantizer w/ step 2k) When s decreases from 2k to 2k−1, more and more cells with size 2k are bisected into cells with size 2k−1 Not “scale-invariant”

Biased Binary Multi-resolution Quantizer (BBMRQ)

0.0 0.2 0.4 0.6 0.8 1.0 10

2

10

1

100

Parameter 0 < α < 1 To obtain Q❜✐❛s,α

s

, divide each cell larger than s into two cells of proportions α and 1 − α resp., repeat until all cells has size ≤ s Scale-invariant as long as α is not a rational power of 1 − α

Biased Binary Multi-resolution Quantizer (BBMRQ)

0.0 0.2 0.4 0.6 0.8 1.0 10

2

10

1

100

Parameter 0 < α < 1 To obtain Q❜✐❛s,α

s

, divide each cell larger than s into two cells of proportions α and 1 − α resp., repeat until all cells has size ≤ s Scale-invariant as long as α is not a rational power of 1 − α

Biased Binary Multi-resolution Quantizer (BBMRQ)

0.0 0.2 0.4 0.6 0.8 1.0 10

2

10

1

100

Parameter 0 < α < 1 To obtain Q❜✐❛s,α

s

, divide each cell larger than s into two cells of proportions α and 1 − α resp., repeat until all cells has size ≤ s Scale-invariant as long as α is not a rational power of 1 − α

Asymptotic Scale-invariance

Cell size cdf of quantizer Q is the cdf of the distribution of the size of the cell containing X ∼ ❯♥✐❢(S), i.e., FQ,S(z) := λ ({x ∈ S : λ ({y ∈ S : Q(y) = Q(x)}) ≤ z}) λ(S) for S ⊆ R Asymptotic cell size cdf FQ: limit (w.r.t. Lévy metric) of FQ,[x0,x1] as x1 − x0 → ∞ For MRQ Qs, asymptotic scale-invariant if the functions x → FQs(xs) are the same for all s > 0

Asymptotic Scale-invariance

Cell size cdf of quantizer Q is the cdf of the distribution of the size of the cell containing X ∼ ❯♥✐❢(S), i.e., FQ,S(z) := λ ({x ∈ S : λ ({y ∈ S : Q(y) = Q(x)}) ≤ z}) λ(S) for S ⊆ R Asymptotic cell size cdf FQ: limit (w.r.t. Lévy metric) of FQ,[x0,x1] as x1 − x0 → ∞ For MRQ Qs, asymptotic scale-invariant if the functions x → FQs(xs) are the same for all s > 0

Asymptotic Scale-invariance

Cell size cdf of quantizer Q is the cdf of the distribution of the size of the cell containing X ∼ ❯♥✐❢(S), i.e., FQ,S(z) := λ ({x ∈ S : λ ({y ∈ S : Q(y) = Q(x)}) ≤ z}) λ(S) for S ⊆ R Asymptotic cell size cdf FQ: limit (w.r.t. Lévy metric) of FQ,[x0,x1] as x1 − x0 → ∞ For MRQ Qs, asymptotic scale-invariant if the functions x → FQs(xs) are the same for all s > 0

Rate of a Quantizer

Number of cells of Q in [x0, x1] is (x1 − x0) ∞ γ−1❞FQ,[x0,x1](γ)

For cell of size γ, prob. that X ∼ ❯♥✐❢[x0, x1] is in that cell is γ/(x1 − x0), its contribution to the integral is (x1 − x0)γ−1(γ/(x1 − x0)) = 1

Log-rate of Q (log of rate of increase of number of cells as [x0, x1] becomes longer): R0(Q) = log2 ∞ γ−1❞FQ(γ)

Rate of a Quantizer

Number of cells of Q in [x0, x1] is (x1 − x0) ∞ γ−1❞FQ,[x0,x1](γ)

For cell of size γ, prob. that X ∼ ❯♥✐❢[x0, x1] is in that cell is γ/(x1 − x0), its contribution to the integral is (x1 − x0)γ−1(γ/(x1 − x0)) = 1

Log-rate of Q (log of rate of increase of number of cells as [x0, x1] becomes longer): R0(Q) = log2 ∞ γ−1❞FQ(γ)

Lp error of a Quantizer

When X ∼ ❯♥✐❢[x0, x1], average Lp error lower-bounded by E[|X −Q(X)|p] ≥ ∞ (γ/2)p p + 1 ❞FQ,[x0,x1](γ)

For cell of size γ, the expected Lp error conditioned on that X is in that cell ≥ (γ/2)p/(p + 1)

If Q is centered (reconstruction level of a cell is at the midpoint), lim

x1−x0→∞ E[|X − Q(X)|p] =

∞ (γ/2)p p + 1 ❞FQ(γ).

Lp error of a Quantizer

When X ∼ ❯♥✐❢[x0, x1], average Lp error lower-bounded by E[|X −Q(X)|p] ≥ ∞ (γ/2)p p + 1 ❞FQ,[x0,x1](γ)

For cell of size γ, the expected Lp error conditioned on that X is in that cell ≥ (γ/2)p/(p + 1)

If Q is centered (reconstruction level of a cell is at the midpoint), lim

x1−x0→∞ E[|X − Q(X)|p] =

∞ (γ/2)p p + 1 ❞FQ(γ).

Rate-error Tradeoff

Theorem The asymptotic cell size cdf of BBMRQ can be arbitrarily close to F2❯♥✐❢[−1,0](x) := min {max{log2(x) + 1, 0}, 1} , i.e., F2❯♥✐❢[−1,0] is the cdf of 2Z where Z ∼ ❯♥✐❢[−1, 0]. Log-rate R0(Qs) can be arbitrarily close to log2 log2 e − log2 s Asymptotic Lp error of Qs can be arbitrarily close to (s/2)p(1 − 2−p) log2 e p(p + 1) There does not exist asymptotically scale-invariant quantizer with R0(Qs) < log2 log2 e − log2 s and Lp error of Qs < (s/2)p(1−2−p) log2 e

p(p+1)

for some s, p

BBMRQ is arbitrarily close to optimal Conjecture: There does not exist asymptotically scale-invariant quantizer achieving equalities in the above two bounds

Rate-error Tradeoff

Theorem The asymptotic cell size cdf of BBMRQ can be arbitrarily close to F2❯♥✐❢[−1,0](x) := min {max{log2(x) + 1, 0}, 1} , i.e., F2❯♥✐❢[−1,0] is the cdf of 2Z where Z ∼ ❯♥✐❢[−1, 0]. Log-rate R0(Qs) can be arbitrarily close to log2 log2 e − log2 s Asymptotic Lp error of Qs can be arbitrarily close to (s/2)p(1 − 2−p) log2 e p(p + 1) There does not exist asymptotically scale-invariant quantizer with R0(Qs) < log2 log2 e − log2 s and Lp error of Qs < (s/2)p(1−2−p) log2 e

p(p+1)

for some s, p

BBMRQ is arbitrarily close to optimal Conjecture: There does not exist asymptotically scale-invariant quantizer achieving equalities in the above two bounds

Rate-error Tradeoff

Theorem The asymptotic cell size cdf of BBMRQ can be arbitrarily close to F2❯♥✐❢[−1,0](x) := min {max{log2(x) + 1, 0}, 1} , i.e., F2❯♥✐❢[−1,0] is the cdf of 2Z where Z ∼ ❯♥✐❢[−1, 0]. Log-rate R0(Qs) can be arbitrarily close to log2 log2 e − log2 s Asymptotic Lp error of Qs can be arbitrarily close to (s/2)p(1 − 2−p) log2 e p(p + 1) There does not exist asymptotically scale-invariant quantizer with R0(Qs) < log2 log2 e − log2 s and Lp error of Qs < (s/2)p(1−2−p) log2 e

p(p+1)

for some s, p

BBMRQ is arbitrarily close to optimal Conjecture: There does not exist asymptotically scale-invariant quantizer achieving equalities in the above two bounds

Rate-error Tradeoff

Theorem The asymptotic cell size cdf of BBMRQ can be arbitrarily close to F2❯♥✐❢[−1,0](x) := min {max{log2(x) + 1, 0}, 1} , i.e., F2❯♥✐❢[−1,0] is the cdf of 2Z where Z ∼ ❯♥✐❢[−1, 0]. Log-rate R0(Qs) can be arbitrarily close to log2 log2 e − log2 s Asymptotic Lp error of Qs can be arbitrarily close to (s/2)p(1 − 2−p) log2 e p(p + 1) There does not exist asymptotically scale-invariant quantizer with R0(Qs) < log2 log2 e − log2 s and Lp error of Qs < (s/2)p(1−2−p) log2 e

p(p+1)

for some s, p

BBMRQ is arbitrarily close to optimal Conjecture: There does not exist asymptotically scale-invariant quantizer achieving equalities in the above two bounds

Rate-error Tradeoff

Theorem The asymptotic cell size cdf of BBMRQ can be arbitrarily close to F2❯♥✐❢[−1,0](x) := min {max{log2(x) + 1, 0}, 1} , i.e., F2❯♥✐❢[−1,0] is the cdf of 2Z where Z ∼ ❯♥✐❢[−1, 0]. Log-rate R0(Qs) can be arbitrarily close to log2 log2 e − log2 s Asymptotic Lp error of Qs can be arbitrarily close to (s/2)p(1 − 2−p) log2 e p(p + 1) There does not exist asymptotically scale-invariant quantizer with R0(Qs) < log2 log2 e − log2 s and Lp error of Qs < (s/2)p(1−2−p) log2 e

p(p+1)

for some s, p

BBMRQ is arbitrarily close to optimal Conjecture: There does not exist asymptotically scale-invariant quantizer achieving equalities in the above two bounds

Rate-error Tradeoff

Theorem The asymptotic cell size cdf of BBMRQ can be arbitrarily close to F2❯♥✐❢[−1,0](x) := min {max{log2(x) + 1, 0}, 1} , i.e., F2❯♥✐❢[−1,0] is the cdf of 2Z where Z ∼ ❯♥✐❢[−1, 0]. Log-rate R0(Qs) can be arbitrarily close to log2 log2 e − log2 s Asymptotic Lp error of Qs can be arbitrarily close to (s/2)p(1 − 2−p) log2 e p(p + 1) There does not exist asymptotically scale-invariant quantizer with R0(Qs) < log2 log2 e − log2 s and Lp error of Qs < (s/2)p(1−2−p) log2 e

p(p+1)

for some s, p

BBMRQ is arbitrarily close to optimal Conjecture: There does not exist asymptotically scale-invariant quantizer achieving equalities in the above two bounds

Rate-error Tradeoff

1 2 3 4 5

Log-rate

10

2

10

1

Asymptotic L1 error Binary MRQ Dithered Binary MRQ Biased Binary MRQ

BBMRQ provides a smooth tradeoff (straight line in the log plot) between rate and error

References

• H. Brunk and N. Farvardin. Fixed-rate successively refinable scalar quantizers. In Proceedings of

Data Compression Conference - DCC ’96, pages 250–259, March 1996. doi: ✶✵✳✶✶✵✾✴❉❈❈✳✶✾✾✻✳✹✽✽✸✸✵. Sorina Dumitrescu and Xiaolin Wu. Algorithms for optimal multi-resolution quantization. Journal

• f Algorithms, 50(1):1 – 22, 2004. ISSN 0196-6774.
• M. Effros and D. Dugatkin. Multiresolution vector quantization. IEEE Transactions on

Information Theory, 50(12):3130–3145, Dec 2004. ISSN 1557-9654. doi: ✶✵✳✶✶✵✾✴❚■❚✳✷✵✵✹✳✽✸✽✸✽✶. William HR Equitz and Thomas M Cover. Successive refinement of information. IEEE Transactions on Information Theory, 37(2):269–275, 1991.

• H. Jafarkhani, H. Brunk, and N. Farvardin. Entropy-constrained successively refinable scalar
• quantization. In Proceedings DCC ’97. Data Compression Conference, pages 337–346, March
• 1997. doi: ✶✵✳✶✶✵✾✴❉❈❈✳✶✾✾✼✳✺✽✷✵✺✼.

Xiaolin Wu and S. Dumitrescu. On optimal multi-resolution scalar quantization. In Proceedings DCC 2002. Data Compression Conference, pages 322–331, April 2002. doi: ✶✵✳✶✶✵✾✴❉❈❈✳✷✵✵✷✳✾✾✾✾✼✵.