On the Compressibility of Affinely Singular Random Vectors Mohammad - - PowerPoint PPT Presentation

▶
on the compressibility of affinely singular random vectors
SMART_READER_LITE
LIVE PREVIEW

On the Compressibility of Affinely Singular Random Vectors Mohammad - - PowerPoint PPT Presentation

Apr 22, 2023 206 likes •849 views

On the Compressibility of Affinely Singular Random Vectors Mohammad Amin Charusaie , Stefano Rini , Arash Amini Sharif University of Technology, National Chiao Tung University Full version: https://arxiv.org/abs/2001.03884

slide-1
SLIDE 1

On the Compressibility of Affinely Singular Random Vectors

Mohammad Amin Charusaie†, Stefano Rini∗, Arash Amini†

†Sharif University of Technology, ∗National Chiao Tung University

Full version: https://arxiv.org/abs/2001.03884 amin.ch90@gmail.com

June 8, 2020

M.A. Charusaie, S.Rini, A. Amini ISIT 2020 June 8, 2020 1 / 22

slide-2
SLIDE 2

A quick overview of the problem

Compressibility measures of random vectors (RVs) based on Shannon’s entropy

M.A. Charusaie, S.Rini, A. Amini ISIT 2020 June 8, 2020 2 / 22

slide-3
SLIDE 3

A quick overview of the problem

Compressibility measures of random vectors (RVs) based on Shannon’s entropy

Shannon Entropy : minimum bits needed to transmit a discrete source

M.A. Charusaie, S.Rini, A. Amini ISIT 2020 June 8, 2020 2 / 22

slide-4
SLIDE 4

A quick overview of the problem

Compressibility measures of random vectors (RVs) based on Shannon’s entropy

Shannon Entropy : minimum bits needed to transmit a discrete source

The R´ enyi information dimension (RID): Shannon compressibility measure defined for general measures

M.A. Charusaie, S.Rini, A. Amini ISIT 2020 June 8, 2020 2 / 22

slide-5
SLIDE 5

A quick overview of the problem

Compressibility measures of random vectors (RVs) based on Shannon’s entropy

Shannon Entropy : minimum bits needed to transmit a discrete source

The R´ enyi information dimension (RID): Shannon compressibility measure defined for general measures

RID(X) =

minimum bits to transmit X with high fidelity minimum bits to transmit any 1D source with the same fidelity

M.A. Charusaie, S.Rini, A. Amini ISIT 2020 June 8, 2020 2 / 22

slide-6
SLIDE 6

A quick overview of the problem

Compressibility measures of random vectors (RVs) based on Shannon’s entropy

Shannon Entropy : minimum bits needed to transmit a discrete source

The R´ enyi information dimension (RID): Shannon compressibility measure defined for general measures

RID(X) =

minimum bits to transmit X with high fidelity minimum bits to transmit any 1D source with the same fidelity

RID for absolutely continuous and discrete RVs ⇒ known

M.A. Charusaie, S.Rini, A. Amini ISIT 2020 June 8, 2020 2 / 22

slide-7
SLIDE 7

A quick overview of the problem

Compressibility measures of random vectors (RVs) based on Shannon’s entropy

Shannon Entropy : minimum bits needed to transmit a discrete source

The R´ enyi information dimension (RID): Shannon compressibility measure defined for general measures

RID(X) =

minimum bits to transmit X with high fidelity minimum bits to transmit any 1D source with the same fidelity

RID for absolutely continuous and discrete RVs ⇒ known But,

RID for measures with singularity, RID-based compressibility measures for stochastic processes (SPs) with singularity, and Relationship between the RID and other compressibility measures

are yet to be determined.

M.A. Charusaie, S.Rini, A. Amini ISIT 2020 June 8, 2020 2 / 22

slide-8
SLIDE 8

What this work is about?

A class of SPs

M.A. Charusaie, S.Rini, A. Amini ISIT 2020 June 8, 2020 3 / 22

slide-9
SLIDE 9

What this work is about?

A class of SPs

Passing a discrete-continuous (DC) white noise from an FIR filter

M.A. Charusaie, S.Rini, A. Amini ISIT 2020 June 8, 2020 3 / 22

slide-10
SLIDE 10

What this work is about?

A class of SPs

Passing a discrete-continuous (DC) white noise from an FIR filter A subclass of moving-average (MA) processes

M.A. Charusaie, S.Rini, A. Amini ISIT 2020 June 8, 2020 3 / 22

slide-11
SLIDE 11

What this work is about?

A class of SPs

Passing a discrete-continuous (DC) white noise from an FIR filter A subclass of moving-average (MA) processes

Linear transformation of independent DC RVs

M.A. Charusaie, S.Rini, A. Amini ISIT 2020 June 8, 2020 3 / 22

slide-12
SLIDE 12

What this work is about?

A class of SPs

Passing a discrete-continuous (DC) white noise from an FIR filter A subclass of moving-average (MA) processes

Linear transformation of independent DC RVs RID of Affinely singular random vectors

M.A. Charusaie, S.Rini, A. Amini ISIT 2020 June 8, 2020 3 / 22

slide-13
SLIDE 13

What this work is about?

A class of SPs

Passing a discrete-continuous (DC) white noise from an FIR filter A subclass of moving-average (MA) processes

Linear transformation of independent DC RVs RID of Affinely singular random vectors

A linear transformation of orthogonally singular random vectors (which include independent DC RVs)

M.A. Charusaie, S.Rini, A. Amini ISIT 2020 June 8, 2020 3 / 22

slide-14
SLIDE 14

What this work is about?

A class of SPs

Passing a discrete-continuous (DC) white noise from an FIR filter A subclass of moving-average (MA) processes

Linear transformation of independent DC RVs RID of Affinely singular random vectors

A linear transformation of orthogonally singular random vectors (which include independent DC RVs)

RID-based compressibility measures for these MA processes

M.A. Charusaie, S.Rini, A. Amini ISIT 2020 June 8, 2020 3 / 22

slide-15
SLIDE 15

What this work is about?

A class of SPs

Passing a discrete-continuous (DC) white noise from an FIR filter A subclass of moving-average (MA) processes

Linear transformation of independent DC RVs RID of Affinely singular random vectors

A linear transformation of orthogonally singular random vectors (which include independent DC RVs)

RID-based compressibility measures for these MA processes RID-based compressibility measures measures = ǫ-achievable rate

M.A. Charusaie, S.Rini, A. Amini ISIT 2020 June 8, 2020 3 / 22

slide-16
SLIDE 16

Contents

1

Affinely singular measures and its applications

2

RID of random variables and processes

3

RID of affinely singular RVs

4

Information dimension and ǫ-achievale rates of discrete-continuous moving-average processes

5

Conclusion

M.A. Charusaie, S.Rini, A. Amini ISIT 2020 June 8, 2020 4 / 22

slide-17
SLIDE 17

Types of Measures

Definition1

The measure µ(·) is

Singular: S ∈ Rn, L´ ebesgue Measure of S = 0, µ(S) > 0.

  • 1W. Rudin, ”Real and complex analysis,” 2006, pp. 121

M.A. Charusaie, S.Rini, A. Amini ISIT 2020 June 8, 2020 5 / 22

slide-18
SLIDE 18

Types of Measures

Definition1

The measure µ(·) is

Singular: S ∈ Rn, L´ ebesgue Measure of S = 0, µ(S) > 0. Absolutely continuous: no such subset exists

  • 1W. Rudin, ”Real and complex analysis,” 2006, pp. 121

M.A. Charusaie, S.Rini, A. Amini ISIT 2020 June 8, 2020 5 / 22

slide-19
SLIDE 19

Types of Measures

Definition1

The measure µ(·) is

Singular: S ∈ Rn, L´ ebesgue Measure of S = 0, µ(S) > 0. Absolutely continuous: no such subset exists Discrete: µ(·) supported on a countable set

  • 1W. Rudin, ”Real and complex analysis,” 2006, pp. 121

M.A. Charusaie, S.Rini, A. Amini ISIT 2020 June 8, 2020 5 / 22

slide-20
SLIDE 20

Types of Measures

Definition1

The measure µ(·) is

Singular: S ∈ Rn, L´ ebesgue Measure of S = 0, µ(S) > 0. Absolutely continuous: no such subset exists Discrete: µ(·) supported on a countable set Discrete-continuous: µ(·) convex combination of discrete and absolutely continuous measures in 1D

  • 1W. Rudin, ”Real and complex analysis,” 2006, pp. 121

M.A. Charusaie, S.Rini, A. Amini ISIT 2020 June 8, 2020 5 / 22

slide-21
SLIDE 21

Affinely Singular Measures

Definition

Zm =

i∈N 1{V =i}

Z(i) ⇒ Affinely Singular RV :

  • Z(i) = Ui [Xc,i; 0m−hi] + bi,

V ⇒ random choice of Zm, Ui: m × m unitary matrix, 0k: k-dimensional column vector of all zeros, bi: fixed vector in Rm, and Xc,i: hi-dimensional absolutely continuous RV.

M.A. Charusaie, S.Rini, A. Amini ISIT 2020 June 8, 2020 6 / 22

slide-22
SLIDE 22

Orthogonally Singular Measures

Definition

Xn: orthogonally singular RV Xi = νiXci + (1 − νi)Xdi Xn

c : absolutely continuous RV,

Xn

d: discrete RV,

Bernoulli νi with P(νi = 1) = αi ⇒ random choice of Xn, and Xn

c independent of Xn d and ν.

2See Lemma 6 of Full version of the paper M.A. Charusaie, S.Rini, A. Amini ISIT 2020 June 8, 2020 7 / 22

slide-23
SLIDE 23

Orthogonally Singular Measures

Definition

Xn: orthogonally singular RV Xi = νiXci + (1 − νi)Xdi Xn

c : absolutely continuous RV,

Xn

d: discrete RV,

Bernoulli νi with P(νi = 1) = αi ⇒ random choice of Xn, and Xn

c independent of Xn d and ν.

Xcis, Xdis, and νis are mutually independent ⇒ Xn is a random vector with independent DC RVs.

2See Lemma 6 of Full version of the paper M.A. Charusaie, S.Rini, A. Amini ISIT 2020 June 8, 2020 7 / 22

slide-24
SLIDE 24

Orthogonally Singular Measures

Definition

Xn: orthogonally singular RV Xi = νiXci + (1 − νi)Xdi Xn

c : absolutely continuous RV,

Xn

d: discrete RV,

Bernoulli νi with P(νi = 1) = αi ⇒ random choice of Xn, and Xn

c independent of Xn d and ν.

Xcis, Xdis, and νis are mutually independent ⇒ Xn is a random vector with independent DC RVs. Linear transformation of Xn has singularities on affine subsets (Affinely Singular RV)2

2See Lemma 6 of Full version of the paper M.A. Charusaie, S.Rini, A. Amini ISIT 2020 June 8, 2020 7 / 22

slide-25
SLIDE 25

An Example: Discrete-Continuous Moving-Average (DC-MA) SPs

Consider an FIR filter: H(s) = l2

i=−l1 aisi

Wm+l1+l2 Ym (ais and lis are fixed constants)

M.A. Charusaie, S.Rini, A. Amini ISIT 2020 June 8, 2020 8 / 22

slide-26
SLIDE 26

An Example: Discrete-Continuous Moving-Average (DC-MA) SPs

Consider an FIR filter: H(s) = l2

i=−l1 aisi

Wm+l1+l2 Ym (ais and lis are fixed constants) Wm+l1+l2 RV with i.i.d. DC elements ⇒ Ym samples of a DC-MA SPs

M.A. Charusaie, S.Rini, A. Amini ISIT 2020 June 8, 2020 8 / 22

slide-27
SLIDE 27

An Example: Discrete-Continuous Moving-Average (DC-MA) SPs

Consider an FIR filter: H(s) = l2

i=−l1 aisi

Wm+l1+l2 Ym (ais and lis are fixed constants) Wm+l1+l2 RV with i.i.d. DC elements ⇒ Ym samples of a DC-MA SPs

Yi ⇒ linear transformation of i.i.d. DC RVs

M.A. Charusaie, S.Rini, A. Amini ISIT 2020 June 8, 2020 8 / 22

slide-28
SLIDE 28

An Example: Discrete-Continuous Moving-Average (DC-MA) SPs

Consider an FIR filter: H(s) = l2

i=−l1 aisi

Wm+l1+l2 Ym (ais and lis are fixed constants) Wm+l1+l2 RV with i.i.d. DC elements ⇒ Ym samples of a DC-MA SPs

Yi ⇒ linear transformation of i.i.d. DC RVs Ym is an affinely singular RV

M.A. Charusaie, S.Rini, A. Amini ISIT 2020 June 8, 2020 8 / 22

slide-29
SLIDE 29

RID for RVs

Uniform Quantization: [Xn]m

  • ⌊mX1⌋

m

, . . . , ⌊mXn⌋

m

  • ,
  • 3A. Renyi,On the dimension and entropy of probability distributions,Acta Mathematica Hungarica, vol. 10, no. 1-2, 1959

M.A. Charusaie, S.Rini, A. Amini ISIT 2020 June 8, 2020 9 / 22

slide-30
SLIDE 30

RID for RVs

Uniform Quantization: [Xn]m

  • ⌊mX1⌋

m

, . . . , ⌊mXn⌋

m

  • ,

Definition 3

For an RV Xn, the R´ enyi Information Dimension (RID) is defined as d(Xn) = lim

m→∞ H([Xn]m) log m

, if the limit exists, where H(·) is the Shannon entropy function.

  • 3A. Renyi,On the dimension and entropy of probability distributions,Acta Mathematica Hungarica, vol. 10, no. 1-2, 1959

M.A. Charusaie, S.Rini, A. Amini ISIT 2020 June 8, 2020 9 / 22

slide-31
SLIDE 31

RID of SPs

Definition 4

For a discrete-domain stationary stochastic process {Xt}, Block-average Information Dimension (BID) is defined as dB

  • {Xt}
  • = lim

n→∞ lim m→∞

H

  • [Xn

1]m

  • n log m

= lim

n→∞

d(Xn

1)

n ,

  • 4S. Jalali and H. V. Poor, ”Universal Compressed Sensing for Almost Lossless Recovery,” in IEEE Transactions on

Information Theory, vol. 63, no. 5

  • 5B. C. Geiger and T. Koch, ”On the Information Dimension of Stochastic Processes,” in IEEE Transactions on Information

Theory, vol. 65, no. 10 M.A. Charusaie, S.Rini, A. Amini ISIT 2020 June 8, 2020 10 / 22

slide-32
SLIDE 32

RID of SPs

Definition 4

For a discrete-domain stationary stochastic process {Xt}, Block-average Information Dimension (BID) is defined as dB

  • {Xt}
  • = lim

n→∞ lim m→∞

H

  • [Xn

1]m

  • n log m

= lim

n→∞

d(Xn

1)

n ,

  • 4S. Jalali and H. V. Poor, ”Universal Compressed Sensing for Almost Lossless Recovery,” in IEEE Transactions on

Information Theory, vol. 63, no. 5

  • 5B. C. Geiger and T. Koch, ”On the Information Dimension of Stochastic Processes,” in IEEE Transactions on Information

Theory, vol. 65, no. 10 M.A. Charusaie, S.Rini, A. Amini ISIT 2020 June 8, 2020 10 / 22

slide-33
SLIDE 33

RID of SPs

Definition 4

For a discrete-domain stationary stochastic process {Xt}, Block-average Information Dimension (BID) is defined as dB

  • {Xt}
  • = lim

n→∞ lim m→∞

H

  • [Xn

1]m

  • n log m

= lim

n→∞

d(Xn

1)

n ,

Definition 5

For a discrete-time stochastic process {Xt}, Information Dimension Rate (IDR) is defined as dI

  • {Xt}
  • = lim

m→∞ lim n→∞

H

  • [Xn

1]m

  • n log m

,

  • 4S. Jalali and H. V. Poor, ”Universal Compressed Sensing for Almost Lossless Recovery,” in IEEE Transactions on

Information Theory, vol. 63, no. 5

  • 5B. C. Geiger and T. Koch, ”On the Information Dimension of Stochastic Processes,” in IEEE Transactions on Information

Theory, vol. 65, no. 10 M.A. Charusaie, S.Rini, A. Amini ISIT 2020 June 8, 2020 10 / 22

slide-34
SLIDE 34

RID of Affinely Singular RVs

Lemma 1

Let Zm be an affinely singular RV which takes hi-dimensional singularities with random choice V . The RID of Zm is d(Zm) = EV [hV ], provided that H(V ) < ∞.

M.A. Charusaie, S.Rini, A. Amini ISIT 2020 June 8, 2020 11 / 22

slide-35
SLIDE 35

RID of Affinely Singular RVs

Lemma 1

Let Zm be an affinely singular RV which takes hi-dimensional singularities with random choice V . The RID of Zm is d(Zm) = EV [hV ], provided that H(V ) < ∞.

Theorem 1

The RID of Ym = AXn, where Xn is orthogonally singular with random choice ν, satisfies d(Ym) = Eν[rank(Aν)], provided that H(ν, Xd) < ∞. Here, Aν stands for the sub-matrix of A formed by the columns i ∈ [n] for which νi = 1.

M.A. Charusaie, S.Rini, A. Amini ISIT 2020 June 8, 2020 11 / 22

slide-36
SLIDE 36

An Illustrative Example

Figure: Measure of independent sparse RVs (Bernoulli-Gaussian model with α = .5 and σ = [0.2, 0.4, 0.8]T ). The RID is equal to d(X, Y, Z) = 3/2.

M.A. Charusaie, S.Rini, A. Amini ISIT 2020 June 8, 2020 12 / 22

slide-37
SLIDE 37

Figure: Transformed by the invertible matrix A =   1 −1 0.3 1 5 1 5 −1 5  . The RID is equal to d

  • A(X, Y, Z)T

= 3/2.

M.A. Charusaie, S.Rini, A. Amini ISIT 2020 June 8, 2020 13 / 22

slide-38
SLIDE 38

Figure: Transformed by the non-invertible matrix A =   1 1 1 1 1 −1  . The RID is equal to d A(X, Y, Z)T = 11/8.

M.A. Charusaie, S.Rini, A. Amini ISIT 2020 June 8, 2020 14 / 22

slide-39
SLIDE 39

Lipschitz functions of orthogonally singular RVs

Lemma

If Xn be an orthogonally singular RV with H(Xd,i) < ∞, i ∈ [n], and Am×n is a matrix that satisfies SPARK(A) = rank(A) + 1, then, for any Lipschitz function f : Rn → Rm we have that d

  • f(Xn)
  • ≤ d(A Xn).

M.A. Charusaie, S.Rini, A. Amini ISIT 2020 June 8, 2020 15 / 22

slide-40
SLIDE 40

Lipschitz functions of orthogonally singular RVs

Lemma

If Xn be an orthogonally singular RV with H(Xd,i) < ∞, i ∈ [n], and Am×n is a matrix that satisfies SPARK(A) = rank(A) + 1, then, for any Lipschitz function f : Rn → Rm we have that d

  • f(Xn)
  • ≤ d(A Xn).

Am×n can be a Vandermonde matrix

M.A. Charusaie, S.Rini, A. Amini ISIT 2020 June 8, 2020 15 / 22

slide-41
SLIDE 41

RID of DC-MA SPs

Lemma 6

Let Yi = l2

j=−l1 ajWi−j for a−l1, al2 = 0, where Wjs are i.i.d.

discrete-continuous RVs with d(Wj) = α. Then we have, dB

  • {Yt}
  • = dI
  • {Yt}
  • = α.

6See Eq. (26) and (27) M.A. Charusaie, S.Rini, A. Amini ISIT 2020 June 8, 2020 16 / 22

slide-42
SLIDE 42

ǫ-achievable rates

Definition7

(fn, gn) ⇒ ǫ-encode-decode pair of {Xt} with rate Rn: fn : Rn → R⌊nRn⌋, gn : R⌊nRn⌋ → Rn, and P

  • gn
  • fn(Xn)
  • = Xn

≤ ǫ. lim infn→∞ Rn ⇒ minimum ǫ-achievable rate.

  • 7Y. Wu and S. Verd´

u, ”R´ enyi Information Dimension: Fundamental Limits of Almost Lossless Analog Compression,” in IEEE Transactions on Information Theory, vol. 56, no. 8

M.A. Charusaie, S.Rini, A. Amini ISIT 2020 June 8, 2020 17 / 22

slide-43
SLIDE 43

ǫ-achievable rates

Definition7

(fn, gn) ⇒ ǫ-encode-decode pair of {Xt} with rate Rn: fn : Rn → R⌊nRn⌋, gn : R⌊nRn⌋ → Rn, and P

  • gn
  • fn(Xn)
  • = Xn

≤ ǫ. lim infn→∞ Rn ⇒ minimum ǫ-achievable rate. f(·) linear ⇒ R∗(ǫ)

  • 7Y. Wu and S. Verd´

u, ”R´ enyi Information Dimension: Fundamental Limits of Almost Lossless Analog Compression,” in IEEE Transactions on Information Theory, vol. 56, no. 8

M.A. Charusaie, S.Rini, A. Amini ISIT 2020 June 8, 2020 17 / 22

slide-44
SLIDE 44

ǫ-achievable rates

Definition7

(fn, gn) ⇒ ǫ-encode-decode pair of {Xt} with rate Rn: fn : Rn → R⌊nRn⌋, gn : R⌊nRn⌋ → Rn, and P

  • gn
  • fn(Xn)
  • = Xn

≤ ǫ. lim infn→∞ Rn ⇒ minimum ǫ-achievable rate. f(·) linear ⇒ R∗(ǫ) g(·) Lipschitz ⇒ R(ǫ)

  • 7Y. Wu and S. Verd´

u, ”R´ enyi Information Dimension: Fundamental Limits of Almost Lossless Analog Compression,” in IEEE Transactions on Information Theory, vol. 56, no. 8

M.A. Charusaie, S.Rini, A. Amini ISIT 2020 June 8, 2020 17 / 22

slide-45
SLIDE 45

ǫ-achievable rates (cont.)

Theorem

For every SP {Zt} with affinely singular samples, taking hi-dimensional singularities with random choice Vm, we have dB

  • {Zt}
  • = lim

m→∞ E

  • hVm

m

  • .

Moreover, if Vm has a finite sample space, and for all ǫ, δ ∈ R+ and large enough m we have that PVm

  • hVm

m − dB

  • {Zt}
  • < δ
  • > 1 − ǫ,

then, for {Zt} (which we call a concentrated source) we know that R∗(ǫ) = R(ǫ) = dB

  • {Zt}
  • .

M.A. Charusaie, S.Rini, A. Amini ISIT 2020 June 8, 2020 18 / 22

slide-46
SLIDE 46

Compression Rates of DC-MA SPs

DC-MA process {Yt} with finite discrete sample space is a concentrated source

  • 8S. Jalali and H. V. Poor, ”Universal Compressed Sensing for Almost Lossless Recovery,” in IEEE Transactions on

Information Theory, vol. 63, no. 5 M.A. Charusaie, S.Rini, A. Amini ISIT 2020 June 8, 2020 19 / 22

slide-47
SLIDE 47

Compression Rates of DC-MA SPs

DC-MA process {Yt} with finite discrete sample space is a concentrated source R∗(ǫ) = R(ǫ) = α.

  • 8S. Jalali and H. V. Poor, ”Universal Compressed Sensing for Almost Lossless Recovery,” in IEEE Transactions on

Information Theory, vol. 63, no. 5 M.A. Charusaie, S.Rini, A. Amini ISIT 2020 June 8, 2020 19 / 22

slide-48
SLIDE 48

Compression Rates of DC-MA SPs

DC-MA process {Yt} with finite discrete sample space is a concentrated source R∗(ǫ) = R(ǫ) = α. For Hk×mYm + Nk with k = O(mα), where H is a Gaussian ensemble with i.i.d. entries distributed according to N(0, 1) and bounded measurement noise, Lagranian-MEP algorithm 8 can address the decoding.

  • 8S. Jalali and H. V. Poor, ”Universal Compressed Sensing for Almost Lossless Recovery,” in IEEE Transactions on

Information Theory, vol. 63, no. 5 M.A. Charusaie, S.Rini, A. Amini ISIT 2020 June 8, 2020 19 / 22

slide-49
SLIDE 49

Conclusion

Affinely singular RVs as a probabilistic structure for compressible RVs

M.A. Charusaie, S.Rini, A. Amini ISIT 2020 June 8, 2020 20 / 22

slide-50
SLIDE 50

Conclusion

Affinely singular RVs as a probabilistic structure for compressible RVs Linear mappings of orthogonally singular RVs (e.g., independent discrete-continuous RVs)

M.A. Charusaie, S.Rini, A. Amini ISIT 2020 June 8, 2020 20 / 22

slide-51
SLIDE 51

Conclusion

Affinely singular RVs as a probabilistic structure for compressible RVs Linear mappings of orthogonally singular RVs (e.g., independent discrete-continuous RVs) RID of affinely singular RVs (linear mapping of orthogonally singular RVs)

M.A. Charusaie, S.Rini, A. Amini ISIT 2020 June 8, 2020 20 / 22

slide-52
SLIDE 52

Conclusion

Affinely singular RVs as a probabilistic structure for compressible RVs Linear mappings of orthogonally singular RVs (e.g., independent discrete-continuous RVs) RID of affinely singular RVs (linear mapping of orthogonally singular RVs) Information dimension of a DC-MA = information dimension of its excitation noise

M.A. Charusaie, S.Rini, A. Amini ISIT 2020 June 8, 2020 20 / 22

slide-53
SLIDE 53

Conclusion

Affinely singular RVs as a probabilistic structure for compressible RVs Linear mappings of orthogonally singular RVs (e.g., independent discrete-continuous RVs) RID of affinely singular RVs (linear mapping of orthogonally singular RVs) Information dimension of a DC-MA = information dimension of its excitation noise Concentrated source ⇒ ǫ-achievable rates = BID

M.A. Charusaie, S.Rini, A. Amini ISIT 2020 June 8, 2020 20 / 22

slide-54
SLIDE 54

Conclusion

Affinely singular RVs as a probabilistic structure for compressible RVs Linear mappings of orthogonally singular RVs (e.g., independent discrete-continuous RVs) RID of affinely singular RVs (linear mapping of orthogonally singular RVs) Information dimension of a DC-MA = information dimension of its excitation noise Concentrated source ⇒ ǫ-achievable rates = BID A sub-class of DC-MA processes ⇒ concentrated sources

M.A. Charusaie, S.Rini, A. Amini ISIT 2020 June 8, 2020 20 / 22

slide-55
SLIDE 55

Conclusion

Affinely singular RVs as a probabilistic structure for compressible RVs Linear mappings of orthogonally singular RVs (e.g., independent discrete-continuous RVs) RID of affinely singular RVs (linear mapping of orthogonally singular RVs) Information dimension of a DC-MA = information dimension of its excitation noise Concentrated source ⇒ ǫ-achievable rates = BID A sub-class of DC-MA processes ⇒ concentrated sources Algorithm that decodes a noisy sampling of these processes

M.A. Charusaie, S.Rini, A. Amini ISIT 2020 June 8, 2020 20 / 22

slide-56
SLIDE 56

Conclusion

Upper-bound on f(Xn) for m-dimensional Lipschitz function f(·) ⇒ Matrix A in which SPARK(A) = rank(A) + 1

M.A. Charusaie, S.Rini, A. Amini ISIT 2020 June 8, 2020 21 / 22

slide-57
SLIDE 57

Conclusion

Upper-bound on f(Xn) for m-dimensional Lipschitz function f(·) ⇒ Matrix A in which SPARK(A) = rank(A) + 1 Dimensional Rate Bias of affinely singular RVs (linear mapping of

  • rthogonally singular RVs)

M.A. Charusaie, S.Rini, A. Amini ISIT 2020 June 8, 2020 21 / 22

slide-58
SLIDE 58

Conclusion

Upper-bound on f(Xn) for m-dimensional Lipschitz function f(·) ⇒ Matrix A in which SPARK(A) = rank(A) + 1 Dimensional Rate Bias of affinely singular RVs (linear mapping of

  • rthogonally singular RVs)

Future directions:

M.A. Charusaie, S.Rini, A. Amini ISIT 2020 June 8, 2020 21 / 22

slide-59
SLIDE 59

Conclusion

Upper-bound on f(Xn) for m-dimensional Lipschitz function f(·) ⇒ Matrix A in which SPARK(A) = rank(A) + 1 Dimensional Rate Bias of affinely singular RVs (linear mapping of

  • rthogonally singular RVs)

Future directions:

Information dimension of continuous-domain processes with compressible structure

M.A. Charusaie, S.Rini, A. Amini ISIT 2020 June 8, 2020 21 / 22

slide-60
SLIDE 60

Conclusion

Upper-bound on f(Xn) for m-dimensional Lipschitz function f(·) ⇒ Matrix A in which SPARK(A) = rank(A) + 1 Dimensional Rate Bias of affinely singular RVs (linear mapping of

  • rthogonally singular RVs)

Future directions:

Information dimension of continuous-domain processes with compressible structure BID and IDR of other possible discrete-domain processes with affinely singular realizations

M.A. Charusaie, S.Rini, A. Amini ISIT 2020 June 8, 2020 21 / 22

slide-61
SLIDE 61

Conclusion

Upper-bound on f(Xn) for m-dimensional Lipschitz function f(·) ⇒ Matrix A in which SPARK(A) = rank(A) + 1 Dimensional Rate Bias of affinely singular RVs (linear mapping of

  • rthogonally singular RVs)

Future directions:

Information dimension of continuous-domain processes with compressible structure BID and IDR of other possible discrete-domain processes with affinely singular realizations IDR of non-stationary processes with affinely singular realizations (e.g., Levy process with DC excitation noise)

M.A. Charusaie, S.Rini, A. Amini ISIT 2020 June 8, 2020 21 / 22

slide-62
SLIDE 62

Thank you!

M.A. Charusaie, S.Rini, A. Amini ISIT 2020 June 8, 2020 22 / 22