Timely Estimation Using Coded Quantized Samples Ahmed Arafa 1 Karim - - PowerPoint PPT Presentation

Timely Estimation Using Coded Quantized Samples

Ahmed Arafa1 Karim Banawan2 Karim G. Seddik3

• H. Vincent Poor4

1Department of Electrical and Computer Engineering, University of North Carolina at Charlotte 2Department of Electrical Engineering, Alexandria University 3Electronics and Communications Engineering Department, American University in Cairo 4Electrical Engineering Department, Princeton University

International Symposium on Information Theory June 2020

Age of Information (AoI)

70 °F

Age of Information (AoI)

70 °F

• bserver

Age of Information (AoI)

70 °F receiver

• bserver

°F

Age of Information (AoI)

70 °F receiver

• bserver

°F

Age of Information (AoI)

70 °F receiver

• bserver

°F

Data is now fresh.

Age of Information (AoI)

72 °F receiver

• bserver

°F

Data has now aged.

Age of Information (AoI) Evolution Curve

Typical age of information evolution versus time curve:

d3 d1 d2 d3

time age

t1 t2 t3 t1 + d1 T t2 + d2 t3 + d3 d1 d2

ti: measurement (and transmission) time of ith update. di: service time for ith update; typically a random quantity.

Age of Information (AoI) Evolution Curve

Typical age of information evolution versus time curve:

d1 d2 d3

time age

t1 t2 t3 t1 + d1 T t2 + d2 t3 + d3 d1 d2 d3

Long Term Average AoI = lim sup

T→∞

1 T E [AT]

Connection to Remote Estimation

Using AoI presumes the monitored process is frequently varying. MMSE may be more suitable for slowly varying processes. AoI is inherently embedded in MMSE estimates. Other variant metrics have also been studied; we focus on MMSE. Related works:

[Nar-Basar (CDC ’14)] [Chakravorty-Mahajan (ISIT ’15)] [Gao-Aykol-Basar (CDC ’15)] [Yun-Joo-Eryilmaz (CDC ’18)] [Ayan-Vilgelm-Klugel-Hirche-Kellerer (ICCPS ’19)] [Mitra-Richards-Bagchi-Sundaram (ACC ’19)] [Chakravorty-Mahajan (T-AC to appear)] [Sun-Polyanskiy-UysalBiyikoglu (T-IT to appear)] [Ornee-Sun (arXiv) ’19] [Huang-Liu-Shirvanimoghaddam-Li-Vucetic (arXiv ’19)] [Maatouk-Kriouile-Assaad-Ephremides (arXiv ’19)] [Ramirez-Erkip-Poor (arXiv ’19)] [Bastopcu-Ulukus (arXiv ’19)]

Connection to Remote Estimation

Weiner process estimation [Sun-Polyanskiy-UysalBiyikoglu (T-IT) to appear]

Communication channel is perfect (distortion-free); introduces random delays. Two sampling schemes: signal-independent and signal-dependent. MMSE = AoI in case of signal-independent sampling; threshold policy on AoI. MMSE = AoI in case of signal-dependent sampling; threshold policy on signal.

Ornstein-Uhlenbeck (OU) process estimation [Ornee-Sun (arXiv ’19)]

Similar results; MMSE = g(AoI) in case of signal-independent sampling.

This work: effects of distortion and coding over noisy channels. . .

We focus on OU processes.

Connection to Remote Estimation

Weiner process estimation [Sun-Polyanskiy-UysalBiyikoglu (T-IT) to appear]

Communication channel is perfect (distortion-free); introduces random delays. Two sampling schemes: signal-independent and signal-dependent. MMSE = AoI in case of signal-independent sampling; threshold policy on AoI. MMSE = AoI in case of signal-dependent sampling; threshold policy on signal.

Ornstein-Uhlenbeck (OU) process estimation [Ornee-Sun (arXiv ’19)]

Similar results; MMSE = g(AoI) in case of signal-independent sampling.

This work: effects of distortion and coding over noisy channels. . .

We focus on OU processes.

System Model

ℓ-bit MMSE quantizer n-bit channel encoder sensor Xt: OU process

transmitter

channel decoder + IIR/FR coding

channel

receiver

ˆ Xt: MMSE estimate β processing time (ACK/NACK) feedback

OU process evolution: Xt = Xse−θ(t−s) + σ √ 2θ e−θ(t−s)We2θ(t−s)−1, t ≥ s

System Model

ℓ-bit MMSE quantizer n-bit channel encoder sensor Xt: OU process

transmitter

channel decoder + IIR/FR coding

channel

receiver

ˆ Xt: MMSE estimate β processing time (ACK/NACK) feedback

OU process evolution: Xt = Xse−θ(t−s) + σ √ 2θ e−θ(t−s)We2θ(t−s)−1, t ≥ s Sensor acquires the ith sample at time Si; signal-independent sampling.

System Model

ℓ-bit MMSE quantizer n-bit channel encoder sensor Xt: OU process

transmitter

channel decoder + IIR/FR coding

channel

receiver

ˆ Xt: MMSE estimate β processing time (ACK/NACK) feedback

OU process evolution: Xt = Xse−θ(t−s) + σ √ 2θ e−θ(t−s)We2θ(t−s)−1, t ≥ s Sensor acquires the ith sample at time Si; signal-independent sampling. MMSE quantizer represents XSi as ˜ XSi using ℓ bits: XSi = ˜ XSi + QSi

System Model

ℓ-bit MMSE quantizer n-bit channel encoder sensor Xt: OU process

transmitter

channel decoder + IIR/FR coding

channel

receiver

ˆ Xt: MMSE estimate β processing time (ACK/NACK) feedback

OU process evolution: Xt = Xse−θ(t−s) + σ √ 2θ e−θ(t−s)We2θ(t−s)−1, t ≥ s Sensor acquires the ith sample at time Si; signal-independent sampling. MMSE quantizer represents XSi as ˜ XSi using ℓ bits: XSi = ˜ XSi + QSi Channel coding schemes:

Infinite Incremental Redundancy (IIR): n-bit codewords, increment if needed. Fixed Redundancy (FR): fixed n-bit codewords, repeat with new samples if needed.

System Model

ℓ-bit MMSE quantizer n-bit channel encoder sensor Xt: OU process

transmitter

channel decoder + IIR/FR coding

channel

receiver

ˆ Xt: MMSE estimate β processing time (ACK/NACK) feedback

OU process evolution: Xt = Xse−θ(t−s) + σ √ 2θ e−θ(t−s)We2θ(t−s)−1, t ≥ s Sensor acquires the ith sample at time Si; signal-independent sampling. MMSE quantizer represents XSi as ˜ XSi using ℓ bits: XSi = ˜ XSi + QSi Channel coding schemes:

Infinite Incremental Redundancy (IIR): n-bit codewords, increment if needed. Fixed Redundancy (FR): fixed n-bit codewords, repeat with new samples if needed.

Decoding consumes fixed β time units for processing and feedback.

System Model

ℓ-bit MMSE quantizer n-bit channel encoder sensor Xt: OU process

transmitter

channel decoder + IIR/FR coding

channel

receiver

ˆ Xt: MMSE estimate β processing time (ACK/NACK) feedback

OU process evolution: Xt = Xse−θ(t−s) + σ √ 2θ e−θ(t−s)We2θ(t−s)−1, t ≥ s Sensor acquires the ith sample at time Si; signal-independent sampling. MMSE quantizer represents XSi as ˜ XSi using ℓ bits: XSi = ˜ XSi + QSi Channel coding schemes:

Infinite Incremental Redundancy (IIR): n-bit codewords, increment if needed. Fixed Redundancy (FR): fixed n-bit codewords, repeat with new samples if needed.

Decoding consumes fixed β time units for processing and feedback. Successfully decoded messages are used to construct an MMSE estimate.

System Model

ℓ-bit MMSE quantizer n-bit channel encoder sensor Xt: OU process

transmitter

channel decoder + IIR/FR coding

channel

receiver

ˆ Xt: MMSE estimate β processing time (ACK/NACK) feedback

OU process evolution: Xt = Xse−θ(t−s) + σ √ 2θ e−θ(t−s)We2θ(t−s)−1, t ≥ s Sensor acquires the ith sample at time Si; signal-independent sampling. MMSE quantizer represents XSi as ˜ XSi using ℓ bits: XSi = ˜ XSi + QSi Channel coding schemes:

Infinite Incremental Redundancy (IIR): n-bit codewords, increment if needed. Fixed Redundancy (FR): fixed n-bit codewords, repeat with new samples if needed.

Decoding consumes fixed β time units for processing and feedback. Successfully decoded messages are used to construct an MMSE estimate. Tradeoff between distortion and timeliness.

System Model: Channel Delays

ℓ-bit MMSE quantizer n-bit channel encoder sensor Xt: OU process

transmitter

channel decoder + IIR/FR coding

channel

receiver

ˆ Xt: MMSE estimate β processing time (ACK/NACK) feedback

Di: reception time of the ith successfully decoded message. For the IIR coding scheme, each message is eventually decoded correctly: Di = Si + Yi

Yi is a random delay incurred due to IR bits added until success: Yi = nTb + β

• ¯

n

+ri(Tb + β), ri ∈ {0, 1, 2, . . . } Channel is memoryless; Yi’s are i.i.d. ∼ Y .

For the FR coding scheme, not every message is decoded: Di = Si,Mi + nTb + β

• ¯

n

Mi is the number of transmission attempts between (i − 1)th and ith successes. Channel is memoryless; Mi’s are i.i.d. ∼ M (geometric).

MMSE Estimates

ℓ-bit MMSE quantizer n-bit channel encoder sensor Xt: OU process

transmitter

channel decoder + IIR/FR coding

channel

receiver

ˆ Xt: MMSE estimate β processing time (ACK/NACK) feedback

For the IIR coding scheme: ˆ Xt = ˜ XSi e−θ(t−Si ), Di ≤ t < Di+1 mse (t, Si) =σ2 2θ

• 1 −
• 1 − 2−2ℓ

e−2θ(t−Si ) hℓ (t − Si) , Di ≤ t < Di+1 For the FR coding scheme: ˆ Xt = ˜ XSi,Mi e−θ(t−Si,Mi ), Di ≤ t < Di+1 mse (t, Si,Mi ) =hℓ (t − Si,Mi ) , Di ≤ t < Di+1 In both schemes, the MMSE is an increasing functional of AoI: t − S.

Problem Formulation: General Age-Penalty

Work with a general increasing age-penalty functional g(·). Goal: choose sampling times {Si} to minimize long term average age-penalty. For the IIR coding scheme: min

{Si }

lim sup

l→∞

l

i=0 E

Di+1

Di

g (t − Si) dt

• l

i=0 E [Di+1 − Di]

Equivalently, one can solve for inter-sampling waiting times {Wi} for which Si = Di−1 + Wi Epoch: time elapsed in between two consecutive successful receptions. Focus on stationary deterministic policies: Wi =f (g (Di−1 − Si−1)) w (Yi−1) , ∀i

Problem Formulation: General Age-Penalty

Work with a general increasing age-penalty functional g(·). Goal: choose sampling times {Si} to minimize long term average age-penalty. For the IIR coding scheme: min

{Si }

lim sup

l→∞

l

i=0 E

Di+1

Di

g (t − Si) dt

• l

i=0 E [Di+1 − Di]

Equivalently, one can solve for inter-sampling waiting times {Wi} for which Si = Di−1 + Wi Epoch: time elapsed in between two consecutive successful receptions. Focus on stationary deterministic policies: Wi =f (g (Di−1 − Si−1)) w (Yi−1) , ∀i

Problem Formulation: General Age-Penalty

Work with a general increasing age-penalty functional g(·). Goal: choose sampling times {Si} to minimize long term average age-penalty. For the IIR coding scheme: min

{Si }

lim sup

l→∞

l

i=0 E

Di+1

Di

g (t − Si) dt

• l

i=0 E [Di+1 − Di]

Equivalently, one can solve for inter-sampling waiting times {Wi} for which Si = Di−1 + Wi Epoch: time elapsed in between two consecutive successful receptions. Focus on stationary deterministic policies: Wi =f (g (Di−1 − Si−1)) w (Yi−1) , ∀i

Problem Formulation: General Age-Penalty

Work with a general increasing age-penalty functional g(·). Goal: choose sampling times {Si} to minimize long term average age-penalty. For the IIR coding scheme: min

{Si }

lim sup

l→∞

l

i=0 E

Di+1

Di

g (t − Si) dt

• l

i=0 E [Di+1 − Di]

Under a stationary deterministic policy: min

w(·)≥0

E D+w(Y)+Y

D

g

• t − S
• dt
• E
• w
• Y
• + Y

Problem Formulation: General Age-Penalty

Work with a general increasing age-penalty functional g(·). Goal: choose sampling times {Si} to minimize long term average age-penalty. For the FR coding scheme, waiting may be possible after failures too: Wi,1 =f

• g
• Di−1 − Si−1,Mi−1
• = w (¯

n) ≡ w1 Wi,2 =w (w1 + ¯ n) ≡ w2 . . . Wi,j =w (w1 + · · · + wj−1 + ¯ n) ≡ wj . . . Under a stationary deterministic policy: min

{wj ≥0}

E D+M

j=1 wj +M ¯

n D

g

• t − S ¯

M

• dt
• E

M

j=1 wj + M ¯

n

Main Results – IIR

min

w(·)≥0

E D+w(Y)+Y

D

g

• t − S
• dt
• E
• w
• Y
• + Y
• Theorem

The optimal waiting policy for the IIR scheme is given by w ∗(¯ y) =

• G −1

¯ y

(λ) + where ¯ y is the realization of the starting AoI ¯ Y , and G¯

y(x) E [g (¯

y + x + Y )]. Threshold policy: take a new sample only if expected age-penalty ≥ λ. Optimal threshold λ∗

IIR can be found via bisection search.

(also = long term average age-penalty.) Result can be shown by [Sun-Cyr (JCN ’19), Theorem 1]. Can also be shown through a different Lagrangian approach.

Main Results – FR

min

{wj ≥0}

E D+M

j=1 wj +M ¯

n D

g

• t − S ¯

M

• dt
• E

M

j=1 wj + M ¯

n

• Theorem

The optimal waiting policy for the FR scheme is given by w ∗

1 =

• G −1(λ)

+ , w ∗

j = 0, j ≥ 2

where G(x) E [g (¯ n + x + M ¯ n)]. In addition, the optimal λ∗

FR is such that w ∗ 1 = 0.

Zero-wait policy: acquire a new sample every ¯ n. λ∗

FR, the optimal long term average age-penalty, is given in closed-form.

Lemma

Using FR, a new message is sent after the previous’s delivery by [β − nTb]+ time units. Just-in-time transmissions & deliveries: receiver is never idle.

Main Results – FR

min

{wj ≥0}

E D+M

j=1 wj +M ¯

n D

g

• t − S ¯

M

• dt
• E

M

j=1 wj + M ¯

n

• Theorem

The optimal waiting policy for the FR scheme is given by w ∗

1 =

• G −1(λ)

+ , w ∗

j = 0, j ≥ 2

where G(x) E [g (¯ n + x + M ¯ n)]. In addition, the optimal λ∗

FR is such that w ∗ 1 = 0.

Zero-wait policy: acquire a new sample every ¯ n. λ∗

FR, the optimal long term average age-penalty, is given in closed-form.

Lemma

Using FR, a new message is sent after the previous’s delivery by [β − nTb]+ time units. Just-in-time transmissions & deliveries: receiver is never idle.

Discussion: Back to Estimation: MDS Coding Over BSC(ǫ)

1 2 3 4 5 6 7 8 9 10 5 10 15 1 2 3 4 5 6 7 8 9 10 0.4 0.5 0.6 0.7 0.8 0.9 1

σ2 = 1, Tb = 0.05 time units, β = 0.15. (dashed ǫ = 0.4; solid ǫ = 0.1)

Discussion: Back to Estimation: MDS Coding Over BSC(ǫ)

0.05 0.1 0.15 0.2 0.25 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 1.1

σ2 = 1, θ = 0.25, Tb = 0.05 time units, ℓ = 3 bits. (dashed ǫ = 0.4; solid ǫ = 0.1)

Conclusions

ℓ-bit MMSE quantizer n-bit channel encoder sensor Xt: OU process

transmitter

channel decoder + IIR/FR coding

channel

receiver

ˆ Xt: MMSE estimate β processing time (ACK/NACK) feedback

Characterized the effects of quantization and coding on remote estimation of OU processes over noisy channels. MMSE = g(AoI) — results in terms of general increasing age-penalty:

IIR coding: sampling policy has a threshold structure. FR coding: just-in-time sampling and transmissions.

Optimal quantization levels (estimates’ qualities) depend on the process’s memory. Processing delay affects which coding scheme (IIR/FR) performs better.