Tracking AR(1) Process with limited communication Rooji Jinan - PowerPoint PPT Presentation

Tracking AR(1) Process with limited communication Rooji Jinan Parimal Parag , Himanshu Tyagi May 21, 2020 1

Remote real-time tracking ˆ X t X t Channel Sampler Encoder Decoder ( R bps) Instantaneous transmission ˆ X t X t s 0 t 0 t With delay in transmission ˆ X t 0 t 2

Fast or Precise? ◮ What is the optimal strategy for real-time tracking of a discrete time process under periodic sampling? ◮ Slow and precise or Fast but loose X t t X t t 3

Application ◮ Many cyber-physical systems often employ tracking of sensor data in real time ◮ Examples: sensing, surveillance, real-time control, ... sensor sensor Remote sensor Channel Application sensor sensor ◮ Communication is limited by the following constraints: ◮ Cost of frequent sampling ◮ Limited channel resources 4

Existing Works Sequential coding for correlated sources ◮ Rate-distortion region characterization [ Viswanathan2000TIT ] ◮ Real-time encoding for Gauss-Markov source [ Khina2017ITW ] Remote estimation under communication constraints ◮ Real-time estimation of Wiener process [ Sun2017ISIT ] ◮ Real-time estimation of AR source [ Chakravorty2017TAC ] Recursive state estimation algorithms under communication constraints ◮ Gaussian AR process [ Stavrou2017ITW ] ◮ Linear system over lossy channel [ Matveev2003TAC ] Current setting ◮ Rate-limited channel with unit delay per channel use ◮ Real-time estimation of AR(1) process 5

Source Process α z − 1 Sample X t Encoder Channel Decoder X t | t = ψ t ( C t − 1 ) ˆ ξ t + at ( φ t ) ( nR bps) ( ψ t ) t = ks zero mean covariance σ 2 (1 − α 2 ) I n Encoder has Decoder has access to received C t − 1 decoder state ◮ Innovation process ξ t ∈ R n is i.i.d. and n -dimensional ◮ Discrete AR(1) n -dimensional source process X t = α X t − 1 + ξ t for all t ≥ 0 ◮ Source process X t is sub-sampled at 1 / s , to obtain samples X ks at t = ks � ◮ sup k ∈ Z + 1 E � X k � 4 2 is bounded n 6

Communication Setting Encoder Channel Decoder X t | t = ψ t ( C t − 1 ) ˆ X ks ( φ t ) ( nR bps) ( ψ t ) Encoder has Decoder has access to received C t − 1 decoder state ◮ Encoder : φ t : X k +1 → { 0 , 1 } nRs at t = ks ◮ Channel : Error free, limited capacity causes delayed transmission ◮ Decoder : ψ t : { 0 , 1 } nR ( t − 1) → X at t = ks ◮ Performance metric : D t ( φ, ψ, X ) � 1 n E � X t − ˆ X t | t � 2 2 . 7

Optimal Decoder Structure Optimal ˆ X t | t = α i E [ X ks | C t − 1 ] From Channel Decoder ◮ Decoder at time t = ks + i for i ∈ { 1 , . . . , s } ◮ For the mean squared error, estimate conditional mean ◮ Utilize the latest information to refine the last sample X ks 8

Encoder Structure Y t X ks − Quantizer Q ( Y t ) ˆ X ks | t Decoder state ◮ Find the error in the decoder estimate of the last sample ◮ Transmit the quantized error 9

Periodic Successive Update Scheme ◮ At t = ks + jp , j ∈ [0 , s / p − 1], encode Y k , j = X ks − ˆ X ks | ks + jp . s = 4 , p = 2 X t 0 4 8 t Q ( Y 0 , 0 ) Q ( Y 0 , 1 ) Q ( Y 4 , 0 ) Q ( Y 4 , 1 ) Q ( Y 8 , 0 ) Q ( Y 8 , 1 ) 10

Encoder at time t = ks + jp Transmit Q ( Y ) no Sample � Y k , j � 2 2 > X t at To channel nM ? t = ks yes Transmit ⊥ 11

( θ, ε )-quantizer Definition Fix 0 < M < ∞ . A quantizer Q : R n → { 0 , 1 } nR constitutes an nR bit ( θ, ε ) -quantizer if for every vector y ∈ R n such that 1 n � y � 2 ≤ M , we have E � y − Q ( y ) � 2 2 ≤ � y � 2 2 θ ( R ) + n ε 2 . for 0 ≤ θ ≤ 1 and 0 ≤ ε . 12

Decoder at time t = ks + jp + i Declare ˆ X t | t = 0 for all subsequent time instants yes Channel Encoded Codeword received ⊥ ? ( nR bps) no ˆ X t | t = α t − ks [ ˆ X ks | ks +( j − 1) p + Q ( Y k , j − 1 )] 13

Performance of Periodic Successive Update Scheme Lemma For t = ks + jp + i, the p-SU scheme employing a nRp bit ( θ, ǫ ) quantizer satisfies D t ( φ p , ψ p , X ) ≤ α 2( t − ks ) θ ( Rp ) j D ks ( φ p , ψ p , X )+ σ 2 (1 − α 2( t − ks ) ) + f ( ǫ, β ) . β : Upperbound on the probability of encoder failure 14

Proof Idea No updation in estimate of X 0 p 1 s T X p | p = α p ( ˆ ˆ + Q ( Y 00 )) X 0 | 0 Transmit Q ( Y 00 ) ◮ X p = α p X 0 + � p u =1 α p − u ξ u X p | p = α p ˆ ◮ When encoding is successful, ˆ X 0 | p , D p = α 2 p 1 n E � X 0 − ˆ X 0 | 0 − Q ( X 0 − ˆ X 0 | 0 ) � 2 2 + σ 2 (1 − α 2 p ) ≤ α 2 p θ 1 2 + ǫ 2 + σ 2 (1 − α 2 p ) n E � X 0 − ˆ X 0 | 0 � 2 ◮ Else, use Cauchy-Schwartz Inequality 15

Performance of Periodic Successive Update Scheme Lemma For a fixed time horizon T, periodic successive update scheme with a ( θ, ǫ ) quantizer gives T g ( s ) α 2 p 1 − ε 2 � �� 1 � � D t ( φ p , ψ p , X ) ≤ σ 2 1 − σ 2 − θ ( Rp ) 1 − α 2 p θ ( Rp ) T t =0 1 − α 2 s for a very low probability of encoder failure and g ( s ) � s (1 − α 2 ) . 16

Example: 1 Uniform Quantizer R = log ⌈ 2 M /ǫ ⌉ ǫ 2 M 2 ≤ √ nM ◮ Say we quantize y , � y � 2 ◮ The quantizer parameters : θ = 0, ǫ 2 = nM 2 2 − 2 R ◮ Optimal p is 1 17

Example:2 Average Distortion Upper Bound for Gain-Shape Quantizer (a) (b) 2 . 00 1 . 95 2 . 8 1 . 90 2 . 6 1 . 85 1 . 80 2 . 4 1 . 75 2 . 2 1 . 70 1 . 65 2 . 0 2 . 5 5 . 0 7 . 5 10 . 0 12 . 5 2 . 5 5 . 0 7 . 5 10 . 0 12 . 5 Figure: (a) gives a case where p =s is the best and in (b) p =1 minimizes the bound 18

A quantizer design Norm Quantizer : Quantizes the norm B = � y � 2 / √ n into ˆ B such that | B − ˆ B | ≤ ε . / ǫ ⌉ n o . o f b i t s = l o g ⌈ M ǫ M Angle Quantizer 1 : A random codebook C consisting of 2 nR independent vectors distributed uniformly over the unit sphere S in R n . For any unit vector y ∈ R n ,the quantizer gives Q ( y ) = √ n cos θ · arg max y ′ ∈C � y , y ′ � . θ chosen to guarantee that there is one codeword y ′ such that, � y , y ′ � > cos θ for all y . 1 Amos Lapidoth. “On the role of mismatch in rate distortion theory”. In: IEEE Trans. Inf. Theory 43.1 (1997), pp. 38–47. 19

Performance of the quantizer For any y ∈ R n ,the quantizer gives Q ( y ) = √ n ˆ B cos θ · arg max y ′ ∈C � y , y ′ � . a vector with norm B Quantized vector ˆ B cos θ Lemma Consider a vector y ∈ R n with � y � 2 2 = nB 2 . Suppose that B ≤ M and let | B − ˆ B | ≤ ε . Then, 1 2 ≤ 2 − 2( R ′ ) B 2 + ε 2 . � 2 � � � y − Q ( y ) n 20

Special Case: Successive Update scheme ◮ Fast and Loose ◮ Set p = 1 X t s = 4 , p = 1 0 4 t 8 Q ( Y 0 , 0 ) Q ( Y 0 , 1 ) Q ( Y 0 , 2 ) Q ( Y 0 , 3 ) 21

Performance of the scheme Lemma Let t = ks + i, for i ∈ [1 , s ] , for n sufficiently large, the successive update scheme used with a ( θ, ǫ ) quantizer realisation with θ ( R ) = 2 − R satisfies D t ( φ, ψ, X ) ≤ α 2 i 2 − 2 Ri D ks ( φ, ψ, X ) + σ 2 (1 − α 2 i ) + f n where f n → 0 for large n. 22

Optimum min-max tracking accuracy Definition We define the accuracy, � T − 1 1 t =0 D t ( φ, ψ, X ) δ T ( φ, ψ, X n ) = 1 − T σ 2 Then, optimum asymptotic maxmin tracking accuracy, � � δ ∗ ( R , s , X ) = lim X ∈ X n δ T ( φ, ψ, X n ) T →∞ lim sup inf . n →∞ ( φ,ψ ) 23

Main Result Theorem (Lower bound for maxmin tracking accuracy: The achievability) For R > 0 and s ∈ N , the asymptotic minmax tracking accuracy is bounded below as δ ∗ ( R , s , X ) ≥ δ 0 ( R ) g ( s ) . for δ 0 ( R ) � α 2 (1 − 2 − 2 R ) (1 − α 2 2 − 2 R ) and g ( s ) � (1 − α 2 s ) s (1 − α 2 ) for all s > 0 . This bound is achieved using successive update scheme for p = 1 and the given realisation of ( θ, ǫ ) quantizer. 24

Theorem (Upper bound for maxmin tracking accuracy: The converse) For R > 0 and s ∈ N , the asymptotic minmax tracking accuracy is bounded above as δ ∗ ( R , s , X ) ≤ δ 0 ( R ) g ( s ) . The upper bound is obtained by considering the Gauss-Markov Processes. 25

Conclusion ◮ We provide an information theoretic upper bound for maxmin tracking accuracy for a fixed rate and sampling frequency. ◮ It is shown that for a fixed rate, high dimensional setting, the strategy of being fast but loose achieves this bound. ◮ We outline the performance requirements of the quantizer needed for achieving the optimal performance. ◮ For non-asymptotic regime our studies show that the optimal strategy might differ. 26

References I Amos Lapidoth. “On the role of mismatch in rate distortion theory”. In: IEEE Trans. Inf. Theory 43.1 (1997), pp. 38–47. 27