Fundamental Estimation and Detection Limits in Linear Non-Gaussian - PowerPoint PPT Presentation

Licentiate’s Presentation Fundamental Estimation and Detection Limits in Linear Non-Gaussian Systems Gustaf Hendeby Automatic Control Department of Electrical Engineering Linköpings universitet AUTOMATIC CONTROL Gustaf Hendeby COMMUNICATION SYSTEMS LINKÖPINGS UNIVERSITET Licentiate’s Presentation

Motivation Estimation and detection are used everywhere Vital functions rely on it Information is expensive AUTOMATIC CONTROL Gustaf Hendeby COMMUNICATION SYSTEMS LINKÖPINGS UNIVERSITET Licentiate’s Presentation

Motivation System description e t u t System y t Measure w t x t AUTOMATIC CONTROL Gustaf Hendeby COMMUNICATION SYSTEMS LINKÖPINGS UNIVERSITET Licentiate’s Presentation

Motivation Noise approximation Gaussian approximation True distribution AUTOMATIC CONTROL Gustaf Hendeby COMMUNICATION SYSTEMS LINKÖPINGS UNIVERSITET Licentiate’s Presentation

Motivation Noise approximation Gaussian approximation True distribution AUTOMATIC CONTROL Gustaf Hendeby COMMUNICATION SYSTEMS LINKÖPINGS UNIVERSITET Licentiate’s Presentation

Motivation Noise approximation True distribution Gaussian approximation AUTOMATIC CONTROL Gustaf Hendeby COMMUNICATION SYSTEMS LINKÖPINGS UNIVERSITET Licentiate’s Presentation

Motivation Noise approximation Gaussian approximation True distribution AUTOMATIC CONTROL Gustaf Hendeby COMMUNICATION SYSTEMS LINKÖPINGS UNIVERSITET Licentiate’s Presentation

Outline 1. Introduction 2. Noise and Information 3. Estimation Limits 4. Detection Limits 5. Conclusions AUTOMATIC CONTROL Gustaf Hendeby COMMUNICATION SYSTEMS LINKÖPINGS UNIVERSITET Licentiate’s Presentation

Outline 1. Introduction 2. Noise and Information 3. Estimation Limits 4. Detection Limits 5. Conclusions AUTOMATIC CONTROL Gustaf Hendeby COMMUNICATION SYSTEMS LINKÖPINGS UNIVERSITET Licentiate’s Presentation

Noise 0.4 0.35 Random effects 0.3 • Measurement noise 0.25 • Process noise p(x) 0.2 More or less informative 0.15 0.1 Description: 0.05 • PDF p ( x ) 0 • Expected value: E ( x ) = µ −4 −3 −2 −1 0 1 2 3 4 x • Variance: var ( x ) = Σ p ( x ) = N ( x ; 0 , 1 ) AUTOMATIC CONTROL Gustaf Hendeby COMMUNICATION SYSTEMS LINKÖPINGS UNIVERSITET Licentiate’s Presentation

Information and Accuracy y t = θ + e t 2 True 1.8 Gauss Approx. 1.6 1.4 1.2 p(e) 1 0.8 0.6 0.4 0.2 0 −4 −3 −2 −1 0 1 2 3 4 e e t AUTOMATIC CONTROL Gustaf Hendeby COMMUNICATION SYSTEMS LINKÖPINGS UNIVERSITET Licentiate’s Presentation

Information and Accuracy y t = θ + e t 2 2 True True 1.8 Approx. Based 1.8 Gauss Approx. Meas. Meas. 1.6 1.6 1.4 1.4 1.2 1.2 p( θ |Y t ) p(e) 1 1 0.8 0.8 0.6 0.6 0.4 0.4 0.2 0.2 0 0 −4 −3 −2 −1 0 1 2 3 4 −4 −3 −2 −1 0 1 2 3 4 θ e θ | Y t e t AUTOMATIC CONTROL Gustaf Hendeby COMMUNICATION SYSTEMS LINKÖPINGS UNIVERSITET Licentiate’s Presentation

Information and Accuracy y t = θ + e t 2 2 True True 1.8 Approx. Based 1.8 Gauss Approx. Meas. Meas. 1.6 1.6 1.4 1.4 1.2 1.2 p( θ |Y t ) p(e) 1 1 0.8 0.8 0.6 0.6 0.4 0.4 0.2 0.2 0 0 −4 −3 −2 −1 0 1 2 3 4 −4 −3 −2 −1 0 1 2 3 4 θ e θ | Y t e t AUTOMATIC CONTROL Gustaf Hendeby COMMUNICATION SYSTEMS LINKÖPINGS UNIVERSITET Licentiate’s Presentation

Information and Accuracy y t = θ + e t 2 2 True True 1.8 Approx. Based 1.8 Gauss Approx. Meas. Meas. 1.6 1.6 1.4 1.4 1.2 1.2 p( θ |Y t ) p(e) 1 1 0.8 0.8 0.6 0.6 0.4 0.4 0.2 0.2 0 0 −4 −3 −2 −1 0 1 2 3 4 −4 −3 −2 −1 0 1 2 3 4 θ e θ | Y t e t AUTOMATIC CONTROL Gustaf Hendeby COMMUNICATION SYSTEMS LINKÖPINGS UNIVERSITET Licentiate’s Presentation

Information and Accuracy y t = θ + e t 2 2 True True 1.8 Approx. Based 1.8 Gauss Approx. Meas. Meas. 1.6 1.6 1.4 1.4 1.2 1.2 p( θ |Y t ) p(e) 1 1 0.8 0.8 0.6 0.6 0.4 0.4 0.2 0.2 0 0 −4 −3 −2 −1 0 1 2 3 4 −4 −3 −2 −1 0 1 2 3 4 θ e θ | Y t e t AUTOMATIC CONTROL Gustaf Hendeby COMMUNICATION SYSTEMS LINKÖPINGS UNIVERSITET Licentiate’s Presentation

Information and Accuracy y t = θ + e t 2 2 True True 1.8 Approx. Based 1.8 Gauss Approx. Meas. Meas. 1.6 1.6 1.4 1.4 1.2 1.2 p( θ |Y t ) p(e) 1 1 0.8 0.8 0.6 0.6 0.4 0.4 0.2 0.2 0 0 −4 −3 −2 −1 0 1 2 3 4 −4 −3 −2 −1 0 1 2 3 4 θ e θ | Y t e t AUTOMATIC CONTROL Gustaf Hendeby COMMUNICATION SYSTEMS LINKÖPINGS UNIVERSITET Licentiate’s Presentation

Definitions: information measures Fisher information (true parameter θ 0 ): � � � ∆ θ I x ( θ ) = − E x θ log p ( x | θ ) � � θ = θ 0 Intrinsic accuracy (true mean µ 0 ): � � ∆ x x log p ( x | µ 0 ) I x = − E x Relative accuracy: Ψ x = var ( x ) I x Kullback-Leibler information: � p ( x ) � � I KL � � p ( · ) , q ( · ) = p ( x ) log dx q ( x ) AUTOMATIC CONTROL Gustaf Hendeby COMMUNICATION SYSTEMS LINKÖPINGS UNIVERSITET Licentiate’s Presentation

Definitions: information measures Fisher information (true parameter θ 0 ): � � � ∆ θ I x ( θ ) = − E x θ log p ( x | θ ) � � θ = θ 0 Intrinsic accuracy (true mean µ 0 ): � � ∆ x x log p ( x | µ 0 ) I x = − E x Relative accuracy: Ψ x = var ( x ) I x Kullback-Leibler information: � p ( x ) � � I KL � � p ( · ) , q ( · ) = p ( x ) log dx q ( x ) AUTOMATIC CONTROL Gustaf Hendeby COMMUNICATION SYSTEMS LINKÖPINGS UNIVERSITET Licentiate’s Presentation

Example: intrinsic accuracy Assume y i = θ + e i , e i ∼ N ( µ = 0 , Σ ) , then the intrinsic accuracy is � � ∆ e I e = − E e e log N ( e ; µ , Σ ) 1 e − ( e − µ ) 2 � � ∆ e √ = − E e e log 2 Σ 2 π Σ √ 2 π Σ + ( e − µ ) 2 � �� ∆ e � = E e log e 2 Σ � 1 ( e − µ ) = 1 1 � � � = E e ∇ e = E e Σ = var ( e ) . Σ Σ AUTOMATIC CONTROL Gustaf Hendeby COMMUNICATION SYSTEMS LINKÖPINGS UNIVERSITET Licentiate’s Presentation

Example: intrinsic accuracy Assume y i = θ + e i , e i ∼ N ( µ = 0 , Σ ) , then the intrinsic accuracy is � � ∆ e I e = − E e e log N ( e ; µ , Σ ) 1 e − ( e − µ ) 2 � � ∆ e √ = − E e e log 2 Σ 2 π Σ √ 2 π Σ + ( e − µ ) 2 � �� ∆ e � = E e log e 2 Σ � 1 ( e − µ ) = 1 1 � � � = E e ∇ e = E e Σ = var ( e ) . Σ Σ AUTOMATIC CONTROL Gustaf Hendeby COMMUNICATION SYSTEMS LINKÖPINGS UNIVERSITET Licentiate’s Presentation

Example: intrinsic accuracy Assume y i = θ + e i , e i ∼ N ( µ = 0 , Σ ) , then the intrinsic accuracy is � � ∆ e I e = − E e e log N ( e ; µ , Σ ) 1 e − ( e − µ ) 2 � � ∆ e √ = − E e e log 2 Σ 2 π Σ √ 2 π Σ + ( e − µ ) 2 � �� ∆ e � = E e log e 2 Σ � 1 ( e − µ ) = 1 1 � � � = E e ∇ e = E e Σ = var ( e ) . Σ Σ AUTOMATIC CONTROL Gustaf Hendeby COMMUNICATION SYSTEMS LINKÖPINGS UNIVERSITET Licentiate’s Presentation

Example: intrinsic accuracy Assume y i = θ + e i , e i ∼ N ( µ = 0 , Σ ) , then the intrinsic accuracy is � � ∆ e I e = − E e e log N ( e ; µ , Σ ) 1 e − ( e − µ ) 2 � � ∆ e √ = − E e e log 2 Σ 2 π Σ √ 2 π Σ + ( e − µ ) 2 � �� ∆ e � = E e log e 2 Σ � 1 ( e − µ ) = 1 1 � � � = E e ∇ e = E e Σ = var ( e ) . Σ Σ AUTOMATIC CONTROL Gustaf Hendeby COMMUNICATION SYSTEMS LINKÖPINGS UNIVERSITET Licentiate’s Presentation

Example: intrinsic accuracy Assume y i = θ + e i , e i ∼ N ( µ = 0 , Σ ) , then the intrinsic accuracy is � � ∆ e I e = − E e e log N ( e ; µ , Σ ) 1 e − ( e − µ ) 2 � � ∆ e √ = − E e e log 2 Σ 2 π Σ √ 2 π Σ + ( e − µ ) 2 � �� ∆ e � = E e log e 2 Σ � 1 ( e − µ ) = 1 1 � � � = E e ∇ e = E e Σ = var ( e ) . Σ Σ AUTOMATIC CONTROL Gustaf Hendeby COMMUNICATION SYSTEMS LINKÖPINGS UNIVERSITET Licentiate’s Presentation

Example: intrinsic accuracy Assume y i = θ + e i , e i ∼ N ( µ = 0 , Σ ) , then the intrinsic accuracy is � � ∆ e I e = − E e e log N ( e ; µ , Σ ) 1 e − ( e − µ ) 2 � � ∆ e √ = − E e e log 2 Σ 2 π Σ √ 2 π Σ + ( e − µ ) 2 � �� ∆ e � = E e log e 2 Σ � 1 ( e − µ ) = 1 1 � � � = E e ∇ e = E e Σ = var ( e ) . Σ Σ AUTOMATIC CONTROL Gustaf Hendeby COMMUNICATION SYSTEMS LINKÖPINGS UNIVERSITET Licentiate’s Presentation

Intrinsic Accuracy: outliers p 1 ( x ; ω , k ) = ( 1 − ω ) N ( x ; 0 , Σ )+ ω N ( x ; 0 , k Σ ) Inverse relative accuracy: Ψ − 1 = ( cov ( x ) I x ) − 1 x Σ − 1 : = 1 +( k − 1 ) ω , to get cov ( x ) = 1 Red is informative, blue is not k = 1 yields Gaussian distribution AUTOMATIC CONTROL Gustaf Hendeby COMMUNICATION SYSTEMS LINKÖPINGS UNIVERSITET Licentiate’s Presentation

Outline 1. Introduction 2. Noise and Information 3. Estimation Limits 4. Detection Limits 5. Conclusions AUTOMATIC CONTROL Gustaf Hendeby COMMUNICATION SYSTEMS LINKÖPINGS UNIVERSITET Licentiate’s Presentation