Lecture 03 Optimal Detection under Noise I-Hsiang Wang - PowerPoint PPT Presentation

Principle of Communications, Fall 2017 Lecture 03 Optimal Detection under Noise I-Hsiang Wang ihwang@ntu.edu.tw National Taiwan University 2017/9/28

x ( t ) Pulse Up { u m } x b ( t ) Shaper Converter discrete baseband passband Noisy sequence waveform waveform Channel y ( t ) { ˆ u m } Sampler y b ( t ) Down + Filter Converter Previous lecture: y ( t ) = x ( t ) ⇒ we can guarantee y b ( t ) = x b ( t ) for all t and u m = ˆ u m , for all m . = 2

x ( t ) Pulse Up { u m } x b ( t ) Shaper Converter discrete baseband passband Noisy sequence waveform waveform Channel Filter + y ( t ) { ˆ u m } Sampler y b ( t ) Down Sampler + + Filter Converter Detection This lecture: additive noise Y ( t ) = x ( t ) + Z ( t ) Questions to be addressed Answer Z ( t ) as white Gaussian process How to model the noise? Equivalent noise after down-conversion, Discrete-time equivalent: filtering, and sampling? V m = u m + Z m Optimal decision rule that minimizes How to find the best from ? V m ˆ u m error probability 3

Outline • Random processes • Statistical model of noise • Hypothesis testing • Optimal detection rules under additive noise • Performance analysis 4

Part I. Random Processes Definition, Gaussian Processes, Stationarity, Power Spectral Density, Filtering 5

Random variables Probability space Random variable ( Ω , F , P ) X : Ω � R , ω �� X ( ω ) . sample sigma probability Distribution space field measure F X ( x ) � P { X ( ω ) ≤ x } , x ∈ R . cumulative distribution function (CDF) X Roughly speaking: ( Ω , F , P ) → ( R , B , F X ) − 6

Random process A collection of of jointly distributed random variables: { X ( ω ; t ) | t ∈ I } I is uncountable ( I = R ) → random waveform { X ( t ) : t ∈ R } → random sequence I is countable ( I = Z ) { X m : m ∈ Z } Distribution is determined by the joint distribution of { X ( ω ; t ) | t ∈ S } for all finite subsets S ⊆ I F X ( t 1 ) ,...,X ( t n ) ( x 1 , . . . , x n ) � P { X ( ω ; t 1 ) ≤ x 1 , . . . , X ( ω ; t n ) ≤ x n } for all positive integer n and finite subset { t 1 , t 2 , . . . , t n } ⊆ I 7

Random process First moment µ X ( t ) � E [ X ( t )] Second moment K X ( s, t ) � Cov ( X ( s ) , X ( t )) = E [( X ( s ) − µ X ( s ))( X ( t ) − µ X ( t ))] (auto-covariance function) R X ( s, t ) � E [ X ( s ) X ( t )] (auto-correlation function) � ∞ F → ˘ Fourier transform X ( ω ; t ) X ( ω ; f ) = X ( ω ; t ) exp ( − j2 π ft ) d t ← −∞ 8

Gaussian random variable X ∼ N ( µ, σ 2 ) mean variance Gaussian probability density function (PDF) � � − | x − µ | 2 1 f X ( x ) � ∂ ∂ xF X ( x ) = 2 πσ 2 exp √ 2 σ 2 9

Jointly Gaussian random variables Definition (Jointly Gaussian) i.i.d. { Z 1 , Z 2 , . . . , Z n } : J.G. ⇐ ∼ N (0 , 1) ⇒ ∃ m ≤ n, W 1 , . . . , W m constant vector b ∈ R n constant matrix a ∈ R n × m such that     Z 1 W 1 . . Z � where W � . .  = a W + b ,  .     . .   Z n W m jointly Gaussian random vector 10

Jointly Gaussian random variables Z ∼ N ( µ , k ) , First moment   E [ Z 1 ] . µ � E [ Z ] = .   .   E [ Z n ] Second moment   Var [ Z 1 ] Cov ( Z 1 , Z 2 ) Cov ( Z 1 , Z n ) · · · Cov ( Z 2 , Z 1 ) Var [ Z 1 ] Cov ( Z 2 , Z n ) · · ·   k � E [( Z − µ )( Z − µ ) � ] = . .   ... . .   . .   Cov ( Z n , Z 1 ) Var [ Z n ] · · · 11

Jointly Gaussian random variables Z ∼ N ( µ , k ) , PDF of jointly Gaussian random vector � � 1 − 1 2( z − µ ) � k − 1 ( z − µ ) f Z ( z ) = exp 2 � n (2 π ) det( k ) Important fact : linear combinations of jointly Gaussian random variables are also jointly Gaussian 12

Gaussian process Definition (Gaussian Process) { Z ( t ) | t ∈ I } is a Gaussian process ⇒ ∀ n ∈ N and { t 1 , . . . , t n } ⊆ I , { Z ( t 1 ) , . . . , Z ( t n ) } : J.G. ⇐ Theorem (Distribution of Gaussian Process) Distribution of a Gaussian process { Z ( t )} is completely determined by its first and second moments µ Z ( t ), K Z ( s, t ) 13

Orthonormal expansion of Gaussian processes • For the Gaussian processes considered in this course, we assume that it can be expanded over an orthonormal basis with independent Gaussian coe ffi cients: ∞ � �� � � : mutually independent. Z k φ k ( t ) , Z k ∼ N (0 , σ 2 Z ( t ) = k ) k =1 ‣ This process is zero-mean, but this is WLOG when modeling the noise ‣ ∥ Z ( t ) ∥ 2 = � ∞ The energy of the random waveform is also random: k =1 Z 2 k • Auto-covariance of this Gaussian process is quite general � ∞ � ∞ ∞ � � � σ 2 K Z ( s, t ) = Cov Z k Z m φ k ( s ) φ m ( t ) = k φ k ( s ) φ k ( t ) . m =1 k =1 k =1 14

Stationary processes • Statistical properties of the noise tend to behave in a time invariant manner • This motivates us to define stationary processes: ‣ ������ is stationary � ������ and ���� � ��� are identically distributed for any time shift � ‣ Hard to check • Definition (Wide-sense stationary) ‣ ������ is wide-sense stationary (WSS) � the first and second moments are time-shift invariant, that is, - � � ��� � � for all � . Mean function is fixed: - Auto-covariance is only a function of time di ff erence: � � ��� � � �� � � � ��� for all � and � . • A Gaussian process is stationary i ff it is WSS. 15

Power spectral density • In the following, we will define the power spectral density of a zero-mean WSS random process as the Fourier transform of the auto-covariance (auto- correlation) function: F R X ( τ ) → S X ( f ) . ← • Several useful properties for a WSS real-valued random process { X ( t )} ‣ Auto-correlation and auto-covariance are both even functions: R X ( τ ) = R X ( − τ ) , K X ( τ ) = K X ( − τ ) ‣ Its PSD is real and even: S X ( f ) = S ∗ X ( f ) = S X ( − f ) � ∞ • � | X ( t ) | 2 � PSD is non-negative, and = S X ( f ) d f. E −∞ 16

� ���� ���� � � � ��� � � �� Energy spectral density of a deterministic waveform For a deterministic signal x ( t ) , its energy spectral density is the energy per unit frequency (hertz) at each frequency energy of output y ( t ; f 0 , ∆ f ) Operational Definition : E x ( f 0 ) � lim ∆ f ∆ f → 0 x ( f 0 ) | 2 . = | ˘ ↑ the square of the freq. response narrow-band filter 17

Auto-correlation of a deterministic waveform For a deterministic signal x ( t ) , its auto-correlation is the inverse Fourier transform of the energy spectral density. � ∞ F R x ( t ) � x ( τ ) x ∗ ( τ − t ) d τ E x ( f ) ← → −∞ Auto-correlation is e ff ectively the convolution of x ( t ) with x * (- t ) In PAM demodulation, the Rx filter q ( t ) = p * (- t ) . The filtering block is called “correlator” in some literatures 18

� ���� � � � Power spectral density of a deterministic waveform For a deterministic signal x ( t ) , its power spectral density is the power per unit frequency (hertz) at each frequency Power = Energy/Time 1 Operational Definition : S x ( f ) � lim E x t 0 ( f ) . t 0 t 0 →∞ � � �� − t 0 2 , t 0 x t 0 ( t ) � x ( t ) 1 t ∈ 2 multiplication with a small time interval 19

PSD of random process vs. deterministic wavform Inverse Fourier transform of the power spectral density is the long-term time average of the auto-correlation! t 0 1 � 2 F R x ( t ) � lim x ( τ ) x ∗ ( τ − t ) d τ S x ( f ) ← → t 0 t 0 →∞ − t 0 2 For an important class of random processes called ergodic processes , the long-term time average is equal to the statistical average: R X ( t ) = E [ X ( τ ) X ∗ ( τ − t )] = R X ( t ) ergodicity = ⇒ S X ( f ) = F { R X ( t ) } = S X ( f ) ↑ the reason why it is called PSD! 20

Filtering of random processes • We are primarily interested in the following properties of a random process after passing through an LTI filter: ‣ First and second moments ‣ Stationarity ‣ Gaussianity • Stationarity/wide-sense stationarity is preserved under LTI filtering • Gaussianity is preserved under LTI filtering • For a WSS process, the PSD of the filtered process is the PSD of the original process times the ESD of the LTI filter 21

First and second moments X ( t ) h ( t ) Y ( t ) = ( X ∗ h )( t ) First moment: µ Y ( t ) = ( µ X ∗ h )( t ) � ∞ � ∞ Second moment: K Y ( s, t ) = h ( s − ξ ) K X ( ξ , τ ) h ( t − τ ) d ξ d τ −∞ −∞ 22

Stationarity WSS WSS X ( t ) h ( t ) Y ( t ) = ( X ∗ h )( t ) Stationary Stationary µ Y = µ X ˘ First moment: h (0) Second moment: K Y ( τ ) = ( h ∗ K X ∗ h rv ) ( τ ) h ( f ) | 2 S X ( f ) S Y ( f ) = | ˘ Power spectral density: 23

Two branches of filtered processes X 1 ( t ) h 1 ( t ) Y 1 ( t ) = ( X 1 ∗ h 1 )( t ) jointly WSS jointly WSS X 2 ( t ) h 2 ( t ) Y 2 ( t ) = ( X 2 ∗ h 2 )( t ) Definition (jointly WSS): { X 1 ( t ) } and { X 2 ( t ) } are jointly WSS if they are both WSS and the cross-covariance K X 1 ,X 2 ( t + τ , t ) � Cov ( X 1 ( s ) , X 2 ( t )) depends on τ only 24