Computing and Communications 2. Information Theory -Gaussian - - PowerPoint PPT Presentation

1896 1920 1987 2006

Computing and Communications

• 2. Information Theory
• Gaussian Channel

Ying Cui Department of Electronic Engineering Shanghai Jiao Tong University, China 2017, Autumn

1

Outline

• Gaussian Channel
• Parallel Gaussian Channels
• Bandlimited Channels

2

Reference

• Elements of information theory, T. M. Cover and J. A.

Thomas, Wiley

3

GAUSSIAN CHANNEL

4

Gaussian Channel

• A time-discrete channel: 𝑍

𝑗 = 𝑌𝑗 + 𝑎𝑗

– output 𝑍

𝑗 at time 𝑗 is sum of input 𝑌𝑗 and noise 𝑎𝑗

– noise 𝑎𝑗~𝒪 0, 𝑂 according to central limit theorem

• cumulative effect of a large number of small random effects

– noise 𝑎𝑗 is assumed to be independent of signal 𝑌𝑗 – common communication channel

• wired and wireless telephone channel, satellite links
• If the noise variance is zero or the input is unconstrained,

the capacity of the channel is infinite

• Average power constraint for codeword (𝑦1, 𝑦2, … 𝑦𝑜)

–

1 𝑜 σ𝑗=1 𝑜

𝑦𝑗

2 ≤ 𝑄

5

Information Capacity

6

Definitions

7

Capacity of Gaussian Channel

8

Sphere Packing Argument

• Reason of being able to construct (2𝑜𝐷, 𝑜) codes with a

low probability of error

• Cannot hope to send at rates greater than C with low

probability of error

9

PARALLEL GAUSSIAN CHANNELS

10

Parallel Gaussian Channels

• 𝑙 independent Gaussian channels in parallel:

𝑍

𝑘 = 𝑌 𝑘 + 𝑎 𝑘, 𝑘 = 1, 2, … , 𝑙,

𝑎

𝑘~𝒪 0, 𝑂 𝑘

– output of each channel is sum of input and Gaussian noise – noise is independent from channel to channel – each parallel component can represent a different frequency – a common power constraint: 𝐹 σ𝑘=1

𝑙

𝑌

𝑘 2 ≤ 𝑄

11

Parallel Gaussian Channels

• Information capacity:

𝐷 = max

𝑔 𝑦1,…,𝑦𝑙 :σ 𝐹𝑌𝑗

2≤𝑄 𝐽(X1, … , 𝑌𝑙; 𝑍

1, … , 𝑍 𝑙)

12

Information Capacity (Water-Filling)

• The objective is to distribute the total power among

the various channels to maximize the total capacity

• Problem formulation
• Optimal solution: 𝑄𝑗 = (𝜉 − 𝑂𝑗)+

– ν is chosen s.t. σ(𝜉 − 𝑂𝑗)+= 𝑄 and (𝑦)+ denotes the positive part of 𝑦

13

BANDLIMITED CHANNELS

14

Bandlimited Channel

• A continuous time channel bandlimited channel with

white noise: Y 𝑢 = 𝑌 𝑢 + 𝑎(𝑢) ∗ ℎ(𝑢)

– 𝑌 𝑢 is signal waveform – 𝑎(𝑢) is white Gaussian noise waveform – ℎ(𝑢) is an ideal bandpass filter impulse response, bandlimited to [-W, W]

• Nyquist–Shannon sampling theorem

– a bandlimited func. has only 2W degrees of freedom/sec

15

Continuous/Discrete-time Channel

• Nyquist rate of bandlimited signal in [-W, W] Hz is 2W

samples per second (i.e., one sample every 1/(2W) sec)

– use this result to understand the capacity of the continuous channel in terms of the corresponding discrete-time channel

• btained by sampling at Nyquist rate
• Power of continuous signal P is measured in watts

(energy/sec), translating into average energy of P/2W per sample

• Noise power is typically specified in terms of Power

Spectrum Density (PSD), say N0/2 watts/Hz, translating to average noise energy of N0/2 per sample

16

Capacity

• Write the capacity of continuous time channel, with

average input signal energy of P/2W per sample and average noise energy of N0/2 per sample, as

• Capacity of bandlimited Gaussian channel with noise

spectral density N0/2 watts/Hz and power P watts

17

Summary

18

cuiying@sjtu.edu.cn iwct.sjtu.edu.cn/Personal/yingcui

19