A Learning Approach to Cooperative Communication System Design Yuxin - - PowerPoint PPT Presentation

▶
a learning approach to cooperative communication system
SMART_READER_LITE
LIVE PREVIEW

A Learning Approach to Cooperative Communication System Design Yuxin - - PowerPoint PPT Presentation

Aug 26, 2023 46 likes •572 views

A Learning Approach to Cooperative Communication System Design Yuxin Lu , Peng Cheng , Zhuo Chen , Wai Ho Mow and Yonghui Li The Hong Kong University of Science and Technology. The University of Sydney. Data 61,

slide-1
SLIDE 1

A Learning Approach to Cooperative Communication System Design

Yuxin Lu⋆, Peng Cheng†, Zhuo Chen∗, Wai Ho Mow⋆ and Yonghui Li†

⋆The Hong Kong University of Science and Technology. †The University of Sydney. ∗Data 61, CSIRO.

The 45th IEEE ICASSP, 2020

Supported by the HK RGC (GRF 16233816)

Yuxin Lu (HKUST) Learning Cooperative Communication System ICASSP 2020 1 / 32

slide-2
SLIDE 2

Outline

1

Background and Motivation

2

Relay-Assisted Cooperative Communication System

3

Learning the Cooperative System

4

Simulation Results

5

Conclusion

Yuxin Lu (HKUST) Learning Cooperative Communication System ICASSP 2020 2 / 32

slide-3
SLIDE 3

Outline

1

Background and Motivation

2

Relay-Assisted Cooperative Communication System

3

Learning the Cooperative System

4

Simulation Results

5

Conclusion

Yuxin Lu (HKUST) Learning Cooperative Communication System ICASSP 2020 3 / 32

slide-4
SLIDE 4

Deep Learning

Artificial Intelligence Machine Learning Deep Learning Deep learning (DL) is a branch

  • f machine learning ⇒ Learn to

make own decisions Structures algorithms in layers ⇒ Create an “artificial neural network”

Yuxin Lu (HKUST) Learning Cooperative Communication System ICASSP 2020 4 / 32

slide-5
SLIDE 5

Deep Learning in Communication

Conventional communication system is optimized in a block-wise manner: source/channel coding, modulation, demodulation, source/channel decoding, equalization

Yuxin Lu (HKUST) Learning Cooperative Communication System ICASSP 2020 5 / 32

slide-6
SLIDE 6

Deep Learning in Communication

Conventional communication system is optimized in a block-wise manner: source/channel coding, modulation, demodulation, source/channel decoding, equalization Deep learning techniques have been applied to replace certain blocks: channel coding/estimation

Yuxin Lu (HKUST) Learning Cooperative Communication System ICASSP 2020 5 / 32

slide-7
SLIDE 7

Deep Learning in Communication

Conventional communication system is optimized in a block-wise manner: source/channel coding, modulation, demodulation, source/channel decoding, equalization Deep learning techniques have been applied to replace certain blocks: channel coding/estimation Individualized component-wise approach might not optimize the overall system function!

Yuxin Lu (HKUST) Learning Cooperative Communication System ICASSP 2020 5 / 32

slide-8
SLIDE 8

Deep Learning in Communication

Can we optimize the communication system in a holistic manner?

Yuxin Lu (HKUST) Learning Cooperative Communication System ICASSP 2020 6 / 32

slide-9
SLIDE 9

Deep Learning in Communication

Can we optimize the communication system in a holistic manner? Joint design of the transmitter and receiver over the channel Expand the optimization space ...

Yuxin Lu (HKUST) Learning Cooperative Communication System ICASSP 2020 6 / 32

slide-10
SLIDE 10

Deep Learning in Communication

Can we optimize the communication system in a holistic manner? Joint design of the transmitter and receiver over the channel Expand the optimization space ...

  • Yes. Communication Autoencoder!

Yuxin Lu (HKUST) Learning Cooperative Communication System ICASSP 2020 6 / 32

slide-11
SLIDE 11

Deep Learning in Communication

Can we optimize the communication system in a holistic manner? Joint design of the transmitter and receiver over the channel Expand the optimization space ...

  • Yes. Communication Autoencoder!

Transmitter and receiver are represented by neural networks (NNs) Promising results have been obtained

Yuxin Lu (HKUST) Learning Cooperative Communication System ICASSP 2020 6 / 32

slide-12
SLIDE 12

Autoencoder

General autoencoder (AE) learns data structure to compress (top)

Encoder Decoder Image, Text, … Image, Text, … Encoder Decoder 00101101 01101001 11010001 00101101 01111001 11010001 Bit Stream Channel Bit Stream Latent Vector Distortion + Noise (Random) Dirty Noisy Latent Vector

General autoencoder (top) v.s. Communication autoencoder. Figure Credit: Zhao, Vuran, Guo and Scott

Yuxin Lu (HKUST) Learning Cooperative Communication System ICASSP 2020 7 / 32

slide-13
SLIDE 13

Autoencoder

General autoencoder (AE) learns data structure to compress (top)

Encoder Decoder Image, Text, … Image, Text, … Encoder Decoder 00101101 01101001 11010001 00101101 01111001 11010001 Bit Stream Channel Bit Stream Latent Vector Distortion + Noise (Random) Dirty Noisy Latent Vector

General autoencoder (top) v.s. Communication autoencoder. Figure Credit: Zhao, Vuran, Guo and Scott

Communication AE learns the channel behavior to improve transmission accuracy

Yuxin Lu (HKUST) Learning Cooperative Communication System ICASSP 2020 7 / 32

slide-14
SLIDE 14

Autoencoder

Most existing applications are for point-to-point communications

Yuxin Lu (HKUST) Learning Cooperative Communication System ICASSP 2020 8 / 32

slide-15
SLIDE 15

More Complicated Scenarios

Can we design an AE to optimize more complicated communication scenarios?

Yuxin Lu (HKUST) Learning Cooperative Communication System ICASSP 2020 9 / 32

slide-16
SLIDE 16

More Complicated Scenarios

Can we design an AE to optimize more complicated communication scenarios? Our focus: Relay-assisted cooperative communication system

Yuxin Lu (HKUST) Learning Cooperative Communication System ICASSP 2020 9 / 32

slide-17
SLIDE 17

Existing Works for AE+Relay

Constellation design for two-way relay networks1 ⇒ Focused on constellation optimization. No detection algorithm was addressed

1T.Matsumine, T.Koike-Akino, and Y.Wang, “Deep learning-based constellation optimization for physical network coding in two-way relay networks,” arXiv preprint arXiv:1903.03713, Mar. 2019. Yuxin Lu (HKUST) Learning Cooperative Communication System ICASSP 2020 10 / 32

slide-18
SLIDE 18

Existing Works for AE+Relay

Constellation design for two-way relay networks1 ⇒ Focused on constellation optimization. No detection algorithm was addressed Our focus: Joint optimization of the constellation and detection algorithm Start with a one-way relay network

1T.Matsumine, T.Koike-Akino, and Y.Wang, “Deep learning-based constellation optimization for physical network coding in two-way relay networks,” arXiv preprint arXiv:1903.03713, Mar. 2019. Yuxin Lu (HKUST) Learning Cooperative Communication System ICASSP 2020 10 / 32

slide-19
SLIDE 19

Outline

1

Background and Motivation

2

Relay-Assisted Cooperative Communication System

3

Learning the Cooperative System

4

Simulation Results

5

Conclusion

Yuxin Lu (HKUST) Learning Cooperative Communication System ICASSP 2020 11 / 32

slide-20
SLIDE 20

System Model

S R D

hSR hSD hRD

MLD/Amplifier MLD/MRC

DF/AF

First phase Second phase

System model of a 3-node relay network. Source (S), Relay (R), Destination (D)

R: half-duplex Source message: mS ∈ {1, 2, · · · , 2k}, encoded as xS of length n k/n bits/independent channel uses

Yuxin Lu (HKUST) Learning Cooperative Communication System ICASSP 2020 12 / 32

slide-21
SLIDE 21

System Model

First Phase: ySJ =

  • EShSJxS + nSJ, J ∈ {R, D},

(1) ES: average source transmit energy hSJ: channel coefficient nSJ: Gaussian noise vector CN(0, 2σ2

SJI)

Yuxin Lu (HKUST) Learning Cooperative Communication System ICASSP 2020 13 / 32

slide-22
SLIDE 22

System Model

First Phase: ySJ =

  • EShSJxS + nSJ, J ∈ {R, D},

(1) ES: average source transmit energy hSJ: channel coefficient nSJ: Gaussian noise vector CN(0, 2σ2

SJI)

Second Phase: yRD =

  • ERhRDxR + nRD,

(2) ER: average relay transmit energy hRD: channel coefficient nRD: Gaussian noise vector CN(0, 2σ2

RDI)

Yuxin Lu (HKUST) Learning Cooperative Communication System ICASSP 2020 13 / 32

slide-23
SLIDE 23

AF Relaying

AF relay node:

Symbol-wise amplifying operation xR =

ySR

√

PS|hSR|2+2σ2

SR , xR ∈ xR,

ySR ∈ ySR, hSR ∈ hSR Drawback: noise amplification ⇐ ySR = √EShSRxS + nSR

Yuxin Lu (HKUST) Learning Cooperative Communication System ICASSP 2020 14 / 32

slide-24
SLIDE 24

AF Relaying

AF relay node:

Symbol-wise amplifying operation xR =

ySR

√

PS|hSR|2+2σ2

SR , xR ∈ xR,

ySR ∈ ySR, hSR ∈ hSR Drawback: noise amplification ⇐ ySR = √EShSRxS + nSR

Destination:

Maximal-ratio combining (MRC) Optimal in the context of AF High complexity: O(n · 2k) per block

Yuxin Lu (HKUST) Learning Cooperative Communication System ICASSP 2020 14 / 32

slide-25
SLIDE 25

DF Relaying

DF relay node:

Maximum-likelihood decoding (MLD) xR = arg minx∈C ySR − hSR √ESx2, where C is code book, |C| = 2k. Drawback: hard decision ⇒ information loss

Yuxin Lu (HKUST) Learning Cooperative Communication System ICASSP 2020 15 / 32

slide-26
SLIDE 26

DF Relaying

DF relay node:

Maximum-likelihood decoding (MLD) xR = arg minx∈C ySR − hSR √ESx2, where C is code book, |C| = 2k. Drawback: hard decision ⇒ information loss

Destination:

Near-optimal decoder (NOD) arg maxxS∈C Pr(ySD|xS)

xR∈C Pr(xS → xR) Pr(yRD|xR)

Near-optimal in the context of DF High complexity: O(n · 2k · 2k) per block

Yuxin Lu (HKUST) Learning Cooperative Communication System ICASSP 2020 15 / 32

slide-27
SLIDE 27

Outline

1

Background and Motivation

2

Relay-Assisted Cooperative Communication System

3

Learning the Cooperative System

4

Simulation Results

5

Conclusion

Yuxin Lu (HKUST) Learning Cooperative Communication System ICASSP 2020 16 / 32

slide-28
SLIDE 28

A typical AE

Channel

Transmitter Receiver A typical AE for a point-to-point communication system

Input: one-hot encoding, e.g., {00, 01, 11, 10} → {1000, 0100, 0010, 0001}

Yuxin Lu (HKUST) Learning Cooperative Communication System ICASSP 2020 17 / 32

slide-29
SLIDE 29

A typical AE

Channel

Transmitter Receiver A typical AE for a point-to-point communication system

Input: one-hot encoding, e.g., {00, 01, 11, 10} → {1000, 0100, 0010, 0001} Output: softmax, i.e., φ(z)i =

ezi k

j=1 ezj , i = 1, 2, . . . , k and

z = [z1, z2, . . . , zk] ∈ Rk

Yuxin Lu (HKUST) Learning Cooperative Communication System ICASSP 2020 17 / 32

slide-30
SLIDE 30

Proposed AE Structure

qR: soft probability Advantage: eliminate noise amplification and hard decision

𝒓𝑆 𝒛𝑇𝑆 𝒚𝑆 𝑛𝑇 𝒓𝑇 𝒓𝐸 𝒛𝑆𝐸 𝒛𝑇𝐸 𝒚𝑇 𝑛𝑇

FC layers FC layers Cconcatenate layer FC layer (Softmax activation) AWGN AWGN AWGN DecoderD: 𝝆𝐸 FC layers Normalization layer EncoderS: 𝝆𝑇 FC layers FC layer (Softmax activation) DecEncR: 𝝆𝑆 FC layers Normalization layer Loss function: ℒ(𝒓𝑇, 𝒓𝐸) One-hot Encoding Argmax Loss function: ℒ(𝒓𝑇, 𝒓𝑆) DecoderR: 𝝆𝑆,𝐸𝐹 EncoderR: 𝝆𝑆,𝐹𝑂

Block diagram of the proposed AE for the cooperative communication system

Yuxin Lu (HKUST) Learning Cooperative Communication System ICASSP 2020 18 / 32

slide-31
SLIDE 31

End-to-End Loss

Expected loss (a large number of data sets) LSD(πS, πR, πD) = EqS[L(qS, qD)] (3)

Yuxin Lu (HKUST) Learning Cooperative Communication System ICASSP 2020 19 / 32

slide-32
SLIDE 32

End-to-End Loss

Expected loss (a large number of data sets) LSD(πS, πR, πD) = EqS[L(qS, qD)] (3) Estimated through sampling LSD(πS, πR, πD) 1 B

B

  • i=1

L(qS,i, qD,i) (4) B: batch size {qS,i, qD,i}: the i-th input output pair of training sample

Yuxin Lu (HKUST) Learning Cooperative Communication System ICASSP 2020 19 / 32

slide-33
SLIDE 33

Objective

(P1) min

πS,πR,πD

LSD(πS, πR, πD)

Yuxin Lu (HKUST) Learning Cooperative Communication System ICASSP 2020 20 / 32

slide-34
SLIDE 34

Proposed AE

How to design the training algorithm?

Yuxin Lu (HKUST) Learning Cooperative Communication System ICASSP 2020 21 / 32

slide-35
SLIDE 35

Proposed AE

How to design the training algorithm? A desirable way: directly train the whole model to minimize LSD

Yuxin Lu (HKUST) Learning Cooperative Communication System ICASSP 2020 21 / 32

slide-36
SLIDE 36

Proposed AE

How to design the training algorithm? A desirable way: directly train the whole model to minimize LSD Experimental results Do Not demonstrate a favorable performance

Yuxin Lu (HKUST) Learning Cooperative Communication System ICASSP 2020 21 / 32

slide-37
SLIDE 37

Proposed AE

How to design the training algorithm? A desirable way: directly train the whole model to minimize LSD Experimental results Do Not demonstrate a favorable performance A novel training algorithm is required!

Yuxin Lu (HKUST) Learning Cooperative Communication System ICASSP 2020 21 / 32

slide-38
SLIDE 38

A Two-Stage Training Scheme

{πS, πR, πD} ⇒ {πS, πR,DE, πR,EN, πD} (5)

Yuxin Lu (HKUST) Learning Cooperative Communication System ICASSP 2020 22 / 32

slide-39
SLIDE 39

A Two-Stage Training Scheme

{πS, πR, πD} ⇒ {πS, πR,DE, πR,EN, πD} (5) (P2) First stage: min

πS,πR,DE

LSR(πS, πR,DE) Second stage: min

πR,EN,πD

LSD(πR,EN, πD)

Yuxin Lu (HKUST) Learning Cooperative Communication System ICASSP 2020 22 / 32

slide-40
SLIDE 40

Training SNR

Fixed SNR: γ Mixed SNR: γ ∈ {γl, γl + ∆, · · · , γu − ∆, γu}

Yuxin Lu (HKUST) Learning Cooperative Communication System ICASSP 2020 23 / 32

slide-41
SLIDE 41

Training Algorithm

Algorithm 1 Two-stage training of the proposed AE model

Input Number of channel uses n, number of information bits (per message) k; SNR parameters ∆, γl and γu First Stage: training of the source-relay link Construct a partial model for the source-relay link; Randomly generate γSR ∈ {γl, γl + ∆, · · · , γu − ∆, γu}; Train this partial model to minimize LSR(πS, πR,DE); Save EncoderS and DecoderR; Second Stage: training of the entire network Load EncoderS and DecoderR; Incorporate the loaded components to construct the complete AE model; Randomly generate γIJ ∈ {γl, γl + ∆, · · · , γu − ∆, γu} for (I, J) ∈ {(S, R), (R, D), (S, D)}; Train the proposed AE model to minimize LSD(πR,EN, πD); Obtain EncoderR and DecoderD.

Yuxin Lu (HKUST) Learning Cooperative Communication System ICASSP 2020 24 / 32

slide-42
SLIDE 42

Outline

1

Background and Motivation

2

Relay-Assisted Cooperative Communication System

3

Learning the Cooperative System

4

Simulation Results

5

Conclusion

Yuxin Lu (HKUST) Learning Cooperative Communication System ICASSP 2020 25 / 32

slide-43
SLIDE 43

Learned Constellations

2 1 1 2 2 1 1 2 2 1 1 2 2 1 1 2

Constellations of xS (red squares) and xR (blue triangles) with an average power constraint for (n, k) = (2, 4)

xS: APSK-like ⇒ Shaping gain xR: Irregular Overlapping Non-conventional

Yuxin Lu (HKUST) Learning Cooperative Communication System ICASSP 2020 26 / 32

slide-44
SLIDE 44

BLER Performance

  • 2
  • 1

1 2 3 4 5 6 7 8

Eb/N0 (dB)

10-6 10-5 10-4 10-3 10-2 10-1 100

BLER

DF-MRC Hamming (7,4) DF-NOD Hamming (7,4) AF-MRC Hamming (7,4) proposed AE (7,4) trained at 4 dB proposed AE (7,4)

BLER performance comparison of the proposed AE and the baseline schemes for (n, k) = (7, 4)

⇒ Competitive BLER performance

Yuxin Lu (HKUST) Learning Cooperative Communication System ICASSP 2020 27 / 32

slide-45
SLIDE 45

Robustness under Non-Gaussian Channels

  • 2
  • 1

1 2 3 4 5 6 7 8

Eb/N0 (dB)

10-6 10-5 10-4 10-3 10-2 10-1 100

BLER

DF-NOD Hamming (7, 4) AF-MRC Hamming (7, 4) proposed AE (7, 4)

BLER performance comparison of the proposed AE and the baseline schemes under the impulse noises

e.g., interference produced by radar signals

Yuxin Lu (HKUST) Learning Cooperative Communication System ICASSP 2020 28 / 32

slide-46
SLIDE 46

Robustness under Non-Gaussian Channels

  • 2
  • 1

1 2 3 4 5 6 7 8

Eb/N0 (dB)

10-6 10-5 10-4 10-3 10-2 10-1 100

BLER

DF-NOD Hamming (7, 4) AF-MRC Hamming (7, 4) proposed AE (7, 4)

BLER performance comparison of the proposed AE and the baseline schemes under the impulse noises

e.g., interference produced by radar signals ⇒ Competitive BLER performance

Yuxin Lu (HKUST) Learning Cooperative Communication System ICASSP 2020 28 / 32

slide-47
SLIDE 47

Outline

1

Background and Motivation

2

Relay-Assisted Cooperative Communication System

3

Learning the Cooperative System

4

Simulation Results

5

Conclusion

Yuxin Lu (HKUST) Learning Cooperative Communication System ICASSP 2020 29 / 32

slide-48
SLIDE 48

Conclusion

Scheme Noise amplification Hard decision Channel estimation Decoding complexity per block DF No Yes Yes O(n · 2k · 2k) AF Yes No Yes O(n · 2k) AE No No No O(n · 2k)

⇒ The proposed AE is a competitive alternative for the conventional relaying techniques DF and AF

Yuxin Lu (HKUST) Learning Cooperative Communication System ICASSP 2020 30 / 32

slide-49
SLIDE 49

Conclusion

Scheme Noise amplification Hard decision Channel estimation Decoding complexity per block DF No Yes Yes O(n · 2k · 2k) AF Yes No Yes O(n · 2k) AE No No No O(n · 2k)

⇒ The proposed AE is a competitive alternative for the conventional relaying techniques DF and AF Future works: A theoretical perspective and performance guarantee need to be provided! Consider other relay networks, e.g., two-way, full-duplex, ...

Yuxin Lu (HKUST) Learning Cooperative Communication System ICASSP 2020 30 / 32

slide-50
SLIDE 50

Take Away

Carefully designed training algorithm, loss functions, and structure ⇒ AE works

Yuxin Lu (HKUST) Learning Cooperative Communication System ICASSP 2020 31 / 32

slide-51
SLIDE 51

Take Away

Carefully designed training algorithm, loss functions, and structure ⇒ AE works More general scenarios, Theoretical perspective ⇒ a longer journal version of this work :)

Yuxin Lu (HKUST) Learning Cooperative Communication System ICASSP 2020 31 / 32

slide-52
SLIDE 52

Thanks!

Email: {ylubg}@ust.hk