An Efficient Neural Network Architecture for Rate Maximization in - - PowerPoint PPT Presentation

Communications & Machine Learning Lab

An Efficient Neural Network Architecture for Rate Maximization in Energy Harvesting Downlink Channels

Presentation for IEEE ISIT 2020

Index

I. Introduction II. System model

• III. Monotonicity of the Optimal Policy
• IV. Numerical results
• V. Conclusion

• I. Introduction

3

An Efficient Neural Network Architecture for Rate Maximization in Energy Harvesting Downlink Channels

Heasung Kim, Taehyun Cho, Jungwoo Lee, Wonjae Shin, and H. Vincent Poor, Communications & Machine Learning Lab, Seoul National University, Korea

• I. Introduction

4 … Thermal energy Solar power Wind energy Piezoelectric

Tx

Rx

Energy harvesting

No extra power source

Rx Rx ❖ Energy harvesting communications

An Efficient Neural Network Architecture for Rate Maximization in Energy Harvesting Downlink Channels

Heasung Kim, Taehyun Cho, Jungwoo Lee, Wonjae Shin, and H. Vincent Poor, Communications & Machine Learning Lab, Seoul National University, Korea

• I. Introduction

5

…

Pros

• To recycle energy
• No external energy source needed
• Green communications

…

Constraints

• Limitation on battery capacity
• Sufficient extra energy is not always

guaranteed ❖ Energy harvesting communications

An Efficient Neural Network Architecture for Rate Maximization in Energy Harvesting Downlink Channels

Heasung Kim, Taehyun Cho, Jungwoo Lee, Wonjae Shin, and H. Vincent Poor, Communications & Machine Learning Lab, Seoul National University, Korea

• I. Introduction

6

Traditional Reinforcement learning algorithms easily fall into local maxima when the transmitter gets inconsistent data.

However…

Using deep neural networks without proper grounds slows down forward propagation and hinders efficient network configuration. ❖ Reinforcement learning with function approximator

An Efficient Neural Network Architecture for Rate Maximization in Energy Harvesting Downlink Channels

Heasung Kim, Taehyun Cho, Jungwoo Lee, Wonjae Shin, and H. Vincent Poor, Communications & Machine Learning Lab, Seoul National University, Korea

• I. Introduction

7

The optimal policy is an increasing function of its input feature

Constructing a function approximator with a monotonic property of the value function.

Main contribution: To eliminate ambiguity in network building

Wasting computation resources required for DNN The learning agent has information about the

• ptimal value function

in advance

• II. System Model

8

An Efficient Neural Network Architecture for Rate Maximization in Energy Harvesting Downlink Channels

Heasung Kim, Taehyun Cho, Jungwoo Lee, Wonjae Shin, and H. Vincent Poor, Communications & Machine Learning Lab, Seoul National University, Korea

• II. System Model Notations

9

Notations 𝑓𝑗

ℎ

harvested energy, i.i.d. 𝐈𝑗 channel gains, i.i.d. 𝑐𝑗 remaining battery 𝑠

𝑙,(𝑗)

rate of 𝑙th user in time slot 𝑗 𝑞(𝑗) total power used in time slot 𝑗 𝑆(𝐈, 𝑞) the Shannon’s channel capacity 𝑊(𝑡𝑗) value function 𝑡 = (𝑓ℎ, 𝐈, 𝑐) state 𝜌(𝑓ℎ, 𝐈, 𝑐) power allocation policy

• Power allocation problem for discounted sum-rate
• n infinite horizon
• System running in discrete slotted time
• Channel State Information at Transmitter (CSIT)

❖ System Model

An Efficient Neural Network Architecture for Rate Maximization in Energy Harvesting Downlink Channels

Heasung Kim, Taehyun Cho, Jungwoo Lee, Wonjae Shin, and H. Vincent Poor, Communications & Machine Learning Lab, Seoul National University, Korea

• II. System Model Broadcast channel

10

Total transmit power at time slot i achievable rate of kth user with SIC

❖ Broadcast channel

An Efficient Neural Network Architecture for Rate Maximization in Energy Harvesting Downlink Channels

Heasung Kim, Taehyun Cho, Jungwoo Lee, Wonjae Shin, and H. Vincent Poor, Communications & Machine Learning Lab, Seoul National University, Korea

• II. System Model Weighted sum-rate

11

• J. Yang, O. Ozel, and S. Ulukus, “Broadcasting with an energy harvesting rechargeable transmitter,” IEEE Transactions on Wireless

Communications, vol. 11, no. 2, pp. 571–583, 2012.

Minimum power required to achieve

❖ Weighted sum-rate maximization problem

An Efficient Neural Network Architecture for Rate Maximization in Energy Harvesting Downlink Channels

Heasung Kim, Taehyun Cho, Jungwoo Lee, Wonjae Shin, and H. Vincent Poor, Communications & Machine Learning Lab, Seoul National University, Korea

• II. System Model Problem Formulation

12

Battery constraints Achievable rate in broadcast channel

determines the transmit power ❖ Weighted sum-rate maximization problem

• III. Monotonicity of the Optimal Policy

13

An Efficient Neural Network Architecture for Rate Maximization in Energy Harvesting Downlink Channels

Heasung Kim, Taehyun Cho, Jungwoo Lee, Wonjae Shin, and H. Vincent Poor, Communications & Machine Learning Lab, Seoul National University, Korea

• III. Monotonicity of the Optimal Policy

14

Proof of increasing property of the

• ptimal policy

Building lightweight monotonic neural network Policy gradient method

Why we prove that the optimal policy is an increasing function?

(𝑓ℎ, 𝐈, 𝑐)

The optimal policy has the equal or greater output (power) as the input variable is greater.

𝜌

𝑞

policy

Total power allocated at one time slot Harvested energy, channel gains, remaining battery

An Efficient Neural Network Architecture for Rate Maximization in Energy Harvesting Downlink Channels

Heasung Kim, Taehyun Cho, Jungwoo Lee, Wonjae Shin, and H. Vincent Poor, Communications & Machine Learning Lab, Seoul National University, Korea

• III. Monotonicity of the Optimal Policy

15 ❖ Increasing Property of the Optimal Power Allocation Policy

• The optimal power allocation policy satisfies the Bellman’s optimality equation.

Condition 1: has increasing difference in . Condition 2: Upper bound and lower bound of the action space are increasing functions for .

• Deriving increasing property of the optimal policy: if 𝐈′ dominates 𝐈, the optimal policy is increasing,

when the following conditions are satisfied [Topkis's theorem]. *Increasing difference in : the change of function from increasing is larger when 𝑞 is larger.

An Efficient Neural Network Architecture for Rate Maximization in Energy Harvesting Downlink Channels

Heasung Kim, Taehyun Cho, Jungwoo Lee, Wonjae Shin, and H. Vincent Poor, Communications & Machine Learning Lab, Seoul National University, Korea

• III. Monotonicity of the Optimal Policy

16 ❖ Increasing Property of the Optimal Power Allocation Policy

Condition 1: has increasing difference in . Condition 2: Upper bound and lower bound of the action space are increasing functions for . Upper & lower bound of the action (transmit power) space does not depend on the channel gain. They only depend on remaining battery of the transmitter.

An Efficient Neural Network Architecture for Rate Maximization in Energy Harvesting Downlink Channels

Heasung Kim, Taehyun Cho, Jungwoo Lee, Wonjae Shin, and H. Vincent Poor, Communications & Machine Learning Lab, Seoul National University, Korea

17

• III. Monotonicity of the Optimal Policy

Since Condition 1 & 2 are satisfied, the optimal power allocation policy is an increasing function for channel gains, , if . Similarly, and .

An Efficient Neural Network Architecture for Rate Maximization in Energy Harvesting Downlink Channels

Heasung Kim, Taehyun Cho, Jungwoo Lee, Wonjae Shin, and H. Vincent Poor, Communications & Machine Learning Lab, Seoul National University, Korea

• III. Monotonicity of the Optimal Policy – Monotonic Network

18

❖ Monotonic Neural Network for the Optimal Policy

…

max

Monotonic neural network [J. Sill, 1998]. Positive weight

…

max min

• Monotonic neural network has only one hidden layer and max-min activation functions.
• The weights should be positive.
• Update the monotonic neural network through the well-known policy gradient scheme.

• VI. Numerical Results

19

An Efficient Neural Network Architecture for Rate Maximization in Energy Harvesting Downlink Channels

Heasung Kim, Taehyun Cho, Jungwoo Lee, Wonjae Shin, and H. Vincent Poor, Communications & Machine Learning Lab, Seoul National University, Korea

• VI. Numerical Results

20

❖ Policy Learning Processes (0, 300, 600 iterations)

Monotonic Neural Network (shallow & optimized for the problem) 128-128-128 size Fully Connected Network (overly complex for the problem)

achieve 0.455 b/s achieve 0.403 b/s

An Efficient Neural Network Architecture for Rate Maximization in Energy Harvesting Downlink Channels

Heasung Kim, Taehyun Cho, Jungwoo Lee, Wonjae Shin, and H. Vincent Poor, Communications & Machine Learning Lab, Seoul National University, Korea

• VI. Numerical Results

21

❖ Sum-rate according to the harvesting probability

• Upper bound : Infinite capacity of the rechargeable

battery is assumed

• MN: Monotonic Network with Policy Gradient

(Proposed)

• TOP: Greedy myopic algorithm. Uses all energy in

the battery and divide it into the users at an optimal rate for the channel states.

• In all cases, our approach is more effective in

learning a transmission policy even with completely random incoming energy and channel processes.

• It also surpasses the performance of the greedy

timeslot-wise optimization policy (TOP).

• V. Conclusion

22

An Efficient Neural Network Architecture for Rate Maximization in Energy Harvesting Downlink Channels

Heasung Kim, Taehyun Cho, Jungwoo Lee, Wonjae Shin, and H. Vincent Poor, Communications & Machine Learning Lab, Seoul National University, Korea

• V. Conclusion

23

❖ Conclusion

• The inefficiency and the instability of conventional deep neural networks can be avoided by

shallow neural networks that are customized for desired optimal policy.

• We showed that a sound network design methodology is critical in applying deep learning

techniques to energy harvesting communication systems.

• Considering other features (e.g. concavity, convexity of the desired function)
• Adding more practical issues on the problem

❖ Future works

Thank you!

heasung1130@snu.ac.kr