A Learning-Based MAC for Energy Efficient Wireless Sensor Networks - - PowerPoint PPT Presentation

▶
a learning based mac for energy efficient wireless sensor
SMART_READER_LITE
LIVE PREVIEW

A Learning-Based MAC for Energy Efficient Wireless Sensor Networks - - PowerPoint PPT Presentation

Mar 26, 2023 310 likes •591 views

A Learning-Based MAC for Energy Efficient Wireless Sensor Networks S. Galzarano 1,2 , Prof. A. Liotta 2 , Prof. G. Fortino 1 (1) University of Calabria, Italy (2) Eindhoven University of Technology, Netherlands Outline WSN & Machine

slide-1
SLIDE 1

A Learning-Based MAC for Energy Efficient Wireless Sensor Networks

  • S. Galzarano1,2, Prof. A. Liotta2, Prof. G. Fortino1

(1)University of Calabria, Italy (2)Eindhoven University of Technology, Netherlands

slide-2
SLIDE 2

Outline

  • WSN & Machine learning
  • Learning-based MAC
  • Simulation results
  • Conclusion & Future work

Stefano Galzarano, Antonio Liotta, Gincarlo Fortino 2

slide-3
SLIDE 3

Challenges in WSN

Stefano Galzarano, Antonio Liotta, Gincarlo Fortino 3

WSN Wireless Ad Hoc Medium:

  • unreliable, asymmetric
  • r unidirectional links
  • restricted broadband

Topology changes and mobillity:

  • Mobile sink and/or nodes
  • failing nodes
  • new node joining

Harsh environment:

  • no physical access to

network once deployed

  • nodes failure

Resource limitations:

  • battery
  • processing
  • memory
slide-4
SLIDE 4

Challenges in WSN

Communication Stack Application level

Stefano Galzarano, Antonio Liotta, Gincarlo Fortino 4

Clustering Routing and neighborood management

Medium Access Control

Physical Layer Data Processing Data Collection Security Event and target detection

slide-5
SLIDE 5

Medium Access Control

  • Protocol layer providing a multiple access control mechanism on a shared

communication medium.

  • A MAC protocol for WSN should use a radio wake-up/sleep scheduling for:

– Energy saving – Reduce collisions (and then also energy and latency) – Reduce idle listening periods – Maximizing throughput – Minimizing latency

Stefano Galzarano, Antonio Liotta, Gincarlo Fortino 5

slide-6
SLIDE 6

Research Objective

  • Using Machine Learning to improve MAC performance in terms of energy

efficiency, throughput and latency.

Stefano Galzarano, Antonio Liotta, Gincarlo Fortino 6

slide-7
SLIDE 7

Machine Learning in WSN

Machine Learning paradigms have been successfully adopted to address various challenges

Stefano Galzarano, Antonio Liotta, Gincarlo Fortino 7

  • Can adapt and operate efficiently in

dynamic environments.

  • Disover important correlation in sensor

data

  • Support more intelligent

decision-making and autonomous control

Reinforcement Learning Decision Tree Genetic Algorithms Swarm Intelligence

  • R. V. Kulkarni, A. Forster, and G. K. Venayagamoorthy,

“Computational Intelligence in Wireless Sensor Networks: A Survey,” Communications Surveys Tutorials, IEEE, vol. 13, no. 1, pp. 68 –96, quarter 2011.

.......

slide-8
SLIDE 8

Reinforcement Learning

  • Usually first choice when solving complex distributed problems in WSNs.
  • Trial and error: learning by interacting with the environment:

– Learning agents – Pool of possible actions – Goodness of actions – A reward function – Select one action – Execute the action – Observe the reward – Correct the goodness of the executed action

Stefano Galzarano, Antonio Liotta, Gincarlo Fortino 8

slide-9
SLIDE 9
  • The achieved total reward (Q-value) of taking a specific action at a given

state is computed using:

Q-Learning

Stefano Galzarano, Antonio Liotta, Gincarlo Fortino 9

slide-10
SLIDE 10
  • The achieved total reward (Q-value) of taking a specific action at a given

state is computed using:

Q-Learning

Stefano Galzarano, Antonio Liotta, Gincarlo Fortino 10

new Q-Value

  • ld Q-Value

learning constant Immediate reward

  • ld Q-Value
slide-11
SLIDE 11

Proposed Q-Learning based MAC

Adapt Q-learning to a radio wake-up/sleep scheduling

  • Learning agent

 Each node in the network

Stefano Galzarano, Antonio Liotta, Gincarlo Fortino 11

slide-12
SLIDE 12

Proposed Q-Learning based MAC

Adapt Q-learning to a radio wake-up/sleep scheduling

  • Learning agent

 Each node in the network

  • State

 Slot sk in a the frame f

Stefano Galzarano, Antonio Liotta, Gincarlo Fortino 12

frame t slot sk

slide-13
SLIDE 13

Proposed Q-Learning based MAC

Adapt Q-learning to a radio wake-up/sleep scheduling

  • Learning agent

 Each node in the network

  • State

 Slot sk in a the frame f

  • Possible actions

 Radio ON/OFF, for each slot

Stefano Galzarano, Antonio Liotta, Gincarlo Fortino 13

frame t slot sk

slide-14
SLIDE 14

Proposed Q-Learning based MAC

Adapt Q-learning to a radio wake-up/sleep scheduling

  • Learning agent

 Each node in the network

  • State

 Slot sk in a the frame f

  • Possible actions

 Radio ON/OFF, for each slot

  • Goodness of actions

 Q(sk)

Stefano Galzarano, Antonio Liotta, Gincarlo Fortino 14

frame t slot sk

Q(sk)

slide-15
SLIDE 15

Reward

Reward signals (per slot)

  • Received packets

+

  • Succesfully transmitted packets

+

  • Over-heard packets
  • Expected packets
  • Stefano Galzarano, Antonio Liotta, Gincarlo Fortino

15

slide-16
SLIDE 16

Simulation results

  • Castalia / OMNET++
  • Comparison with other 2 different RL-based MAC

Stefano Galzarano, Antonio Liotta, Gincarlo Fortino 16

slide-17
SLIDE 17

Simulation results

  • Linear, star and mesh topologies
  • Packets inter-arrival time between 1 and 10 seconds
  • Max throughput is between 20 and 200 byte/sec (200 bytes length

payload)

  • Performance metrics: throughput, latency, energy efficiency

Stefano Galzarano, Antonio Liotta, Gincarlo Fortino 17

Liu, Z., Elhanany, I.: RL-MAC: A reinforcement learning based MAC protocol for wireless sensor networks. International Journal of Sensor Networks 1, 117–124 (2006)

slide-18
SLIDE 18

Simulation results

Stefano Galzarano, Antonio Liotta, Gincarlo Fortino 18

slide-19
SLIDE 19

Simulation results

Stefano Galzarano, Antonio Liotta, Gincarlo Fortino 19

slide-20
SLIDE 20

Simulation results

Stefano Galzarano, Antonio Liotta, Gincarlo Fortino 20

slide-21
SLIDE 21

Simulation results

Stefano Galzarano, Antonio Liotta, Gincarlo Fortino 21

slide-22
SLIDE 22

Simulation results

  • A nodes-to-sink communication pattern has been used.
  • 2 pkt/sec
  • Multipath ring routing
  • Performance metrics: latency, packet delivery, energy efficiency

Stefano Galzarano, Antonio Liotta, Gincarlo Fortino 22

Mihaylov, M., Tuyls, K., Nowé, A.: Decentralized learning in wireless sensor networks. In: Taylor, M.E., Tuyls, K. (eds.) ALA 2009. LNCS (LNAI), vol. 5924, pp. 60–73. Springer, Heidelberg (2010)

slide-23
SLIDE 23

Simulation results

Stefano Galzarano, Antonio Liotta, Gincarlo Fortino 23

Mihaylov, M., Tuyls, K., Nowé, A.: Decentralized learning in wireless sensor networks. In: Taylor, M.E., Tuyls, K. (eds.) ALA 2009. LNCS (LNAI), vol. 5924, pp. 60–73. Springer, Heidelberg (2010)

slide-24
SLIDE 24

Conclusions

  • A Q-Learning approach has been successfully employed for a

self-adapting MAC layer on WSNs;

  • Simulation results show that it outperforms others RL-based

MAC

Stefano Galzarano, Antonio Liotta, Gincarlo Fortino 24

slide-25
SLIDE 25

Future work

  • Ongoing work:

– implementation on real sensor platforms; – extensive experiments with varying real deployment.

  • Dynamically update both frame length and slot number on

the basis of the network traffic.

Stefano Galzarano, Antonio Liotta, Gincarlo Fortino 25

slide-26
SLIDE 26

Thank you!!! Any Questions?

Stefano Galzarano, Antonio Liotta, Gincarlo Fortino 26

A Learning-Based MAC for Energy Efficient Wireless Sensor Networks

  • S. Galzarano, Prof. A. Liotta, Prof. G. Fortino

University of Calabria, Italy & Eindhoven University of Technology, Netherlands