On the Impact of Clustering for IoT Analytics and Message Broker - PowerPoint PPT Presentation

On the Impact of Clustering for IoT Analytics and Message Broker Placement across Cloud and Edge Daniel Happ (TU Berlin, happ@tkn.tu-berlin.de) Suzan Bayhan (University of Twente) EdgeSys 2020 April 27, 2020 IoT Analytics and Message Broker Placement D. Happ/S. Bayhan 1 / 15

Motivation Internet of Things (IoT) becoming sharing economy Multiple Applications use same sensor data Sensor data processing needed for meaningful insights Storage, distribution, and processing usually in cloud Plentiful resources Flexible (on-demand, pay-as-you-go) Cloud has latency and privacy issues, preventing certain use-cases Moving more processing to the edge IoT Analytics and Message Broker Placement D. Happ/S. Bayhan 2 / 15

The case for Publish/Subscribe in IoT ”Sense once, notify many” translates well to publish/subscribe pattern Decoupling properties: subscribe • Time Subscriber notify • Synchronization publish notify Publisher Broker Subscriber • Space notify Subscriber Enable seamless processing operator offloading scheme 1 1D. Happ and A. Wolisz, ”Towards gateway to cloud offloading in IoT publish/subscribe systems,” in 2017 Second International Conference on Fog and Mobile Edge Computing (FMEC), May 2017, pp. 101–106. IoT Analytics and Message Broker Placement D. Happ/S. Bayhan 3 / 15

Research questions 1. How to place operators and brokers jointly across a cloud/fog/edge topology? 2. What is the impact of clustering of publishers and subscribers on the placement? 3. What is the impact of the network size? IoT Analytics and Message Broker Placement D. Happ/S. Bayhan 4 / 15

JOI deploys Operators & Message Brokers 2 Application users Input: network and application Network and Cloud layer application constraints & constraints application graph Application Fog layer deployment JOI Broker deployment Output: where to deploy IoT application operators and brokers Fog-service provider Edge layer Sensors 2Daniel Happ, Suzan Bayhan, and Vlado Handziski. 2020. JOI: Joint Placementof IoT Analytics Operators and Pub/Sub Message Brokers in Fog-centric IoT Platforms. Future Generation Comp. Sys. (2020). under review IoT Analytics and Message Broker Placement D. Happ/S. Bayhan 5 / 15

JOI: Optimal Solution Sketch Variables: x i , j : Operator i placed on node j y i , j : Operator i publishes to broker on node j Constraints: Node resources (CPU, memory) Node-to-node bandwidth Node access network bandwidth Objective: Minimize sum of subscriber delays IoT Analytics and Message Broker Placement D. Happ/S. Bayhan 6 / 15

Greedy Heuristic for JOI Preparation: Sort operators by hops to sink (stratum) Depth-first search: Place ops with low stratum first Place op optimally (with current knowledge) Place its broker optimally (with current knowledge) Finally: Join brokers until ”maximum number of brokers” constraint met IoT Analytics and Message Broker Placement D. Happ/S. Bayhan 7 / 15

Tabu Heuristic for JOI Starting-Point: Best solution of cloud and greedy In each step, try to improve: 1. Placement of one operator 2. Placement of one broker 3. Operator to broker association Tabu: Short term memory IoT Analytics and Message Broker Placement D. Happ/S. Bayhan 8 / 15

Degrees of Clustering in IoT Cloud Layer Fog Layer Edge Layer P 1 S 2 P 2 S 2 S 2 S 1 High degree of clustering for IoT publisher P 1 Low degree of clustering for publisher P 2 IoT Analytics and Message Broker Placement D. Happ/S. Bayhan 9 / 15

Von Mises Distribution models Clustering Properties: uniform for β = 0 , approaches the positive half of a normal distribution Nodes numbered according to delay to pivot node Fixed operators follow modified von Mises function IoT Analytics and Message Broker Placement D. Happ/S. Bayhan 10 / 15

System-level Simulations Application graphs: Random, fanout, sequence Fixed, but random, network topology: Edge, fog, cloud Realistic delays and bandwidth (public route servers) Metric: cloud gap IoT Analytics and Message Broker Placement D. Happ/S. Bayhan 11 / 15

Impact of Clustering on Cloud Gap (Random) clustering factor 0–9: cloud deployment best 2.5 2.0 clustering factor 12: Cloud Gap transition 1.5 1.0 clustering factor 15+: improvement by GREEDY 0.5 TABU greedy/tabu 0.0 0 3 6 9 12 15 18 enables joint heuristic Clustering Factor based on clustering factor IoT Analytics and Message Broker Placement D. Happ/S. Bayhan 12 / 15

Placement of Operators (Clustering Factor 18) Random Fanout Sequence Greedy places most 1.0 (approx. 80%) operators in 0.8 the fog 0.6 Share Tabu improves by putting 0.4 less operator on edge & 0.2 mostly more in cloud 0.0 CLOUD TABU GREEDY CLOUD TABU GREEDY CLOUD TABU GREEDY hard to give easy ”rules of thumb” IoT Analytics and Message Broker Placement D. Happ/S. Bayhan 13 / 15

Impact of Network Size (Greedy) Cluster Factor 0 25 Difference between greedy Cluster Factor 18 and cloud more pronounced 20 in smaller topologies Cloud Gap 15 10 5 Observations hold true for 0 all the sizes considered 3 9 30 95 300 Topology Size IoT Analytics and Message Broker Placement D. Happ/S. Bayhan 14 / 15

Conclusion Contributions: We proposed two heuristics for joint operator & broker placement We proposed modified von Mises distributions for modelling clustering We conducted simulations to evaluate impact of clustering Main Result: Increasing clustering leads to placement towards the edge Future Work: Cluster-aware placement heuristic Adaptive/dynamic scheme IoT Analytics and Message Broker Placement D. Happ/S. Bayhan 15 / 15