Tasty Topics! Novel Approaches Using Topic Filtering Tom Fairbairn, - - PowerPoint PPT Presentation

▶
tasty topics
SMART_READER_LITE
LIVE PREVIEW

Tasty Topics! Novel Approaches Using Topic Filtering Tom Fairbairn, - - PowerPoint PPT Presentation

Jul 24, 2023 525 likes •872 views

Tasty Topics! Novel Approaches Using Topic Filtering Tom Fairbairn, Solace Systems tom.fairbairn@solace.com dev.solace.com https://www.linkedin.com/in/tomfairbairn Pub/Sub revision Distributed Decoupled Fanin/Fanout Persistence

slide-1
SLIDE 1

Tasty Topics!

Novel Approaches Using Topic Filtering

Tom Fairbairn, Solace Systems

tom.fairbairn@solace.com dev.solace.com https://www.linkedin.com/in/tomfairbairn

slide-2
SLIDE 2
slide-3
SLIDE 3

Pub/Sub revision

  • Distributed
  • Decoupled
  • Fanin/Fanout
  • Persistence
  • Register interest in Topic

Image credit: pubnub.com

slide-4
SLIDE 4

Topics

Topic ≠ 𝑈𝑏𝑕!

food/apple/slices */*/slices => food/apple/slices food/ham/slices food_apple_slices List {food_apple_slices, food_ham_slices} String search?

slide-5
SLIDE 5

Topic Dynamism

Where is the topic

  • Created?
  • Administered?

On Send? At the broker? At the receiver?

slide-6
SLIDE 6

Who Cares?

  • Simpler
  • Consistent
  • Reduces unnecessary data copies

– E.g. In IoT reduces unnecessary sensor reads

slide-7
SLIDE 7

Use Case 1

Migrating Data Format

slide-8
SLIDE 8

Case 1: Migrating Your Data Format

{ “Person” : “Tom”, “Team” : “Magicians”, “Mobile” : “07746 244422”, “EmployeeId” : 6 } { “Employee” : [“Id” : “0000732006”, “Name” : “Tom Fairbairn”, “PhoneNum” : “+44(0)7746244422”, “DirectReports”: [], ”ReportsTo”: “Ben Taieb” ] }

slide-9
SLIDE 9

CI/CD

“Employee” : [“Id” : “0000732006”, “Name” : “Tom Fairbairn”,

Tag

Build Test Release

slide-10
SLIDE 10

Data Format – read/write

Gson gson = new Gson(); empolyeeData = gson.fromJson(data, employee.class); public class employee { private String Person; private String Team; private String Mobile; public int EmployeeId; … } public class employee { private class employeeData { private String Id; private String Name; private String PhoneNum; private String[] DirectReports; private String ReportsTo; } … }

Tag: v1.0 Tag: v2.0

slide-11
SLIDE 11

Data Format topic

private String versionedTopic = “london/employee/json/$GIT_TAG_NAME/[…]”; session.subscribe(versionedTopic); producer.send(message, versionedTopic);

slide-12
SLIDE 12

Use Case 2

Authorisation

slide-13
SLIDE 13

Authorisation

Pub/Sub Any app Subscription app

  • 1. Request account balance
  • 2. Service calculates subscriptions
  • 3. Service subscribes for app
  • 4. App receives matching data

Topic: balance/request Data: tom balance/update/tom balance/update/dick balance/update/harry

slide-14
SLIDE 14

Use Case 3

Monitoring

slide-15
SLIDE 15

Case 3: Monitoring

slide-16
SLIDE 16

Pub/Sub Monitoring over Pub/Sub!

Pub/Sub Any app Monitoring app

Monitoring/Management API MQTT ReST

slide-17
SLIDE 17
slide-18
SLIDE 18

Use Case 4

Replay

slide-19
SLIDE 19

Case 4: Replay

DB DB DB DB DB DB DB DB DB DB DB

Dealing with shared state

– Ployglotpersistence? – Replay “state of the world” from message stream

slide-20
SLIDE 20

Replay – queues that can subscribe

Queue app/control app/config data/app/…

  • Queue Browser
  • TTL
  • LVQ
slide-21
SLIDE 21

Use Case 5

Find the nearest… Geo-location using topics

slide-22
SLIDE 22

Case 2:Find The Nearest… In Real Time

Source: ICOMP 2016, A.L. Lee, Ranged Filtering of Streaming Numeric Data… using Topic-Based Pub/Sub Messaging Map data from Google Maps; maps.google.com

slide-23
SLIDE 23

Geo-filtering topic

Publish to topic with location:

<app>/<type>/<lat>/<long>/<vehicle>/<id> geo/sim/51.520150/-00.097330/CAR/00021

Where is CAR00021?

subscribe(“geo/sim/*/*/CAR/00021”);

slide-24
SLIDE 24

Geo-filtering location

subscribe(“geo/sim/51.52015*/-00.09733*/>”);

Match: lat 51.520150 to 51.520159 long -000.097330 to -000.097339

slide-25
SLIDE 25

Geo-filtering location

subscribe(“geo/sim/51.52*/-00.09*/>”); Match: lat 51.520 to 51.529999 long -0.090 to -0.099999

slide-26
SLIDE 26

Geo-filtering location

subscribe(“geo/sim/51.52*/-00.09*/>”, “geo/sim/51.516*/-00.092*/>”, “geo/sim/51.516*/-00.093*/>”, “geo/sim/51.516*/-00.094*/>”, “geo/sim/51.516*/-00.096*/>”, “geo/sim/51.517*/-00.092*/>”, “geo/sim/51.517*/-00.093*/>”, “geo/sim/51.517*/-00.094*/>”, “geo/sim/51.517*/-00.096*/>”, // repeat for 51.518 and .519 );

slide-27
SLIDE 27

Geo-filtering location

  • Create any polygon

– Accuracy at metre level – Circles, arcs…

  • Subscriptions

generated once

  • Matches then

stream in with no extra computation

slide-28
SLIDE 28

Geo-filtering location algorithm

  • Divide space into rectangles aligned to subscriptions
  • Throw away rectangles with no match
slide-29
SLIDE 29

Geo-filtering location algorithm

  • Repeat: divide remaining rectangles by 10
  • Throw away rectangles with no match
slide-30
SLIDE 30

Geo-filtering location algorithm deployment

  • Library?

Pub/Sub Geo-filtering app Subscription app

  • 1. Request subscriptions for shape
  • 2. Service calculates subscriptions
  • 3. Service subscribes for app
  • 4. App receives matching data
slide-31
SLIDE 31

Last code snippet

if

Topic

then

slide-32
SLIDE 32

Monitoring

Come and see us at Booth 15!

slide-33
SLIDE 33