BEATING THE TRAFFIC JAM using embedded devices, OPC UA, Akka, NoSQL - - PowerPoint PPT Presentation

▶
beating the traffic jam
SMART_READER_LITE
LIVE PREVIEW

BEATING THE TRAFFIC JAM using embedded devices, OPC UA, Akka, NoSQL - - PowerPoint PPT Presentation

Jul 30, 2023 264 likes •647 views

BEATING THE TRAFFIC JAM using embedded devices, OPC UA, Akka, NoSQL Kristoffer Dyrkorn Scientist, BEKK BACKGROUND 2 Public Roads Administration BEKK Norwegian consulting firm Responsible for state and county roads Private and public sector

slide-1
SLIDE 1

BEATING THE TRAFFIC JAM

Kristoffer Dyrkorn Scientist, BEKK

using embedded devices, OPC UA, Akka, NoSQL

slide-2
SLIDE 2

BACKGROUND

Responsible for state and county roads Planning, construction, operation 7500 employees Spending: GBP 4.1 Billion (2013) Norwegian consulting firm Private and public sector enterprises Strategy, technology, digital services 370 employees

2

Public Roads Administration BEKK

slide-3
SLIDE 3

ROAD NETWORKS

3

Population (mill) Area (1000 km2) Roads (1000 km)

Germany 81 360 640 UK 64 240 400 Norway 5 390 100

slide-4
SLIDE 4
slide-5
SLIDE 5

5

E18 Vestfold – Undrumsdal. Photo: Hans A. Rosbach

slide-6
SLIDE 6

6

Atlanterhavsveien, www.nasjonaleturistveger.no. Photo: Harald Mowinckel

slide-7
SLIDE 7

7

Trollstigen, www.nasjonaleturistveger.no. Photo: Steinar Skaar

slide-8
SLIDE 8

8

Aurlandsfjellet, www.nasjonaleturistveger.no. Photo: Bjørn Andresen

slide-9
SLIDE 9

ROAD MAINTENANCE IS EXPENSIVE Climate Chemicals Traffic

Snow plowing Studded tires Frost heave Road salting Sea spray Bird droppings Vehicle count Vehicle weight

slide-10
SLIDE 10
slide-11
SLIDE 11

A TRAFFIC EVENT

"measure_point_number": 1601436, "county_id": 16, "region_id": 2, "server_local_timestamp": "2015-01-31T01:58:44.330+02:00", "server_utc_timestamp": "2015-01-30T23:58:44.330Z", "client_utc_timestamp": "2015-01-30T23:58:45.229Z", "event_number": 2319762, "vehicle_type": 3, "vehicle_type_raw": "9", "vehicle_type_quality": 22228, "vehicle_number": 2319762, "speed": 80.9, "length": 16.46, "lane": 1, "gap": 10.3

slide-12
SLIDE 12

12

slide-13
SLIDE 13

13

slide-14
SLIDE 14

14

slide-15
SLIDE 15
slide-16
SLIDE 16

DATA FLOW: ROADSIDE EQUIPMENT

Sensor Sensor Data logger 3G modem

slide-17
SLIDE 17

DATA FLOW: APPLICATION

17

Support libraries Application logic Java VM Data logger Server push OPC UA Storage Java VM Queries/results Reports REST API Admin GUI Other backends Install GUI

slide-18
SLIDE 18

DEPLOYMENT PIPELINE

18 Tanuki service wrapper Embedded Jetty Application Elasticsearch wrapper Elasticsearch

GitHub Jenkins Nexus RedHat Satellite Server Chef Solo Chef Solo

Environments Server roles Service URLs RPMs

Config files Config files

slide-19
SLIDE 19

TOPOLOGY AND SCALING

19

ES Data Data loggers

Application

ES Data ES Data ES Data

Application

ES Master ES Master

Voting node

DC 1 DC 2 DC 3

Application Application

slide-20
SLIDE 20

(INTERLUDE)

slide-21
SLIDE 21

DATA ANALYSIS

21

10000 20000 30000 40000 50000 60000 70000 80000 90000 100000

Traffice ven ts per day,

  • n

average, across 800 data loggers (2013)

slide-22
SLIDE 22

TRAFFIC PATTERNS

22

slide-23
SLIDE 23

STORAGE ESTIMATES

23

Current system New system

Data loggers 800 5000 Events/day 1 mill 6 mill Events/year 365 mill 2.2 bill Storage/year 2 TB

slide-24
SLIDE 24

TRAFFIC PATTERNS: VEHICLE FLOW (10 MIN AVERAGE)

24

slide-25
SLIDE 25

TRAFFIC PATTERNS: VEHICLE SPEED (1 MIN AVERAGE)

25

slide-26
SLIDE 26
slide-27
SLIDE 27
slide-28
SLIDE 28
slide-29
SLIDE 29

29

(INTERLUDE END)

slide-30
SLIDE 30

ELK = Elasticsearch, Logstash, Kibana

slide-31
SLIDE 31

EXAMPLE REPORT

31

For a given time interval, provide:

  • The total vehicle count and average speed,
  • The 85 and 95 percentile speeds,
  • and, in each of 5 vehicle length categories: the vehicle count and average speed,
  • and, in each of 12 vehicle speed categories: the vehicle count,
  • among the eligible traffic events

...and all of this for:

  • each traffic lane connected to a data logger,
  • each data logger within a geographical region
slide-32
SLIDE 32

EVOLVING ARCHITECTURES

32

From Play to Embedded Jetty + Akka + various libraries From MongoDB to Elasticsearch From SVN to Github From deployment by file copying to RedHat Satellite Server and Chef Solo From 3 identical server nodes to 11 server nodes having 4 roles From 1 to 3 data centers From Java 7 to Java 8 From ElasticSearch 0.90.3 to Elasticsearch 1.4.2 From Elasticsearch faceting and scripts to Elasticsearch aggregation framework From 0 to 100 data loggers In production for 1 year Stable, no data loss Handles 1200 data loggers

slide-33
SLIDE 33

FORTUNATE PROPERTIES

Started small - few moving parts Replacing an existing system Mindset: Exploration Transaction rates are predictable Built-in event replay support Simplified changes (We now need to split up the application) Scaling targets are known Slow ramp up Evaluate and change No sudden, uncontrolled peaks Robustness

33

slide-34
SLIDE 34

FUN WITH EMBEDDED DEVICES & OPC UA

34

Can process incoming traffic OR send buffered data Has clock drift, despite containing a GPS Has GPS, but reports the wrong position! Sends GPS coordinates out as mysterious byte arrays Has a non-correct implementation of OPC UA (and so did we) Has response times longer than response timeouts Pushes events round robin among all connected clients

slide-35
SLIDE 35

USE CASES

35

Reports Real-time Forecasts

Maintenance Measuring changes Weather impact Expected travel time Routing Alternatives to cars Traffic forecast Advise on routing Shape commuting habits

slide-36
SLIDE 36

?

slide-37
SLIDE 37

THANK YOU!

Kristoffer Dyrkorn Scientist, BEKK kristoffer.dyrkorn@bekk.no