Data for Official Statistics Marco Puts, Piet Daas, Martijn - - PowerPoint PPT Presentation

data for official statistics
SMART_READER_LITE
LIVE PREVIEW

Data for Official Statistics Marco Puts, Piet Daas, Martijn - - PowerPoint PPT Presentation

High Frequency Road Sensor Data for Official Statistics Marco Puts, Piet Daas, Martijn Tennekes Road sensors Road sensor data Passing vehicle counts for each minute (24/7) at about 60.000 sensors in the Netherlands Types of sensors:


slide-1
SLIDE 1

Marco Puts, Piet Daas, Martijn Tennekes

High Frequency Road Sensor Data for Official Statistics

slide-2
SLIDE 2

Road sensors

Road sensor data

– Passing vehicle counts for each minute (24/7) at about 60.000 sensors in the Netherlands – Types of sensors: ‐ Induction loop ‐ Camera ‐ Bluetooth – Length categories (e.g. small, medium, long vehicles) – Large volume: approx. 230 mln records/day

2

slide-3
SLIDE 3

Dutch highways

3

slide-4
SLIDE 4

Dutch highways with road sensors

4

slide-5
SLIDE 5

Final Process

Clean Transform & Select Aggregate Frame

slide-6
SLIDE 6

Big Data Processes Quality feedback loop

6

Process

Qi Qo Compare

input

  • utput
slide-7
SLIDE 7

Low quality of data

7

slide-8
SLIDE 8

Quality of the Data

8

slide-9
SLIDE 9

Cleaning the data

Hidden Markov Model

9

X1 X2 X3 X4 Y1 Y2 Y3 Y4

  • bservation

state

slide-10
SLIDE 10

Cleaning the Data Recursive Bayesian Estimation

10

X1 X2 X3 X4 Y1 Y2 Y3 Y4

  • bservation

state

Update

slide-11
SLIDE 11

Cleaning the Data Recursive Bayesian Estimation

11

X1 X2 X3 X4 Y1 Y2 Y3 Y4

  • bservation

state

Prediction

slide-12
SLIDE 12

Cleaning the Data

Recursive Bayesian Estimation

12

X1 X2 X3 X4 Y1 Y2 Y3 Y4

  • bservation

state

Missing Data

slide-13
SLIDE 13

Cleaning the Data

Recursive Bayesian Estimation

13

X1 X2 X3 X4 Y1 Y2 Y3 Y4

  • bservation

state

Smoothing

slide-14
SLIDE 14

Results of the filter

Time [minutes] Vehicle count

14

slide-15
SLIDE 15

Precision/Accuracy

The filter does not introduce extra errors: – Precision: 3.6% – Accuracy:+0.13%

15

slide-16
SLIDE 16

Questions

?

16