Data for Official Statistics Marco Puts, Piet Daas, Martijn - - PowerPoint PPT Presentation
Data for Official Statistics Marco Puts, Piet Daas, Martijn - - PowerPoint PPT Presentation
High Frequency Road Sensor Data for Official Statistics Marco Puts, Piet Daas, Martijn Tennekes Road sensors Road sensor data Passing vehicle counts for each minute (24/7) at about 60.000 sensors in the Netherlands Types of sensors:
Road sensors
Road sensor data
– Passing vehicle counts for each minute (24/7) at about 60.000 sensors in the Netherlands – Types of sensors: ‐ Induction loop ‐ Camera ‐ Bluetooth – Length categories (e.g. small, medium, long vehicles) – Large volume: approx. 230 mln records/day
2
Dutch highways
3
Dutch highways with road sensors
4
Final Process
Clean Transform & Select Aggregate Frame
Big Data Processes Quality feedback loop
6
Process
Qi Qo Compare
input
- utput
Low quality of data
7
Quality of the Data
8
Cleaning the data
Hidden Markov Model
9
X1 X2 X3 X4 Y1 Y2 Y3 Y4
- bservation
state
Cleaning the Data Recursive Bayesian Estimation
10
X1 X2 X3 X4 Y1 Y2 Y3 Y4
- bservation
state
Update
Cleaning the Data Recursive Bayesian Estimation
11
X1 X2 X3 X4 Y1 Y2 Y3 Y4
- bservation
state
Prediction
Cleaning the Data
Recursive Bayesian Estimation
12
X1 X2 X3 X4 Y1 Y2 Y3 Y4
- bservation
state
Missing Data
Cleaning the Data
Recursive Bayesian Estimation
13
X1 X2 X3 X4 Y1 Y2 Y3 Y4
- bservation
state
Smoothing
Results of the filter
Time [minutes] Vehicle count
14
Precision/Accuracy
The filter does not introduce extra errors: – Precision: 3.6% – Accuracy:+0.13%
15
Questions
?
16