DIVIDE: ADAPTIVE CONTEXT-AWARE QUERY DERIVATION FOR IOT DATA STREAMS - - PowerPoint PPT Presentation
DIVIDE: ADAPTIVE CONTEXT-AWARE QUERY DERIVATION FOR IOT DATA STREAMS - - PowerPoint PPT Presentation
DIVIDE: ADAPTIVE CONTEXT-AWARE QUERY DERIVATION FOR IOT DATA STREAMS Mathias De Brouwer Sensors and Actuators on the Web (SAW) workshop ISWC 2019 Auckland, New Zealand 26 October 2019 CONTEXT-AWARE MONITORING IN I O T Wearable Motion
DIVIDE: ADAPTIVE CONTEXT-AWARE QUERY DERIVATION FOR IOT DATA STREAMS
Mathias De Brouwer Sensors and Actuators on the Web (SAW) workshop – ISWC 2019 Auckland, New Zealand – 26 October 2019
CONTEXT-AWARE MONITORING IN IOT
3
Sound sensor A0 Light sensor A1 Temperature sensor A2 … Localization Door/window sensor Motion sensor Wearable
CONTEXT-AWARE MONITORING IN IOT
4
Sound sensor A0 Light sensor A1 Temperature sensor A2 … Localization Door/window sensor Motion sensor Wearable
Electronic Health Record of patients Medical domain knowledge Hospital lay-out Care staff
CONTEXT-AWARE MONITORING IN IOT
5
Sound sensor A0 Light sensor A1 Temperature sensor A2 … Localization Door/window sensor Motion sensor Wearable
Electronic Health Record of patients Medical domain knowledge Hospital lay-out Care staff
?
CONTEXT-AWARE MONITORING IN IOT
6
Sound sensor A0 Light sensor A1 Temperature sensor A2 … Localization Door/window sensor Motion sensor Wearable
Electronic Health Record of patients Medical domain knowledge Hospital lay-out Care staff
REQUIRES INTELLIGENT
CONSOLIDATION & ANALYSIS OF HETEROGENEOUS, VOLUMINOUS, HIGH VELOCITY DATA
?
CONTEXT-AWARE MONITORING IN IOT
7
Sound sensor A0 Light sensor A1 Temperature sensor A2 … Localization Door/window sensor Motion sensor Wearable
Electronic Health Record of patients Medical domain knowledge Hospital lay-out Care staff
REQUIRES INTELLIGENT
CONSOLIDATION & ANALYSIS OF HETEROGENEOUS, VOLUMINOUS, HIGH VELOCITY DATA
RDF STREAM PROCESSING
EXAMPLE USE CASE – PATIENT BOB
8
Sound sensor A0 Light sensor A1 Temperature sensor A2 … Localization Door/window sensor Motion sensor Wearable
Electronic Health Record of patients Medical domain knowledge Hospital lay-out Care staff
RDF STREAM PROCESSING
EXAMPLE USE CASE – PATIENT BOB
9
Sound sensor A0 Light sensor A1 Temperature sensor A2 … Localization Door/window sensor Motion sensor Wearable
Electronic Health Record of patients Medical domain knowledge Hospital lay-out Care staff
Diagnosis Bob: epilepsy attack
RDF STREAM PROCESSING
EXAMPLE USE CASE – PATIENT BOB
10
Sound sensor A0 Light sensor A1 Temperature sensor A2 … Localization Door/window sensor Motion sensor Wearable
Electronic Health Record of patients Medical domain knowledge Hospital lay-out Care staff
Diagnosis Bob: epilepsy attack Epilepsy patients are sensitive to light Threshold: light should not go > 250 lumen
RDF STREAM PROCESSING
EXAMPLE USE CASE – PATIENT BOB
11
Sound sensor A0 Light sensor A1 Temperature sensor A2 … Localization Door/window sensor Motion sensor Wearable
Electronic Health Record of patients Medical domain knowledge Hospital lay-out Care staff
Diagnosis Bob: epilepsy attack Epilepsy patients are sensitive to light Threshold: light should not go > 250 lumen Query 1: generate alarm if value measured by sensor A1 is > 250
RDF STREAM PROCESSING
EXAMPLE USE CASE – PATIENT BOB
12
Sound sensor A0 Light sensor A1 Temperature sensor A2 … Localization Door/window sensor Motion sensor Wearable
Electronic Health Record of patients Medical domain knowledge Hospital lay-out Care staff
Query 1: generate alarm if value measured by sensor A1 is > 250 Diagnosis Bob: concussion
RDF STREAM PROCESSING
EXAMPLE USE CASE – PATIENT BOB
13
Sound sensor A0 Light sensor A1 Temperature sensor A2 … Localization Door/window sensor Motion sensor Wearable
Electronic Health Record of patients Medical domain knowledge Hospital lay-out Care staff
Concussion patients are sensitive to light & sound Thresholds: light should not go > 170 lumen, sound should not go > 30 decibels Query 1: generate alarm if value measured by sensor A1 is > 250 Diagnosis Bob: concussion
RDF STREAM PROCESSING
EXAMPLE USE CASE – PATIENT BOB
14
Sound sensor A0 Light sensor A1 Temperature sensor A2 … Localization Door/window sensor Motion sensor Wearable
Electronic Health Record of patients Medical domain knowledge Hospital lay-out Care staff
Concussion patients are sensitive to light & sound Thresholds: light should not go > 170 lumen, sound should not go > 30 decibels Query 1: generate alarm if value measured by sensor A1 is > 250 170 Diagnosis Bob: concussion
RDF STREAM PROCESSING
EXAMPLE USE CASE – PATIENT BOB
15
Sound sensor A0 Light sensor A1 Temperature sensor A2 … Localization Door/window sensor Motion sensor Wearable
Electronic Health Record of patients Medical domain knowledge Hospital lay-out Care staff
Concussion patients are sensitive to light & sound Thresholds: light should not go > 170 lumen, sound should not go > 30 decibels Query 1: generate alarm if value measured by sensor A1 is > 250 Query 2: generate alarm if value measured by sensor A0 is > 30 170 Diagnosis Bob: concussion
RDF STREAM PROCESSING
HOW TO DEAL WITH CONTEXT CHANGES?
16
Approach Real-time reasoning? How to configure? Manual reconfiguration when context changes? FIXED GENERIC
QUERIES
expressive manually no
HOW TO DEAL WITH CONTEXT CHANGES?
17
Approach Real-time reasoning? How to configure? Manual reconfiguration when context changes? FIXED GENERIC
QUERIES
expressive manually no
HOW TO DEAL WITH CONTEXT CHANGES?
18
Approach Real-time reasoning? How to configure? Manual reconfiguration when context changes? FIXED GENERIC
QUERIES
expressive manually no
HOW TO DEAL WITH CONTEXT CHANGES?
19
Approach Real-time reasoning? How to configure? Manual reconfiguration when context changes? FIXED GENERIC
QUERIES
expressive manually no MULTIPLE QUERIES (1 FOR EACH
RELEVANT SENSOR)
(almost) none manually yes
HOW TO DEAL WITH CONTEXT CHANGES?
20
Approach Real-time reasoning? How to configure? Manual reconfiguration when context changes? FIXED GENERIC
QUERIES
expressive manually no MULTIPLE QUERIES (1 FOR EACH
RELEVANT SENSOR)
(almost) none manually yes
HOW TO DEAL WITH CONTEXT CHANGES?
21
Approach Real-time reasoning? How to configure? Manual reconfiguration when context changes? FIXED GENERIC
QUERIES
expressive manually no MULTIPLE QUERIES (1 FOR EACH
RELEVANT SENSOR)
(almost) none manually yes
HOW TO DEAL WITH CONTEXT CHANGES?
22
Approach Real-time reasoning? How to configure? Manual reconfiguration when context changes? FIXED GENERIC
QUERIES
expressive manually no MULTIPLE QUERIES (1 FOR EACH
RELEVANT SENSOR)
(almost) none manually yes DIVIDE none automatically no
GENERAL ISSUE VS. GOAL
(Changed) context
23
GENERAL ISSUE VS. GOAL
(Changed) context
24
Queries of interest are automatically derived & (re)configured
GENERAL ISSUE VS. GOAL
(Changed) context
25
Queries of interest are automatically derived & (re)configured
By performing reasoning on (changed) context instead of
- n the real-time data streams
GENERAL ISSUE VS. GOAL
(Changed) context
26
Queries of interest are automatically derived & (re)configured
By performing reasoning on (changed) context instead of
- n the real-time data streams
DIVIDE
DIVIDE – BUILDING BLOCKS
Logic: Notation3 Logic (N3)
27
DIVIDE – BUILDING BLOCKS
Logic: Notation3 Logic (N3) Reasoner: EYE reasoner ➔ OWL 2 RL reasoning ➔ define goal (= which triples EYE should look for evidence) ➔ EYE produces a proof with the goal as the last applied rule
28
OVERVIEW OF DIVIDE
29
INSTANTIATED QUERY QUERY DERIVATION PREPROCESSED ONTOLOGY (IN EYE IMAGE) REASONER’S GOAL SENSOR QUERY RULE CONTEXT
New or changed (e.g., for a patient or room) Output: filtering queries to run on RSP engine ONTOLOGY PREPROCESSING
ONTOLOGY
ONTOLOGY PREPROCESSING
Goal: speed up query derivation
30
ONTOLOGY PREPROCESSING
Goal: speed up query derivation How?
31
ONTOLOGY PREPROCESSING
Goal: speed up query derivation How?
32
Create N3 copy
- f ontology
ONTOLOGY PREPROCESSING
Goal: speed up query derivation How?
33
Create N3 copy
- f ontology
Create specialized
- ntology-specific
rules from OWL 2 RL rules
ONTOLOGY PREPROCESSING
Goal: speed up query derivation How?
34
Create N3 copy
- f ontology
Create specialized
- ntology-specific
rules from OWL 2 RL rules Create compiled prolog image of EYE (with ontology & specialized rules)
DIVIDE – BUILDING BLOCKS
35
Ontology: ACCIO ontology
DIVIDE – BUILDING BLOCKS
36
LightIntensityAboveThresholdFault ≡ (hasSymptom some LightIntensityAboveThresholdSymptom) and (madeBySensor some (isSubsystemOf some (hasLocation some (isLocationOf some ( (hasDiagnosis some (hasMedicalSymptom some SensitiveToLight)) and (hasRole some PatientRole))))))
Ontology: ACCIO ontology
DIVIDE – BUILDING BLOCKS
37
HandleHighLightInRoomAction ≡ LightIntensityAboveThresholdFault and (madeBySensor some (isSubsystemOf some (hasLocation some (isLocationOf some LightingDevice))))
Ontology: ACCIO ontology
INPUTS OF QUERY DERIVATION
38
Reasoner goal: look for an AboveThresholdAction
{?x a :AboveThresholdAction.} => {?x a : AboveThresholdAction.}.
INPUTS OF QUERY DERIVATION
39
Reasoner goal: look for an AboveThresholdAction
{?x a :AboveThresholdAction.} => {?x a : AboveThresholdAction.}.
Accio ontology – definitions (previous slides) & individuals:
:Concussion a :Diagnosis; :hasMedicalSymptom :ConcussionSensitivenessToLight . :ConcussionSensitivenessToLight :hasThreshold :ConcussionLightThreshold . :ConcussionLightThreshold :hasDataValue "170"^^xsd:float ; :isThresholdOnProperty [ a :LightIntensity ] .
INPUTS OF QUERY DERIVATION
40
Reasoner goal: look for an AboveThresholdAction
:Bob a :Person ; :hasRole [ a :PatientRole ] ; :hasLocation :R101 ; :hasDiagnosis :Concussion . :40-a5-ef-05-a4-a6-A0 a :SoundSensor ; :hasLocation :R101 . :40-a5-ef-05-a4-a6-A1 a :LightSensor ; :hasLocation :R101 . :40-a5-ef-05-a4-a6-A2 a :TemperatureSensor ; :hasLocation :R101 . ... {?x a :AboveThresholdAction.} => {?x a : AboveThresholdAction.}.
Accio ontology – definitions (previous slides) & individuals: Context – part about patient Bob and his room:
:Concussion a :Diagnosis; :hasMedicalSymptom :ConcussionSensitivenessToLight . :ConcussionSensitivenessToLight :hasThreshold :ConcussionLightThreshold . :ConcussionLightThreshold :hasDataValue "170"^^xsd:float ; :isThresholdOnProperty [ a :LightIntensity ] .
INPUTS OF QUERY DERIVATION
41
Reasoner goal: look for an AboveThresholdAction
:Bob a :Person ; :hasRole [ a :PatientRole ] ; :hasLocation :R101 ; :hasDiagnosis :Concussion . :40-a5-ef-05-a4-a6-A0 a :SoundSensor ; :hasLocation :R101 . :40-a5-ef-05-a4-a6-A1 a :LightSensor ; :hasLocation :R101 . :40-a5-ef-05-a4-a6-A2 a :TemperatureSensor ; :hasLocation :R101 . ... {?x a :AboveThresholdAction.} => {?x a : AboveThresholdAction.}.
Accio ontology – definitions (previous slides) & individuals: Context – part about patient Bob and his room:
:Concussion a :Diagnosis; :hasMedicalSymptom :ConcussionSensitivenessToLight . :ConcussionSensitivenessToLight :hasThreshold :ConcussionLightThreshold . :ConcussionLightThreshold :hasDataValue "170"^^xsd:float ; :isThresholdOnProperty [ a :LightIntensity ] .
Sensor query rule – ???
WHAT IS THIS SENSOR QUERY RULE ?
{ ?p :hasRole [ a :PatientRole ] ; :hasLocation ?l ; :hasDiagnosis [ :hasMedicalSymptom [ :hasThreshold [ :hasDataValue ?th ; :isThresholdOnProperty [ a ?prop ] ] ] ] .
?s a :Sensor ; :observes [ a ?prop ] ; :hasLocation ?l .
} => { _:x a :Query ; :pattern :pattern-1 ; :inputVariables (("?th" ?th) ("?s" ?s) ("?prop" ?prop)) ; _:oo a :observation ; :madeBySensor ?s ; :hasResult [ :hasDataValue _:v ] ; :hasSymptom [ a :ThresholdSymptom ; :forProperty [ a ?prop ] ] . } .
42
WHAT IS THIS SENSOR QUERY RULE ?
{ ?p :hasRole [ a :PatientRole ] ; :hasLocation ?l ; :hasDiagnosis [ :hasMedicalSymptom [ :hasThreshold [ :hasDataValue ?th ; :isThresholdOnProperty [ a ?prop ] ] ] ] .
?s a :Sensor ; :observes [ a ?prop ] ; :hasLocation ?l .
} => { _:x a :Query ; :pattern :pattern-1 ; :inputVariables (("?prop" ?prop) ("?th" ?th) ("?s" ?s)) ; _:oo a :observation ; :madeBySensor ?s ; :hasResult [ :hasDataValue _:v ] ; :hasSymptom [ a :ThresholdSymptom ; :forProperty [ a ?prop ] ] . } .
43
Representation
- f generic query
WHAT IS THIS SENSOR QUERY RULE ?
{ ?p :hasRole [ a :PatientRole ] ; :hasLocation ?l ; :hasDiagnosis [ :hasMedicalSymptom [ :hasThreshold [ :hasDataValue ?th ; :isThresholdOnProperty [ a ?prop ] ] ] ] .
?s a :Sensor ; :observes [ a ?prop ] ; :hasLocation ?l .
} => { _:x a :Query ; :pattern :pattern-1 ; :inputVariables (("?prop" ?prop) ("?th" ?th) ("?s" ?s)) ; _:oo a :observation ; :madeBySensor ?s ; :hasResult [ :hasDataValue _:v ] ; :hasSymptom [ a :ThresholdSymptom ; :forProperty [ a ?prop ] ] . } .
44
Representation
- f generic query
:pattern-1 a QueryPattern ; sh:prefixes :prefixes ; sh:construct """ CONSTRUCT { ?o a AboveThresholdAction ; forProperty ?prop . } FROM NAMED WINDOW :win ON <http://idlab.ugent.be/grove> [ RANGE PT1S TUMBLING ] WHERE { WINDOW :win { ?o a Observation ; madeBySensor ?s ; hasResult [ DUL : hasDataValue ?v ] . FILTER (xsd:float(?v) > xsd:float(?th)) } } ORDER BY DESC (?t) LIMIT 1""".
WHAT IS THIS SENSOR QUERY RULE ?
{ ?p :hasRole [ a :PatientRole ] ; :hasLocation ?l ; :hasDiagnosis [ :hasMedicalSymptom [ :hasThreshold [ :hasDataValue ?th ; :isThresholdOnProperty [ a ?prop ] ] ] ] .
?s a :Sensor ; :observes [ a ?prop ] ; :hasLocation ?l .
} => { _:x a :Query ; :pattern :pattern-1 ; :inputVariables (("?prop" ?prop) ("?th" ?th) ("?s" ?s)) ; _:oo a :observation ; :madeBySensor ?s ; :hasResult [ :hasDataValue _:v ] ; :hasSymptom [ a :ThresholdSymptom ; :forProperty [ a ?prop ] ] . } .
45
Representation
- f generic query
HOW ARE THE QUERIES DERIVED ?
Instantiated relevant queries appear in this proof Can be extracted & substituted with additional reasoning steps
46
<#lemma22> a r:Inference; r:gives { _:sk_0 a sd:Query. _:sk_0 sd:pattern ns6:pattern-1. _:sk_0 sd:inputVariables (("?prop" SSNiot:LightIntensity) ("?th" "170.0"^^xsd:float) ("?s" <http://occs.intec.ugent.be/ontology/entity#40-a5-ef-05-a4-a6-A1>)). _:sk_2 a sosa:Observation. _:sk_2 sosa:madeBySensor <http://occs.intec.ugent.be/ontology/entity#40-a5-ef-05-a4-a6-A1>. _:sk_2 sosa:hasResult _:sk_3. _:sk_3 DUL:hasDataValue _:sk_1. _:sk_2 SSNiot:hasSymptom _:sk_4. _:sk_4 a SSNiot:ThresholdSymptom. _:sk_4 ssn:forProperty _:sk_5. _:sk_5 a SSNiot:LightIntensity. }; r:evidence ( <#lemma33> ... <#lemma47> ); r:rule <#lemma48>.
➔
DIVIDE PERFORMANCE EVALUATION
(a) Ontology preprocessing (b) Query derivation
Set-up: 1-person hospital room with concussion patient & 10 sensors ➔ output: 2 queries (for light sensor & sound sensor)
COMPARISON OF DIVIDE WITH REAL-TIME REASONING APPROACHES
Fair comparison: OWL 2 RL reasoning Filtering the same event type Context: 1-person hospital room with concussion patient & 10 sensors
48
(a) DIVIDE (c) Pipe C-SPARQL & RDFox (b) StreamFox
49
COMPARISON OF DIVIDE WITH REAL-TIME REASONING APPROACHES
50
COMPARISON OF DIVIDE WITH REAL-TIME REASONING APPROACHES
DIVIDE APPROACH ON RASPBERRY PI
Context: 1-person hospital room with concussion patient & 10 sensors
SUMMARY
52
SUMMARY
53
No manual query construction & configuration
SUMMARY
54
No manual query construction & configuration Automatic reconfiguration when context changes (~ reduction of # reasoning steps)
SUMMARY
55
No manual query construction & configuration Automatic reconfiguration when context changes (~ reduction of # reasoning steps) Efficient query evaluation
SUMMARY
56
No manual query construction & configuration Automatic reconfiguration when context changes (~ reduction of # reasoning steps) Efficient query evaluation No high-end hardware needed
ADDITIONAL ADVANTAGES OF DIVIDE
57
…
Many IoT devices & sensors
DIVIDE perfectly supports the vision of cascading reasoning & edge computing
ADDITIONAL ADVANTAGES OF DIVIDE
58
Local autonomy & increased responsiveness
…
Many IoT devices & sensors
DIVIDE perfectly supports the vision of cascading reasoning & edge computing
ADDITIONAL ADVANTAGES OF DIVIDE
59
Only relevant data is sent over network ➔ reduced bandwidth usage & delays
…
Many IoT devices & sensors
DIVIDE perfectly supports the vision of cascading reasoning & edge computing
Local autonomy & increased responsiveness
ADDITIONAL ADVANTAGES OF DIVIDE
60
Only perform more complex back-end reasoning for important events ➔ much more scalable Only relevant data is sent over network ➔ reduced bandwidth usage & delays
…
Many IoT devices & sensors
DIVIDE perfectly supports the vision of cascading reasoning & edge computing
Local autonomy & increased responsiveness
ADDITIONAL ADVANTAGES OF DIVIDE
61
Only perform more complex back-end reasoning for important events ➔ much more scalable Flexible privacy management Only relevant data is sent over network ➔ reduced bandwidth usage & delays
…
Many IoT devices & sensors
DIVIDE perfectly supports the vision of cascading reasoning & edge computing
Local autonomy & increased responsiveness
ADDITIONAL ADVANTAGES OF DIVIDE
62
Only perform more complex back-end reasoning for important events ➔ much more scalable Flexible privacy management Only relevant data is sent over network ➔ reduced bandwidth usage & delays
…
Many IoT devices & sensors
DIVIDE perfectly supports the vision of cascading reasoning & edge computing
No synchronization of context data to local filtering devices Local autonomy & increased responsiveness
ADDITIONAL ADVANTAGES OF DIVIDE
63
Only perform more complex back-end reasoning for important events ➔ much more scalable Flexible privacy management Only relevant data is sent over network ➔ reduced bandwidth usage & delays
…
Many IoT devices & sensors
➔ DIVIDE helps in solving the trade-off between reasoning expressivity & data velocity DIVIDE perfectly supports the vision of cascading reasoning & edge computing
No synchronization of context data to local filtering devices Local autonomy & increased responsiveness
ALTERNATIVE USE CASE: TRANSPORT
64
how to organize car traffic flow in a big city
TOPIC
ALTERNATIVE USE CASE: TRANSPORT
65
how to organize car traffic flow in a big city
TOPIC
real-time car movement
STREAMS
ALTERNATIVE USE CASE: TRANSPORT
66
how to organize car traffic flow in a big city
TOPIC
real-time car movement
STREAMS
disruptions, event, time of day/week/year, …
CONTEXT
ALTERNATIVE USE CASE: TRANSPORT
67
how to organize car traffic flow in a big city
TOPIC
real-time car movement
STREAMS
disruptions, event, time of day/week/year, …
CONTEXT
traffic lights ~ query filters
QUERIES
ALTERNATIVE USE CASE: TRANSPORT
68
how to organize car traffic flow in a big city
TOPIC
real-time car movement
STREAMS
disruptions, event, time of day/week/year, …
CONTEXT
traffic lights ~ query filters
QUERIES
Extra challenging because of dependencies between “queries” Note: this is still an idea – needs to be fully worked out
FUTURE RESEARCH STEPS
69
FUTURE RESEARCH STEPS
70
Usability: make the configuration easier and accessible to non-experts
FUTURE RESEARCH STEPS
71
Usability: make the configuration easier and accessible to non-experts Improve framework around key reasoning steps with EYE
FUTURE RESEARCH STEPS
72
Usability: make the configuration easier and accessible to non-experts Improve framework around key reasoning steps with EYE Investigate energy-efficiency on low-end devices
FUTURE RESEARCH STEPS
73
Usability: make the configuration easier and accessible to non-experts Improve framework around key reasoning steps with EYE Investigate energy-efficiency on low-end devices Generalize and implement DIVIDE for other use case domains
FUTURE RESEARCH STEPS
74
Usability: make the configuration easier and accessible to non-experts Improve framework around key reasoning steps with EYE Investigate energy-efficiency on low-end devices Generalize and implement DIVIDE for other use case domains Extend query instantiation to other query parameters (e.g. window)
- ir. Mathias De Brouwer
PhD Student
Ghent University – imec IDLab – Internet Technology and Data Science Lab E mrdbrouw.DeBrouwer@UGent.be T +32 9 331 49 38 M +32 484 97 43 15 http://idlab.ugent.be http://idlab.technology
Ghent University @ugent Ghent University