Continuous queries Daniele DellAglio dellaglio@ifi.uzh.ch - PowerPoint PPT Presentation

How to Build a Stream Reasoning Application D. Dell'Aglio, E. Della Valle, T. Le-Pham, A. Mileo, and R. Tommasini http://streamreasoning.org/events/streamapp2017 Continuous queries Daniele Dell’Aglio dellaglio@ifi.uzh.ch http://dellaglio.org @dandellaglio

Share, Remix, Reuse — Legally This work is licensed under the Creative Commons  Attribution 3.0 Unported License. Your are free:  to Share — to copy, distribute and transmit the work to Remix — to adapt the work Under the following conditions  Attribution — You must attribute the work by inserting a credits slide stating – These slides are partially based on “ How to Build a Stream Reasoning Application 2017 ” by D. Dell'Aglio, E. Della Valle, T. Le-Pham, A. Mileo, and R. Tommasini available online at http://streamreasoning.org/events/streamapp2017 To view a copy of this license, visit  http://creativecommons.org/licenses/by/3.0/ http://streamreasoning.org/events/streamapp2017

Continuous query evaluation  From SPARQL • One query, one answer • The query is sent after that the data is available  To a continuous query language • One query, multiple answers • The query is registered in the query engine • The registration usually happens before that the data arrives • Real-time responsiveness is usually required http://streamreasoning.org/events/streamapp2017 3

Let’s process the RDF streams!  In literature there are two different main approaches to process streams  Data Stream Management Systems (DSMSs) • Roots in DBMS research • Aggregations and filters  Complex Event Processors (CEPs) • Roots in Discrete Event Simulation • Search of relevant patterns in the stream • Non-equi-join on the timestamps (after, before, etc.)  Current systems implements feature of both of them • EPL (e.g. Esper, ORACLE CEP)  Now we focus on the CQL/STREAM model • Developed in the DSMS research • C-SPARQL (and others) is inspired to this model http://streamreasoning.org/events/streamapp2017 4

Querying data streams – The CQL model Sliding windows Relational algerbra stream-to-relation relation-to-relation Streams Relations … relation-to-stream <s 1 > infinite < s,τ > <s 2 > unbounded finite … sequence bag <s 3 > Stream Relation R(t) Mapping: T  R *Stream operators http://streamreasoning.org/events/streamapp2017 5

CQL extension for querying RDF data streams Sliding windows S2R operators SPARQL operators RDF Mappings Streams R2S operators *Stream operators http://streamreasoning.org/events/streamapp2017 6

Time-based sliding window width slide R2R operator ω β W(ω,β) S 1 S 3 S 6 S 8 S 11 S S 2 S 4 S 5 S 7 S 9 S 10 S 12 t http://streamreasoning.org/events/streamapp2017 7

SPARQL: a quick recap http://streamreasoning.org/events/streamapp2017 8

The query output S2R operators SPARQL operators RDF RDF Streams Mappings R2S operators  Which is the format of the answer?  We can distinguish two cases 1. No R2S operator: the output is a relation (that changes during the time) 2. R2S operator: a stream. – An RDF stream? It depends by the Query Form http://streamreasoning.org/events/streamapp2017 9

No R2S operator: relation a  … b  … [t  1] a  … b  … SELECT ?a ?b … FROM …. a  … b  … [t  3] WHERE …. a  … b  … [t  5] a  … b  … [t  7] RSP bindings queries <… :prop … > [t  1] <… :prop … > CONSTRUCT {?a :prop ?b } <… :prop … > [t  3] FROM …. WHERE …. <… :prop … > [t  5] <… :prop … > [t  7] triples http://streamreasoning.org/events/streamapp2017 10

R2S operator: stream  stream R2S operators … <… :prop … > [t  1] CONSTRUCT RSTREAM {?a :prop ?b } <… :prop … > [t  1] FROM …. <… : prop … > [t  3] WHERE …. RSP <… : prop … > [t  5] query < …: prop … > [t  7] …  Three operators:  Rstream: streams out all data in the last step  Istream: streams out data in the last step that wasn’t on the previous step, i.e. streams out what is new  Dstream: streams out data in the previous step that isn’t in the last step, i.e. streams out what is old http://streamreasoning.org/events/streamapp2017 11

Some existing RSP systems (oversimplified!)  C-SPARQL: RDF Store + Stream processor • Combined architecture RDF Store C-SPARQL continuous translator query results Stream processor  CQELS: Implemented from scratch. Focus on performance • Native + adaptive joins for static-data and streaming data continuous CQELS Native RSP results query http://streamreasoning.org/events/streamapp2017 12

Some existing RSP systems (oversimplified!)  SPARQL stream : Ontology-based stream query answering • Virtual RDF views, using R2RML mappings • SPARQL stream queries over the original data streams. continuous SPARQLStream rewriter DSMS/CEP results query R2RML mappings  Instans: RETE-based evaluation http://streamreasoning.org/events/streamapp2017 13

Classification of existing systems Triple window Union, Join, Time window R2S operator Continuous Aggregates Optional, execution Model Filter ✔ TA-RDF Limited TA-SPARQL ✔ tRDF tSPARQL ✔ ✔ ✔ ✔ RDF Stream Streaming SPARQL ✔ ✔ ✔ ✔ ✔ RDF Stream Rstream C-SPARQL only ✔ ✔ ✔ ✔ ✔ RDF Stream Istream CQELS only (Virtual) RDF ✔ ✔ ✔ ✔ ✔ SPARQLStream Stream ✔ ✔ ✔ RDF Instans Disclaimer: only a partial view http://streamreasoning.org/events/streamapp2017 14

Similar models, similar (not equals!) query languages SELECT ?sensor FROM NAMED STREAM <http://www.cwi.nl/SRBench/observations> [NOW-3 HOURS SLIDE 10 MINUTES] SPARQL stream WHERE { ?observation om-owl:procedure ?sensor ; om-owl:observedProperty weather:WindSpeed ; om-owl:result [ om-owl:floatValue ?value ] . } GROUP BY ?sensor HAVING ( AVG(?value) >= "74"^^xsd:float ) SELECT ?sensor FROM STREAM <http://www.cwi.nl/SRBench/observations> [RANGE 1h STEP 10m] WHERE { C-SPARQL ?observation om-owl:procedure ?sensor ; om-owl:observedProperty weather:WindSpeed ; om-owl:result [ om-owl:floatValue ?value ] . } GROUP BY ?sensor HAVING ( AVG(?value) >= "74"^^xsd:float ) SELECT ?sensor WHERE { STREAM <http://www.cwi.nl/SRBench/observations> [RANGE 10800s SLIDE 600s] { ?observation om-owl:procedure ?sensor ; om-owl:observedProperty weather:WindSpeed ; CQELS om-owl:result [ om-owl:floatValue ?value ] .} } GROUP BY ?sensor HAVING ( AVG(?value) >= "74"^^xsd:float ) http://streamreasoning.org/events/streamapp2017 15

The problem (1) {:alice :isIn :hall} {:alice :isIn :kitchen} {:bob :isIn :hall} {:bob :isIn :kitchen} S e 1 e 2 e 3 e 4 1 3 6 9 t  Where are Alice and Bob, Executi 1° 2° answer when they are together? on answer 1 :hall [6] :kitchen [11]  Let’s consider a tumbling 2 :hall [5] :kitchen [10] window W(ω=β=5) 3 :hall [6] :kitchen [11]  Let’s execute the experiment 4 - [7] - [12] 4 times Which is the correct answer? http://streamreasoning.org/events/streamapp2017 16

The problem (2) {:alice :isIn :hall} {:alice :isIn :kitchen} {:bob :isIn :hall} {:bob :isIn :kitchen} S e 1 e 2 e 3 e 4 1 3 6 9 t CSPARQL CQELS Executi 1° 2° answer Executi 1° 2° answer on answer on answer 1 :hall [6] :kitchen [11] 1 :hall [3] :kitchen [9] 2 :hall [5] :kitchen [10] 2 No answers 3 :hall [6] :kitchen [11] 3 :hall [3] :kitchen [9] 4 - [7] - [12] 4 No answers Which system behaves in the correct way? http://streamreasoning.org/events/streamapp2017 17

Understanding the RSPs  They share similar models, but they behave in different ways  The C-SPARQL, CQELS and SPARQL stream models do not allow to determine in a unique way which should be the answer given the inputs and the query • There are missing parameters (encoded in the implementations)  Why is it important to understand those behaviours? • To assess the correct implementation of the systems • To improve the comprehension of the benchmarking  W3C RDF stream processor community group started to jointly work out a recommendation in 2014  http://www.w3.org/community/rsp/ http://streamreasoning.org/events/streamapp2017 18

The problem (3)  In the context of continuous query answering over RDF streams, how can the behaviour of existing systems be captured, compared and contrasted?  Why do we need it? • Comparison and contrast • Interoperability • Study RDF Stream Processing related problems • Standard RSP query language http://streamreasoning.org/events/streamapp2017 19

RSEP-QL  A reference model that formally defines the semantics of RDF Stream Processing engines Event Pattern detection operators Applications Model to express continous RSEP-QL queries SPARQL BGP evaluation RSP-QL over background data Backgroun RDF Streams d data BGP evaluation over streams http://streamreasoning.org/events/streamapp2017 20

RSEP-QL From SPARQL to RSEP-QL Continuous QF Evaluator ET SE E Q Q Q Q Query Evaluator ( SE , SDS , ET , QF) (E, SDS , ET , QF) (E, SDS, QF) (E, DS, QF) Interface Result DS SDS Formatter Data layer RDF graphs RDF graphs Ans(Q) RDF streams http://streamreasoning.org/events/streamapp2017 21