Machine Learning for Complex Event Recognition Nikos Katzouris 1 , - - PowerPoint PPT Presentation
Machine Learning for Complex Event Recognition Nikos Katzouris 1 , Alexander Artikis 1 , Georgios Paliouras 1 http://cer.iit.demokritos.gr 1 NCSR Demokritos, Athens, Greece Team Members Tenured Researcher: Georgios Paliouras Research
1NCSR Demokritos, Athens, Greece
◮ Georgios Paliouras
◮ Elias Alevizos ◮ Alexander Artikis ◮ Nikos Katzouris ◮ Ioannis Kontopoulos ◮ Vagelis Michelioudakis ◮ Efthimis Tsilionis ◮ Christos Vlassopoulos
◮ http://cer.iit.demokritos.gr
Input ◮ Recognition ◮ Output
Event Recognition System Complex Event Definitions
Simple Events . . . . . . . . . . . .
happensAt(active(id0), 10) holdsAt(coord(id0, 20.88, 11.90), 10) happensAt(active(id1), 10) holdsAt(coord(id1, 22.34, 15.23), 10)
Complex Events . . . . . . . . . . . .
holdsAt(meeting(id0, id1), 11) holdsAt(meeting(id0, id1), 12) holdsAt(meeting(id0, id1), 13)
Input ◮ Recognition ◮ Output
Event Recognition System Complex Event Definitions
Simple Events . . . . . . . . . . . .
initiatedAt(meeting(X, Y ), T) ← happensAt(active(X), T), happensAt(active(Y ), T), holdsAt(close(X, Y , 25), T). terminatedAt(meeting(X, Y ), T) ← happensAt(walking(X), T), not holdsAt(close(X, Y , 25), T). happensAt(active(id0), 10) holdsAt(coord(id0, 20.88, 11.90), 10) happensAt(active(id1), 10) holdsAt(coord(id1, 22.34, 15.23), 10)
Complex Events . . . . . . . . . . . .
holdsAt(meeting(id0, id1), 11) holdsAt(meeting(id0, id1), 12) holdsAt(meeting(id0, id1), 13)
Input ◮ Recognition ◮ Output
Event Recognition System Complex Event Definitions
Simple Events . . . . . . . . . . . .
initiatedAt(meeting(X, Y ), T) ← happensAt(active(X), T), happensAt(active(Y ), T), holdsAt(close(X, Y , 25), T). terminatedAt(meeting(X, Y ), T) ← happensAt(walking(X), T), not holdsAt(close(X, Y , 25), T). happensAt(active(id0), 10) holdsAt(coord(id0, 20.88, 11.90), 10) happensAt(active(id1), 10) holdsAt(coord(id1, 22.34, 15.23), 10)
Complex Events . . . . . . . . . . . .
holdsAt(meeting(id0, id1), 11) holdsAt(meeting(id0, id1), 12) holdsAt(meeting(id0, id1), 13)
Input ◮ Recognition ◮ Output
Event Recognition System Complex Event Definitions
Simple Events . . . . . . . . . . . .
initiatedAt(meeting(X, Y ), T) ← happensAt(active(X), T), happensAt(active(Y ), T), holdsAt(close(X, Y , 25), T). terminatedAt(meeting(X, Y ), T) ← happensAt(walking(X), T), not holdsAt(close(X, Y , 25), T). happensAt(active(id0), 10) holdsAt(coord(id0, 20.88, 11.90), 10) happensAt(active(id1), 10) holdsAt(coord(id1, 22.34, 15.23), 10)
Complex Events . . . . . . . . . . . .
holdsAt(meeting(id0, id1), 11) holdsAt(meeting(id0, id1), 12) holdsAt(meeting(id0, id1), 13)
Input ◮ Recognition ◮ Output
Event Recognition System Complex Event Definitions
Simple Events . . . . . . . . . . . .
initiatedAt(meeting(X, Y ), T) ← happensAt(active(X), T), happensAt(active(Y ), T), holdsAt(close(X, Y , 25), T). terminatedAt(meeting(X, Y ), T) ← happensAt(walking(X), T), not holdsAt(close(X, Y , 25), T). happensAt(active(id0), 10) holdsAt(coord(id0, 20.88, 11.90), 10) happensAt(active(id1), 10) holdsAt(coord(id1, 22.34, 15.23), 10)
Complex Events . . . . . . . . . . . .
holdsAt(meeting(id0, id1), 11) holdsAt(meeting(id0, id1), 12) holdsAt(meeting(id0, id1), 13)
City Transport Management Control Centre
IRM Demonstrator
SENSOR & GEO DATA PROCESSING EVENT RECOGNITION INFORMATION EXTRACTION SENSOR NETWORK RESOURCE DATA & DIGITAL MAPS
Data Communication High-Level Events Low-level Events
Training/ Debriefing Actual Operation Operator Operator Driver Driver
◮ Formal, declarative semantics. ◮ Representation of complex temporal phenomena. ◮ Representation of complex atemporal phenomena. ◮ Very efficient reasoning → RTEC.
◮ Formal, declarative semantics. ◮ Representation of complex temporal phenomena. ◮ Representation of complex atemporal phenomena. ◮ Very efficient reasoning → RTEC. ◮ Direct connections to machine learning → Inductive Logic
◮ Input: ◮ Positive and negative examples. ◮ Background knowledge. ◮ Language bias. ◮ Output: ◮ A logical theory that entails as many positive and as few
initiatedAt(meet(X, Y ), T) ← initiatedAt(meet(X, Y ), T) ← happensAt(active(X), T). initiatedAt(meet(X, Y ), T) ← happensAt(active(X), T), happensAt(inactive(Y ), T). initiatedAt(meet(X, Y ), T) ← happensAt(active(X), T), happensAt(inactive(Y ), T), holdsAt(close(X, Y , 25), T).
initiatedAt(meet(X, Y ), T) ← happensAt(active(X), T), holdsAt(close(X, Y , 25), T).
initiatedAt(meet(X, Y ), T) ← happensAt(inactive(Y ), T).
initiatedAt(meet(X, Y ), T) ← holdsAt(orientation(X, Y , 45), T).
initiatedAt(meet(X, Y ), T) ← happensAt(active(X), T), happensAt(inactive(Y ), T), holdsAt(close(X, Y , 25), T), holdsAt(close(Y , X, 25), T), not happensAt(inactive(X), T), not happensAt(abrupt(X), T), not happensAt(running(X), T), happensAt(inactive(Y ), T), not happensAt(active(Y ), T), not happensAt(running(Y ), T), not happensAt(abrupt(Y ), T), holdsAt(orientation(X, Y , 45), T).
Rules generated by adding one condition from ⊥ to the body of ⊤
Bottom Rule ⊥ : Top Rule ⊤:
initiatedAt(meet(X, Y ), T) ← initiatedAt(meet(X, Y ), T) ← happensAt(active(X), T). initiatedAt(meet(X, Y ), T) ← happensAt(active(X), T), happensAt(inactive(Y ), T). initiatedAt(meet(X, Y ), T) ← happensAt(active(X), T), happensAt(inactive(Y ), T), holdsAt(close(X, Y , 25), T).
initiatedAt(meet(X, Y ), T) ← happensAt(active(X), T), holdsAt(close(X, Y , 25), T).
initiatedAt(meet(X, Y ), T) ← happensAt(inactive(Y ), T).
initiatedAt(meet(X, Y ), T) ← holdsAt(orientation(X, Y , 45), T).
initiatedAt(meet(X, Y ), T) ← happensAt(active(X), T), happensAt(inactive(Y ), T), holdsAt(close(X, Y , 25), T), holdsAt(close(Y , X, 25), T), not happensAt(inactive(X), T), not happensAt(abrupt(X), T), not happensAt(running(X), T), happensAt(inactive(Y ), T), not happensAt(active(Y ), T), not happensAt(running(Y ), T), not happensAt(abrupt(Y ), T), holdsAt(orientation(X, Y , 45), T).
Rules generated by adding one condition from ⊥ to the body of ⊤
Bottom Rule ⊥ : Top Rule ⊤: Ordered by θ-subsumption : If r1r2 then covers(r2, e) ⇒ covers(r1, e) for all examples e
initiatedAt(meet(X, Y ), T) ← initiatedAt(meet(X, Y ), T) ← happensAt(active(X), T). initiatedAt(meet(X, Y ), T) ← happensAt(active(X), T), happensAt(inactive(Y ), T).
initiatedAt(meet(X, Y ), T) ← happensAt(active(X), T), happensAt(inactive(Y ), T), holdsAt(close(X, Y , 25), T).
initiatedAt(meet(X, Y ), T) ← happensAt(active(X), T), holdsAt(close(X, Y , 25), T).
initiatedAt(meet(X, Y ), T) ← happensAt(inactive(Y ), T).
initiatedAt(meet(X, Y ), T) ← happensAt(active(X), T), happensAt(inactive(Y ), T), holdsAt(close(X, Y , 25), T), holdsAt(close(Y , X, 25), T), not happensAt(inactive(X), T), not happensAt(abrupt(X), T), not happensAt(running(X), T), happensAt(inactive(Y ), T), not happensAt(active(Y ), T), not happensAt(running(Y ), T), not happensAt(abrupt(Y ), T), holdsAt(orientation(X, Y , 45), T).
Bottom Rule ⊥ : Top Rule ⊤:
Covered examples
initiatedAt(meet(X, Y ), T) ← initiatedAt(meet(X, Y ), T) ← happensAt(active(X), T).
initiatedAt(meet(X, Y ), T) ← happensAt(active(X), T), happensAt(inactive(Y ), T).
initiatedAt(meet(X, Y ), T) ← happensAt(active(X), T), happensAt(inactive(Y ), T), holdsAt(close(X, Y , 25), T).
initiatedAt(meet(X, Y ), T) ← happensAt(active(X), T), holdsAt(close(X, Y , 25), T).
initiatedAt(meet(X, Y ), T) ← happensAt(inactive(Y ), T).
initiatedAt(meet(X, Y ), T) ← happensAt(active(X), T), happensAt(inactive(Y ), T), holdsAt(close(X, Y , 25), T), holdsAt(close(Y , X, 25), T), not happensAt(inactive(X), T), not happensAt(abrupt(X), T), not happensAt(running(X), T), happensAt(inactive(Y ), T), not happensAt(active(Y ), T), not happensAt(running(Y ), T), not happensAt(abrupt(Y ), T), holdsAt(orientation(X, Y , 45), T).
Bottom Rule ⊥ : Top Rule ⊤:
Covered examples
initiatedAt(meet(X, Y ), T) ← initiatedAt(meet(X, Y ), T) ← happensAt(active(X), T).
initiatedAt(meet(X, Y ), T) ← happensAt(active(X), T), happensAt(inactive(Y ), T).
initiatedAt(meet(X, Y ), T) ← happensAt(active(X), T), happensAt(inactive(Y ), T), holdsAt(close(X, Y , 25), T).
initiatedAt(meet(X, Y ), T) ← happensAt(active(X), T), holdsAt(close(X, Y , 25), T).
initiatedAt(meet(X, Y ), T) ← happensAt(inactive(Y ), T).
initiatedAt(meet(X, Y ), T) ← happensAt(active(X), T), happensAt(inactive(Y ), T), holdsAt(close(X, Y , 25), T), holdsAt(close(Y , X, 25), T), not happensAt(inactive(X), T), not happensAt(abrupt(X), T), not happensAt(running(X), T), happensAt(inactive(Y ), T), not happensAt(active(Y ), T), not happensAt(running(Y ), T), not happensAt(abrupt(Y ), T), holdsAt(orientation(X, Y , 45), T).
Bottom Rule ⊥ : Top Rule ⊤:
Covered examples
initiatedAt(meet(X, Y ), T) ← initiatedAt(meet(X, Y ), T) ← happensAt(active(X), T). initiatedAt(meet(X, Y ), T) ← happensAt(active(X), T), happensAt(inactive(Y ), T).
initiatedAt(meet(X, Y ), T) ← happensAt(active(X), T), happensAt(inactive(Y ), T), holdsAt(close(X, Y , 25), T).
initiatedAt(meet(X, Y ), T) ← happensAt(active(X), T), holdsAt(close(X, Y , 25), T).
initiatedAt(meet(X, Y ), T) ← happensAt(inactive(Y ), T).
initiatedAt(meet(X, Y ), T) ← happensAt(active(X), T), happensAt(inactive(Y ), T), holdsAt(close(X, Y , 25), T), holdsAt(close(Y , X, 25), T), not happensAt(inactive(X), T), not happensAt(abrupt(X), T), not happensAt(running(X), T), happensAt(inactive(Y ), T), not happensAt(active(Y ), T), not happensAt(running(Y ), T), not happensAt(abrupt(Y ), T), holdsAt(orientation(X, Y , 45), T).
Bottom Rule ⊥ : Top Rule ⊤:
Covered examples
◮ Inductive Logic Programming algorithms are batch learners.
◮ Each candidate in the search space is evaluated on the entire
◮ Inductive Logic Programming algorithms are batch learners.
◮ Each candidate in the search space is evaluated on the entire
◮ Online learning:
◮ Examples arrive in a stream. ◮ Each example is “seen” once.
◮ Inductive Logic Programming algorithms are batch learners.
◮ Each candidate in the search space is evaluated on the entire
◮ Online learning:
◮ Examples arrive in a stream. ◮ Each example is “seen” once.
◮ Make decisions from subsets of the stream:
◮ Decisions are optimal “locally”. ◮ Decisions are optimal “globally”... ◮ within an error margin ǫ, ◮ with probability 1−δ.
◮ X is a random variable. ◮ X1, . . . , XN are N independent observations of X’s values. ◮ Let ¯
◮ Let ˆ
◮ X is a random variable. ◮ X1, . . . , XN are N independent observations of X’s values. ◮ Let ¯
◮ Let ˆ
◮ Then:
R1: 0.345 R2: 0.232 R3: 0.145 R4: 0.612 R5: 0.325 Candidate Rules
¯ X − ǫ ≤ ˆ X ≤ ¯ X + ǫ, where ǫ =
2N
Training stream
Find the best candidate across the stream As examples stream in... Monitor ¯ X = scoreBestRule − scoreSecondBestRule Continue until the number N of examples makes ¯ X > ǫ =
2N Then ¯ X − ǫ > 0 ⇒ ˆ X > 0 ⇒ BestRule is indeed the best rule, with probability 1−δ.
R1: 0.345 R2: 0.232 R3: 0.145 R4: 0.612 R5: 0.325 Candidate Rules
¯ X − ǫ ≤ ˆ X ≤ ¯ X + ǫ, where ǫ =
2N
Training stream
Find the best candidate across the stream As examples stream in... Monitor ¯ X = scoreBestRule − scoreSecondBestRule Continue until the number N of examples makes ¯ X > ǫ =
2N Then ¯ X − ǫ > 0 ⇒ ˆ X > 0 ⇒ BestRule is indeed the best rule, with probability 1−δ.
R1: 0.345 R2: 0.232 R3: 0.145 R4: 0.612 R5: 0.325 Candidate Rules
¯ X − ǫ ≤ ˆ X ≤ ¯ X + ǫ, where ǫ =
2N
Training stream
Find the best candidate across the stream As examples stream in... Monitor ¯ X = scoreBestRule − scoreSecondBestRule Continue until the number N of examples makes ¯ X > ǫ =
2N Then ¯ X − ǫ > 0 ⇒ ˆ X > 0 ⇒ BestRule is indeed the best rule, with probability 1−δ.
R1: 0.345 R2: 0.232 R3: 0.145 R4: 0.612 R5: 0.325 Candidate Rules
¯ X − ǫ ≤ ˆ X ≤ ¯ X + ǫ, where ǫ =
2N
Training stream
Find the best candidate across the stream As examples stream in... Monitor ¯ X = scoreBestRule − scoreSecondBestRule Continue until the number N of examples makes ¯ X > ǫ =
2N Then ¯ X − ǫ > 0 ⇒ ˆ X > 0 ⇒ BestRule is indeed the best rule, with probability 1−δ.
initiatedAt(meet(X, Y ), T) score: 0.987 initiatedAt(meet(X, Y ), T) ← happensAt(active(Y ), T). score: 0.312 initiatedAt(meet(X, Y ), T) ← happensAt(active(X), T). score: 0.534 initiatedAt(meet(X, Y ), T) ← holdsAt(close(X, Y , 25), T). score: 0.023 initiatedAt(meet(X, Y ), T) ← happensAt(active(Y ), T), happensAt(active(X), T). score: 0.312 initiatedAt(meet(X, Y ), T) ← happensAt(active(Y ), T), holdsAt(close(X, Y , 25), T). score: 0.312 initiatedAt(meet(X, Y ), T) ← happensAt(active(Y ), T), happensAt(active(X), T), holdsAt(close(X, Y , 25), T). score: 0.312
initiatedAt(meet(X, Y ), T) score: 0.732 initiatedAt(meet(X, Y ), T) ← happensAt(active(Y ), T). score: 0.307 initiatedAt(meet(X, Y ), T) ← happensAt(active(X), T). score: 0.568 initiatedAt(meet(X, Y ), T) ← holdsAt(close(X, Y , 25), T). score: 0.048 initiatedAt(meet(X, Y ), T) ← happensAt(active(Y ), T), happensAt(active(X), T). score: 0.312 initiatedAt(meet(X, Y ), T) ← happensAt(active(Y ), T), holdsAt(close(X, Y , 25), T). score: 0.312 initiatedAt(meet(X, Y ), T) ← happensAt(active(Y ), T), happensAt(active(X), T), holdsAt(close(X, Y , 25), T). score: 0.312
initiatedAt(meet(X, Y ), T) score: 0.216 initiatedAt(meet(X, Y ), T) ← happensAt(active(Y ), T). score: 0.418 initiatedAt(meet(X, Y ), T) ← happensAt(active(X), T). score: 0.632 initiatedAt(meet(X, Y ), T) ← holdsAt(close(X, Y , 25), T). score: 0.122 initiatedAt(meet(X, Y ), T) ← happensAt(active(Y ), T), happensAt(active(X), T). score: 0.312 initiatedAt(meet(X, Y ), T) ← happensAt(active(Y ), T), holdsAt(close(X, Y , 25), T). score: 0.312 initiatedAt(meet(X, Y ), T) ← happensAt(active(Y ), T), happensAt(active(X), T), holdsAt(close(X, Y , 25), T). score: 0.312
initiatedAt(meet(X, Y ), T) score: 0.216 initiatedAt(meet(X, Y ), T) ← happensAt(active(Y ), T). score: 0.418 initiatedAt(meet(X, Y ), T) ← happensAt(active(X), T). score: 0.632 initiatedAt(meet(X, Y ), T) ← holdsAt(close(X, Y , 25), T). score: 0.122 initiatedAt(meet(X, Y ), T) ← happensAt(active(Y ), T), happensAt(active(X), T). score: 0.312 initiatedAt(meet(X, Y ), T) ← happensAt(active(Y ), T), holdsAt(close(X, Y , 25), T). score: 0.312 initiatedAt(meet(X, Y ), T) ← happensAt(active(Y ), T), happensAt(active(X), T), holdsAt(close(X, Y , 25), T). score: 0.312
initiatedAt(meet(X, Y ), T) score: 0.216 initiatedAt(meet(X, Y ), T) ← happensAt(active(Y ), T). score: 0.418 initiatedAt(meet(X, Y ), T) ← happensAt(active(X), T). score: 0.644 initiatedAt(meet(X, Y ), T) ← holdsAt(close(X, Y , 25), T). score: 0.122 initiatedAt(meet(X, Y ), T) ← happensAt(active(Y ), T), happensAt(active(X), T). score: 0.618 initiatedAt(meet(X, Y ), T) ← happensAt(active(Y ), T), holdsAt(close(X, Y , 25), T). score: 0.588 initiatedAt(meet(X, Y ), T) ← happensAt(active(Y ), T), happensAt(active(X), T), holdsAt(close(X, Y , 25), T). score: 0.312
initiatedAt(meet(X, Y ), T) score: 0.216 initiatedAt(meet(X, Y ), T) ← happensAt(active(Y ), T). score: 0.418 initiatedAt(meet(X, Y ), T) ← happensAt(active(X), T). score: 0.689 initiatedAt(meet(X, Y ), T) ← holdsAt(close(X, Y , 25), T). score: 0.122 initiatedAt(meet(X, Y ), T) ← happensAt(active(Y ), T), happensAt(active(X), T). score: 0.656 initiatedAt(meet(X, Y ), T) ← happensAt(active(Y ), T), holdsAt(close(X, Y , 25), T). score: 0.522 initiatedAt(meet(X, Y ), T) ← happensAt(active(Y ), T), happensAt(active(X), T), holdsAt(close(X, Y , 25), T). score: 0.312
initiatedAt(meet(X, Y ), T) score: 0.216 initiatedAt(meet(X, Y ), T) ← happensAt(active(Y ), T). score: 0.418 initiatedAt(meet(X, Y ), T) ← happensAt(active(X), T). score: 0.672 initiatedAt(meet(X, Y ), T) ← holdsAt(close(X, Y , 25), T). score: 0.122 initiatedAt(meet(X, Y ), T) ← happensAt(active(Y ), T), happensAt(active(X), T). score: 0.693 initiatedAt(meet(X, Y ), T) ← happensAt(active(Y ), T), holdsAt(close(X, Y , 25), T). score: 0.503 initiatedAt(meet(X, Y ), T) ← happensAt(active(Y ), T), happensAt(active(X), T), holdsAt(close(X, Y , 25), T). score: 0.312
initiatedAt(meet(X, Y ), T) score: 0.216 initiatedAt(meet(X, Y ), T) ← happensAt(active(Y ), T). score: 0.418 initiatedAt(meet(X, Y ), T) ← happensAt(active(X), T). score: 0.672 initiatedAt(meet(X, Y ), T) ← holdsAt(close(X, Y , 25), T). score: 0.122 initiatedAt(meet(X, Y ), T) ← happensAt(active(Y ), T), happensAt(active(X), T). score: 0.623 initiatedAt(meet(X, Y ), T) ← happensAt(active(Y ), T), holdsAt(close(X, Y , 25), T). score: 0.503 initiatedAt(meet(X, Y ), T) ← happensAt(active(Y ), T), happensAt(active(X), T), holdsAt(close(X, Y , 25), T). score: 0.312
initiatedAt(meet(X, Y ), T) score: 0.216 initiatedAt(meet(X, Y ), T) ← happensAt(active(Y ), T). score: 0.418 initiatedAt(meet(X, Y ), T) ← happensAt(active(X), T). score: 0.672 initiatedAt(meet(X, Y ), T) ← holdsAt(close(X, Y , 25), T). score: 0.122 initiatedAt(meet(X, Y ), T) ← happensAt(active(Y ), T), happensAt(active(X), T). score: 0.693 initiatedAt(meet(X, Y ), T) ← happensAt(active(Y ), T), holdsAt(close(X, Y , 25), T). score: 0.503 initiatedAt(meet(X, Y ), T) ← happensAt(active(Y ), T), happensAt(active(X), T), holdsAt(close(X, Y , 25), T). score: 0.618
OLED
Theory Expan- sion Clause Eval- uation Clause Exa- pansion Clause Pruning EC Axioms holdsAt(F, T + 1) ← initiatedAt(F, T). holdsAt(F, T + 1) ← holdsAt(F, T), not terminatedAt(F, T). Learnt Hypothesis Ht: initiatedAt(moving(X , Y ), T) ← happensAt(walking(X ), T), happensAt(walking(Y ), T), holdsAt(close(X , Y , 34), T). terminatedAt(moving(X , Y ), T) ← happensAt(inactive(X ), T), not holdsAt(close(X , Y , 34), T). Training example It holdsAt(moving(id1 , id2 ), 10) happensAt(walking(id1 ), 9), happensAt(walking(id2 ), 9), holdsAt(close(id1 , id2 , 34), 9), holdsAt(orientation(id1 , id2 , 45), 9) Data Stream/Training Examples
Training example It′ not holdsAt(moving(id1 , id2 ), 20) happensAt(active(id1 ), 19), happensAt(running(id2 ), 19), not holdsAt(close(id1 , id2 , 34), 19), holdsAt(orientation(id1 , id2 , 120), 19)
◮ Activity recognition using a benchmark dataset (CAVIAR).
◮ 28 surveillance videos.
◮ Input: short-term activities per video frame+contextual
◮ walking, active, inactive, running. ◮ coordinates, orientation, occlusion.
◮ Learn concepts for Move and Meet. ◮ 10-fold cross-validation.
◮ Statistical Relational Learning
◮ Logic + Probability ◮ Online structure + parameter learning.
◮ Distributed/parallel learning.
◮ N. Katzouris, A. Artikis and G. Paliouras. Online Learning of
◮ N. Katzouris, A. Artikis and G. Paliouras. Parallel Online
◮ http://github.com/nkatzz/OLED
◮ Common problem in Big data applications:
◮ Incomplete annotation of activities of ‘special significance’. ◮ Providing complete annotation is time-consuming. ◮ The process of complex event pattern construction is hindered.
◮ Solution: Online supervision completion
◮ Use given labels to complete the missing ones as data arrive. ◮ Perform supervised structure learning.
Micro-Batch Dt HappensAt(stop start(228394000), 99) HappensAt(slow motion end(228394000), 99) HappensAt(isInArea(228394000, Area1166285600), 99) HappensAt(velocity lt 2(228394000), 99) InitiatedAt(stopped(228394000), 100) . . . HappensAt(stop start(228394000), 150) HappensAt(slow motion end(228394000), 150) HappensAt(isInArea(228394000, Area1166285600), 150) HappensAt(velocity lt 2(228394000), 150) ? InitiatedAt(stopped(228394000), 150) . . . Micro-Batch Dt+1 HappensAt(stop end(228394000), 200) HappensAt(slow motion start(228394000), 200) HappensAt(leavesArea(228394000, Area1166285600), 200) HappensAt(velocity gt 2(228394000), 200) Terminated(stopped(228394000), 200) . . . HappensAt(stop end(228394000), 220) HappensAt(slow motion start(228394000), 220) HappensAt(leavesArea(228394000, Area1166285600), 220) HappensAt(velocity gt 2(228394000), 200) ? Terminated(stopped(228394000), 220) . . .
. . . . . . . . .
Data Stream/Training Examples Data Partitioning Graph Construction Supervision Completion OSLα
Micro-Batch Dt HappensAt(stop start(228394000), 5) HappensAt(slow motion end(228394000), 5) HappensAt(isInArea(228394000, Area1166285600), 5) HappensAt(velocity lt 2(228394000), 5) InitiatedAt(stopped(228394000), 5) . . . HappensAt(stop end(228394000), 20) HappensAt(slow motion start(228394000), 20) HappensAt(leavesArea(228394000, Area1166285600), 20) HappensAt(velocity gt 2(228394000), 20) ¬InitiatedAt(stopped(228394000), 20) . . . HappensAt(stop start(228394000), 50) HappensAt(slow motion end(228394000), 50) HappensAt(isInArea(228394000, Area1166285600), 50) HappensAt(velocity lt 2(228394000), 50) ? InitiatedAt(stopped(228394000), 50) . . . Training Sequence
Micro-Batch Dt HappensAt(stop start(228394000), 5) HappensAt(slow motion end(228394000), 5) HappensAt(isInArea(228394000, Area1166285600), 5) HappensAt(velocity lt 2(228394000), 5) InitiatedAt(stopped(228394000), 5) . . . HappensAt(stop end(228394000), 20) HappensAt(slow motion start(228394000), 20) HappensAt(leavesArea(228394000, Area1166285600), 20) HappensAt(velocity gt 2(228394000), 20) ¬InitiatedAt(stopped(228394000), 20) . . . HappensAt(stop start(228394000), 50) HappensAt(slow motion end(228394000), 50) HappensAt(isInArea(228394000, Area1166285600), 50) HappensAt(velocity lt 2(228394000), 50) ? InitiatedAt(stopped(228394000), 50) . . . Training Sequence
Micro-Batch Dt HappensAt(stop start(228394000), 5) HappensAt(slow motion end(228394000), 5) HappensAt(isInArea(228394000, Area1166285600), 5) HappensAt(velocity lt 2(228394000), 5) InitiatedAt(stopped(228394000), 5) . . . HappensAt(stop end(228394000), 20) HappensAt(slow motion start(228394000), 20) HappensAt(leavesArea(228394000, Area1166285600), 20) HappensAt(velocity gt 2(228394000), 20) ¬InitiatedAt(stopped(228394000), 20) . . . HappensAt(stop start(228394000), 50) HappensAt(slow motion end(228394000), 50) HappensAt(isInArea(228394000, Area1166285600), 50) HappensAt(velocity lt 2(228394000), 50) ? InitiatedAt(stopped(228394000), 50) . . . Training Sequence HappensAt(stop start(228394000), 5) HappensAt(slow motion end(228394000), 5) HappensAt(isInArea(228394000, Area1166285600), 5) HappensAt(velocity lt 2(228394000), 5) . . . Vertex: InitiatedAt(stopped(228394000), 5)
Micro-Batch Dt HappensAt(stop start(228394000), 5) HappensAt(slow motion end(228394000), 5) HappensAt(isInArea(228394000, Area1166285600), 5) HappensAt(velocity lt 2(228394000), 5) InitiatedAt(stopped(228394000), 5) . . . HappensAt(stop end(228394000), 20) HappensAt(slow motion start(228394000), 20) HappensAt(leavesArea(228394000, Area1166285600), 20) HappensAt(velocity gt 2(228394000), 20) ¬InitiatedAt(stopped(228394000), 20) . . . HappensAt(stop start(228394000), 50) HappensAt(slow motion end(228394000), 50) HappensAt(isInArea(228394000, Area1166285600), 50) HappensAt(velocity lt 2(228394000), 50) ? InitiatedAt(stopped(228394000), 50) . . .
Training Sequence
HappensAt(stop start(228394000), 5) HappensAt(slow motion end(228394000), 5) HappensAt(isInArea(228394000, Area1166285600), 5) HappensAt(velocity lt 2(228394000), 5) . . .
Vertex: InitiatedAt(stopped(228394000), 5)
HappensAt(stop end(228394000), 20) HappensAt(slow motion start(228394000), 20) HappensAt(leavesArea(228394000, Area1166285600), 20) HappensAt(velocity gt 2(228394000), 20) . . .
Vertex: ¬InitiatedAt(stopped(228394000), 20)
Micro-Batch Dt HappensAt(stop start(228394000), 5) HappensAt(slow motion end(228394000), 5) HappensAt(isInArea(228394000, Area1166285600), 5) HappensAt(velocity lt 2(228394000), 5) InitiatedAt(stopped(228394000), 5) . . . HappensAt(stop end(228394000), 20) HappensAt(slow motion start(228394000), 20) HappensAt(leavesArea(228394000, Area1166285600), 20) HappensAt(velocity gt 2(228394000), 20) ¬InitiatedAt(stopped(228394000), 20) . . . HappensAt(stop start(228394000), 50) HappensAt(slow motion end(228394000), 50) HappensAt(isInArea(228394000, Area1166285600), 50) HappensAt(velocity lt 2(228394000), 50) ? InitiatedAt(stopped(228394000), 50) . . .
Training Sequence
HappensAt(stop start(228394000), 5) HappensAt(slow motion end(228394000), 5) HappensAt(isInArea(228394000, Area1166285600), 5) HappensAt(velocity lt 2(228394000), 5) . . .
Vertex: InitiatedAt(stopped(228394000), 5)
HappensAt(stop end(228394000), 20) HappensAt(slow motion start(228394000), 20) HappensAt(leavesArea(228394000, Area1166285600), 20) HappensAt(velocity gt 2(228394000), 20) . . .
Vertex: ¬InitiatedAt(stopped(228394000), 20)
HappensAt(stop start(228394000), 50) HappensAt(slow motion end(228394000), 50) HappensAt(isInArea(228394000, Area1166285600), 50) HappensAt(velocity lt 2(228394000), 50) . . .
Vertex: ? InitiatedAt(stopped(228394000), 50)
HappensAt(stop start(228394000), 5) HappensAt(slow motion end(228394000), 5) HappensAt(isInArea(228394000, Area1166285600), 5) HappensAt(velocity lt 2(228394000), 5) . . . Vertex: InitiatedAt(stopped(228394000), 5) HappensAt(stop end(228394000), 20) HappensAt(slow motion start(228394000), 20) HappensAt(leavesArea(228394000, Area1166285600), 20) HappensAt(velocity gt 2(228394000), 20) . . . Vertex: ¬InitiatedAt(stopped(228394000), 20) HappensAt(stop start(228394000), 50) HappensAt(slow motion end(228394000), 50) HappensAt(isInArea(228394000, Area1166285600), 50) HappensAt(velocity lt 2(228394000), 50) . . . Vertex: ? InitiatedAt(stopped(228394000), 50)
HappensAt(stop start(228394000), 5) HappensAt(slow motion end(228394000), 5) HappensAt(isInArea(228394000, Area1166285600), 5) HappensAt(velocity lt 2(228394000), 5) . . . Vertex: InitiatedAt(stopped(228394000), 5) HappensAt(stop end(228394000), 20) HappensAt(slow motion start(228394000), 20) HappensAt(leavesArea(228394000, Area1166285600), 20) HappensAt(velocity gt 2(228394000), 20) . . . Vertex: ¬InitiatedAt(stopped(228394000), 20) HappensAt(stop start(228394000), 50) HappensAt(slow motion end(228394000), 50) HappensAt(isInArea(228394000, Area1166285600), 50) HappensAt(velocity lt 2(228394000), 50) . . . Vertex: ? InitiatedAt(stopped(228394000), 50)
HappensAt(stop start(228394000), 5) HappensAt(slow motion end(228394000), 5) HappensAt(isInArea(228394000, Area1166285600), 5) HappensAt(velocity lt 2(228394000), 5) . . . Vertex: InitiatedAt(stopped(228394000), 5) HappensAt(stop end(228394000), 20) HappensAt(slow motion start(228394000), 20) HappensAt(leavesArea(228394000, Area1166285600), 20) HappensAt(velocity gt 2(228394000), 20) . . . Vertex: ¬InitiatedAt(stopped(228394000), 20) HappensAt(stop start(228394000), 50) HappensAt(slow motion end(228394000), 50) HappensAt(isInArea(228394000, Area1166285600), 50) HappensAt(velocity lt 2(228394000), 50) . . . Vertex: ? InitiatedAt(stopped(228394000), 50) 0.1
HappensAt(stop start(228394000), 5) HappensAt(slow motion end(228394000), 5) HappensAt(isInArea(228394000, Area1166285600), 5) HappensAt(velocity lt 2(228394000), 5) . . . Vertex: InitiatedAt(stopped(228394000), 5) HappensAt(stop end(228394000), 20) HappensAt(slow motion start(228394000), 20) HappensAt(leavesArea(228394000, Area1166285600), 20) HappensAt(velocity gt 2(228394000), 20) . . . Vertex: ¬InitiatedAt(stopped(228394000), 20) HappensAt(stop start(228394000), 50) HappensAt(slow motion end(228394000), 50) HappensAt(isInArea(228394000, Area1166285600), 50) HappensAt(velocity lt 2(228394000), 50) . . . Vertex: ? InitiatedAt(stopped(228394000), 50) 0.1
HappensAt(stop start(228394000), 5) HappensAt(slow motion end(228394000), 5) HappensAt(isInArea(228394000, Area1166285600), 5) HappensAt(velocity lt 2(228394000), 5) . . . Vertex: InitiatedAt(stopped(228394000), 5) HappensAt(stop end(228394000), 20) HappensAt(slow motion start(228394000), 20) HappensAt(leavesArea(228394000, Area1166285600), 20) HappensAt(velocity gt 2(228394000), 20) . . . Vertex: ¬InitiatedAt(stopped(228394000), 20) HappensAt(stop start(228394000), 50) HappensAt(slow motion end(228394000), 50) HappensAt(isInArea(228394000, Area1166285600), 50) HappensAt(velocity lt 2(228394000), 50) . . . Vertex: ? InitiatedAt(stopped(228394000), 50) 0.1 0.85
HappensAt(stop start(228394000), 5) HappensAt(slow motion end(228394000), 5) HappensAt(isInArea(228394000, Area1166285600), 5) HappensAt(velocity lt 2(228394000), 5) . . . Vertex: InitiatedAt(stopped(228394000), 5) HappensAt(stop end(228394000), 20) HappensAt(slow motion start(228394000), 20) HappensAt(leavesArea(228394000, Area1166285600), 20) HappensAt(velocity gt 2(228394000), 20) . . . Vertex: ¬InitiatedAt(stopped(228394000), 20) HappensAt(stop start(228394000), 50) HappensAt(slow motion end(228394000), 50) HappensAt(isInArea(228394000, Area1166285600), 50) HappensAt(velocity lt 2(228394000), 50) . . . Vertex: ? InitiatedAt(stopped(228394000), 50) 0.1 0.85
HappensAt(stop start(228394000), 5) HappensAt(slow motion end(228394000), 5) HappensAt(isInArea(228394000, Area1166285600), 5) HappensAt(velocity lt 2(228394000), 5) . . . Vertex: InitiatedAt(stopped(228394000), 5) HappensAt(stop end(228394000), 20) HappensAt(slow motion start(228394000), 20) HappensAt(leavesArea(228394000, Area1166285600), 20) HappensAt(velocity gt 2(228394000), 20) . . . Vertex: ¬InitiatedAt(stopped(228394000), 20) HappensAt(stop start(228394000), 50) HappensAt(slow motion end(228394000), 50) HappensAt(isInArea(228394000, Area1166285600), 50) HappensAt(velocity lt 2(228394000), 50) . . . Vertex: ? InitiatedAt(stopped(228394000), 50) 0.1 0.85 0.1
HappensAt(stop start(228394000), 5) HappensAt(slow motion end(228394000), 5) HappensAt(isInArea(228394000, Area1166285600), 5) HappensAt(velocity lt 2(228394000), 5) . . . Vertex: InitiatedAt(stopped(228394000), 5) HappensAt(stop end(228394000), 20) HappensAt(slow motion start(228394000), 20) HappensAt(leavesArea(228394000, Area1166285600), 20) HappensAt(velocity gt 2(228394000), 20) . . . Vertex: ¬InitiatedAt(stopped(228394000), 20) HappensAt(stop start(228394000), 50) HappensAt(slow motion end(228394000), 50) HappensAt(isInArea(228394000, Area1166285600), 50) HappensAt(velocity lt 2(228394000), 50) . . . Vertex: ? InitiatedAt(stopped(228394000), 50) 0.1 0.85 0.1
HappensAt(stop start(228394000), 5) HappensAt(slow motion end(228394000), 5) HappensAt(isInArea(228394000, Area1166285600), 5) HappensAt(velocity lt 2(228394000), 5) . . . Vertex: InitiatedAt(stopped(228394000), 5) HappensAt(stop end(228394000), 20) HappensAt(slow motion start(228394000), 20) HappensAt(leavesArea(228394000, Area1166285600), 20) HappensAt(velocity gt 2(228394000), 20) . . . Vertex: ¬InitiatedAt(stopped(228394000), 20) HappensAt(stop start(228394000), 50) HappensAt(slow motion end(228394000), 50) HappensAt(isInArea(228394000, Area1166285600), 50) HappensAt(velocity lt 2(228394000), 50) . . . Vertex: ? InitiatedAt(stopped(228394000), 50) 0.85
HappensAt(stop start(228394000), 5) HappensAt(slow motion end(228394000), 5) HappensAt(isInArea(228394000, Area1166285600), 5) HappensAt(velocity lt 2(228394000), 5) . . . Vertex: InitiatedAt(stopped(228394000), 5) HappensAt(stop end(228394000), 20) HappensAt(slow motion start(228394000), 20) HappensAt(leavesArea(228394000, Area1166285600), 20) HappensAt(velocity gt 2(228394000), 20) . . . Vertex: ¬InitiatedAt(stopped(228394000), 20) HappensAt(stop start(228394000), 50) HappensAt(slow motion end(228394000), 50) HappensAt(isInArea(228394000, Area1166285600), 50) HappensAt(velocity lt 2(228394000), 50) . . . Vertex: InitiatedAt(stopped(228394000), 50) 0.85
20 40 60 80 0.2 0.4 0.6 0.8 1 % supervision kept F1-score 20 40 60 80 10 20 30 40 % supervision kept Runtime (seconds) 20 40 60 80 100 0.2 0.4 0.6 0.8 1 % supervision kept F1-score 20 40 60 80 100 1 2 3 4 % supervision kept Runtime (minutes) SPLICE-OSLα OSLα incomplete SPLICE-OLED OLED incomplete
20 40 60 80 0.2 0.4 0.6 0.8 1 % supervision kept F1-score 20 40 60 80 10 20 30 40 % supervision kept Runtime (seconds) 20 40 60 80 100 0.2 0.4 0.6 0.8 1 % supervision kept F1-score 20 40 60 80 100 10 20 30 % supervision kept Runtime (minutes) SPLICE-OSLα OSLα incomplete SPLICE-OLED OLED incomplete