Unsupervised Recognition of Interleaved Activities of Daily Living - - PowerPoint PPT Presentation

▶
unsupervised recognition of interleaved activities of
SMART_READER_LITE
LIVE PREVIEW

Unsupervised Recognition of Interleaved Activities of Daily Living - - PowerPoint PPT Presentation

Apr 05, 2024 350 likes •567 views

Unsupervised Recognition of Interleaved Activities of Daily Living through Ontological and Probabilistic Reasoning Gabriele Civitarese Daniele Riboni Timo Sztyler Heiner Stuckenschmidt Univ. of Cagliari Univ. of Milano Univ. of Mannheim

slide-1
SLIDE 1

Unsupervised Recognition of Interleaved Activities of Daily Living through Ontological and Probabilistic Reasoning

IEEE International Conference on Pervasive Computing and Communications 2016 1 14.09.2016

Daniele Riboni

  • Univ. of Cagliari

Italy

Timo Sztyler

  • Univ. of Mannheim

Germany

Gabriele Civitarese

  • Univ. of Milano

Italy

Heiner Stuckenschmidt

  • Univ. of Mannheim

Germany

slide-2
SLIDE 2

MOTIVATION

14.09.2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing 2016 2

slide-3
SLIDE 3

Gabriele Civitarese 14.09.2016 3 IEEE International Conference on Pervasive Computing and Communications 2016

Scenario

Recognizing activities of daily living in a smart-home to support healthcare, home automation, a

  • e idepedet life, …

We rely on unobtrusive sesos …

slide-4
SLIDE 4

Gabriele Civitarese 14.09.2016 4 ACM International Joint Conference on Pervasive and Ubiquitous Computing 2016

State of the Art and Open Issues

Most atiit eogitio sstes el o … acquire expensive labeled data sets

  • ften user/environment-specific

We propose an unsupervised method to recognize complex/interleaved ADLs Based on hybrid ontological – probabilistic reasoning

… supeised-based approaches: unfeasible to enumerate all activity patterns … koledge-based approaches:

slide-5
SLIDE 5

Gabriele Civitarese

Our approach …

5 14.09.2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing 2016

… oeoes daaks of supeised-based approach … elies o seati elatios atiities↔ eets … eogizes iteleaed atiities

derived from ontological reasoning inferred by a probabilistic model not user/environment-speifi, o epesie data set, …

slide-6
SLIDE 6

MODEL AND SYSTEM

14.09.2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing 2016 6

slide-7
SLIDE 7

Gabriele Civitarese 14.09.2016 7 ACM International Joint Conference on Pervasive and Ubiquitous Computing 2016

System overview 1. 2. 3.

Semantic correlation reasoner Semantic integration layer Statistical analysis of events Markov Logic Network (MLN) / MAP Inference

MLN knowledge base Event(se1,et1,t1) semantic correlations

Recognized activity instances

slide-8
SLIDE 8

Gabriele Civitarese 14.09.2016 8 ACM International Joint Conference on Pervasive and Ubiquitous Computing 2016

  • 1. Semantic Correlation Reasoner

Why do we use Ontology (OWL2)? to deie seati oelatios eet tpe ↔ atiit lass

stove silverware_drawer freezer Hot meal 0.5 0.33 0.5 Cold meal 0.0 0.33 0.5 Tea 0.5 0.33 0.0

prepare interact Ontology / Axioms {turn on stove} is a predictive sensor event type for {Prepare hot meal} and {Prepare tea} OWL2 Reasoner infers

PPM Matrix

slide-9
SLIDE 9

Gabriele Civitarese 14.09.2016 9 ACM International Joint Conference on Pervasive and Ubiquitous Computing 2016

  • 2. Statistical Analysis of Events

Input: PPM matrix and temporally ordered events infers most probable activity class for each event allows to define activity boundaries (activity instance candidate) activity instance candidate Events Temporal extension

  • f MLN (MLNNC )

Knowledge Base

Our ontology is translated into the MLNNC model

slide-10
SLIDE 10

Gabriele Civitarese 14.09.2016 10

  • 3. MLN / MAP Inference

Hidden predicates Observed predicates

Event 1: opens freezer (1:00pm) Event 2: turns on stove (1:02pm)

hot meal? cold meal? tea?

ADL

Sensor Event Stove

Hot meal

belong to ADL  0.5: hot meal  0.5: cold meal  0.0: tea Sensor Event Freezer

&

 0.5: hot meal  0.0: cold meal  0.5: tea

slide-11
SLIDE 11

EXPERIMENTS

14.09.2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing 2016 11

slide-12
SLIDE 12

Gabriele Civitarese 14.09.2016 12 IEEE International Conference on Pervasive Computing and Communications 2016

Data Sets

We consider two well-ko data sets …

  • 1. CASAS (controlled environment)
  • 2. SmartFaber (uncontrolled environment)
  • Interleaved ADLs of twenty-one subjects
  • Sensors: movement, water, interaction, door, phone
  • Activities: fill medications dispenser, watch DVD, water plants,

ase the phoe, lea, hoose outfit, …

  • An elderly woman diagnosed with Mild Cognitive Impairment
  • Sensors: magnetic, motion, presence, temperature
  • Atiities: takig ediies, ookig, …

ACM International Joint Conference on Pervasive and Ubiquitous Computing 2016

slide-13
SLIDE 13

Gabriele Civitarese 14.09.2016 13 IEEE International Conference on Pervasive Computing and Communications 2016

CASAS (1/2)

0.6 0.65 0.7 0.75 0.8 0.85 0.9 ac1 ac2 ac3 ac4 ac5 ac6 ac7 ac8 MLNNC (Dataset) MLNNC (Ontology) HMM (related work)

  • Our approach outperforms HMM
  • ntological reasoning is effective

ACM International Joint Conference on Pervasive and Ubiquitous Computing 2016

0.5 1 1.5 2 2.5 3 Delta-Start Delta-Dur

F-Measure Minutes

Candidate Refined

  • Refinement improves boundary precision
slide-14
SLIDE 14

Gabriele Civitarese 14.09.2016 14 ACM International Joint Conference on Pervasive and Ubiquitous Computing 2016

SmartFaber (2/2)

0.6 0.65 0.7 0.75 0.8 0.85 0.9 ac9 ac10 ac11 MLNNC (Dataset) MLNNC (Ontology) Supervised / SmartFarber 5 10 15 20 25 Delta-Start Delta-Dur

Minutes F-Measure

Candidate Refined

  • unsupervised and supervised-based

results are comparable

  • results were penalized by a poor

choice of sensors

slide-15
SLIDE 15

DISCUSSION / FUTURE WORK

14.09.2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing 2016 15

slide-16
SLIDE 16

Gabriele Civitarese 14.09.2016 16 IEEE International Conference on Pervasive Computing and Communications 2016

Discussion

Results with two large datasets of interleaved ADLs were positive, but...

  • … koledge egieeig is euied uild otolog

existing smart-home ontologies can be reused

  • … it is uestioale if oe otolog a oe ee hoe

adaptation/extension should be performed (semi-) automatically

ACM International Joint Conference on Pervasive and Ubiquitous Computing 2016

slide-17
SLIDE 17

Gabriele Civitarese 14.09.2016 17 ACM International Joint Conference on Pervasive and Ubiquitous Computing 2016

Future Work

Ca atie leaig allo to … … fie-tue eistig odels? use’s eioet/haits … eole the otolog aodig to the uet otet? Extensive real-old epeiets should sho … … if ad ho the otolog has to e adapted … hat happes i a ulti-user environment

slide-18
SLIDE 18

THANK YOU FOR YOUR ATTENTION

14.09.2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing 2016 18

slide-19
SLIDE 19

BACKUP SLIDES

14.09.2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing 2016 19

slide-20
SLIDE 20

Gabriele Civitarese 14.09.2016 20 ACM International Joint Conference on Pervasive and Ubiquitous Computing 2016

Semantic Integration Layer

  • collects events data from a sensor network
  • applies preprocessing rules to detect operations

<Event(se1, et1, t1, …, Eetsek,etk,tk)>

fidge doo seso sigaled 1  the opeatio is opeig the fidge Example

slide-21
SLIDE 21

Gabriele Civitarese 14.09.2016 21

MLN Model (detailed)

time-aware inference temporal knowledge-based

Ontological constraints

*PriorProbability (SenEvent, ADL, ActivClass, p) *Event (SenEvent, EventType, Time) *InstanceCandidate (ADL, Start, Stop)

Observed predicates

OccursIn (SenEvent, ADL) InstanceClass (ActivClass, ADL)

Hidden predicates PPM Matrix *PriorProbability Statistical analysis of events *InstanceCandidate / *Event