Online Personalization of Cross-Subjects based Activity Recognition - - PowerPoint PPT Presentation

▶
online personalization of cross subjects based activity
SMART_READER_LITE
LIVE PREVIEW

Online Personalization of Cross-Subjects based Activity Recognition - - PowerPoint PPT Presentation

May 12, 2023 771 likes •1.09k views

Online Personalization of Cross-Subjects based Activity Recognition Models on Wearable Devices Timo Sztyler, Heiner Stuckenschmidt 15.03.2017 Timo Sztyler 1 15.03.2017 Content P ER C OM 2017 1. Motivation 2. Data & Features 3. Methods

slide-1
SLIDE 1

Timo Sztyler, Heiner Stuckenschmidt

Online Personalization of Cross-Subjects based Activity Recognition Models on Wearable Devices

15.03.2017 1 Timo Sztyler

slide-2
SLIDE 2

Timo Sztyler 15.03.2017 2 IEEE International Conference on Pervasive Computing and Communications 2017

Content

  • 1. Motivation
  • 2. Data & Features
  • 3. Methods
  • 1. Online Random Forest
  • 2. Cross-Subjects Activity Recognition
  • 3. Personalization: Online and Active Learning
  • 4. Results
  • 5. Conclusion / Future Work

PERCOM 2017

slide-3
SLIDE 3

Timo Sztyler 15.03.2017 3 IEEE International Conference on Pervasive Computing and Communications 2017

  • 1. Motivation

PERCOM 2017

slide-4
SLIDE 4

Timo Sztyler

PERCOM 2017

15.03.2017 4 IEEE International Conference on Pervasive Computing and Communications 2017

Most of the existing works target subject-specific activity recognition

Motivation

requires training data for each subject is not available immediately behavior changes are often not considered evolving cross-subjects based activity recognition

slide-5
SLIDE 5

Timo Sztyler

PERCOM 2017

15.03.2017 5 IEEE International Conference on Pervasive Computing and Communications 2017

Idea

use online learning to avoid retraining or storing all data

  • 1. Build a cross-subjects activity recognition model

focus on specific groups of people (child vs. elder) reduces data collection and training effort is available at hand

  • 2. Personalize the base model

use active learning to query the user (uncertainty)

slide-6
SLIDE 6

Timo Sztyler 15.03.2017 6 IEEE International Conference on Pervasive Computing and Communications 2017

  • 2. Data & Features

PERCOM 2017

slide-7
SLIDE 7

Timo Sztyler

  • 15 subjects (8 males / 7 females)
  • seven wearable devices / positions
  • chest, forearm, head, shin, thigh,

upper arm, waist

  • acceleration, GPS, gyroscope, light,

magnetic field, and sound level

  • climbing stairs up/down, jumping,

lying, standing, sitting, running, walking

  • each subject performed each activity ≈10 minutes

Data Set

15.03.2017

PERCOM 2017

7 IEEE International Conference on Pervasive Computing and Communications 2017

slide-8
SLIDE 8

Timo Sztyler

PERCOM 2017

15.03.2017

Feature Extraction

Methods Time

Correlation coefficient (Pearson), entropy (Shannon), gravity (roll, pitch), mean, mean absolute deviation, interquartile range (type R-5), kurtosis, median, standard deviation, variance Frequency Energy (Fourier, Parseval), entropy (Fourier, Shannon), DC mean (Fourier)

  • time and frequency-based features
  • gravity-based features (low-pass filter)
  • derive device orientation (roll, pitch)

Previous experiments have shown … … splitting the recorded data into small overlapping segments has been shown to be the best setting.

IEEE International Conference on Pervasive Computing and Communications 2017 8

slide-9
SLIDE 9

Timo Sztyler 15.03.2017 IEEE International Conference on Pervasive Computing and Communications 2017

3.1. Online Random Forest

PERCOM 2017

9

slide-10
SLIDE 10

Timo Sztyler

PERCOM 2017

15.03.2017

Online Random Forest

Considering online mode, the main differences are … bagging (generation of subsamples) growing of the individual trees replace sample with replacement with Poisson(1) Select thresholds and features randomly (Extreme Randomized Forest) Training Sample Prediction k=Poisson (1) k=Poisson (1) . . . Tree #1 Tree #n . . . k-times

IEEE International Conference on Pervasive Computing and Communications 2017 10

slide-11
SLIDE 11

Timo Sztyler 15.03.2017

3.2. Cross-Subjects Activity Recognition

PERCOM 2017

IEEE International Conference on Pervasive Computing and Communications 2017 11

slide-12
SLIDE 12

Timo Sztyler

PERCOM 2017

15.03.2017

Cross-Subjects Activity Recognition (1/2)

Recognition model relies on labeled data of several people expect target person most common approach: leave-one-out Problem: Children and elders walk differently Model only covers most dominant behavior across all people

IEEE International Conference on Pervasive Computing and Communications 2017 12

slide-13
SLIDE 13

Timo Sztyler

PERCOM 2017

15.03.2017

fitness

14 9 6 10 3 1,12 11 5 8,15 2,4,7 13

Cross-Subjects Activity Recognition (2/2)

We aim to build a model that considers physical characteristics … same/similar gender and physique (walking) … similar fitness level (running) We follow a group-based approach … Rely only on specific people …

13 IEEE International Conference on Pervasive Computing and Communications 2017

slide-14
SLIDE 14

Timo Sztyler 15.03.2017

3.3. Personalization: Online and Active Learning

PERCOM 2017

IEEE International Conference on Pervasive Computing and Communications 2017 14

slide-15
SLIDE 15

Timo Sztyler

PERCOM 2017

Active Learning Smoothing classification result Ask User aggregate uncertain recognitions Online Learning update update Body Sensor Network Labeled data set for base model New labeled data set Updatable Model

Personalization: Online and Active Learning

IEEE International Conference on Pervasive Computing and Communications 2017 15 15.03.2017

slide-16
SLIDE 16

Timo Sztyler

PERCOM 2017

15.03.2017

Personalization: Online and Active Learning

Smoothing classification result Online Learning update Updatable Model

Smoothing adjusts the classification result of a single window if it is surrounded by another activity adjusted window is used to update the model focuses on minor classification errors

i i+1 i+2 i-1 i-2

IEEE International Conference on Pervasive Computing and Communications 2017 16

slide-17
SLIDE 17

Timo Sztyler

PERCOM 2017

15.03.2017

Personalization: Online and Active Learning

Active Learning classification result Ask User aggregate uncertain recognitions Online Learning update

New labeled data set

Updatable Model

User-Feedback queries the user regarding uncertain classification results infeasible to ask for a specific window (1 sec) focuses on major classification errors specified a duration of uncertainty query result is a new data set

IEEE International Conference on Pervasive Computing and Communications 2017 17

slide-18
SLIDE 18

Timo Sztyler 15.03.2017

  • 4. Results

PERCOM 2017

IEEE International Conference on Pervasive Computing and Communications 2017 18

slide-19
SLIDE 19

Timo Sztyler

PERCOM 2017

15.03.2017

Cross-Subject Activity Recognition

Class Randomly Leave-one-out Our approach stairs up 0.62 0.66 0.69 stairs down 0.63 0.67 0.69 jumping 0.79 0.88 0.87 lying 0.81 0.83 0.86 standing 0.71 0.73 0.79 sitting 0.59 0.63 0.68 running 0.88 0.90 0.96 walking 0.60 0.67 0.70 avg. 0.69 0.74 0.78

Inspecting the individual activities … static and dynamic perform comparable (~78%) walking and climbing stairs have the lowest rates

IEEE International Conference on Pervasive Computing and Communications 2017 19

slide-20
SLIDE 20

Timo Sztyler

PERCOM 2017

15.03.2017

Personalization (1/3)

Base + Smoothing + User-Feedback + Both static 0.76 0.76 0.79 0.79 dynamic 0.76 0.80 0.86 0.87

  • w. avg.

0.76 0.78 0.83 0.84

Using online and active learning …

  • nline vs. offline learning  lower recognition rate

user-feedback  walking, stairs are mostly resolved smoothing  minor errors decrease rapidly

IEEE International Conference on Pervasive Computing and Communications 2017 20

slide-21
SLIDE 21

Timo Sztyler

PERCOM 2017

15.03.2017

Personalization (2/3)

Watch & Phone Glasses & Phone Class Precision Recall F1 Precision Recall F1 static 0.75 0.73 0.73 0.80 0.80 0.80 dynamic 0.87 0.85 0.86 0.88 0.87 0.87

  • w. avg.

0.81 0.80 0.80 0.84 0.84 0.84

Focusing on interesting combinations …

  • ffline mode: phone and (watch 69% or glasses 72%)

improved significantly, especially walking

IEEE International Conference on Pervasive Computing and Communications 2017 21

slide-22
SLIDE 22

Timo Sztyler

PERCOM 2017

15.03.2017

Personalization (3/3)

Personalization is a continuous process … especially dynamic activities improve significantly most improvement in the first two time intervals first iteration +4%, five iterations +8% number of windows with a low confidence value decrease with each iteration

IEEE International Conference on Pervasive Computing and Communications 2017 22

slide-23
SLIDE 23

Timo Sztyler

PERCOM 2017

15.03.2017

Parameter

Considering different confidence thresholds … Considering a different number of trees… turning point  t=0.5 10 questions  +8% 10 trees vs. 100 trees a smaller forest is more feasible concerning wearable devices

IEEE International Conference on Pervasive Computing and Communications 2017 23

slide-24
SLIDE 24

Timo Sztyler 15.03.2017

  • 5. Conclusion and Future Work

PERCOM 2017

IEEE International Conference on Pervasive Computing and Communications 2017 24

slide-25
SLIDE 25

Timo Sztyler

PERCOM 2017

15.03.2017

Conclusion

Our results show that … … physical characteristics allow to build promising cross-subjects models (78%) … personalization is significantly less effort than creating a labeled data set (10 questions) … personalized model achieves a recognition rate of 84%, for dynamic activities even 87% personalized cross-subjects based models are feasible (online and active learning)

IEEE International Conference on Pervasive Computing and Communications 2017 25

slide-26
SLIDE 26

Timo Sztyler

PERCOM 2017

15.03.2017

Future Work

  • Data Set

We got access to a large data set (~150 people), including vital parameter.

  • User Acceptance (Scenario)

error rate, emotional condition, environment

  • HAR vs. ADL

physical activities are often insufficient

IEEE International Conference on Pervasive Computing and Communications 2017 26

slide-27
SLIDE 27

Timo Sztyler

PERCOM 2017

15.03.2017

Thank you for your attention :)

IEEE International Conference on Pervasive Computing and Communications 2017 27

slide-28
SLIDE 28

Timo Sztyler

PERCOM 2017

15.03.2017

1 2 3 4 5 6 Randomly 0.61 0.69 0.75 0.77 0.79 0.80 Leave-one-out 0.65 0.74 0.79 0.82 0.83 0.85 Our Method 0.68 0.78 0.82 0.85 0.87 0.88

Cross-Subject Activity Recognition

  • ur group-based approach performs better

at least a four-sensor setup is necessary not feasible in a real world scenario We trained a single classifier for each subject …

IEEE International Conference on Pervasive Computing and Communications 2017 28

slide-29
SLIDE 29

Timo Sztyler 15.03.2017

Offline Random Forest

PERCOM 2017

Considering offline mode, typically … … for each tree sample with replacement is applied … at each node features are selected at random … a quality measure is used to determine best split … majority vote over the individual results is applied … after a split samples are propagated to child nodes Training Data S&R #1 S&R #n . . . Tree #1 Tree #n Prediction . . .

29 IEEE International Conference on Pervasive Computing and Communications 2017

slide-30
SLIDE 30

Timo Sztyler

PERCOM 2017

15.03.2017

Online learning enables …

Personalization: Online and Active Learning

Active learning enables … … to delete already seen/processed data/records … to adapt a model to new behavior … to gather the most informative unlabeled data … to weight new information higher (unlearn)

30 IEEE International Conference on Pervasive Computing and Communications 2017