Assistive Wearable Technology Stephan Koster - - PowerPoint PPT Presentation

▶
assistive wearable technology
SMART_READER_LITE
LIVE PREVIEW

Assistive Wearable Technology Stephan Koster - - PowerPoint PPT Presentation

Aug 27, 2022 100 likes •543 views

Assistive Wearable Technology Stephan Koster skoster@student.ethz.ch 21.05.2013 Distributed Systems Seminar 1 Assistive Wearable Technology: Principles Measure Sensors Deduce Activity/Environment Walking? Dancing? Reading? Driving?

slide-1
SLIDE 1

Assistive Wearable Technology

Stephan Koster

skoster@student.ethz.ch

21.05.2013 1 Distributed Systems Seminar

slide-2
SLIDE 2

Assistive Wearable Technology: Principles

Application

Help React Record

Deduce Activity/Environment

Walking? Dancing? Reading? Driving?

Measure

Sensors

Datum 2 Informatik II

slide-3
SLIDE 3

21.05.2013 3 Informatik II

Assistive Wearable Technology

Portability

  • Lightweight
  • Low-Power
  • Unobtrusive

Sensors

  • Cameras
  • Microphones
  • Electrodes
  • Accelerometers
  • Gyros
  • Bending Strips

Applications

  • Traffic
  • Office
  • Home
  • Industry
  • Enhance

Gestures

  • Disabilities
  • Sports
  • Medicine
  • Military
slide-4
SLIDE 4

Assistive Wearable Technology: Papers

  • Wearable EOG Goggles
  • Improving Hearing Aids
  • Activity Tracking in Car Manufacturing

21.05.2013 4 Informatik II

slide-5
SLIDE 5

Wearable EOG goggles

21.05.2013 5 Informatik II

Andreas Bulling, Daniel Roggen, Gerhard Troster Wearable EOG goggles: Seamless sensing and context- awareness in everyday environments

slide-6
SLIDE 6

Assistive Wearable Technology

Portability

  • Lightweight
  • Low-Power
  • Unobtrusive

Sensors

  • Cameras
  • Microphones
  • Electrodes
  • Accelerometers
  • Gyros
  • Bending strips

Applications

  • Traffic
  • Office
  • Home
  • Industry
  • Gestures
  • Disabilities
  • Sports
  • Medicine
  • Military

21.05.2013 6 Informatik II

slide-7
SLIDE 7

Eye tracking: Different Methods

21.05.2013 7 Informatik II

Eye see cam 2

Video based Electrooculography Eye attached reflector

Contact Lens 3 Bulling, 2011

slide-8
SLIDE 8

Electrooculography

  • Eye as electrical dipole
  • Varies with light
  • Electrodes can determine eye movements
  • Only rough directions

21.05.2013 8 Informatik II

slide-9
SLIDE 9

Aside: How Eyes Work

  • Periods of gaze fixed on target (Fixations)
  • Very fast twitch between fixation points (Saccades)
  • “Random” twitches for scanning
  • Blinks

21.05.2013 9 Informatik II

Eye movement when scanning a face 2

slide-10
SLIDE 10

Wearable EOG goggles: Hardware

  • Sensors:
  • 4 Dry electrodes
  • 1 Light sensor
  • 1 Acceleration sensor
  • Data Processing:
  • Signal amplifier in goggles
  • Data processing unit on body

Datum 10 Informatik II

slide-11
SLIDE 11

Eye Tracking: Data Analysis

Datum 11 Informatik II

Classification

Activity Gesture

Event detection

Blinks Fixations Saccades

Preprocessing

Calibration Drift Compensation

Data Acquisition

4 Electrodes per eye Accelerometer Light Sensor

slide-12
SLIDE 12

Experiment: Eye gestures

21.05.2013 12 Informatik II

  • Event detection

generates String

  • Match String to

defined Gestures

slide-13
SLIDE 13

Experiment: Recognize Reading Activity

Datum 13 Informatik II

slide-14
SLIDE 14

Experiment: Recognize Reading Activity

21.05.2013 14 Informatik II

slide-15
SLIDE 15

Other Recognizable Activities

  • Recognize
  • Web browsing
  • Watching video
  • Driving
  • Remembered face  New face

21.05.2013 15 Informatik II

slide-16
SLIDE 16

Wearable EOG Goggles: Conclusions

21.05.2013 16 Informatik II

+ Many activities detectable + Good Precision + Cognitive process visible

  • Glasses,

cables, belt

  • Size limit

to electrodes

  • No “Killer

app”

slide-17
SLIDE 17

Improving Hearing Aids with Wearable Sensors

21.05.2013 17 Informatik II

Bernd Tessendorf, Andreas Bulling, Daniel Roggen, Thomas Stiefmeier, Manuela Feilner, Peter Derleth, and Gerhard Troster Recognition of Hearing Needs from Body and Eye Movements to Improve Hearing Instruments

slide-18
SLIDE 18

Assistive Wearable Technology

Portability

  • Lightweight
  • Low-Power
  • Unobtrusive

Sensors

  • Cameras
  • Microphones
  • Electrodes
  • Accelerometers
  • Gyros
  • Bending strips

Applications

  • Traffic
  • Office
  • Home
  • Industry
  • Gestures
  • Disabilities
  • Sports
  • Medicine
  • Military

21.05.2013 18 Informatik II

slide-19
SLIDE 19

How Hearing Aids Work: Hardware

  • Many microphones
  • Audio filters
  • Several Profiles

21.05.2013 19 Informatik II

slide-20
SLIDE 20

How Hearing Aids Work: Hearing Profiles

  • Speech
  • Speech in noise
  • Noise
  • Music

21.05.2013 20 Informatik II

slide-21
SLIDE 21

Selecting Hearing Profiles

  • Users prefer automatic switching if reliable
  • Decision based on sound only

→ Ambiguity

21.05.2013 21 Informatik II

slide-22
SLIDE 22

Improving Hearing Aids: Additional Sensors

21.05.2013 22 Informatik II

  • 9 Accel+Gyro

units

  • EOG on one

eye

  • Microphone of

he hearing aid

slide-23
SLIDE 23

Improving Hearing Aids: Data Analysis

Apply settings

DSP in hearing aid

Select settings

Directional on? Filters?

SVM classifier (person indep.)

Determine situation

Data Collection + Feature Extraction

Body movement Eye movement Sound

21.05.2013 23 Informatik II

slide-24
SLIDE 24

Improving Hearing Aids: Experiment

  • Artificial situation in lab
  • Overlay standard office noise
  • Alternate scenarios
  • Work
  • Work while others talk
  • Talk to someone

21.05.2013 24 Informatik II

Noise Noise Speech in noise

slide-25
SLIDE 25

Improving Hearing Aids: Experiment

21.05.2013 25 Informatik II

Working in office

slide-26
SLIDE 26

Improving Hearing Aids: Experiment

21.05.2013 26 Informatik II

Conversation in office

slide-27
SLIDE 27

Improving Hearing Aids: Experiment

21.05.2013 27 Informatik II

Trying to concentrate

slide-28
SLIDE 28

Improving Hearing Aids: Experiment Results

21.05.2013 28 Informatik II

slide-29
SLIDE 29

Improving Hearing Aids: Conclusions

21.05.2013 29 Informatik II

+ Solves a real problem + Good Precision

  • Bulky

Sensors + Potential for improvements

  • No Hearing

impaired test subjects

slide-30
SLIDE 30

Activity Tracking in Car Manufacturing

21.05.2013 30 Informatik II Hyundai assembly line 4

Thomas Stiefmeier, Daniel Roggen, Georg Ogris, Paul Lukowicz, Gerhard Troster Wearable Activity Tracking in Car Manufacturing

slide-31
SLIDE 31

Assistive Wearable Technology

Portability

  • Lightweight
  • Low-Power
  • Unobtrusive

Sensors

  • Cameras
  • Microphones
  • Electrodes
  • Accelerometers
  • Gyros
  • Bending strips

Applications

  • Traffic
  • Office
  • Home
  • Industry
  • Gestures
  • Disabilities
  • Sports
  • Medicine
  • Military

21.05.2013 31 Informatik II

slide-32
SLIDE 32

Activity Tracking in Car Manufacturing

  • Training for assembly line: “Learning island”
  • Tracking on the job: Quality control

21.05.2013 32 Informatik II

slide-33
SLIDE 33

Scenario1: Learning Island

  • Theory sessions
  • Specially prepped car
  • Supervisor judges performance

21.05.2013 33 Informatik II

slide-34
SLIDE 34

Scenario1: Learning Island

  • Automatically judge performance
  • Context-sensitive help replaces theory session
  • Specific task: exchange front lamp

21.05.2013 34 Informatik II

slide-35
SLIDE 35

Datum 35 Informatik II

slide-36
SLIDE 36

Learning Island: Sensors

  • Bending strips in

bracelet

  • Accelerometer on

glove

  • RFID reader in

glove

  • Switches on car

21.05.2013 36 Informatik II

slide-37
SLIDE 37

Scenario2: Quality Control

  • Checklist on finished car
  • Doors, hood, trunk etc
  • Write up faults
  • Improvement: direct data entry

21.05.2013 37 Informatik II

slide-38
SLIDE 38

Quality Control: Sensors used

  • Bending strips in

sleeves

  • Accelerometers on

Gloves

  • Position relative to car:

Tags on worker’s shoulder

Datum 38 Informatik II

slide-39
SLIDE 39

Quality Control: In Action

21.05.2013 39 Informatik II

slide-40
SLIDE 40

Activity Tracking In Car Manufacturing: Conclusions

21.05.2013 40 Informatik II

+ Useful for training

  • Vague on

usefulness in checking

  • Switches
  • n Car

needed

  • Task

model not fault tolerant + Sensors not disruptive + Thorough acceptance study

slide-41
SLIDE 41

Is the vision fulfilled?

21.05.2013 41 Informatik II

Eye Tracking with Electrodes Improving Hearing Aids Activity Tracking in Car Manufacturing Helpful? potentially ++ + Unobtrusive?

  • potentially

+ Reliable? + + +

slide-42
SLIDE 42

Discussion

21.05.2013 42 Informatik II

slide-43
SLIDE 43

Sources

1) EyeSee Cam www.eyeseecam.com 2) Image with saccades: http://en.wikipedia.org/wiki/File:Szakkad.jpg 3) Image contact lens: http://en.wikipedia.org/wiki/File:Contact_Lens_Ayala.jpg 4) Image Hyundai plant: http://commons.wikimedia.org/wiki/File:Hyundai_car_assembly_line.jpg

Paper1, EOG goggles: Bulling et al, 2009 http://dl.acm.org/citation.cfm?id=1520468 Paper2, Hearing aids: Tessendorf, Bulling et al, 2011 https://www.andreas- bulling.de/fileadmin/docs/tessendorf11_pervasive.pdf Paper3, Activity tracking: Stiefmeier et al, 2008 http://dl.acm.org/citation.cfm?id=1399121 Additional papers: Bulling et al.: Eye Movement Analysis for Activity Recognition Using Electrooculography (2011) Bulling, Troster: What’s in the Eyes for Context-Awareness? (2011)

21.05.2013 43 Informatik II