Device-Free activity recognition Stephan Sigg Department of - - PowerPoint PPT Presentation

▶
device free activity recognition
SMART_READER_LITE
LIVE PREVIEW

Device-Free activity recognition Stephan Sigg Department of - - PowerPoint PPT Presentation

May 20, 2023 452 likes •931 views

Device-Free activity recognition Stephan Sigg Department of Communications and Networking Aalto University, School of Electrical Engineering stephan.sigg@aalto.fi Bad Worishofen, 10.07.2017 Stephan Sigg July 23, 2017 2 / 42 WiFi

slide-1
SLIDE 1

Device-Free activity recognition

Stephan Sigg

Department of Communications and Networking Aalto University, School of Electrical Engineering stephan.sigg@aalto.fi Bad Worishofen, 10.07.2017

slide-2
SLIDE 2

Stephan Sigg July 23, 2017 2 / 42

slide-3
SLIDE 3

Stephan Sigg July 23, 2017 3 / 42

WiFi Fingerprinting

Seifeldin et. al: Nuzzer: A Large-Scale Device-Free Passive Localization System for Wireless Environments, IEEE TMC 2013 Bong et. al: Reasonable Resolution of Fingerprint Wi-Fi Radio Map for Dense Map Interpolation, FRTA, 2014

slide-4
SLIDE 4

Stephan Sigg July 23, 2017 4 / 42

Seifeldin et. al: Nuzzer: A Large-Scale Device-Free Passive Localization System for Wireless Environments, IEEE TMC 2013

slide-5
SLIDE 5

Stephan Sigg July 23, 2017 5 / 42

slide-6
SLIDE 6

Stephan Sigg July 23, 2017 6 / 42

slide-7
SLIDE 7

Stephan Sigg July 23, 2017 7 / 42

slide-8
SLIDE 8

Stephan Sigg July 23, 2017 8 / 42

Exploiting the RF-channel for environmental preception

◮ Multi-path propagation ◮ Signal superimposition ◮ Scattering ◮ Signal Phase ◮ Reflection ◮ Blocking of signal paths ◮ Doppler Shift ◮ Fresnel effects

slide-9
SLIDE 9

Stephan Sigg July 23, 2017 9 / 42

Aspects of the mobile radio channel

e

j(2π f t +γi)

i

cos( ) ϕ

i

i i

e

j ( +γi) 2π f t γ i 1 ϕ

i

−δ δi

i

ϕ

e

j ( 2π f +γ t ) j ϕ cos( ) G a i n

slide-10
SLIDE 10

Stephan Sigg July 23, 2017 10 / 42

Aspects of the mobile radio channel

Superimposition of RF signals

◮ At a receiver, all incoming signals add up to one

superimposed sum signal

◮ Constructive and destructive interference ◮ Normally: Heavily distorted sum signal

slide-11
SLIDE 11

Stephan Sigg July 23, 2017 11 / 42

Aspects of the mobile radio channel

Superimposition of RF signals

◮ The wireless medium is

a broadcast channel

◮ Multipath transmission

◮ Reflection ◮ Diffraction ◮ Different path lengths ◮ Signal components

arrive at different times

◮ Interference

ζsum =

ι

  • i=1

ℜ

  • ej(fit+γi)
slide-12
SLIDE 12

Stephan Sigg July 23, 2017 12 / 42

RF-based activity recognition

slide-13
SLIDE 13

Stephan Sigg July 23, 2017 13 / 42

slide-14
SLIDE 14

Stephan Sigg July 23, 2017 14 / 42

Time-domain signal strength fluctuation

◮ Recognition of environmental situation (presence,

movement (speed))

◮ Non-intrusive ◮ Arbitrary antenna placement ◮ Pre-training possible ◮ Limited gesture recognition accuracy ◮ Noisy, information source

slide-15
SLIDE 15

Stephan Sigg July 23, 2017 15 / 42

Device-Free recognition (DFL / DFAR)

Time domain features – Situation awareness Frequency domain features – Gesture recognition Fresnel effects DFAR on COTS hardware

slide-16
SLIDE 16

Stephan Sigg July 23, 2017 16 / 42

Aspects of the mobile radio channel

α

Movement direction Receive node Transmit node signal propagation relative speed between transmitter and receiver (v)

Doppler Shift

◮ Frequency of received and transmitted signal may differ ◮ Dependent on relative speed between transmitter and

receiver

◮ fd = v λ · cos(α)

slide-17
SLIDE 17

Stephan Sigg July 23, 2017 17 / 42

Whole-Home Gesture Recognition Using Wireless Signals, Q. Pu, S. Gupta, S. Gollakota, S. Patel, Mobicom’13 See Through Walls with Wi-Fi!, F Adib, D. Katabi, SIGCOMM’13

slide-18
SLIDE 18

Stephan Sigg July 23, 2017 18 / 42

Micro doppler variations

See Through Walls with Wi-Fi!, F Adib, D. Katabi, SIGCOMM’13

slide-19
SLIDE 19

Stephan Sigg July 23, 2017 19 / 42

Micro doppler variations

◮ Recognition of fine-grained gestures ◮ Potentially directional recognition from multiple sources

simultaneously

◮ Binary information (towards/away) ◮ Potentially also speed but noisy ◮ Accuracy dependent on direction of movement (towards

Antenna)

◮ Requires non-standard hardware (e.g. software radios)

slide-20
SLIDE 20

Stephan Sigg July 23, 2017 20 / 42

Device-Free recognition (DFL / DFAR)

Time domain features – Situation awareness Frequency domain features – Gesture recognition Fresnel effects DFAR on COTS hardware

slide-21
SLIDE 21

Stephan Sigg July 23, 2017 21 / 42

slide-22
SLIDE 22

Stephan Sigg July 23, 2017 22 / 42

Human Respiration Detection with Commodity WiFi Devices: Do User Location and Body Orientation Matter?, Wang et al., Ubicomp 2016

slide-23
SLIDE 23

Stephan Sigg July 23, 2017 23 / 42

.

Human Respiration Detection with Commodity WiFi Devices: Do User Location and Body Orientation Matter?, Wang et al., Ubicomp 2016

slide-24
SLIDE 24

Stephan Sigg July 23, 2017 24 / 42

Fresnel effets for DFAR

◮ Fine-grained centimer-scale accuracy ◮ Fragile instrumentation requirements ◮ Requires non-standard hardware (e.g. software radios)

slide-25
SLIDE 25

Stephan Sigg July 23, 2017 25 / 42

Device-Free recognition (DFL / DFAR)

Time domain features – Situation awareness Frequency domain features – Gesture recognition Fresnel effects DFAR on COTS hardware

slide-26
SLIDE 26

Stephan Sigg July 23, 2017 26 / 42

Can we do this with standard hardware?

RSSI

Passive

slide-27
SLIDE 27

Stephan Sigg July 23, 2017 27 / 42

Measure signal strength on a phone

◮ Approx. 1 sample/sec

slide-28
SLIDE 28

Stephan Sigg July 23, 2017 27 / 42

Measure signal strength on a phone

◮ How to obtain this data on a phone?

◮ root access ◮ Firmware does not support such access

slide-29
SLIDE 29

Stephan Sigg July 23, 2017 27 / 42

Measure signal strength on a phone

slide-30
SLIDE 30

Stephan Sigg July 23, 2017 27 / 42

Measure signal strength on a phone

slide-31
SLIDE 31

Stephan Sigg July 23, 2017 27 / 42

Measure signal strength on a phone

slide-32
SLIDE 32

Stephan Sigg July 23, 2017 27 / 42

Measure signal strength on a phone

raw data

grouped by sender multidimensional

data points

Sample Sender +

raw data still access- ible for visualisation

.pcap Data Point

  • timespan
  • feature 1
  • feature 2, ...

.tab .pic kle .pdf .png

interactive

plot

video

receiver

Radio signal

Capturing Processing Post processing

tcpdump

  • windowing
  • feature calculation

matplotlib video analysis

  • range data

mining toolkit

◮ http://www.stephansigg.de/DeviceFree/pcapTools.tar.gz

slide-33
SLIDE 33

Stephan Sigg July 23, 2017 28 / 42

Sampled RSSI over time

81.4 81.6 81.8 82 82.2 82.4 82.6 82.8 83 83.2 83.4 −95 −94 −93 −92 −91 −90 −89 −88 −87 −86 Time [seconds] RSSI [dBm]

RSSI samples over time

◮ Only use simple time-domain features ◮ Pre-processing?

slide-34
SLIDE 34

Stephan Sigg July 23, 2017 29 / 42

Which sample rate can we expect?

Suburban flat channel

Packets/sec 1 2 3 4 5 6 7 8 9 10 11 University (ETH) channel Packets/sec 1 2 3 4 5 6 7 8 9 10 11 34.06 0.61 4.45 0.10 53.97 0.14 0.28 0.11 26.46 0.06 0.05

g r

  • u

n d 3 r d fl . 14.88 5.02 0.19 0.04 192.29 0.04 0.03 0.07 2.88 0.09 0.02

Dormitory channel

Packets/sec 1 2 3 4 5 6 7 8 9 10 11 10.28 10.03 12.13 9.92 9.30 1.77 0.09 0.19 6.92 0.47 0.36 10.45 9.18 9.01 21.91 23.70 22.31 21.34 21.58 0.55 0.62 14.51 Train station channel Packets/sec 1 2 3 4 5 6 7 8 9 10 11 0.85 0.35 0.32 0.20 0.85 3.10 2.59 11.85 4.46 2.05 9.61 15.29 8.86 11.06 1.41 2.15 10.99 4.45 1.23 11.09 10.79 23.30 Café in center channel Packets/sec 1 2 3 4 5 6 7 8 9 10 11

slide-35
SLIDE 35

Stephan Sigg July 23, 2017 30 / 42

Case studies

slide-36
SLIDE 36

Stephan Sigg July 23, 2017 31 / 42

Results

slide-37
SLIDE 37

Stephan Sigg July 23, 2017 32 / 42

Abdelnasser et. al: WiGest: A Ubiquitous WiFi-based Gesture Recognition System, INFOCOM, 2015

slide-38
SLIDE 38

Stephan Sigg July 23, 2017 33 / 42

Abdelnasser et. al: WiGest: A Ubiquitous WiFi-based Gesture Recognition System, INFOCOM, 2015

slide-39
SLIDE 39

Stephan Sigg July 23, 2017 34 / 42

RSSI-based

◮ COTS hardware ◮ Ubiquitously available ◮ low accuracy ◮ dependent on environmental traffic situation

slide-40
SLIDE 40

Stephan Sigg July 23, 2017 35 / 42

CSI-based DFAR

slide-41
SLIDE 41

Stephan Sigg July 23, 2017 36 / 42

The received vector y is expressed in terms of the channel transmission matrix H, the input vector x and noise vector n as y = Hx + n

slide-42
SLIDE 42

Stephan Sigg July 23, 2017 37 / 42

802.11n – CSI

The CSI matrix

The MIMO control field in the 802.11n Management frame (used to manage the exchange of MIMO channel state or transmit beamforming feedback information) contains a CSI cotrol field in which the CSI matrix for all carriers is stored. Example (3x3) – complex amplitude and phase:

slide-43
SLIDE 43

Stephan Sigg July 23, 2017 38 / 42

Open CSI tools

Atheros CSI tool http://pdcc.ntu.edu.sg/wands/Atheros/ Intel 5300 tool https://dhalperi.github.io/linux-80211n-csitool/

slide-44
SLIDE 44

Stephan Sigg July 23, 2017 39 / 42

CSI-based gait recognition

Wang et. al: WiGest: Gait Recognition Using WiFi Signals, Ubicomp, 2016

slide-45
SLIDE 45

Stephan Sigg July 23, 2017 40 / 42

CSI-based

◮ CSI phase fine-grained recognition of movement ◮ Available from COTS hardware ◮ Binary information ◮ Constant after change in distance conducted ◮ Recognition accuracy dependent on direction of movement

wrt Rx antenna

slide-46
SLIDE 46

Stephan Sigg July 23, 2017 41 / 42

Device-Free recognition (DFL / DFAR)

Time domain features – Situation awareness Frequency domain features – Gesture recognition Fresnel effects DFAR on COTS hardware

slide-47
SLIDE 47

Stephan Sigg July 23, 2017 42 / 42

Thank you!

Stephan Sigg stephan.sigg@aalto.fi