The Telepathic Phone: Frictionless Activity Recognition from WiFi - - PowerPoint PPT Presentation

▶

Aug 16, 2022 25 likes •256 views

The Telepathic Phone: Frictionless Activity Recognition from WiFi RSSI Stephan Sigg, Ulf Blanke, Gerhard Troester Computer Networks Group, University of Goettingen Wearable Computing Lab, ETH Zurich PerCom 2014, 27.03.2014 Introduction Impact

SLIDE 1

The Telepathic Phone: Frictionless Activity Recognition from WiFi RSSI

Stephan Sigg, Ulf Blanke, Gerhard Troester

Computer Networks Group, University of Goettingen Wearable Computing Lab, ETH Zurich

PerCom 2014, 27.03.2014

SLIDE 2

Introduction Impact on RF Related Work AR on a mobile phone Results Conclusion

Activity recognition ...

2 / 14 The Telepathic Phone: Frictionless Activity Recognition from WiFi RSSI

SLIDE 3

Introduction Impact on RF Related Work AR on a mobile phone Results Conclusion

Activity recognition ...

2 / 14 The Telepathic Phone: Frictionless Activity Recognition from WiFi RSSI

SLIDE 4

Introduction Impact on RF Related Work AR on a mobile phone Results Conclusion

Is there another way?

Scholz, Sigg, Shihskova, von Zengen, Bagshik, Guenther, Beigl, Ji: SenseWaves: Radiowaves for context recognition, in Video Proceedings of Pervasive 2011 3 / 14 The Telepathic Phone: Frictionless Activity Recognition from WiFi RSSI

SLIDE 5

Introduction Impact on RF Related Work AR on a mobile phone Results Conclusion

Recent related work

.

Whole-Home Gesture Recognition Using Wireless Signals, Q. Pu, S. Gupta, S. Gollakota, S. Patel, Mobicom’13 See Through Walls with Wi-Fi!, F Adib, D. Katabi, SIGCOMM’13 4 / 14 The Telepathic Phone: Frictionless Activity Recognition from WiFi RSSI

SLIDE 6

Introduction Impact on RF Related Work AR on a mobile phone Results Conclusion

Can we do this with standard hardware?

RSSI

Passive

5 / 14 The Telepathic Phone: Frictionless Activity Recognition from WiFi RSSI

SLIDE 7

Introduction Impact on RF Related Work AR on a mobile phone Results Conclusion

Measure signal strength on a phone

◮ Approx. 1 sample/sec

6 / 14 The Telepathic Phone: Frictionless Activity Recognition from WiFi RSSI

SLIDE 8

Introduction Impact on RF Related Work AR on a mobile phone Results Conclusion

Measure signal strength on a phone

◮ How to obtain this data on a phone?

◮ root access ◮ Firmware does not support such access 6 / 14 The Telepathic Phone: Frictionless Activity Recognition from WiFi RSSI

SLIDE 9

Introduction Impact on RF Related Work AR on a mobile phone Results Conclusion

Measure signal strength on a phone

6 / 14 The Telepathic Phone: Frictionless Activity Recognition from WiFi RSSI

SLIDE 10

Introduction Impact on RF Related Work AR on a mobile phone Results Conclusion

Measure signal strength on a phone

6 / 14 The Telepathic Phone: Frictionless Activity Recognition from WiFi RSSI

SLIDE 11

Introduction Impact on RF Related Work AR on a mobile phone Results Conclusion

Measure signal strength on a phone

6 / 14 The Telepathic Phone: Frictionless Activity Recognition from WiFi RSSI

SLIDE 12

Introduction Impact on RF Related Work AR on a mobile phone Results Conclusion

Measure signal strength on a phone

raw data

grouped by sender multidimensional

data points

Sample Sender +

raw data still access- ible for visualisation

.pcap Data Point

timespan
feature 1
feature 2, ...

.tab .pic kle .pdf .png

interactive

plot

video

receiver

Radio signal

Capturing Processing Post processing

tcpdump

windowing
feature calculation

matplotlib video analysis

range data

mining toolkit

◮ http://www.stephansigg.de/DeviceFree/pcapTools.tar.gz

6 / 14 The Telepathic Phone: Frictionless Activity Recognition from WiFi RSSI

SLIDE 13

Introduction Impact on RF Related Work AR on a mobile phone Results Conclusion

Sampled RSSI over time

81.4 81.6 81.8 82 82.2 82.4 82.6 82.8 83 83.2 83.4 −95 −94 −93 −92 −91 −90 −89 −88 −87 −86 Time [seconds] RSSI [dBm]

RSSI samples over time

◮ Only use simple time-domain features ◮ Pre-processing?

7 / 14 The Telepathic Phone: Frictionless Activity Recognition from WiFi RSSI

SLIDE 14

Introduction Impact on RF Related Work AR on a mobile phone Results Conclusion

Which sample rate can we expect?

Suburban flat channel

P a c k e t s / s e c 1 2 3 4 5 6 7 8 9 10 11 University (ETH) channel P a c k e t s / s e c 1 2 3 4 5 6 7 8 9 10 11 34.06 0.61 4.45 0.10 53.97 0.14 0.28 0.11 26.46 0.06 0.05

ground 3 r d fl . 14.88 5.02 0.19 0.04 192.29 0.04 0.03 0.07 2.88 0.09 0.02

Dormitory channel

P a c k e t s / s e c 1 2 3 4 5 6 7 8 9 10 11 10.28 10.03 12.13 9.92 9.30 1.77 0.09 0.19 6.92 0.47 0.36 10.45 9.18 9.01 21.91 23.70 22.31 21.34 21.58 0.55 0.62 14.51 Train station channel P a c k e t s / s e c 1 2 3 4 5 6 7 8 9 10 11 0.85 0.35 0.32 0.20 0.85 3.10 2.59 11.85 4.46 2.05 9.61 15.29 8.86 11.06 1.41 2.15 10.99 4.45 1.23 11.09 10.79 23.30 Café in center channel P a c k e t s / s e c 1 2 3 4 5 6 7 8 9 10 11

8 / 14 The Telepathic Phone: Frictionless Activity Recognition from WiFi RSSI

SLIDE 15

Introduction Impact on RF Related Work AR on a mobile phone Results Conclusion

Classification accuracy

9 / 14 The Telepathic Phone: Frictionless Activity Recognition from WiFi RSSI

SLIDE 16

Introduction Impact on RF Related Work AR on a mobile phone Results Conclusion

Classification accuracy

9 / 14 The Telepathic Phone: Frictionless Activity Recognition from WiFi RSSI

SLIDE 17

Introduction Impact on RF Related Work AR on a mobile phone Results Conclusion

Case studies

10 / 14 The Telepathic Phone: Frictionless Activity Recognition from WiFi RSSI

SLIDE 18

Introduction Impact on RF Related Work AR on a mobile phone Results Conclusion

Another pressing issue we solve ...

11 / 14 The Telepathic Phone: Frictionless Activity Recognition from WiFi RSSI

SLIDE 19

Introduction Impact on RF Related Work AR on a mobile phone Results Conclusion

Another pressing issue we solve ...

11 / 14 The Telepathic Phone: Frictionless Activity Recognition from WiFi RSSI

SLIDE 20

Introduction Impact on RF Related Work AR on a mobile phone Results Conclusion

Results

12 / 14 The Telepathic Phone: Frictionless Activity Recognition from WiFi RSSI

SLIDE 21

Introduction Impact on RF Related Work AR on a mobile phone Results Conclusion

Conclusion

◮ Recognition of Activities and Gestures from passive RSSI ◮ On a standard mobile phone ◮ Implementation: Pcap processing for AR (Python)

http://www.stephansigg.de/DeviceFree/pcapTools.tar.gz

13 / 14 The Telepathic Phone: Frictionless Activity Recognition from WiFi RSSI

SLIDE 22

Introduction Impact on RF Related Work AR on a mobile phone Results Conclusion

Conclusion

13 / 14 The Telepathic Phone: Frictionless Activity Recognition from WiFi RSSI

SLIDE 23

Introduction Impact on RF Related Work AR on a mobile phone Results Conclusion

Thank you!

14 / 14 The Telepathic Phone: Frictionless Activity Recognition from WiFi RSSI