Aspects of Pervasive Sensing: Perception and Security from ambient - - PowerPoint PPT Presentation

Aspects of Pervasive Sensing: Perception and Security from ambient noise

Stephan Sigg

Department of Communications and Networking Aalto University, School of Electrical Engineering stephan.sigg@aalto.fi TU-BS, 27.04.2017

Stephan Sigg April 27, 2017 2 / 31

Cheap collaboration Radio Vision Security from ambient signals

Stephan Sigg April 27, 2017 3 / 31

Stephan Sigg April 27, 2017 3 / 31

Stephan Sigg April 27, 2017 4 / 31

Feedback-based distributed adaptive beamforming

Stephan Sigg April 27, 2017 5 / 31

Feedback-based distributed adaptive beamforming

◮ Weak multimodal fitness function ◮ Single local=global optimum

e

j(2π f t +γi)

i

cos( ) ϕ

i

i i

e

j ( +γi) 2π f t γ i 1 ϕ

i

−δ δi

i

ϕ

e

j ( 2π f +γ t ) j ϕ cos( ) G a i n

Stephan Sigg April 27, 2017 6 / 31

Stephan Sigg April 27, 2017 6 / 31

Stephan Sigg April 27, 2017 6 / 31

Stephan Sigg April 27, 2017 7 / 31

Feedback-based distributed adaptive beamforming

◮ Weak multimodal fitness function ◮ Single local=global optimum

e

j(2π f t +γi)

i

cos( ) ϕ

i

i i

e

j ( +γi) 2π f t γ i 1 ϕ

i

−δ δi

i

ϕ

e

j ( 2π f +γ t ) j ϕ cos( ) G a i n

Stephan Sigg April 27, 2017 8 / 31

Feedback-based distributed adaptive beamforming

Stephan Sigg April 27, 2017 8 / 31

Feedback-based distributed adaptive beamforming

Stephan Sigg April 27, 2017 8 / 31

Feedback-based distributed adaptive beamforming

Stephan Sigg April 27, 2017 9 / 31

Cheap collaboration Radio Vision Security from ambient signals

Stephan Sigg April 27, 2017 10 / 31

g

Stephan Sigg April 27, 2017 10 / 31

Stephan Sigg April 27, 2017 11 / 31

RF-sensing for environmental perception

◮ Multi-path propagation ◮ Signal superimposition ◮ Scattering ◮ Signal Phase ◮ Reflection ◮ Blocking of signal paths ◮ Doppler Shift ◮ Fresnel effects

Stephan Sigg April 27, 2017 12 / 31

RF-based activity recognition

Sensewaves Video

Stephan Sigg April 27, 2017 13 / 31

RF-based device-free activity recognition

L y i n g empty S t a n d i n g Crawling W a l k i n g

Stephan Sigg April 27, 2017 13 / 31

RF-based device-free activity recognition

L y i n g empty S t a n d i n g Crawling W a l k i n g

Stephan Sigg April 27, 2017 14 / 31

– Video –

Stephan Sigg April 27, 2017 15 / 31

Cheap collaboration Radio Vision Security from ambient signals

Stephan Sigg April 27, 2017 16 / 31

Motivation

6

Stephan Sigg April 27, 2017 16 / 31

Motivation

Trust and proximity

We will use audio as a source of common information in proximity 6

Security from environmental stimuli

Real-time implementation on android mobile phonesa

aStephan Sigg, et al., AdhocPairing: Spontaneous audio-based secure device pairing for Android mobile devices, IWSSI 2012

◮ Hardware noise cancellation on some phones ◮ Hardware originated synchronisation offset

Stephan Sigg April 27, 2017 19 / 31

Audio-based ad-hoc secure pairing1

◮ Use audio to generate secret

key

◮ high Entropy, fuzzy

cryptography, case studies, attack scenarios

Clap Music Snap Speak Whistle 0.5 0.55 0.6 0.65 0.7 0.75 0.8

Hamming distance in created fingerprints (loud audio source in 1.5m and 3m)

Audio sequence class

Median percentage of identical bits in fingerprints Fingerprints created for matching audio samples Fingerprints created for non−matching audio samples 2 4 6 8 10 12 14 16 18 20 0.91 0.93 0.95 0.97 0.99 1.01 Test run Percentage of passed tests

Percentage of tests in one test run that passed at >5% for Kuiper KS p−values

1.01947 (confidence value at α = 0.03) 0.92053 (confidence value at α = 0.03) Only music Only whistle Only snap Only speak Only clap

• 1S. Sigg et al., Secure Communication based on Ambient Audio, IEEE Transactions on Mobile Computing

Stephan Sigg April 27, 2017 20 / 31

Secure pairing from noisy data

possible messages X possible codewords C

x c′

Decoding Encoding

c

C

Stephan Sigg April 27, 2017 21 / 31

Device-to-Device Authentication

Stephan Sigg April 27, 2017 22 / 31

Accelerometer Reading

1 2 3 4 5 6 7 −5 5 Time [s] Acceleration [m/s2]

◮ Accelerometer reading on z-axis only

Stephan Sigg April 27, 2017 23 / 31

Rotated Signal

1 2 3 4 5 6 7 10 20 Time [s] Acceleration [m/s2]

◮ Orientation relative to ground using

Madgwick’s Algorithm

◮ Notice influence of gravity g

z y x g

Stephan Sigg April 27, 2017 24 / 31

Noise-Reduced Signal

1 2 3 4 5 6 7 −5 5 Time [s] Acceleration [m/s2]

◮ Apply a bandpass filter to keep frequencies between 0.5

and 12 Hz

Stephan Sigg April 27, 2017 25 / 31

Gait-Cycle Detection

1 2 3 4 5 6 −5 5 Time [s] Acceleration [m/s2]

◮ Partition data into gait cycles ◮ Resample gait cycles to equal length ◮ Calculate average gait cycle

Stephan Sigg April 27, 2017 26 / 31

Quantization

−5 5 Acceleration [m/s2] −5 5 Acceleration [m/s2] −5 5 Acceleration [m/s2]

Cycle Average Cycle 1 0 0 1

◮ Average gait cycle overlaid on each original gait cycle ◮ 4 bits per cycle

Stephan Sigg April 27, 2017 27 / 31

Quantization

−5 5 Acceleration [m/s2]

a) 1001 0100 1001 1010 1010 1001 0101 0110 b) 1001 0100 1001 1010 1010 1001 0101 0110 c) 0111 1000 1001 0101 1000 1100 1011 1000

◮ Average gait cycle overlaid on each original gait cycle ◮ 4 bits per cycle

Stephan Sigg April 27, 2017 28 / 31

Comparison between Locations

−5 5 Acceleration [m/s2]

forearm: 0111

1000 1001 0101 1000 1100 1011 1000

−5 5 Acceleration [m/s2]

waist: 0110

1000 1001 0001 1001 1001 1100 1010

Stephan Sigg April 27, 2017 29 / 31

Evaluation

I n t r a

• b
• d

y c h e s t f

• r

e a r m h e a d s h i n t h i g h u p p e r a r m w a i s t 0.2 0.4 0.6 0.8 1 Inter-body Similarity

Stephan Sigg April 27, 2017 30 / 31

Cheap collaboration Radio Vision Security from ambient signals

Stephan Sigg April 27, 2017 31 / 31

Thank you!

Stephan Sigg stephan.sigg@aalto.fi