Supervisors : Prof A. L. Ananda , Prof Chan Mun Choon , Prof - - PowerPoint PPT Presentation

Students: Kartik Sankaran†, Guo Xiang Fa†, Minhui Zhu† Supervisors: Prof A. L. Ananda†, Prof Chan Mun Choon†, Prof Li-Shiuan Peh#

† School of Computing, National University of Singapore # Electrical Engineering and Computer Science, Massachusetts Institute of Technology

▪ Introduction ▪ Related work ▪ Motivation ▪ Methodology ▪ Evaluation

2

Smartphones are becoming “intelligent”, silently understanding what the user is doing and helping in tasks Goog

• gle

le Now

• w

http://www.google.com/landing/now/#cards

Cover er App pp

https://www.coverscreen.com/ 3

Key ingredient of this intelligence is “Context-awareness” derived from phone’s sensors Sensor

• r

da data Con

• ntext

t de detection ion Algor

• rit

ithm hm User’s Con

• ntext

Context-Awar ware Apps Intelli llige gent nt beh ehavio ior

Runs al all t the he time, must be low-powe

• wer

http://mobihealthnews.com/26977/google-adds-low-power-step-counting-to-android-4-4/ http://en.wikipedia.org/wiki/Apple_M7 4

Transportation context detection: Phone automatically understands user’s daily commute

https://play.google.com/store/apps/details?id=com.protogeo.moves

Activit ity Di Diaries Traffic fic managem ement nt Redu ducing ing pe peak hou

• ur

waiting tim ing time Urban an Planning ning

5

Accelerometer is the predominant sensor used for transportation context detection

Sensor Nexus 5 Nexus 4 Proximity 12.675 12.675 Rotation Vector 8.65 4.1 Magnetometer 5 5 Linear Accelerometer 3.65 4.1 Gyroscope 3.2 3.6 Significant Motion 0.45 0.5 Accelerometer 0.45 0.5 Light 0.175 0.175 Barometer 0.004 0.003

Relati tively ly low

• w-po

power er sensor

• r

(as reported by Android’s Sensor Manager)

Current nt dr drawn in mA mA Measur ures s acceler leratio ation n (m/sec2) ) in all 3 a axial direction ions

6

Although low-power, accelerometer-based approaches have many problems Accl Data Featur ure e Extractio action Featur ures Supe perv rvise ised d machine ine learning rning Con

• ntext

t classifi sification ation

High h sam ampling rat ate (1 (10 Hz or

• r more)

Statistical Mean, variance, min, max, … Frequency FFT (1 to 5 Hz), … … …

Expensive computat ation

• n

Wai ait an and v d vehi hicl cle detect ction

• n problems

(c (covered lat ater) Position n dependenc nce Extensi sive trai aining

7

We present an altern ernati tive e app pproa

• ach to

transportation context detection using on

• nly barom
• meter

er Me Measur ures s air pr pressure e (mill lliba ibar) ) Altit itude ude (metres)

http://www.hko.gov.hk/education/edu01met/wxobs/pressure/pres-fig2e.jpg http://www.calctool.org/CALC/phys/default/pres_at_alt.png

Present t in Ne Nexus 4/5 /5/6 /6, , Gal alax axy S3/4 /4/5 /5, Gal alax axy No Note 1/2 /2/3 /3/4 /4, man any more!

8

Current uses of smartphone’s barometer

9

Faster er GPS fix Fitne ness ss app pps Weather her pr predi diction tion Floor

• r-change

hange de detection tion

http://cdn.appspirate.com/wp-content/uploads/2013/09/Fix-Android-GPS.jpg http://weather.phillipmartin.info/science_air_pressure.gif https://play.google.com/store/apps/details?id=com.opensignal.weathersignal http://img.howcast.com/thumbnails/506920/36_running_uphill_xxxlarge.jpg

Can an sense even 1 metre ch chan ange in he height ht!

This is the first wor

• rk using on
• nly barom
• meter

er for IDLE, WALKING, and VEHICLE detection Position ion and or d orientatio tation n inde depe pende ndent nt

10

Low

• w sampl

pling ng rate e of

• f 1 Hz

1 Hz Simple ple pr proc

• cessing

ing T er errain ain de depe pende dent nt Not affected d by hand d movem ements nts

Similar ar overal all ac accurac acy to both 26 26% lower energy tha han Google Compar ared with h widely-deploye

• yed

Google le (Accl cl-Phon hone) ) + FMS (G (GPS-Server)

http://developer.android.com/training/location/activity-recognition.html https://fmsurvey.sg/pages/home_mobile

Bar arometer al algo implement nted on Android

Available even on old Android versions 1000+ users in Singapore and Boston

Popular sensors used for context detection: Accelerometer, GPS, Cellular, WiFi [Incel et al. BIONANOSCIENCE’13] GPS

+

Accele lerom

• meter

er Velo loci city y change e rate, e, Stop

• p rate,

e, He Headi ding ng change e rate Wher ere am e am I on

• n earth?

11 http://1.bp.blogspot.com/-96e_AJrC3IU/TbZb-gPt55I/AAAAAAAAKIU/2CAKZ0K1NZc/s1600/sign.jpg

Better er Accuracy

[Zheng et al. UBICOMP’08] [Ryder et al. CSE’09] [Reddy et al. TOSN’10] [Future Mobility Survey TRB’13]

Popular sensors used for context detection: Accelerometer, GPS, Cellular, WiFi [Incel et al. BIONANOSCIENCE’13] Cellul lular/ r/WiFi iFi Beacon

• n

Recep eption ion Ratio Signa gnal l Strengt ngth De Detect t movem ement nt

12 http://article.sapub.org/image/10.5923.j.ijnc.20120204.02_005.gif

Beacon

• n IDs

[Anderson et al. CSTR’06] [Sohn et al. UBICOMP’06] [ParkSense MOBICOM’13] [BeaconPrint UBICOMP’05]

Popular sensors used for context detection: Accelerometer, GPS, Cellular, WiFi [Incel et al. BIONANOSCIENCE’13] Accele lerome

• meter

er Training ning and d classifi sification on ation online line Training ning of

• ffline

ne Classifi sification tion on

• nline

ne Featur ures extract acted d and d fed t d to

• machine

ine learning ning

13

Training ning and d classifi sification o ation offline line

[Siirtola et al. IJIMAI’12] [Hemminki et al. SENSYS’13] [Wang et al. APWCS’10]

28 8 out of f 36 pap apers listed use A Accelerometer

[Reddy et al. TOSN’10] [Future Mobility Survey TRB’13] [Google IO’13] [Gomes et al. MDM’12]

Accelerometer can an do fi fine-gr grai aine ned d vehi hicl cle-mode de detect ction,

• n, use ba

barometer as as lo low-po powe wer r trigger

Consumes 85 mW (excludes base power)

Barometer has mainly been used for floor-change detection Rem emoving ing accele lerom

• meter

er dr drift Stair irs/Ele /Elevat ator

• r

de detection tion Barom

• meter

er

14

Aidi ding ng GPS fix

[Tanigawa et al. WPNC’08] [Vanini et al. PERCOM’13] [Kartik et al. HOTMOBILE’14] [Zhang et al. PLANS’12] [Lester et al. PERVASIVE’06]

Why is barometer advantageous over other sensors?

Sensor Limitations Barometer advantage GPS Lack of indoor/underground coverage High power usage Usable everywhere Ultra-low power Cellular/WiFi Requires dense access points/cellular towers No external infrastructure required Accelerometer Position dependence Training required Classification complexity Affected by hand movements [WAIT detection] Does not work well in “smooth” vehicles Inherently position independent Simple calibration based on terrain Simple processing Unaffected by hand movements Does not depend on vehicle vibrations

15

More details in evaluation Don’t put phone in something air-tight! Causes false positives when used to trigger GPS Depends on where phone placed (hand/bag/pocket) Each user handles phone differently

Power advantage over other sensors

16

Sensor Nexus 5 Nexus 4 Proximity 12.675 12.675 Rotation Vector 8.65 4.1 Magnetometer 5 5 Linear Accelerometer 3.65 4.1 Gyroscope 3.2 3.6 Significant Motion 0.45 0.5 Accelerometer 0.45 0.5 Light 0.175 0.175 Barometer 0.004 0.003

100 00 times low

• wer

er than acceler erom

• meter

er

(as reported by Android’s Sensor Manager)

Current nt dr drawn in mA mA

Power (mW) Increase over base power CPU Asleep 25 x CPU Awake (base) 108 x Accl (2 Hz) 164 51% Accl (10 Hz) 180 67% Accl (20 Hz) 230 112% Baro (1 Hz) 110 2%

Pow

• wer

er con

• nsum

umed d at di differ erent nt sampl pling ing rates (inclu clude des s base e po power er)

Low sam ampling rat ate + simple processing g : L Low power

What question is our paper trying to answer?

17

Accele lerom

• meter

er da data is bo both a b boon

• n and a b

d a bane: e: Mo More e infor

• rma

mation, tion, but but requ quires s com

• mpl

plex x pr proc

• cessing

ing Do Does 1D hei D height t da data pr provide de sufficie cient nt infor

• rmatio

mation n for

• r transpor

porta tation tion con

• ntext

t de detection ion? In other er wor

• rds

ds, is l less da data enou

• ugh?

Understanding the barometer, its strengths and potential sources of error Cross-section ction of

• f

ME MEMS MS barom

• meter

er

18 (image from Stephen Ming-Chang Hou’s PhD thesis, MIT 2012)

• 1. Vibrat

ation

• 2. T

emperat ature

• 3. Instal

allat ation

• n bias

as 4.

• 4. Aging dr

drift

• 5. Sunlight

ht an and d Wind

• 6. Weat

athe her drift (3 (3 mtr in 10 mi min in storms)

Un Unaffect cted Compe mpensat sated Rem emoved at end of produ duction ction line Ver ery y long ter erm m (mo months/ye hs/year ars) s) Ch Chip p protect cted d by phone e casing The e only thing that can affect ct

• ur

r context t detect ction

• n algorit

ithm hm

http://static.sparkfun.com/images/products/09694-04.jpg

19

Chip Phones Temp Sensor? Compensation Oversampling Noise filter LPS331AP (STM) Galaxy S3 Yes Linear (chip) Yes No BMP180 (Bosch) Galaxy N/S4, Nexus 4 Yes Quad (driver) Yes No BMP280 (Bosch) Nexus 5 Yes Quad (driver) Yes Yes

All except Nexus 5 require filtering Helps reduce noise Driver part of AOSP

Weakness sses: : Poor r absolut

• lute

e accuracy acy withou

• ut

t good sea level reference (but that’s ok) Strengt gths hs: : Excellent t relative e accuracy! cy! Sensiti tive e to even 1 1 me metre!

Understanding the barometer, its strengths and potential sources of error

Clarifications in activity definitions: IDLE, WALKING, and VEHICLE VEHI HICLE CLE: : includ udes s motor

• rize

zed a d and nd non

• n-motor
• rize

ized d veh ehicle les s (like e cycling) ing)

20

IDL DLE: : include udes walking ing in same e room

• m
• r
• r f

floor

• r of
• f a b

buildi ding

Several al ap applicat ations (s (such ch as as C Cover, Moves, HeyDay ay) ) use t the hese bas asic s c stat ates Man any ap applicat ations just wan ant to know “is user in transport” or “is user in the same place”

Are e just 3 st states enou

• ugh?

h? Yes!

Less strict and “higher” level definition than accelerometer’s IDLE.

Thi his ID IDLE defi finition n is more useful to ap applicat ations + yields fewer fal alse p positives for “movement”

https://www.coverscreen.com/ http://www.moves-app.com/ http://www.hey.co/

Intuition behind barometer context detection VEHI HICLE CLE: : Rapi pid d ht change, e, mor

• re

e ups ps an and d do downs

21

IDL DLE: : (ide deall lly) y) no

• ht change

Road ads ar are no not perfect ctly flat at. Bar arometer is se sensitive enough h to meas asure the he ch chan ange in he height ht.

WALK LK: g : gradual dual ht changes (except ept stairs/e /ele levat ator)

• r)

Overview of barometer context detection

22

Pre-processing: Filter to remove noise Con

• nver

erting ting pr pressure into

• hei

eight

23 http://hyperphysics.phy-astr.gsu.edu/hbase/kinetic/imgkin/barf2.gif

Filtering removes noisy values (Nexus 5 does this on the chip)

currentHeight = α * sensorHeight + (1 – α) * prevHeight

Jumpdet: Detecting high-speed vehicles or highly-sloped roads

24

“Jump”: more than 0.8 m ht change in 5 se secon

• nds

ds Track k num number ber of

• f jumps

ps in 20 200 secon

• nd

slidi ding ng windo dow w (with h sign) If ratio

• of
• f +

+ve ve to

• -ve

ve jumps ps is 30 to

• 70 %, t

then classify as VEHI HICLE LE

We typically go only up or only down while using stairs/elevator

Difficult to produce “jumps” while walking on ing on the e roa

• ad

d (except ept stairs/e /ele levator/

• r/downhill)

downhill) WALK LKING NG rarely y pr produ duces both +ve ve and d -ve ve jumps ps

Peakdet: Detecting low-speed vehicles or less-sloped roads

25 http://212.126.36.179/~ntrfound/uploads/images/media/road-over-hills.jpg

Slow

• w traffic

ic or

• r f

frequ quent nt stop

• ppi

ping ng can prevent “jumps” Veh ehicle icles s still l see e mor

• re

e ups ps an and d do downs du due e to h

• higher

er spe peed If number er of

• f pe

peaks (size e 1 m) in 20 200 secon

• nd

d slidi ding ng windo dow w is 2 or more, then it’s VEHICLE

Takes more than 200 sec to travel between peaks while walking

Walk detection: Observe gradual changes in height

26

Find nd stdd ddev of

• f he

height ht in 20 200 secon

• nd

d slidi ding ng window. dow. If mor

• re

e than 0.3 metres, , then classify y as W WALK LKING. G.

A lower value cannot differentiate between weather drift and walking Need a reference barometer to remove weather drift

27

Capt pture e da daily com

• mmute

Col

• lle

lect cted d traces from

• m 3 c

cou

• untrie

ies s and d 13 vol

• lunt

nteer ers s (47 hou

• urs transpo

sporta tation) tion)

Compar ared with h widely-deploye

• yed

Google le (Accl cl-Phon hone) ) + FMS (G (GPS-Server)

http://developer.android.com/training/location/activity-recognition.html https://fmsurvey.sg/pages/home_mobile

Update interval of 10 seconds

City Ppl Vehicle Walk Idle Total Singapore 7 6.5 6.4 2.1 15 Boston 6 3.75 7.8 44.4 55.95 China 1 22 1.5 85 108.5

Singapore: FMS, Google, Baro China: Google, Baro Boston: Baro

Baro FMS Google GoogleSmooth Idle 76% 33% 76% 76% Walk 54% 46% 79% 91% Vehicle 81% 90% 31% 34% Overall 69% 68% 56% 62%

Smoothing google output does not help much for vehicle

Idle Walk Vehicle Idle 76% 19% 5% Walk 19% 54% 27% Vehicle 6% 13% 81%

Accuracy comparison with Google and FMS Accuracy

28

Con

• nfusion

sion matrix x for

• r Baro

Overall accuracy similar, each approach has different strengths/weaknesses FMS’s GPS good for VEHICLE detection, poor for IDLE, WALKING Accl good for WALKING and “stationary” IDLE, poor for VEHICLE Baro good for IDLE and VEHICLE, poor for WALKING Due to walking slowly on “flatter” roads Due to latency of long sliding window.

Location dependence: Accuracy in Boston and China Accuracy

29

Singapore Boston China Idle 76% 85% 99% Walk 54% 40% 23% Vehicle 81% 72% 78% Overall 69% 79% 93%

Boston has different terrain, freezing weather (during polar vortex) IDLE and VEHICLE accuracy good, WALKING still poor

Simulation using Google Maps elevation data from 5 places T er errain ain character eristic istics

30

Avg Elevation Change (m) Boston 0.56 (0.66) Singapore 0.69 (0.65) Kansas City 0.84 (0.99) Lausanne 1.04 (1.19) San Francisco 1.05 (1.17)

Height data at 30 m points along 900 km of roads

Avg Peak Distance (m) San Francisco 645 (709) Kansas City 479 (494) Boston 476 (435) Lausanne 395 (536) Singapore 332 (252)

Simulation using Google Maps elevation data from 5 places Accuracy y using g 30,000 000 elevati ation

• n da

data po points

31

Vehicle (50 kmph) Vehicle (25 kmph) Walk (5 kmph) Walk (8 kmph) Kansas City 96% 89% 73% 56% San Francisco 92% 76% 74% 66% Lausanne 84% 79% 58% 50% Singapore 99% 98% 63% 32% Boston 99% 91% 66% 58%

Thresholds are sufficiently robust for different terrains Walk detection not as good, fixed using fusion

Weather dependence: Rainy Rainy ny da day (Singa ingapo pore, e, 28 28 hou

• urs)

32

Accuracy is 96% Weather drift is gradual, follows diurnal cycles Sudden storms can cause WALKING false positives, but only just before the storm

Weather dependence: Windy Windy ndy da day (Singa ingapo pore e Changi beach, 20 20 min du during recor

• rd

d windy dy weather) er)

33

Also tested in front

• f Air-con and fan

Phone casing protects from wind

Journe ney trac aces collect cted d du during record dry/r /rai ainy (S (Singap apore) ) + + winter r weat athe her (B (Boston)

WAIT detection: Barometer v/s Accelerometer Waiting ting at a b a bus st stop

• p,

not walking, ing, ph phon

• ne

e in hand (30 0 min)

34

Barometer is unaffected, accuracy 100% Minor hand movements cause Google’s algo to report UNKNOWN, accuracy 25%

VEHICLE detection: Barometer v/s Accelerometer Subway way in Singap gapor

• re
• n
• n Circle

e Li Line (1 hou

• ur)

35

Barometer is unaffected, accuracy almost 100% Since subway is so smooth, Google reports IDLE most of the time, accuracy 15%

Also observed in Chi hina a trac ace in the he trai ain

Power consumption: Barometer v/s Accelerometer

36

Google runs for 5 seconds every time it is triggered High sampling rate and complex processing increase power

Power consumption: Barometer v/s Accelerometer

37

Barometer runs constantly

Power usag age c can an be r reduced further her with h sensor bat atch ching

Low sampling rate and simple processing reduce power

Power consumption: Barometer v/s Accelerometer Pow

• wer

er meter er measureme ements ts (screen of

• ff, inclu

lude des s base e CPU awake e po power er)

38

Power (mW) CPU Idle 25 CPU Awake 85 Google 120 Baro 88

If exclude base power, reduction is 35 to 3 mW (91% lower energy) Baro consumes 26% lower energy than Google

Fusing Baro and Google algorithms together Accuracy y with Fusion

• n

39

Baro Google Baro + Google Idle 76% 76% 76% Walking 54% 79% 84% Vehicle 81% 31% 73% Overall 69% 56% 78%

Google’s walk detection accuracy is high Combining strengths of both approaches Baro’s vehicle and idle detection is high

▪ Using temperature with barometer ▪ Sensor batching ▪ Integration with FMS App

40

41

42