SurroundSense: Mobile Phone Localization via Ambience Fingerprinting - - PowerPoint PPT Presentation

Based on a paper: „SurroundSense: Mobile Phone Localization via Ambience Fingerprinting” and Romit Roy Choudhury’s presentation: http://people.ee.duke.edu/~romit/courses/s10/mat erial/surroundsense.ppt

 Takes advantage of phone’s hardware.  Starting to be popular:

AppStore: 3000 LBAs Android: 500 LBAs

 Find my iphone: provide location of your

phone on computer.

 Geolife: display shopping list when the phone

Is detected near Wal-Mart.

 Microblog: ask user to update blog when

visiting art gallery.

 Starbucks: voucher for a person, who enters

coffee shop.

 Consider information about particular client:

latitude 52.2317028164831644 longtitude 21.005795001983643 vs palace of culture and science in Warsaw.

 Most of the location-based application needs

logical, not physical location.

 Unfortnately, most existing solutions are

physical.

• Gsm
• GPS
• SkyHook
• Google Latitude
• Radar
• …

Physical Location Error

Pizza Hut Starbucks Physical Location Error

Pizza Hut Starbucks Physical Location Error

The dividing-wall problem

 Deus ex machina

 GPS / GSM alone are error prone, but

combined with other sensors, they might produce an unique fingerprint.

SurroundSense

 Multi-dimensional fingerprint

• Based on ambient sound/light/color/movement/WiFi

Starbucks Wall Pizza Hut

QuickTim eﾪ and TIFF (Uncom pressed) are needed to see th

B A C D E

Should Ambiences be Unique Worldwide?

F G H J I L M N O P Q Q R K

Should Ambiences be Unique Worldwide?

B A C D E F G H J I K L M N O P Q Q R

GSM provides macro location (strip mall) SurroundSense refines to Starbucks

 It is unprofitable to have identical businesses

aroud, with the same light, music, color, layout, etc.

 SurroundSense takes advantage of that fact.

+

Ambience Fingerprinting Test Fingerprint Sound

Acc.

Color/Light

WiFi

Logical Location

Matching Fingerprint Database

=

Candidate Fingerprints GSM Macro Location

SurroundSense Architecture

Fingerprints

 Sound: (via phone microphone)  Color: (via phone camera)

Amplitude Values

• 1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1

Normalized Count

0.14 0.12 0.1 0.08 0.06 0.04 0.02

Acoustic fingerprint (amplitude distribution) Color and light fingerprints on HSL space Lightness

1 0.5

Hue

0.5 1 0 0.2 0.4 0.6 0.8 1

Saturation

Fingerprints

 Movement: (via phone accelerometer)

Cafeteria Clothes Store Grocery Store

Static Moving

Fingerprints

 Movement: (via phone accelerometer)

Cafeteria Clothes Store Grocery Store

Static

Queuing Seated

Moving

Fingerprints

 Movement: (via phone accelerometer)

Cafeteria Clothes Store Grocery Store

Static

Pause for product browsing Short walks between product browsing

Moving

Fingerprints

 Movement: (via phone accelerometer)

Cafeteria Clothes Store Grocery Store

Static

Walk more Quicker stops

Moving

Fingerprints

 Movement: (via phone accelerometer)  WiFi: (via phone wireless card)

Cafeteria Clothes Store Grocery Store

Static

ƒ(overheard WiFi APs)

Moving

Discussion

 Time varying ambience  What if phones are in pockets?  Fingerprint Database

Discussion

 Time varying ambience

• Collect ambience fingerprints over different time windows

 What if phones are in pockets?  Fingerprint Database

Discussion

 Time varying ambience

• Collect ambience fingerprints over different time windows

 What if phones are in pockets?

• Use sound/WiFi/movement

 Fingerprint Database

Discussion

 Time varying ambience

• Collect ambience fingerprints over different time windows

 What if phones are in pockets?

• Use sound/WiFi/movement
• Opportunistically take pictures

 Fingerprint Database

Discussion

 Time varying ambience

• Collect ambience fingerprints over different time windows

 What if phones are in pockets?

• Use sound/WiFi/movement
• Opportunistically take pictures

 Fingerprint Database

• War-sensing

Architecture: Filtering & Matching

Candidate Fingerprints

 May vary over time.  May vary over the circumstances.  Much noise.

 Amplitude divided into 100 equal

intervals

 50 positive and 50 negative  Frequency = 8 KHz (8k samples / s)  Normalized – divided by total number of

samples in the recording.

 Filter Metric: Euclidean distance

discard candidate fingerprint if metric > threshold r

 Individual pace  Some people do shopping in hurry  large noise floor

 Totally 10 Samples, 4 times per second.  Two sample states:

• Stationary
• Motion

 support vector machines for detecting each

state (libSVM).

 Discard candidates with different

classification.

static moving

t t R =

0.0 ≤ R ≤ 0.2 sitting 0.2 ≤ R ≤ 2.0 slow browsing 2.0 ≤ R ≤ ∞ speed-walking

 Stationary devices with

unique MAC address

 If they are in neighbourhood…  Recive beacon every 5

seconds.

 M as the union of MAC addresses in f1 and f2  f(m) – how many times MAC address m was

seen.

 Add a large value when m occurs frequently

in both f1 and f2

 Rich diversity across different

locations.

 Uniformity at the same location.  Unique floor.  K-means clustering algorithm

(approximated).

 Pictures in Hue Saturation Lightness space.

 SizeOf(Cij) – number of pixels in cluster Cij  Ti – total number of pixels in Ci  δ(i, j) – centroid distance between ith cluster F1

and jth cluster F2

 The similarity between the fingerprints is a

sum of all pairwise similarities.

Evaluation Methodology

 51 business locations

• 46 in Durham, NC
• 5 in India

 Data collected by 4 people

• 12 tests per location

 Mimicked customer behavior

Evaluation: Per-Cluster Accuracy

Cluster

• No. of Shops

1 2 3 4 5 6 7 8 9 10 4 7 3 7 4 5 5 6 5 5 Accuracy (%) Cluster Localization accuracy per cluster

Limitations and Future Work

 Energy-Efficiency

• Continuous sensing likely to have a large energy draw

 Localization in Real Time

• User’s movement requires time to converge

 Non-business locations

• Ambiences may be less diverse

Ambience can be a great clue about location

Ambient Sound, light, color, movement …

None of the individual sensors good enough Combined they may be unique Uniqueness facilitated by economic incentive Businesses benefit if they are mutually diverse in ambience Ambience diversity helps SurroundSense Current accuracy of 89%

Conclusion