Rajin Darko 1 Ca CarT rTel: el: A Distributed Mobile Sensor - - PowerPoint PPT Presentation

Rajin Darko

1

Ca CarT rTel: el: A Distributed Mobile Sensor Computing System

Bret Hull, Vladimir Bychkovsky, Yang Zhang, Kevin Chen, Michel Goraczko, Allen Miu, Eugene Shih, Hari Balakrishnan and Samuel Madden

MIT Computer Science and Artificial Intelligence Laboratory, Cambridge

2

Context

• 1. Motivation
• 2. Overview and Contribution
• 3. ICEDB
• 4. CafNet
• 5. The Portal
• 6. Case Studies
• 7. The Implementation

3

Introduction

 Over 600 milion cars in the world  More than100 sensors per car  Source of sensor data

4

Introduction - What is a CarTel

 CarTel - mobile sensor computing

system designed to collect, process, deliver and visualize data from sensors located on mobile units such as cars, trains...

 It is a mobile embedded computer

connected to a set of sensors.

5

Motivation

 CarTel is motivated by the hypothesis

that an important and emerging category

• f sensor networks is mobile and

involves heterogeneous sensor data

 That motivation comes from “technology

push” and “application pull”

6

Motivation Overview and contribution ICEDB CafNet The Portal Case Studies The implementation

Motivation

 Technology push is rapidly making the

underlying hardware components available

 Application pull generates demands for

such systems.

7

Motivation Overview and contribution ICEDB CafNet The Portal Case Studies The implementation

Motivation

 When sensor equipped computers or

mobile phones are connected to cars or carried by people they form sensor network system, which can sense the environment at the much finer fidelity then any other static sensors

8

Motivation Overview and contribution ICEDB CafNet The Portal Case Studies The implementation

Motivation

9

Motivation Overview and contribution ICEDB CafNet The Portal Case Studies The implementation

Motivation - Example

 If we would want to monitor the delays on

some roads around the city, one solution would be to deploy static sensor network on all the major roads.

 This solution is possible but considering

expansion of backroads that many drivers use this solution would come across as expensive and impractical way of covering roads around the city.

 Better way would be to equip each car with a

GPS sensor to obtain info about traffic delays so we can use that information for route planning and traffic monitoring applications

10

Motivation Overview and contribution ICEDB CafNet The Portal Case Studies The implementation

Motivation – usage of sensors

 Except traffic monitoring, mobile sensors

can be used for other purposes such as

 Environmental monitoring (mobile chemical

and pollution sensors)

 Civil infrastructure monitoring (vibration

sensors for monitoring state of roads)

 Automotive diagnostics (system is obtaining

info from cars on-board sensors, or can detect road rage)

 Geo-imaging  Data muling

11

Motivation Overview and contribution ICEDB CafNet The Portal Case Studies The implementation

Overview and contribution

 CarTel provides a reusable software

platform that can be used to build many mobile sensing applications. Each node is a mobile, embedded, computer coupled to a set of sensors

12

Motivation Overview and contribution ICEDB CafNet The Portal Case Studies The implementation

Overview and contribution

Goals of CarTel technical design:

 Provide a simple programming interface CarTel applications should be as easy to write as standard Web applications, and application developers should not have to deal with distribution

• r mobility.

 Handle large amounts of heterogeneous

sensor data.

CarTel should not constrain sensor data types, and should make it easy to integrate new kinds of sensors, new mobile nodes, and new data types into the system.

13

Motivation Overview and contribution ICEDB CafNet The Portal Case Studies The implementation

Overview and contribution

Goals of CarTel technical design:



Handle intermittent connectivity



CarTel systems connect to the network using different types of opportunistic wireless (Wi-Fi, Bluetooth..)



Internet users in their homes and other locations can allow CarTel nodes to communicate with Internet hosts.



In most cases those networks will go from good connectivity to no connectivity at all, thus making those connection intermittent for some period of time.



CarTel node can also use other mobile devices, such as mobile phones, or usb keys and flash memory to deliver data as mules

14

Motivation Overview and contribution ICEDB CafNet The Portal Case Studies The implementation

Overview and Contribution

• CarTel Components

The system is designed to have three main components:

• 1. The portal - central location that hosts

CarTel applications and functions as the point of control and configuration for the distributed system.

 It’s a software that requests data from

remote nodes, and aggregates reports arriving from those nodes into a coherent picture of current conditions, and visualizes that data.

15

Motivation Overview and contribution ICEDB CafNet The Portal Case Studies The implementation

Overview and Contribution

• CarTel Components

2.

ICEDB – intermittently connected database - a delay-tolerant continuous query processor, and

3.

CafNet - carry-and-forward network, a delay- tolerant network stack.

 The CarTel programming model is centralized

and simple.

 Applications running on the portal issue

continuous queries using an API exported by

• ICEDB. These queries cause mobile nodes to

stream responses using CafNet’s data delivery mechanism.

16

Motivation Overview and contribution ICEDB CafNet The Portal Case Studies The implementation

Overview and Contribution

• CarTel Components

1.

Queries specify:



What sensor data must be acquired



At what rate should they be acquired



How the data should be sub-sampled, filtered and summarized on the mobile node



On what order should the results be prioritized for returning back to the portal.

The ICEDB continuous query interface allows an application to express intra and inter query priorities.

2.

Query results stream in across an intermittently connected network and populate a relational database at the portal.

3.

Application issue SQL queries on the portal’s relational database to retrieve data they need for further analysis, visualization, etc. These are snapshot queries that run on whatever data is currently available.

17

Motivation Overview and contribution ICEDB CafNet The Portal Case Studies The implementation

Overview and Contribution

• Contributions

 CarTel handles heterogeneous sensor data,

allowing the set of sensors to be expanded without requiring major software changes on the remote nodes.

 Each sensor has an adapter running on the

node

 Adapter handles

 the details of configuring  extracting information from that sensor  converting it into a normalized form.

 To ease management and deployment, when

a new sensor is added, or when the functions

• f an adapter need to be modified, only the

adapter module needs to change.

18

Motivation Overview and contribution ICEDB CafNet The Portal Case Studies The implementation

ICEDB - Introduction

 ICEDB distributes query execution and result

delivery between the ICEDB server on the portal and the remote nodes.

 The ICEDB server holds a list of continuous queries,

submitted by applications, that are sent to the remote nodes using CafNet

 The nodes in the field run ICEDB remote, to process

the sensor data and return the query results using CafNet, prioritizing the result streams in order of importance.

 Finally, when ICEDB server receives results from

remote nodes, it places them into a per-query result table in the relational database at the portal.

19

Motivation Overview and contribution ICEDB CafNet The Portal Case Studies The implementation

ICEDB – Data model

 ICEDB supports heterogeneous data types  Makes the addition and removal of sensors relatively

easy.

 Because all results are eventually stored in a

relational database, this requirement implies that the system

 must be able to parse and store information that sensors

produce and

 must be able to evolve schemas as users add new

sensors and application developers introduce new data types.

 Adapter is a meta-data package used for adding new

sensor types and managing schemas

 Adapter has attributes of a sensor and executable

program that interfaces with a sensors and triggers data collection

20

Motivation Overview and contribution ICEDB CafNet The Portal Case Studies The implementation

ICEDB – Data model

 Purpose of attributes is to provide

information to ICEDB to:

1.

automatically create local tables to store sensor readings

2.

acquire tuples from the sensor, and

3.

parse sensor readings to store them in the database and process them as specified by subsequent continuous queries.

21

Motivation Overview and contribution ICEDB CafNet The Portal Case Studies The implementation

ICEDB – Data model



A typical adapter includes the following attributes:

1.

ID and name: Each adapter must be uniquely identified.

2.

Type: An adapter can either push data to ICEDB, or ICEDB can pull data by invoking the executable at a specified rate.



ICEDB invokes the executable for each push adapter once, so that it runs as a daemon in the background.



Pull adapters produce readings each time ICEDB invokes the executable program. 3.

Rate: For pull type adapters, the rate at which ICEDB should invoke the executable.

4.

Forwarding flag: Specifies whether or not raw data should be forwarded to the portal. Setting this flag is a shortcut for specifying a continuous query that selects all attributes from the sensor.

5.

Schema: A list of (name, type) pairs that specifies the attributes produced by the sensor. In our current implementation, the type field must be a valid PostgreSQL data type.

6.

Priority: The priority assigned to forwarded tuples when the forwarding flag is set



When attributes are defined, adapters reside inside the ICEDB server on the portal and are pushed out to remote nodes using CafNet

22

Motivation Overview and contribution ICEDB CafNet The Portal Case Studies The implementation

ICEDB – Continuous Query Model

 Queries in ICEDB are written in SQL with several

extensions for continuous queries and prioritization.

 These queries are run over data as it is produced

by the adapters. To support continuous queries in ICEDB, queries include a sample rate specified by a keyword RATE

SELECT carid,traceid,time,location FROM gps

WHERE gps.time BETWEEN now()-1 mins AND now() RATE 5 mins

23

Motivation Overview and contribution ICEDB CafNet The Portal Case Studies The implementation

ICEDB – Continuous Query Model - Prioritization

 A “Score” is assigned to each result by the ICEDB query

language, thus prioritizing data for delivery.

 Local prioritization produces scorings of data that can

dynamically change over time based on other data present at the local node. Local prioritization is limited because it cannot receive feedback from the portal, which has a global view of the data and can make better choice of how the data should be prioritized.

 Global prioritization is a scoring of results by getting

feedbacks from the portal. In order to achieve global prioritization, each time a node establishes a connection, it sends to the portal a synopsis of its query results, and the portal responds with a global prioritization of this coarse representation of the data.

24

Motivation Overview and contribution ICEDB CafNet The Portal Case Studies The implementation

ICEDB – Continuous Query Model

Local Prioritization

 To locally prioritize query results we use two

language extensions: PRIORITY and DELIVERY ORDER.

 The PRIORITY keyword is specified at the end of

a query and assigns a numeric priority to the query’s result buffer.

 ICEDB transmits query result buffers strictly in order of

priority, ensuring that high priority queries are transmitted before low priority queries.

 The DELIVERY ORDER clause allows the remote

node to locally determine the transmission order

• f results within a given query buffer.

25

Motivation Overview and contribution ICEDB CafNet The Portal Case Studies The implementation

ICEDB – Continuous Query Model

Global Prioritization

 ICEDB applications express global priorities using

the SUMMARIZE AS clause.

 It specifies a query that will compute a summary, which

consists of a set of tuples that summarize the entire buffer of result tuples.

 When connectivity occurs, before any query

results are transferred, this summary is sent to the

• portal. The portal applies a user-specified function

to order the tuples in the summary, and send this prioritization back to the node, which it then uses to order the entire set of result tuples.

26

Motivation Overview and contribution ICEDB CafNet The Portal Case Studies The implementation

CafNet

 CafNet is a general-purpose network stack

for delay-tolerant communication. Applications can use it to send messages across an intermittently connected network. Its mechanisms allow messages to be delivered across 2 kinds of intermittency:

1.

when end-to-end connectivity is available between the sending and receiving application, but is intermittent;

2.

when the only form of connectivity is via one or more intermediate mules.

27

Motivation Overview and contribution ICEDB CafNet The Portal Case Studies The implementation

CafNet

 CafNet offers a message-oriented data

transmission and reception API to applications

 The unit of data transport in CafNet is an

Application Data Unit (ADU)

 Each ADU has its own identifier – the combination

• f source, destination and ADU ID which is

unique.

 CafNet has a job of informing the application when

connectivity is available or changes, and in response, the application decides what data to send “at the last moment”, rather than committing that data to the network in advance.

28

Motivation Overview and contribution ICEDB CafNet The Portal Case Studies The implementation

CafNet

 CafNet defines a three-layer protocol stack:

1.

CafNet Transport Layer (CTL),

2.

CafNet Network Layer (CNL),

3.

Mule Adaptation Layer (MAL)



CafNet applications use application-level framing and schedule, and later transmit application data units (ADUs) to the CTL.



The CTL passes the message to the CNL



CNL then buffers the message until it can send the message to an appropriate node. At that point, the CNL passes the message to the appropriate MAL,



At the end MAL sends it over the appropriate network interface.

29

Motivation Overview and contribution ICEDB CafNet The Portal Case Studies The implementation

APPLICATION

CAFNET TRANSPORT LAYER

• Registers data to be transmitted
• Delivers incoming data
• Requests data from the application
• Notifies application of successful delivery

CAFNET NETWORK LAYER

• Notifies transport layer of free buffers
• Schedules data for transmission
• Selects routes
• Buffers data for transmission

MULE ADAPTATION LAYER

• Provides uniform neighbor discovery

DEVICE DRIVER

30

CafNet

Motivation Overview and contribution ICEDB CafNet The Portal Case Studies The implementation

31

Portal

Motivation Overview and contribution ICEDB CafNet The Portal Case Studies The implementation

 The portal includes a geo-spatial data

visualization system that stores sensor data from cars.

 Organizes data in terms of traces, which are

sets of sensor readings collected during a particular drive.

 Users are given a simple graphical query

interface for selecting the traces they are interested in and visualizing various summaries, statistics, and maps within or across the traces they select.

32

Portal

Motivation Overview and contribution ICEDB CafNet The Portal Case Studies The implementation

The portal shows users where they've recently driven.

33

Portal

Motivation Overview and contribution ICEDB CafNet The Portal Case Studies The implementation

Each trip is organized into a trace, which when viewed allows access to all accompanying sensor data

34

Portal

Motivation Overview and contribution ICEDB CafNet The Portal Case Studies The implementation

On this picture you can see the speed measured during drive. The red, orange, and yellow segments are slow, the greens are about average, and the blues are real fast (no blue below).

35

Portal

Motivation Overview and contribution ICEDB CafNet The Portal Case Studies The implementation

We use traffic flow data collected from our users' drives to determine congestion-prone road segments.

36

Portal

Motivation Overview and contribution ICEDB CafNet The Portal Case Studies The implementation

Users can graphically search through their traces to analyze their drives visually.

37

Portal

Motivation Overview and contribution ICEDB CafNet The Portal Case Studies The implementation

Users can also view the Wi-Fi’s that were detected on their drives by selecting the right sensor overlay on the tab on the left (Wi-Fi in this case).

38

Case Studies

Motivation Overview and contribution ICEDB CafNet The Portal Case Studies The implementation

 There are three case studies conducted

using CarTel

• 1. Road traffic monitoring
• 2. Wide area Wi-Fi measurements
• 3. Automotive diagnostics

39

Case Studies – Road Traffic Monitoring

Motivation Overview and contribution ICEDB CafNet The Portal Case Studies The implementation

 Each car was equipped with a GPS can

be used as a fine-grained probe of the roads.

 It allows to measure delays on the road

segments and find congestion hot spots

 Using attached cameras on the car we

can make applications that serve with a top-notch navigation to drivers on unfamiliar terrains.

40

Case Studies – Road Traffic Monitoring

Motivation Overview and contribution ICEDB CafNet The Portal Case Studies The implementation

Commuting Time Analysis

 Most people have tons of heuristics that

help them what route to choose on their daily travel

 CarTel system chooses its route

according to various measurements done by comparing times in different parts of the day and on different routes

41

Case Studies – Road Traffic Monitoring

Motivation Overview and contribution ICEDB CafNet The Portal Case Studies The implementation

 Using the GPS adapter and a continuous

ICEDB query, the commute time application keeps an accurate record of the routes a driver takes and the delays along those routes.

 Users can display a detailed view of any

trip that shows a trace color coded by vehicle speed

 These visualizations help users identify

heavily congested segments within a given route that should be avoided.

42

Case Studies – Road Traffic Monitoring

Motivation Overview and contribution ICEDB CafNet The Portal Case Studies The implementation

 During the initial deployment one of the users

decided to optimize his commute. His traveling gave following results:

 From his own perspective this times didn’t

exactly match with his own assumption

 Small deviation times shows that routs are

quite predictable

Route Avg Dist Avg Time Std-dev Freeway City Streets Frontage Road 9.94 miles 9.83 miles 9.27 miles 19m 52s 29m 34s 31m 51s 2m 14s 2m 19s 3m 54s

43

Case Studies – Road Traffic Monitoring

Motivation Overview and contribution ICEDB CafNet The Portal Case Studies The implementation

 This predictability shows that it is easy

to precisely build models of traffic delays by aggregating GPS traces across all users.

 It can precisely predict time needed for

traveling different routes at different parts of the day for the drivers who never used those certain routes before.

44

Case Studies – Road Traffic Monitoring

Motivation Overview and contribution ICEDB CafNet The Portal Case Studies The implementation

Hot Spot Heuristics

 We calculate traffic hot spots from the GPS records

collected once per second.

 First, we define a grid on the map (approximately 100

meters by 80 meters) and assign to the each cell, GPS point, where the GPS is located.

 Next, we examine the velocity of the car at each

point, as reported by the GPS unit, and compute the standard deviation of the velocities over all GPS records in a given cell.

 After filtering out cells with an insufficient number of

samples, we place markers on a map at the center of the top ten cells with the greatest variation in velocity.

45

Case Studies – Road Traffic Monitoring

Motivation Overview and contribution ICEDB CafNet The Portal Case Studies The implementation



Picture below shows the top ten hot spots this application calculated for the areas around Boston and Seattle. Not surprisingly, in Seattle, many sections of Interstate-5 show high variation in speed during commute times.

46

Case Studies – Road Traffic Monitoring

Motivation Overview and contribution ICEDB CafNet The Portal Case Studies The implementation



Likewise, in Boston, Interstate-93 is a key area of congestion. However, in both areas some intersections on back-roads display congestion too.

47

Case Studies – Road Traffic Monitoring

Motivation Overview and contribution ICEDB CafNet The Portal Case Studies The implementation

Image Acquisition

 Driving to an unfamiliar location can be difficult, even with

detailed driving directions. Streets are often unmarked and bad weather can reduce visibility.

 One way of making turn-by-turn driving directions more

useful would be to include photographs of landmarks preceding each turn.

 People often verbally give these types of directions: “turn

left at the red barn” or “you’ve gone too far if you see the McDonald’s.”

 CarTel makes it easy to collect the photographs needed to

augment driving directions with the requisite landmarks. Cars have cameras integrated in sensors, so every few seconds along the drive picture is taken.

48

Case Studies – Road Traffic Monitoring

Motivation Overview and contribution ICEDB CafNet The Portal Case Studies The implementation



The portal uses these images to automatically generate a large repository of geo-coded, street-level images. This archive can be integrated with a driving direction application to provide images just prior to every turn

49

Case Studies – Wide area Wi-Fi Measurements

Motivation Overview and contribution ICEDB CafNet The Portal Case Studies The implementation

 According to Jupiter Research, over 65% of

households with internet connection have Wi-Fi accesses point (AP) in home.

 Since this is reasonably high density of APs, some of

those households could provide others with Internet accesses.

 If such thing would occur the main question beside

security would be performance of such network, especially for the users moving at vehicular speeds.

50

Case Studies – Wide area Wi-Fi Measurements

Motivation Overview and contribution ICEDB CafNet The Portal Case Studies The implementation

 For that research an earlier version of CarTel system

(without ICEDB) has been used to collect over 290 drive hours of data about Wi-Fi connectivity in urban environments over several months.

 Our data collection program continually scanned for APs,

attempted associations, and collected statistics about the following events as our users traveled on their daily paths:

 Wi-Fi scans, which are reports that list nearby APs.  Wi-Fi associations, which are attempts to establish link-layer

connectivity with APs.

 IP address acquisitions, which are attempts to acquire an IP

address using DHCP

 Ping and upload statistics, which are connectivity and throughput

statistics of TCP uploads through open APs.

51

Case Studies – Wide area Wi-Fi Measurements

Motivation Overview and contribution ICEDB CafNet The Portal Case Studies The implementation



Research discovered 32,000 distinct APs, of which we were able to associate with about 5,000 and acquire an IP address from about 2,000.



Moreover, the likelihood of a successful association was the same across a large range of urban vehicular speeds up to 60 km/hour.



These findings suggest that such unplanned Wi-Fi networks can be used with delay tolerant protocol stack CafNet to provide connectivity required for mobile users at vehicular speeds using CarTel.

Mean association duration 25 seconds Mean time between connections to the Internet 260 seconds Median upload throughput 30 Kbytes/s

52

Case Studies – Automotive Diagnostics (Driver Patterns)

Motivation Overview and contribution ICEDB CafNet The Portal Case Studies The implementation



Environmental Protection Agency (EPA) has come up with a Federal Test Procedure (FTP75) that rates cars for fuel economy and emission standards.



The procedure performs measurements as the car is driven along a particular schedule of speeds.



It shows representative typical urban driving patterns including highway driving. 

Later on, that test has been criticized for assuming that braking was too gentle, and acceleration as well, and it doesn’t represents real-world driving patterns



So, In 1996, the EPA introduced a new driving cycle (US06) that includes harder acceleration and higher speeds, but this test is not used for fuel economy purposes.

 According to research in the fields of air and waste

management, strong correlations exist between emission levels and both speed and acceleration.

53

Case Studies – Automotive Diagnostics (Driver Patterns)

Motivation Overview and contribution ICEDB CafNet The Portal Case Studies The implementation



CarTel providers made test by taking two experienced CarTel drivers and made them drive according to Federal Standards. This is what they came up with: Speed Acceleration The graph shows the cumulative distribution functions of speed and acc where the accelerations are grouped into discrete ranges including 1mph per second.

54

Case Studies – Automotive Diagnostics (On-Board Diagnostic Data)

Motivation Overview and contribution ICEDB CafNet The Portal Case Studies The implementation



CarTel system can collect large amount of data regarding emissions, engine failures, fuel consumptions over time with the OBD-II interface.



For just a few months system collected over 60 000 records about:



Fuel consumption



Oil pressure



Troubleshooting codes



Air intake



Engine temperature



Engine throttle position and RPM 

CarTel can use this data to compare data taken from the same car, on the same route over different periods of time so it can measure lack of performance

55

Implementation

Motivation Overview and contribution ICEDB CafNet The Portal Case Studies The implementation 

CarTel runs with a



small embedded computer running Linux 2.6



Soekris net4801



586-class processor running at 266 MHz



128 MB of RAM



1 GB or more of flash memory



802.11b miniPCI Wi-Fi card



Senao NL-2511MP with Prism 2.5 chipset flashed with firmware v 1.7.4



Rayming TN200 GPS unit 

CarTel system gets its power from the cigarette lighter socket so when the ignition is on, system starts to work.



System was tried out at on ten cars, six of them were used on regular basis

Thank you for the attention

56