[PDF] - 1.1 Problem Description and System Model at multiple no des PDF Document

SLIDE 1 Information Dissemination in P artitionable Mobile Ad Ho c Net w

rks

Goutham Karumanc hi Sriniv asan Muralidharan Ra vi Prak ash Departmen t

f

Computer Science Univ ersit y

f

T exas at Dallas Ric hardson, TX 75083-0688. E-mail: fgoutham,msrini,ra vipg@utdallas.edu Abstract A d-ho c wir eless networks have no wir e d c

mp
nent,

and may have unpr e dictable mobility p attern. Such networks c an get p artitione d and r e c

nne

cte d sever al times. One p

ssible

appr

ach

for information dissemination in such networks is to r eplic ate information at multiple no des acting as r ep

sitories,

and employ quorum b ase d str ate gies to up date and query information. We pr

p
se

thr e e such str ate gies that also use lo c al know le dge ab

ut

the r e achability

f

r ep

sitories

to judiciously sele ct quorums. The primary go al is high availability

f

information in the fac e

f

network p ar- titioning. We also c

nsider

four p

licies

to determine the appr

priate

time to p erform up dates. Exp erimental r esults indic ate that a hybrid information management str ate gy and an absolute c

nne

ctivity-b ase d up date trigger p

licy

ar e most suite d for p artitionable ad-ho c networks. 1 In tro duction Existing solutions for dissemination

f

information in static

r

cellular net w

rks

ma y not b e applicable to ad-ho c net w

rks.

First, these solutions usually do not consider c hanging top

logy
f

the net w

rk

bac kb

ne

that con tains the information serv ers. Second, the p

s-

sibilit y

f

partitioning means that some no des ma y not b e able to comm unicate up dates to

ther

no des and/or ma y b e unable to retriev e the latest information

n

queries. Third, earlier solutions that do consider net- w

rk

partitioning approac h the problem from the di- rection

f

replica consistency in distributed databases. While the problems are similar, sev eral instances

f

information dissemination problem in ad-ho c net w

rks

are simpler in nature. Emplo ying the sophisticated replica consistency solutions w

uld

result in reduced a v ailabilit y

f

data, while also incurring unacceptably high comm unication

v

erheads. F

urth,

unlik e tradi- tional distributed database systems where the timing

f

up dates is indep enden t

f

net w

rk

top

logy

, in ad- ho c net w

rks

lo cation sensitiv e information should b e up dated as a function

f

net w

rk

top

logy

. So, it is imp

rtan

t to determine: 1. When to up date information? 2. Where to send up dates? 3. Whic h no des to query for information?

1.1 Problem Description and System Model

Let us consider a building

n

re. A large n um b er, sa y N ,

f

regh ters ha v e b een assigned to extinguish the re. Among the N regh ters are a small n um b er, sa y n,

f
cers

whose job is to collectiv ely manage the en tire

p

eration. All the

cers

need to b e able to comm unicate amongst themselv es. Also, an

cer

can act as a pro xy for all

rdinary

regh ters with whom it can comm unicate. T

increase

the eciency

f

the

p

eration, and for the safet y

f

the regh ters, it is imp

rtan

t to: 1. trac k the lo cation

f

eac h regh ter, and 2. gather information ab

ut

the surroundings

f

eac h regh ter. A regh ter is resp

nsible

for up dating his/her lo cation and state information. Also ev ery regh ter should b e able to retriev e the latest information ab

ut

ev ery

ther

regh ter. Let eac h regh ter ha v e a small, ligh t- w eigh t wireless comm unication device with the same range to send and receiv e information. The devices

f

all the regh ters collectiv ely form a connected m ulti- hop wireless net w

rk.

The

cers

ha v e additional memory in their devices to main tain the state information. Th us, the

cers'

devices collectiv ely act as information serv ers for

ther

regh ters. 1 The m ulti-hop wireless net w

rk

formed b y the regh ters ma y get partitioned in to disjoin t comp

nen

ts. The duration for whic h the net w

rk

sta ys partitioned, the iden tities

f

no des in eac h partition, and the w a ys in whic h partitions merge are non-deterministic. 1 While it is p

ssible

to add more memory to ev ery regh ter's device without signican tly increasing the w eigh t, broadcasting ev ery up date to ev ery no de w

uld

incur high comm unication costs and drain the batteries quic kly .

SLIDE 2 Th us, a regh ter should c ho

se

the

cer(s)

to receiv e its up dates without an y prior kno wledge

f

future access patterns to its data. Similarly , a querying no de should send its query to some

cer(s)

without an y prior kno wledge

f

the mobilit y pattern

f

the no de whose information it is trying to retriev e. The goal is to maximize the a v ailabilit y

f

latest state information ab

ut

no des in a partitionable net w

rk

while incurring reasonable comm unication

v

erheads. The set

f

regh ters constitute an ad-ho c net w

rk
f

N no des. The n

cers

(n << N ) corresp

nd

to designated serv ers in the net w

rk.

The remaining regh ters are

rdinary

no des

r

clien ts. There is no a pri-

ri

asso ciation b et w een a clien t and a serv er, and the net w

rk

ma y get partitioned. T

mitigate

the impact

f

partitioning w e prop

se

to use a data replication sc heme for information managemen t. Eac h up date b y a clien t is sen t to a subset

f

serv ers. Similarly , eac h query is also sen t to a subset

f

serv ers. Designing subsets (quorums) that alw a ys in tersect has b een ex- tensiv ely studied in the con text

f

connected net w

rks.

The c hallenge is to construct the query and up date subsets so as to maximize the probabilit y

f

a hit (a query returning latest state information ab

ut

a no de) ev en when the net w

rk

is partitioned. The prop

sed

solution has to p erform ecien tly b

th

when the net w

rk

is connected and when it is partitioned. Successiv e up dates/queries b y a no de need not b e sen t to the same subset

f

serv ers. This is b ecause the set

f

reac hable serv ers c hanges with time. Ho w ev er, a no de could emplo y recen t information ab

ut

serv er reac habilit y to determine the subset

f

serv ers to whic h queries and up dates are sen t. 2 Related W

rk

Da vidson, Garcia-Molina and Sk een [4 ] state that data replication results in high a v ailabilit y

f

information in the presence

f

failures and net w

rk

partitioning. Ho w ev er, if unconstrained up dates b y dieren t no des are p ermitted in m ultiple partitions

f

a net- w

rk,

the replica v alues ma y div erge. Consequen tly , queries in dieren t partitions w

uld

return inconsis- ten t v alues. T

ensure

correctness,

ne

w

uld

need to prev en t up dates in all but

ne

partition. Th us, a v ailabilit y and correctness are m utually conicting goals. Basically , the causes

f

error are the write-write conicts and read-write conicts among replicas in a partitioned net w

rk.

A detailed description can b e found in [4]. In

rder

to a v

id

conicts v arious m utual exclusion based solutions are prop

sed.

In the v

ting

approac h prop

sed

b y Giord [6] eac h replica is assigned a n um- b er

f

v

tes.

A ma jorit y

f

v

tes

is needed for up dates. Also, the sum

f

v

tes

needed for queries and up dates should exceed the total n um b er

f

v

tes.

This ensures that t w

up

dates cannot happ en concurren tly . Also, if the net w

rk

is partitioned, the same data item cannot b e up dated in t w

dieren

t partitions. Ho w ev er, if the net w

rk

is partitioned in to more than t w

par-

titions suc h that no partition has a ma jorit y

f

v

tes

the up date

p

erations are not p

ssible.

Th us, data a v ailabilit y is reduced. T

address

this problem, dynamic voting sc hemes ha v e b een prop

sed

b y P aris and Long [13 ], Ja jo dia and Mutc hler [8 ], Barbara, Garcia-Molina and Spauster [3 ], among

thers.

In [13 ], dynamic v

ting

allo ws data access to pro ceed as long as strict ma jorit y

f

curren t aliv e ph ysical copies are accessible. If the n um b er

f

accessible copies is equal to the n um b er

f

inacces- sible copies data accesses are not p

ssible.

Ho w ev er, this problem can b e resolv ed b y lexic

gr

aphic dynamic voting wherein all no des are totally

rdered

b y no de iden tit y whic h is used to break the ties [8]. In [3 ] t w

dynamic

v

ting

sc hemes ha v e b een describ ed, namely the gr

up

c

nsensus

approac h and the autonomous r e- assignment approac h. In the group consensus approac h the no des in the ma jorit y agree up

n

a new v

te

as- signmen t either in a distributed fashion

r

b y electing a leader. Then, the t w

-phase

commit proto col is emplo y ed among the ma jorit y no des to install the new v

tes.

In the autonomous reassignmen t approac h eac h no de indep enden tly pic ks a new v

te

v alue whic h can b e installed

nly

if the no de can

btain

a ma jorit y

f

the v

tes.

These dynamic v

ting

sc hemes require ex- tensiv e comm unication b et w een serv ers for v

te

reassignmen t. This is a serious concern for ad-ho c wireless net w

rks

with scarce comm unication bandwidth. El Abbadi, Sk een and Cristian [1] prop

sed

the ac c essible c

pies

algorithm that emplo ys the r e ad-

ne/write-al

l proto col. A datum is accessible if a ma- jorit y

f

its replicas are presen t in the same partition. A query

n

accessible datum is p erformed b y reading its nearest cop y . An up date writes to all copies

f

the datum in the partition. Th us, queries and up dates can b e p erformed in

nly
ne

partition. Also, all copies

f

the datum remain consisten t. Instead

f

relying

n

strict ma jorities, quorum based solutions create a set

f

subsets

f

the no des in the system. Eac h subset is called a quorum. T

p

erform an up date

r

a query p ermission from all no des in at least

ne

quorum is sucien t. The quorums need not b e a ma jorit y

f

no des. Herlih y [7 ] prop

sed

a dynamic quorum adjustment sc heme in whic h quorums can b e adjusted

n

partitioning and mergers. If an

p

eration is unable to progress using

ne

quorum it ma y b e able to mak e progress b y using another more fa v

rable

quorum. Suc h a selection is aided b y hin ts pro vided b y the underlying system ab

ut

no de accessibilit y . Previous Researc h: Though the problem w e set ab

ut

solving app ears v ery similar to the replica consistency problem, there are a few subtle dierences. The algorithms describ ed ab

v

e seek to a v

id

write- write conicts across partitions, and ensure serializ- abilit y among all the up date transactions. F

r

some w eak consistency solutions it is acceptable if read-only

SLIDE 3 transactions are not serializable with resp ect to eac h

ther.

In the problem at hand, write-write conicts are simply not p

ssible.

A datum is up dated b y

nly
ne

no de. Let a no de p erform successiv e writes to t w

dif-

feren t quorums

f

replicas. If w e assume that eac h no de has a monotonically increasing lo cal clo c k, lik e Lamp

rt's

clo c k [10 ], the t w

writes

can b e easily seri- alized if eac h replica stores the data item as a <value, writer's timestamp> tuple. Let a query b e sen t to a set

f

replicas, returning dieren t v alues for the datum. The most recen t

f

these v alues can b e iden tied using the timestamp receiv ed with the v alue. W e emplo y a v ariation

f

the quorum based sc heme. As in [1, 7 ] w e, to

,

use previous kno wledge ab

ut

no de accessibilit y to select sets

f

replicas that receiv e queries and up dates. Ho w ev er, unlik e [1], w e do not assume an ideal net w

rk

in whic h no des detect net w

rk

partitioning immediately . In most replica consistency solutions,

n

merger

f

partitions the replicas in the merging partitions are sync hronized so that they ha v e the same v alue. This incurs v ery high comm unication

v

erheads whic h are not acceptable in lo w bandwidth wireless net w

rks.

The sync hronization trac

n

mergers ma y preclude all

ther

up dates and queries in the net w

rk

for some time. So, w e in tend to adopt an

ptimistic

approac h. W e do not p erform sync hronization b et w een copies at the time

f

mergers. This sa v es

n

comm unication cost without signican tly degrading the accuracy

f

information returned b y queries. Moreo v er, when new links are established w e do not ha v e to run tests to determine if hitherto disjoin t partitions are merging. Also, the simplicit y

f

the solution is imp

rtan

t for its fast and error-free implemen tation in an actual net w

rk.

Our approac h is guided b y similar principles as the Ba y

u

pro ject [16 ]. Ho w ev er, there are also some im- p

rtan

t dierences. In Ba y

u

serv ers engage in anti- entr

py

sessions to exc hange information and reac h consensus ab

ut

the

rder
f

up dates. W e do not use suc h a sc heme for reasons describ ed ab

v

e. Also, in Ba y

u

eac h datum has a primary serv er resp

nsible

for committing up dates. W e do not ha v e the notion

f

primary serv ers. Also, Ba y

u

is not designed for real-time applications and has ten tativ e up dates that are later committed

r

rolled bac k. W e are targeting real-time applications where inaccuracy

f

information is sometimes preferable to no information at all. 3 Quorum Based Solution Giv en a set S

f

serv ers, a quorum system is a set

f

m subsets

f

S , namely S , S 1 ; : : : ; S m1 , suc h that: 1. [ m1 i=0 S i = S . 2. S i \ S j 6= , for

i;

j

m
1.

The sets S i are constructed a priori and ev ery no de kno ws the mem b ership

f

these sets. Giv en n serv ers, it is p

ssible

to form quorums

f

size O ( p n) [11 ]. Information Up date: When a no de x wishes to update some information it timestamps the datum with its lo cal clo c k v alue. W e assume lo

sely

sync hronized clo c ks suc h that the time b et w een successiv e up dates is greater than the maxim um clo c k sk ew. Then, the follo wing actions are p erformed: 1. No de x randomly selects a quorum S i from the set

f

quorums and sends UPD A TE message, timestamp ed with its lo cal clo c k v alue, to all serv ers in the quorum. 2. The serv ers,

n

receiving the UPD A TE message,

v

erwrite their

ld

cop y

f

the data item with the new cop y . If they do not ha v e an

ld

cop y

f

that data item, they simply add the information receiv ed in the message to their database. Option- ally , the serv ers ma y also send a p

sitiv

e ac kno wledgmen t to x. Let an UPD A TE message b e rst sen t to quorum S i . Later, let another UPD A TE, for the same data item, b e sen t to quorum S j . Then, all serv ers in the set S i

S

j ha v e

utdated

v ersions

f

the data item, while all serv ers in S j ha v e the latest v ersion

f

the data item. An alternativ e w

uld

ha v e b een to send a DELETE message to serv ers in the set S i

S

j . While this alternativ e w

uld

reduce the memory requiremen ts, it w

uld

increase the comm unication

v

erheads. Information Query: When no de y wishes to mak e a query the follo wing actions are p erformed: 1. No de y randomly selects a quorum S j and sends a QUER Y message to all serv ers in the quorum. 2. When a serv er receiv es a QUER Y for a datum and has a cop y

f

it, the serv er sends a REPL Y con- taining the information along with the timestamp asso ciated with the datum. Otherwise, the serv er sends a NULL reply . 3. Receiving all the REPL Y messages,y selects the v alue

f

the datum with the greatest timestamp. As t w

quorums

alw a ys in tersect in the xed net- w

rk,

the set

f

queried serv ers is b

und

to con tain at least

ne

serv er that b elonged to the quorum that receiv ed the latest up date. Hence, eac h query returns the latest v alue

f

the queried data item. Suc h a query and up date strategy has b een previously emplo y ed for lo cation managemen t in cellular net w

rks

[9, 14 , 15 ]. Wh y random selection

f

quorums: Let a quorum b e initially selected b y a mobile no de suc h that all serv ers in the quorum are reac hable from that no de. Subsequen tly , if the same quorum is selected whenev er the no de has to mak e an up date

r

a query , there is no guaran tee that all serv es in that quorum w

uld

b e reac hable. So, deterministic quorum selection is not a go

d

idea. Moreo v er, random selection giv es a b etter c hance

f

load balancing among the serv ers ev en if some no de suddenly starts generating up dates and queries at a higher rate than

ther

no des.

SLIDE 4 4 Prop

sed

Solution W e need to address the issues

f

when to trigger up dates, and where to send up dates and queries so as to mitigate the impact

f

net w

rk

partitioning.

4.1 When to Update?

Our aim is to propagate information ab

ut

no des suc h that

ther

no des,

n

querying for information, get as recen t a v ersion as p

ssible.

Had there b een a means to predict net w

rk

partitioning, up dates could b e p erformed just b efore suc h partitions

ccur.

Ho w ev er, in the absence

f

suc h an

racle,

a no de ma y ha v e to

b-

serv e c hanges in net w

rk

top

logy

and mak e guesses ab

ut

reac habilit y to/from

ther

no des. W e prop

se

four dieren t p

licies

to trigger up dates in increasing

rder
f

sophistication: 1. In the time-b ase d strategy the time b et w een successiv e lo cation up date attempts b y a no de is ex- p

nen

tially distributed, with a mean

f

t units. F

r
ur

sim ulations w e set t to 1:0 unit. 2. The time and lo c ation-b ase d strategy is similar to the time based strategy . Ho w ev er, in this strategy a no de remem b ers its lo cation when it last sen t an up date. If the no de's lo cation unc hanged since the last successful up date, no up date requests are sen t. 3. In the absolute c

nne

ctivity-b ase d strategy a no de sends an up date when a certain pr e-sp e cie d num- b er

f

links inciden t

n

it ha v e b een established

r

brok en since the last up date. 4. In the p er c entage c

nne

ctivity-b ase d strategy an up date is triggered when a pr e-sp e cie d p er c ent

f

the links inciden t

n

it ha v e c hanged since the last up date. The last t w

up

date strategies are motiv ated b y the feeling that in ad-ho c net w

rks

the frequency

f

updates should b e a function

f

the dynamism exhibited b y the net w

rk.

In [2], Bar-No y , Kessler and Sidi sho w ed that mo v emen t-based and distance-based up dates p erformed b etter than time-based up dates in a cellular net w

rk.

As there is no notion

f

cells in ad-ho c net w

rks

the mo v emen t-based strategy is not applicable. Also, the relativ e p

sition
f

no des (and the resultan t top

logy)

is a more imp

rtan

t piece

f

information than their absolute lo cation

r

absolute distance mo v ed. Hence, distance-based up dates ma y ha v e limited usefulness in ad-ho c net w

rks.

The absolute connectivit y- and the p ercen tage connectivit y- based approac hes are an attempt to capture the relativ e c hange in top

logy

.

4.2 Where to Update and Query?

There is an implicit assumption in the quorum based sc heme that all the serv ers in the selected quorum are reac hable from the no de that sends the up date

r

query . This assumption is v alid for cellular net w

rks

where base stations and serv ers main taining lo cation information b elong to the wired bac kb

ne

net w

rk

and are reac hable from eac h

ther

virtually all the time. Ho w ev er, in an ad-ho c net w

rk

there is a p

ssibilit

y that the net w

rk

gets partitioned as describ ed earlier in the con text

f

regh ters in a building. Let no de x select quorum S i = fs x1 ; s x2 ; : : : ; s xn g to up date the v alue

f

a data item. If some elemen ts

f

S i are in a dieren t partition from x at the time

f

the up date. They do not receiv e the up date. Only the serv ers in S i

S

i receiv e the up date. Subsequen tly , let another no de y select a quorum S j = fs y 1 ; s y 2 ; : : : ; s y n g to query the v alue

f

the data item. Some elemen ts

f

S j ma y b e in a dieren t partition from y at the time

f

the query . So,

nly

serv ers in S j

S

j receiv e the query . Th us, there is a p

ssibilit

y that the query ma y not return an y information,

r

return stale information. This is due to the fact that ev en though S i \ S j 6= , it is p

ssible

that S i \ S j is empt y . One p

ssible

solution w

uld

b e to send the query

nce

again to another quorum S k hoping that S i \ S k is non-empt y . Ho w ev er, this solution is not acceptable for t w

reasons:

1. It increases the comm unication

v

erheads in a bandwidth p

r

net w

rk,

without an y guaran tees

f

returning the latest information. 2. The querying no de ma y not b e able to realize that it has stale information. If at least

ne

serv er returns a non-NULL reply the no de accepts the v alue with the highest timestamp. There ma y b e no w a y for the querying no de to kno w that there exists at least

ne

cop y

f

the data item with a higher timestamp, somewhere in the net w

rk.

T

alleviate

the problem

f

query failures w e need to mak e informed selection

f

quorums at the time

f

up dates and queries. The idea is to select quorums so that the sets S i

S

i and S j

S

j are as small as p

ssible.

The smaller these sets, the greater the probabilit y

f

the set

f

reac hable queried serv ers in tersecting with the set that receiv ed the latest up date. Disqualied List: W e prop

se

to use a heuristic to select serv ers for up dates and queries. No de x main- tains a disqualie d list, D QL x , con taining serv ers that x b eliev es to b e unreac hable. Note that D QL x represen ts x 's p erception

f

unreac habilit y . There ma y b e serv ers that are not reac hable from x, y et not in D QL x . Similarly , there ma y b e serv ers that are reac hable from x, but also in D QL x . The instance

f

the heuristic running at no de x p erforms the follo wing tasks: 1. Main tenance

f

D QL x . 2. Selection

f

serv ers for up dates/queries

n

the basis

f

D QL x 's comp

sition.

SLIDE 5 3. Comm unication with selected serv ers for up dates and queries. One

f

the goals

f

the heuristic is to b e able to up date the disqualied list as part

f

the query and up date

p

erations, without incurring an y extra com- m unication

v

erheads and dela ys.

4.3 Heuristic

Initially , disqualie d list, D QL x is empt y . W e pro- p

se

three strategies to select serv ers for up dates and queries using information in the disqualied list, namely: (i) Sele ct then Eliminate (STE), (ii) Elimi- nate then Sele ct (ETS), and (iii) Hybrid strategies. Select then Eliminate (STE) Strategy: In this strategy , no de x randomly selects a quorum S i . F rom the set

f

serv ers S i , x eliminates those in D QL x . Let S i = S i

D

QL x . Up date/query messages are sen t to all serv ers in S i . No de x exp ects to receiv e a reply from ev ery serv er to whic h a message is sen t within time T timeout . An up date is considered to b e successful if at least

ne

serv er sends an ac kno wledgmen t

f

receipt

f

the update message. In the case

f

queries, if

ne
r

more serv ers send non-NULL timestamp ed information, the receiv ed cop y with the greatest timestamp is selected, and the

p

eration is considered to b e successful. This pro cess is rep eated un til there is success. If a reply is not receiv ed from a serv er s within T timeout , no de x concludes that s is not reac hable and adds s to D QL x . Once a serv er s has b een added to D QL x it sta ys in it for a disqualic ation dur a- tion,

D

QL . A t the end

f
D

QL , s is remo v ed from D QL x .The v alue

f
D

QL is a parameter

f

the heuristic and should b e determined

n

the basis

f

the connectivit y c haracteristics

f

the net w

rk.

If all the no des are concen trated in a small area,

r

if the no des ha v e a large wireless range, the probabilit y

f

net w

rk

partitioning is lo w. Ev en if the net w

rk

gets partitioned, the mean time b efore the partitions merge with eac h

ther

is also exp ected to b e small. In suc h cases ha ving a small v alue

f
D

QL w

uld

b e advisable. If a m uc h smaller v alue

f
D

QL is selected, a serv er ma y b e remo v ed from D QL x when it is still unreac h- able from x . So, x will end up trying to prob e serv ers that are still not reac hable, increasing the comm unication

v

erheads. On the

ther

hand, if a v ery large v alue

f
D

QL is selected, some previously unreac hable serv ers will not b e prob ed for a long time ev en after they are

nce

again reac hable. This ma y result in un- a v ailabilit y

f

information, reduced utilization

f

some serv ers, and excessiv e load

n
ther

serv ers. Eliminate then Select (ETS) Strategy: In this strategy , no de x rst eliminates all quorums that ha v e at least

ne

no de in D QL x . One

f

the remaining quorums is randomly selected and messages are sen t to serv ers in this quorum. If at least

ne

serv er sends an ac kno wledgmen t in the case

f

an up date,

r

a non-NULL v alue in the case

f

a query , the

p

eration is said to succeed. Let some serv ers not resp

nd

within T timeout , while all

thers

send a NULL reply . Then the serv ers that did not resp

nd

are added to D QL x for

D

QL time. Also, quorums con taining at least

ne

serv er in the expanded D QL x are eliminated, and

ne
f

the remaining quorums is selected for the up date/query

p

eration. This pro cess is rep eated un til the up date/query succeeds. If all quorums get eliminated b efore there is success, the up date/query

p

eration is said to fail. Hybrid Strategy: The h ybrid strategy uses ETS for up dates and STE for queries. The idea is to maximize the n um b er

f

serv ers receiving the up dates with the hop e that this will result in more accurate information retriev al b y queries. The motiv ation for the h ybrid strategy will b ecome clear in the follo wing paragraphs. Comparison

f

STE, ETS and Hybrid Strate- gies: STE tries to maximize a v ailabilit y

f

information, while ETS tries to maximize accuracy

f

queries at the exp ense

f

a v ailabilit y . In the ETS strategy an attempt is made to iden tify a quorum with no serv er in the disqualied list. This reduces the n um b er

f

quorums a v ailable for up date

r

query while maximiz- ing the n um b er

f

serv ers receiving the query/up date. In the STE strategy all the quorums are a v ailable for up date

r

query th us increasing a v ailabilit y . Ho w ev er, the increased a v ailabilit y

f

STE ma y actually reduce the accuracy

f

information. One

f

the concerns

f

a transaction pro cessing system for partitioned net w

rks

is c

rr

e ctness within p artition. Cor- rectness within partition means that the queries in a partition should return the v alues stored b y the latest successful up date in that partition to the same data item. When up dates are sen t to few er serv ers, the probabilit y

f

queries hitting at least

ne

serv er with the latest information is also lo w er. This probabilit y is further reduced as the queries ma y also b e sen t to few er serv ers. Ev en though the query and up date sets in tersect, their in tersection ma y b elong to another net w

rk

partition. STE seems to run coun ter to the desired goal

f

minimizing the size

f

the set S i

S

i , where S i is the set

f

reac hable serv ers

f

quorum S i . Hence, the probabilit y

f
btaining

stale information is greater. Let us consider the situation when D QL x accurately represen ts the serv ers that are not reac hable from no de x. Then, due to the reason stated ab

v

e, queries in the STE strategy ma y return stale information

r

no information at all. Ho w ev er, in the ETS strategy , due to the w a y the quorums are eliminated a priori, all serv ers in the quorums selected for an up date and a subsequen t query b elong to the same partition, and are indeed reac hable. Hence, correctness within partition is guaran teed for ETS in this situation.

SLIDE 6 Ho w ev er, the ETS strategy pro vides a lo w er a v ailabilit y

f

data than STE. Once again, let D QL x accurately represen t reac habilit y information, and let updates

r

queries succeed in

ne

partition. This means that all serv ers

f

at least

ne

quorum are within that partition. Due to the non-empt y in tersection prop ert y

f

quorum pairs, all quorums will b e eliminated in

ther

partition(s) b y the ETS strategy . So, queries and updates cannot b e p erformed in those partition(s). If a v ailabilit y

f

stale information is preferable to no information at all, STE ma y b e preferred

v

er ETS. Of course, if

D

QL is dieren t from the elapsed time b et w een partition detection and merger, D QL x will b e inaccurate, and ev en ETS ma y return stale information while STE ma y ha v e reduced a v ailabilit y . The h ybrid strategy attempts to increase the accuracy

f

future queries b y p erforming up dates using the ETS strategy . A t the same time it striv es to maximize the a v ailabilit y

f

information during queries b y using the STE strategy . It is hop ed that p erforming up dates at a greater n um b er

f

sites will actually increase the probabilit y

f

a query returning fresh information, ev en though the queries are sen t to few er serv ers. This ma y b e esp ecially useful in systems where there are man y more queries than up dates. Sa vings in comm unication

v

erheads during the more frequen t queries ma y more than

set

for the extra comm unication

v

erheads incurred b y the less frequen t up dates.

4.4 Communication and Storage Overheads

A cop y

f

ev ery data item, fresh

r

stale, ma y b e residen t at eac h serv er. So, ev ery data item requires O (n) storage. This can b e reduced if, at the time

f

up date, the no de also sends delete messages to serv ers that w ere in its previous up date quorum, and are not in its presen t up date quorum. Ho w ev er, this will require additional comm unication. Giv en n serv ers, there exist quorum formation sc hemes that giv e quorums

f

size O ( p n) [11 ]. So, ev en if an up date

r

query requires a constan t n um b er

f

re- tries b efore success, the comm unication complexit y is O ( p n). Note that n is usually m uc h smaller than the total n um b er

f

no des N . Hence, the comm unication and storage

v

erheads are reasonable. 5 Sim ulation Exp erimen ts W e conducted sim ulation exp erimen ts using the CSIM18 sim ulation engine [12]. W e compared the p erformance

f

the STE,ETS and Hybrid strategies with the simple quorum based strategy that do es not ac- coun t for partitioning and do es not use the disqualied list. In the simple quorum based sc heme, a no de con tin ues to randomly select a quorum and send queries/up dates un til either the

p

eration is p erformed successfully ,

r

all quorums ha v e b een tried without success. In the latter case, the

p

eration is said to fail. W e assumed a system comp

sed
f

100 mobile no des. Of these, 25 no des w ere assumed to act as serv ers. Initially , the 100 no des are randomly distributed in a square region with edges

f

1000 length units. During the sim ulation exp erimen t all the no des are allo w ed to mo v e

nly

within this square. In

rder

to mo del mobilit y

f

no des w e cycle through the follo wing t w

steps:

(i) the duration for whic h a no de sta ys at a lo cation is exp

nen

tially distributed with mean 0:5 time units, (ii) at the end

f

this duration, the no de randomly selects another lo cation within the square region that is at most 75 length units a w a y from its curren t lo cation, and instan taneously mo v es to the new lo cation. A wireless link is assumed to b e presen t b et w een a pair

f

no des if they are within 150 length units

f

eac h

ther.

Eac h time a new link is established,

r

an

ld

link is brok en, the sim ulator executes Flo yd- W arshall's algorithm [5 , 17 ] to determine the reac habilit y

f

no des, and the presence

f

partitions in the net w

rk.

Note that this computation

f

reac habilit y do es not have to b e p erforme d b y no des as part

f

their up date and query function. In subsequen t exp erimen ts w e v ary the wireless range to v alues

ther

than 150 and measure the impact

f

partitioning

n

the information dissemination strategies. The 25 serv ers are lo gic al ly arranged in a 5

5

square grid, and 25 quorums are formed: eac h quorum is com- p

sed
f

the union

f

a ro w and a column

f

serv ers in this logical grid. Th us, eac h quorum consists

f

9 serv ers. The quorums are assigned distinct sequence n um b ers in the range 1

25.

Ev ery pair

f

quorums has t w

serv

ers in common. The up date and query

p

erations w

rk
n

lo cation dep enden t information. 2 As eac h no de has its unique lo cation, there are 100 distinct pieces

f

information. The time b et w een successiv e queries b y a no de is ex- p

nen

tially distributed, with a mean

f

0:5 time units. A querying no de randomly selects

ne
f

the 100 no des as the no de whose lo cation it wishes to query . Query messages are sen t to serv ers based

n

the strategy in use: ETS, STE,

r

Hybrid. If a no de disco v ers that a serv er is unreac hable, the no de places the serv er in its disqualied list for a p erio d

D

QL = 0:25 time units. W e also v aried the v alue

f
D

QL to measure its impact

n

serv er a v ailabilit y , utilization and load balancing. In eac h sim ulation run, no data collection w as p erformed un til the rst 10; 000 queries w ere nished (successfully

r

unsuccessfully). This w as done to eliminate the impact

f

an y initial transien t eects. Subse- quen tly , data collection w as p erformed un til the next 100; 000 queries w ere completed. Eac h data p

in

t in the graphs represen ts the mean v alue

btained

from v e sim ulation runs with iden tical input parameters, but dieren t random n um b er generation seeds. 2 Note that eac h regh ter sends its lo cation and the status

f

the re, etc. in his/her lo cation as part

f

the up date.

SLIDE 7

5.1 Simulation Results

W e measured v arious p erformance c haracteristics

f

the ETS, STE and Hybrid strategies v ersus the simple quorum based strategy that do es not use the disqualied list. F

r

up dates the time and lo c ation-b ase d p

licy

w as emplo y ed. Query success and freshness: The impact

f

wireless range

f

no des

n

the n um b er

f

successful queries and the n um b er

f

queries returning fr esh information (information stored b y the latest up date for that datum) is sho wn in Figure 1. When the wireless range is equal to 250,

r

150 length units almost all the queries succeed regardless

f

the strategy emplo y ed, as seen in the graph

n

the left hand side

f

Figure 1. This is b e- cause for these v alues

f

wireless range there are v ery few partitions in the net w

rk.

Ev en when partitions do

ccur,

the partitions get merged quic kly . Ho w ev er, there is a signican t drop in the n um b er

f

successes for wireless range

f

100 length units. This is due to increased probabilit y

f

net w

rk

partitioning. The ETS strategy suers the most due to increased partitioning probabilit y . This is due to the fact that when the net- w

rk

is partitioned the ETS strategy ma y not b e able to nd an y quorum b ecause at least

ne

serv er

f

ev ery quorum is in the querying no de's disqualied list. When the wireless range is equal to 250

r

150 most

f

the queries return latest information. But, when the range is reduced to 100, the n um b er

f

queries returning the latest information is signican tly reduced. Once again, this is due to increased lik eliho

d
f

net- w

rk

partitioning. Moreo v er, the fr action

f

successful queries that yield the latest information is m uc h higher for the ETS strategy than the STE and simple quorum based strategies, as

bserv

ed b y lo

king

at the graphs

n

the left as w ell as

n

the righ t in Figure 1. This is consisten t with

ur

prior analysis

f

the STE strategy emphasizing a v ailabilit y

v

er accuracy , and ETS emphasizing accuracy

v

er a v ailabilit y . In Figure 1, the v alue

f
D

QL is set to 0:25. Similar results w ere

btained

for

ther

v alues

f
D

QL as w ell. Among the four strategies, ETS has few est queries returning the latest information. Also, ETS is signif- ican tly p

rer

than the

ther

strategies in terms

f

query success. This leads us to b eliev e that ETS is not a go

d

strategy for information dissemination. Comm unication Ov erheads: Figure 2 sho ws that the ETS, STE, Hybrid, and simple quorum based strategies incur comparable comm unication costs for larger v alues

f

wireless range. Ho w ev er, for wireless range

f

100 length units when the probabilit y

f

net- w

rk

partitioning is more, the simple quorum based strategy incurs v ery high comm unication cost. F

r

the same wireless range the comm unication costs are com- parativ ely lo w for the STE, Hybrid, and ETS strategies, as w as exp ected. The ETS strategy has the lo w est cost b ecause quite

ften

it cannot ev en nd a quorum to send the queries to. The simple quorum based strategy has the highest

v

erheads b ecause the no des try to send their queries to ev en those serv ers that are not reac hable. Also, when there is a failure, the no des k eep trying

ther

quorums un til either some information is retriev ed

r

all quorums ha v e b een tried.

500000 1e+06 1.5e+06 2e+06 2.5e+06 3e+06 3.5e+06 4e+06 4.5e+06 5e+06 250 150 125 100 Total no.of query messages sent Wireless Range Impact of wireless range and DQL on communication

verheads (delta = 0.25)

Without DQL STE ETS Hybrid

Figure 2. Impact of wireless range and quo- rum selection policy on communication over- heads.

Load Distribution: W e measured the distribution

f

load (total n um b er

f

messages receiv ed b y a serv er)

n

all serv ers when

D

QL w as set to 0:25 and the wireless range w as 100. In all the four strategies dieren t serv ers receiv e a dieren t n um b er

f

query messages, with the simple quorum based sc heme (without DQL) exp eriencing the maxim um trac and also the maxim um v ariance in trac. The reason for the load to b e unev enly distributed is that the net w

rk

gets partitioned. Due to this ev ery serv er is no longer equally lik ely to receiv e the query messages. Ho w ev er, when the wireless range w as increased to 150 and 250 distance units net w

rk

partitioning w as a rare

ccurrence

and the load w as ev enly distributed across all serv ers. Num b er

f

attempts: Figure 3 sho ws the cum ula- tiv e n um b er

f

successful queries v ersus the n um b er

f

query attempts, where a query attempt corresp

nds

to selection

f

a quorum and sending messages to reac h- able serv ers in that quorum. The STE, Hybrid, and the simple quorum based strategies require almost same n um b er

f

attempts for a success. The ETS strategy requires few er attempts b ecause the n um b er

f

quorums a v ailable after eliminating those that con tain serv ers in D QL is less. Similar results w ere

btained

for

ther

v alues

f

wireless range and

D

QL . Most

f

the successful queries require

nly
ne

attempt. As the curv es plateau after ab

ut

four to v e attempts it can b e safely said that if a query has not

SLIDE 8

20000 40000 60000 80000 100000 120000 250 150 125 100 No.of successes Wireless Range Impact of wireless range and DQL on successful queries (delta = 0.25) Without DQL STE ETS Hybrid 5000 10000 15000 20000 25000 30000 35000 40000 45000 50000 250 150 125 100 No.of queries returning latest information Wireless Range Impact of wireless range and DQL on retrieval of latest data (delta = 0.25) Without DQL STE ETS Hybrid

Figure 1. Impact of wireless range and quorum selection policy on the number of successful and accurate queries.

succeeded after four to v e attempts, there is no p

in

t in trying further. The small increase in the n um b er

f

successes ma y not b e w

rth

the signican t increase in the comm unication cost incurred in doing so.

20000 40000 60000 80000 100000 5 10 15 20 25 No.of queries No.of attempts Cumulative no. of successful queries vs. no. of attempts (delta=0.25,range=100) Without DQL STE ETS Hybrid

Figure 3. Number of attempts required by a successful query.

Impact

f

Up date P

licy:

Figure 4 sho ws that the up date p

licy

can ha v e a signican t impact

n

the n um b er

f

information up dates p erformed b y no des during the lifetime

f

the sim ulation. As the Hybrid strategy p erforms signican tly b etter than ETS, and has lo w er comm unication

v

erheads than simple quorum based strategy , w e

nly

studied the impact

f

the up date p

licy
n

the Hybrid strategy . F

r

the absolute connectivit y-based p

licy

, an up date is triggered when ve edges inciden t

n

the no de ha v e c hanged since the last up date. F

r

the p ercen tage connectivit y-based strategy , an up date is triggered when 20%

f

the links ha v e c hanged since the last up date. Maxim um n um b er

f

up dates w ere p erformed when the p ercen tage connectivit y-based p

licy

w as emplo y ed. This is b ecause with 100 no des scattered

v

er the en tire region, the net w

rk

is v ery sparsely connected. So,

nly

a few links need to c hange to exceed the 20% threshold. F

r

the same reason, it tak es a long time for v e links to c hange. Hence, the absolute connectivit y based p

licy

triggers v ery few up dates. Ho w ev er, inspite

f

the signican tly few er up dates, the n um b er

f

successful, fresh and stale queries for the absolute connectivit y-based up date p

licy

is comparable to that

f

the

ther

p

licies.

Ho w ev er, b efore w e rush to judgmen t, let us also consider the error in the lo cation information returned b y all queries (fresh as w ell as stale). Note that a query returns the lo cation

f

the queried no de at the time that no de up dated its lo cation and state information. Error indicates the distance b et w een the actual lo cation

f

the queried no de at the time

f

the query , and the lo cation information returned to the querying no de. Recalling

ur

example, a large error in lo cation determination will mislead the regh ters ab

ut

the p

si-

tion

f

their colleagues, and seriously jeopardize b

th

their safet y and the success

f

their

p

eration. Figure 4 sho ws the n um b er

f

queries

n

the y-axis with lo cation error no more than the v alue

n

the x-axis. The p ercen tage connectivit y-based p

licy

has the b est p erformance with ab

ut

20,000 queries returning lo cation information that is within 25 distance units

f

the actual lo cation

f

the queried no de. Ab

ut

40,000 queries return lo cation information that is no more than 100 distance units a w a y from the actual lo cation

f

the queried no de, and so

n.

The graph also indicates that inspite

f

signican tly few er up dates, the error in lo cation determination for the absolute connectivit y-based

SLIDE 9 p

licy

is comparable to

ther

p

licies.

This se ems to indic ate that, at le ast for sp arse networks, the absolute c

nne

ctivity-b ase d p

licy

is most suitable. Ev en though the

ther

p

licies

p erform more up dates, due to the highly partitioned nature

f

the net w

rk

most

f

the up dates are visible to a small subset

f

querying no des. A ma jorit y

f

no des in the

ther

partitions do not see the up dates and con tin ue to return

utdated

lo cation information. Impact

f

Net w

rk

Densit y: W e increased the n um b er

f

no des in the net w

rk

from 100 to 200, while k eeping the n um b er

f

serv ers unc hanged at 25. As can b e seen in Figure 5, with increased densit y (due to the presence

f

200 no des in the same area), net w

rk

connectivit y impro v ed and there w ere few er partitions. As a result, the error in lo cation information returned b y queries w as reduced. This is b ecause the serv ers receiving the latest up date w ere reac hable from more no des in the net w

rk.

Also, with increased connectivit y the absolute connectivit y-based up date p

licy

triggered more lo cation up dates. Greater the n um b er

f

links inciden t

n

a no de, the less time it tak es for v e links to c hange since the last up date. Still the absolute connectivit y- based up date p

licy

p erforms few er up dates than the time-based and p ercen tage connectivit y-based up date p

licies,

and

nly

sligh tly more up dates than the time and lo cation-based up date p

licy

. Just as in the case

f

sparse net w

rks,

the n um b er

f

successful, fresh and stale queries are comparable for all the four p

licies.

Th us,

ur

sim ulation exp erimen ts seem to indicate that the absolute c

nne

ctivity-b ase d up date p

licy

is the most suitable among the four p

licies

considered. When the net w

rk

is sparse and up dates are going to ha v e less

f

an impact it triggers few er up dates. When the net w

rk

is dense, the top

logy

is c hanging more rapidly , and up dates can b e seen b y more no des it triggers a greater n um b er

f

up dates. Impact

f

disqualication p erio d: F

r

v alues

f
D

QL greater than 0:25 (w e tried v alues

f

0:50 and 1:00) the simple quorum based strategy (without DQL) exhibited a higher n um b er

f

successes than the ETS, STE and Hybrid strategies. This ma y b e due to b e due to the fact that with a larger v alue

f
D

QL , if a serv er is found to b e unreac hable, a no de do es not send up dates and queries to that serv er for quite some time ev en after the serv er has

nce

again b ecome reac hable. A

D

QL v alue lo w er than 0.25 yielded the same p erformance in terms

f

query success, freshness, and staleness. Ho w ev er, the comm unication

v

erheads increased. This seems to b e due to the p

ssibilit

y that no des ma y resume sending up dates and queries to serv ers prematurely: the serv ers ma y still b e in a dif- feren t partition. 6 Conclusion A scenario for the use

f

an ad-ho c net w

rk
f

mobile no des w as presen ted, where the net w

rk

can get partitioned and reconnected sev eral times. The problem

f

information dissemination in suc h situations w as formalized. In

rder

to increase the a v ailabilit y

f

information, data replication w as considered. Three quorum based strategies STE, ETS and Hybrid (ETS for up dates, STE for queries) ha v e b een prop

sed.

The STE strategy

ptimizes

a v ailabilit y

f

information, whereas ETS tries to maximize accuracy

f

information at the exp ense

f

a v ailabilit y . This is also indicated b y the sim ulation results. The Hybrid strategy exploits the adv an tages

f

b

th

the strategies. F

ur

p

licies

to trigger up dates w ere considered. Of these the absolute connectivit y-based p

licy

w as found to ha v e the b est p erformance. The absolute connectivit y-based p

licy

triggers more up dates in dense net w

rks

where the impact

f

up dates will b e visible to a greater n um b er

f

no des. In sparse net w

rks

this p

licy

triggers few er up dates as the up dates ha v e a lo w er p

ten

tial

f

b eing visible to

ther

no des. References [1] A. El Abbadi, D. Sk een, and F. Cristian. An Ecien t F ault-T

leran

t Algorithm for Replicated Data Man- agemen t. In Pr

c

e e dings

f

the 5 th A CM SIGA CT- SIGMOD Symp

sium
n

the Principles

f

Datab ase Systems, pages 215{229, 1985. [2] A. Bar-No y , I. Kessler, and M. Sidi. Mobile Users: T

Up

date

r

not to Up date? In Pr

c

e e dings

f

IEEE INF OCOM, pages 570{576, 1994. [3] D. Barbara, H. Garcia-Molina, and A. Spauster. In- creasing Av ailabilit y Under Mutual Exclusion Con- strain ts with Dynamic V

te

Reassignmen t. A CM T r ansactions

n

Computer Systems, 7(4):394{426, No- v em b er 1989. [4] S.B. Da vidson, H. Garcia-Molina, and D. Sk een. Con- sistency in P artitioned Net w

rks.

A CM Computing Surveys, 17(3):341{370, Septem b er 1985. [5] R.W. Flo yd. Algorithm 97 (SHOR TEST P A TH). Communic ations

f

the A CM, 5(6):345, 1962. [6] D.K. Giord. W eigh ted V

ting

for Replicated Data. In Pr

c

e e dings

f

the 7 th Symp

sium
n

Op er ating Sys- tems Principles, pages 150{162. A CM, 1979. [7] M. Herlih y . Dynamic Quorum Adjustmen t for P artitioned Data. A CM T r ansactions

n

Datab ase Systems, 12(2):170{194, June 1987. [8] S. Ja jo dia and D. Mutc hler. In tegrating Static and Dynamic V

ting

Proto cols to Enhance File Av ailabilit y . In Pr

c

e e dings

f

the 4 th International Confer enc e

n

Data Engine ering, pages 144{153. IEEE, 1988. [9] G. Krishnam urthi, M. Azizo glu, and A.K. Somani. Optimal Lo cation Managemen t Algorithms for Mo- bile Net w

rks.

In Pr

c

e e dings

f

The F

urth

A nnual A CM/IEEE International Confer enc e

n

Mobile Com- puting and Networking (MobiCom'98), pages 223{232, Octob er 1998.

SLIDE 10

20000 40000 60000 80000 100000 Time&Loc Time Absolute Percentage Number Update policy Impact of update policy on performance for a sparse network Successes Fresh Stale Updates 20000 40000 60000 80000 100000 200 400 600 800 1000 1200 1400 Cumulative no.of queries Error range Location error distribution for different update policies using hybrid query strategy Time & Location Time Based Absolute Connectivity Percentage Connectivity

Figure 4. Impact of update policy on freshness and accuracy of information returned by queries.

20000 40000 60000 80000 100000 200 400 600 800 1000 1200 1400 Cumulative no.of queries Error range Location error distribution for sparse & dense networks using percentage connectivity policy for updates Sparse Network Dense Network 20000 40000 60000 80000 100000 Time&Loc Time Absolute Percentage Number Update Strategy Impact of update strategy on performance for a dense network Successes Fresh Stale Updates

Figure 5. Impact of network density on freshness and accuracy of information returned by queries.

[10] L. Lamp

rt.

Time, Clo c ks and the Ordering

f

Ev en ts in a Distributed System. Communic ations

f

the A CM, 21(7):558{565, July 1978. [11] M. Maek a w a. A p N Algorithm for Mutual Exclusion in Decen tralized Systems. A CM T r ansactions

n

Com- puter Systems, pages 145{159, Ma y 1985. [12] Mesquite Soft w are, Inc., 3925 W est Brak er Lane, Austin, TX 78759. CSIM18 Simulation Engine, 1998. [13] J.-F. P aris and D.D.E. Long. Ecien t Dynamic V

t-

ing Algorithms. In Pr

c

e e dings

f

the 4 th Interna- tional Confer enc e

n

Data Engine ering, pages 268{ 275. IEEE, 1988. [14] R. Prak ash, Z. J. Haas, and M. Singhal. Load Bal- anced Lo cation Managemen t for Mobile Systems using Dynamic Hashing and Quorums. T ec hnical Rep

rt

UTDCS-05-97, The Univ ersit y

f

T exas at Dallas, Oc- tob er 1997. [15] R. Prak ash and M. Singhal. A Dynamic Approac h to Lo cation Managemen t in Mobile Computing Systems. In Pr

c

e e dings

f

the 8 th International Confer enc e

n

Softwar e Engine ering and Know le dge Engine ering (SEKE'96), pages 488{495, Lak e T aho e, Nev ada, June 1996. [16] D. B. T erry , M. M. Theimer, K. P eterson, A. J. De- mers, M. J. Spreitzer, and C. H. Hauser. Managing Up date Conicts in Ba y

u,

a W eakly Connected Repli- cated Storage System. In Pr

c

e e dings

f

SIGOPS'95, pages 172{183, 1995. [17] S. W arshall. A Theorem

n

Bo

lean

Matrices. Journal

f

the A CM, 9(1):11{12, 1962.