Presenter: Yunfeng Gu Supervisor: Azzedine Boukerche PARADISE - - PowerPoint PPT Presentation

SLIDE 1

Supporting Multi-dimensional Range Query in the P2P Network

Presenter: Yunfeng Gu Supervisor: Azzedine Boukerche PARADISE Research Laboratory University of Ottawa, Canada

SLIDE 2

 P2P A

Appli lications ns

 Di

Distributed d directory s y sys ystems ms

 P2P d

doma main na n name me s services

 Network f

k file le s sys ystems ms

 Massively p

ly paralle llel s l sys ystems ms

 Di

Distributed e e-ma

• mail s

l sys ystems ms

 Massive-s

• scale

le f fault lt-t

• tole

lerant nt s sys ystem m

 ...

...

2 DS-RT 2011 .

SLIDE 3

Ge Get Remo move Pu Put Return n

Ge Get k key y Remo move key y Put k key y data data Return n data data Pe Peer Pe Peer Pe Peer Pe Peer Pe Peer Pe Peer Pe Peer Request t to no nodes Respons nse f from m no nodes

3 DS-RT 2011 .

SLIDE 4

 Appli

lications ns

 Mult

lti-p

• pla

layer g game mes

 Gr

Grid c computing ng

 Publi

lish/ h/Subscribe s sys ystems ms

 Gr

Group c commu mmuni nications ns

 Name

me s services

 P2P d

data s sha haring ng

 Glo

Global s l storage

 ...

...

4 DS-RT 2011 .

SLIDE 5

Ge Get k key y Return n data data Remo move Key y Put k key y Da Data

Pe Peer Pe Peer Pe Peer Pe Peer Pe Peer Pe Peer Pe Peer

Ge Get rang nges Remo move rang nges Put r rang nges data data Return n data data

Request t to no nodes Respons nse f from m no nodes

5 DS-RT 2011 .

SLIDE 6

Return n data data

Pe Peer Pe Peer Pe Peer Pe Peer Pe Peer Pe Peer Pe Peer

Ge Get rang nges Remo move rang nges Put r rang nges data data

Request t to no nodes Respons nse f from m no nodes

6 DS-RT 2011 .

SLIDE 7

7

 Go

Goal l

 Better p

preserve d data lo locali lities

 Sche

heme mes

 K-d

• d t

tree[1]

[1]

 Quad t

tree[2. 3

. 3]]

DS-RT 2011 .

SLIDE 8

 Go

Goal l

 Better p

preserve d data lo locali lities

 Sche

heme mes

 K-d

• d t

tree[1]

[1]

 Quad t

tree[2. 3

. 3]

 Z-o

• order S

SFC[4. 6

. 6]

 Hilb

lbert S SFC[5]

[5]

8

Copy from Wiki

DS-RT 2011 .

SLIDE 9

9

1-D d

• D data s

space 2-D d

• D data s

space 3-D d

• D data s

space

 Observations

ns 1 1

 Da

Data lo locali lities e expand nd e expone nent ntially a lly as t the he d dime mens nsiona nali lity o y of d data s space inc ncreases

DS-RT 2011 .

SLIDE 10

10

 Observations

ns 2 2

 Da

Data lo locali lities e extend nd e expone nent ntially a lly as t the he o

• rder o
• f t

the he r recursive decomposition i n inc ncreases

DS-RT 2011 .

SLIDE 11

 Cha

halle lleng nge

 How t

to a accommo mmodate a and nd ma maint ntain d n data lo locali lities w with a h an n expone nent ntially e lly expand nding ng a and nd e extend nding ng r rate a at P P2P la layer

 Go

Goal l

 MDR

DRQ c can b n be d done ne a at a a r reasona nable le r routing ng c cost

11 DS-RT 2011 .

SLIDE 12

P2P S Sys ystems ms Partitioni ning ng Und nderlyi lying ng a archi hitecture Di Direct ma mapping ng DHT-based ... DHT ... ... SCARP[10] Z-order/Hilbert SFC Skip graphs[7] No MURK[10] K-d tree d-torus (CAN)[8] No ZNet[11] Z-order/Quad tree Skip graphs No Skipindex[12] K-d tree Skip graphs No Squid[13] Hilbert SFC Ring (Chord)[9] No SONAR[14] K-d tree d-torus (CAN) No SkipNet[15] Naming tree Skip graphs No P-Grid[16] Binary search tree Flat graphs No Mercury[17] Random sampling Ring No

12 DS-RT 2011 .

SLIDE 13

13 DS-RT 2011 .

SLIDE 14

Pe Peer Pe Peer Pe Peer Pe Peer Pe Peer Pe Peer Pe Peer

Return n data data Ge Get rang nges Remo move rang nges Put r rang nges data data

Request t to no nodes Respons nse f from m no nodes

14 DS-RT 2011 .

SLIDE 15

15

R R 1 1 2 2 3 3 00 0 01 1 02 2 03 3 10 0 11 1 12 2 13 3 20 0 21 1 22 2 23 3 30 0 31 1 32 2 33 3

DS-RT 2011 .

SLIDE 16

16

00 001 1 00 00 01 01 00 001 1 00 0010 000 001 1 DP DParent nt HParent nt DC DChi hild ldren HChi hild ldren 100 001 1 200 001 1 300 001 1 00 0011 1 00 0012 2 00 0013 3 000 000 00 00 SPEER SPEER 0000 0000 HParent nt & & DP DParent nt DC DChi hild ldren HChi hild ldren 1000 000 2000 000 3000 000 000 0001 1 000 0002 2 000 0003 3

DS-RT 2011 .

SLIDE 17

17

000 0001 1 000 000 00 001 1 000 0001 1 DP DParent nt HParent nt Ro Root 0000 0000 000 000 SPEER SPEER HParent nt & & DP DParent nt

DS-RT 2011 .

SLIDE 18

18

R R 1 1 2 2 3 3 00 0 01 1 02 2 03 3 10 0 11 1 12 2 13 3 20 0 21 1 22 2 23 3 30 0 31 1 32 2 33 3

DS-RT 2011 .

SLIDE 19

19

Da Data s structure Routing ng t table le s size (Neighb hbors) Total li l links nks HD Tree ≤ 2(k + 1) 2(n – 1) – k × h Tree ≤ k + 1 n – 1 Chord log(n) ? CAN 2d ? Skip graphs O(m · log(m)) ? k: k-ary tree h: height of tree n: total nodes in the system d: d-torus, number of dimensions m: total number of data elements

DS-RT 2011 .

SLIDE 20

20

00 001 1 00 00 01 01 00 001 1 00 0010 000 001 1 DP DParent nt HParent nt DC DChi hild ldren HChi hild ldren

 A no

node i in k n k-ar ary H HD t D tree c can ha n have a at mo most k k HChi hild ldren a and nd k k Dc Dchi hild ldren

 But o

• nly 1

nly 1 HParent nt a and nd 1 1 Dp Dparent nt 100 001 1 200 001 1 300 001 1 00 0011 1 00 0012 2 00 0013 3 00 001 1 00 00 01 01 00 001 1

 2HP a

and nd 2 2DP DP ha has o

• nly o

nly one ne o

• ption

n

 2HC a

and nd 2 2DC DC ha has a at mo most k o k options ns 00 0010 00 0011 1 00 0012 2 00 0013 3 000 001 1 100 001 1 200 001 1 300 001 1

DS-RT 2011 .

SLIDE 21

21

dbc bca bc bcd abc bcd bc bc bc bca dbc bcac ac dbc bca abc bc abc bcd ab dbc bcac ac a R d d db b dbc bc abc bcd dbc bcac ac

 Hierarchi

hical R l Routing ng ( (HR) c cons nsists o

• f

 a s

series o

• f 2

2HP o

• r 2

2DP DP o

• perations

ns, o , or

 a s

series o

• f 2

2HC o

• r 2

2DC DC o

• perations

ns

DS-RT 2011 .

SLIDE 22

22

01 012 01 01 00 001 1 200 200 00 00 00 001 1

 A d

distributed o

• peration c

n cons nsists o

• f t

two hi hierarchi hical o l operations ns

00 0012 12 200 2001 1

DS-RT 2011 .

SLIDE 23

23

201 01 01 01 001 01 00 003 3 00 00 00 001 1

 A d

distributed o

• peration c

n cons nsists o

• f t

two hi hierarchi hical o l operations ns

DS-RT 2011 .

SLIDE 24

24

 Di

Distributed r routing ng ( (DR DR) i is a a s series o

• f d

distributed o

• perations

ns

 DR

DR c can r n reach t h the he DS DST no node i in ( n (2 2 х d) ho hops t time me a at w worst

abc abc cde de bc bc cd bc bcd de de def def abc abc ef efa fab ab def def ab ab fa ef ef abc abc cef ef bc bc ce bc bce ef ef def def abc abc de dea eab ab def def ab ab ea a de de abc abc def def

Note: d d = depth of node in HD tree = length of the code

DS-RT 2011 .

SLIDE 25

25

ab ab wxyz yz

 DR

DROCR i is a a DR DR o

• rient

nted c comb mbine ned r routing ng a alg lgorithm hm

 DR

DROCR c can r n reach a h any no y node i in n (dSR

SRC +

+ dDS

DST

• 2
• 2 х m) ho

hops t time me

 Many r

y routing ng s strategies c can b n be b built lt o

• ver d

different nt r routing ng

• ptions

ns i in H n HD T D Tree

ab abwx wx ab abw bwx wx bwxy wxy wxy wxy wxyz yz ab ab

Note: dSR

SRC ,

, dDS

DST = depth of SRC/DST node in HD tree = length of the SRC/DST code

m = length of the max. match between SRC and DST code

yz yzab ab zab ab yz yza xyz yza xyz yz wxyz yz ab ab a R y y xy xy b R x x xy xy

DS-RT 2011 .

SLIDE 26

26

>: performs better <: performs worse =: equivalent HR: 2HP/2DP/2HC/2DC routing Tree: routing in the equivalent tree structure Tr Tree D2 D2H D2 D2D D H2D D H2 H2H H HR HR DR DROCR DR DR D2 D2H > = < = < < < D2 D2D D > > = > = < < H2D D > = < = < < < H2 H2H H > > = > = < < HR > > > > > = < DR DROCR HR + + DR DR > > > > > > =

DS-RT 2011 .

SLIDE 27

 Match e

h exists b between S n SRC a and nd DS DST c codes

 Try a

y all p ll possible le o

• ptions

ns t tha hat c can e n extend nd e existing ng ma match h towards DS DST c code

 Try a

y all o ll options ns t towards DS DST no node

 Try S

y SDP DP & & S SDC DC

 No ma

match c h can b n be f found nd b between S n SRC a and nd DS DST c codes

 Try a

y all p ll possible le o

• ptions

ns t towards DS DST no node

 Try S

y SDP DP & & S SDC DC

27 DS-RT 2011 .

SLIDE 28

Return n data data

Pe Peer Pe Peer Pe Peer Pe Peer Pe Peer Pe Peer Pe Peer

Ge Get rang nges Remo move rang nges Put r rang nges data data

Request t to no nodes Respons nse f from m no nodes

28 DS-RT 2011 .

SLIDE 29

2 0 1 3 2 0 1 3

21 21 02 02 00 00 01 01 03 03 12 12 10 10 11 11 13 13 22 22 20 20 23 23 32 32 30 30 31 31 33 33

29

1st

st o

• rder o
• f r

recursive d decomposition 2nd

nd o

• rder o
• f r

recursive d decomposition

DS-RT 2011 .

SLIDE 30

2 2 1 1 3 3 R R 00 00 01 01 02 02 03 03 10 10 11 11 12 12 13 13 20 20 21 21 22 22 23 23 30 30 31 31 32 32 33 33

02 02 00 00 01 01 03 03 12 12 10 10 11 11 13 13 20 20 32 32 30 30 31 31 33 33 21 21 22 22 23 23

2 0 1 3

30 DS-RT 2011 .

SLIDE 31

21 21 02 02 00 00 01 01 03 03 12 12 10 10 11 11 13 13 22 22 20 20 23 23 32 32 30 30 31 31 33 33 2 2 1 1 3 3 R R 00 00 01 01 02 02 03 03 10 10 11 11 12 12 13 13 20 20 21 21 22 22 23 23 30 30 31 31 32 32 33 33

02 02 00 00 01 01 03 03 12 12 10 10 11 11 13 13 20 20 32 32 30 30 31 31 33 33 21 21 22 22 23 23

01 01 03 03 12 12 21 21 30 30 03 03

31 DS-RT 2011 .

SLIDE 32

21 21 02 02 00 00 01 01 03 03 12 12 10 10 11 11 13 13 22 22 20 20 23 23 32 32 30 30 31 31 33 33 2 2 1 1 3 3 R R 00 00 01 01 02 02 03 03 10 10 11 11 12 12 13 13 20 20 21 21 22 22 23 23 30 30 31 31 32 32 33 33

02 02 00 00 01 01 03 03 12 12 10 10 11 11 13 13 20 20 32 32 30 30 31 31 33 33 21 21 22 22 23 23

01 01 03 03 12 12 21 21 30 30 03 03

32 DS-RT 2011 .

SLIDE 33

2 0 1 3

21 21 02 02 00 00 01 01 03 03 12 12 10 10 11 11 13 13 22 22 20 20 23 23 32 32 30 30 31 31 33 33 2 2 1 1 3 3 R R 00 00 01 01 02 02 03 03 10 10 11 11 12 12 13 13 20 20 21 21 22 22 23 23 30 30 31 31 32 32 33 33

02 02 00 00 01 01 03 03 12 12 10 10 11 11 13 13 20 20 32 32 30 30 31 31 33 33 21 21 22 22 23 23

01 01 03 03 12 12 21 21 30 30 03 03

33 DS-RT 2011 .

SLIDE 34

2 0 1 3

1 02 02 00 00 01 01 3 2 10 10 11 11 13 13 22 22 20 20 23 23 32 32 0 31 31 33 33 2 2 1 1 3 3 R R 00 00 01 01 02 02 03 03 10 10 11 11 12 12 13 13 20 20 21 21 22 22 23 23 30 30 31 31 32 32 33 33

02 02 00 00 01 01 03 03 12 12 10 10 11 11 13 13 20 20 32 32 30 30 31 31 33 33 21 21 22 22 23 23

01 01 03 03 12 12 21 21 30 30 03 03

34 DS-RT 2011 .

SLIDE 35

2 0 1 3

1 02 02 00 00 01 01 3 2 10 10 11 11 13 13 22 22 20 20 23 23 32 32 0 31 31 33 33 2 2 1 1 3 3 R R 00 00 01 01 02 02 03 03 10 10 11 11 12 12 13 13 20 20 21 21 22 22 23 23 30 30 31 31 32 32 33 33

02 02 00 00 01 01 03 03 12 12 10 10 11 11 13 13 20 20 32 32 30 30 31 31 33 33 21 21 22 22 23 23

01 01 03 03 12 12 21 21 30 30 03 03

35 DS-RT 2011 .

SLIDE 36

2 0 1 3

1 02 02 00 00 01 01 3 2 10 10 11 11 13 13 22 22 20 20 23 23 32 32 0 31 31 33 33 2 2 1 1 3 3 R R 00 00 01 01 02 02 03 03 10 10 11 11 12 12 13 13 20 20 21 21 22 22 23 23 30 30 31 31 32 32 33 33

02 02 00 00 01 01 03 03 12 12 10 10 11 11 13 13 20 20 32 32 30 30 31 31 33 33 21 21 22 22 23 23

01 01 03 03 12 12 21 21 30 30 03 03

36 DS-RT 2011 .

SLIDE 37

R R

6 7 4 2 2 5 3 0 1 1

2 2 1 1 3 3

37 DS-RT 2011 .

SLIDE 38

 Simu

mula lator - ns

• ns2

 Network S

k Size

 2-a

• ary H

y HD t D tree o

• f he

height ht 9 9 ( (n = n = 1 1023)

 4-a

• ary H

y HD t D tree o

• f he

height ht 5 5 ( (n = n = 1 1365)

 8-a

• ary H

y HD t D tree o

• f he

height ht 3 3 ( (n = n = 5 585)

 Mult

lti-d

• dime

mens nsiona nal R l Rang nge Q Queries

 Z-O

• Order S

SFC & & DR DROCR

 AVG #

G # o

• f no

nodes a affected / / t total # l # o

• f no

nodes i in t n the he s sys ystem m

 Di

Dime mens nsiona nali lity( y(dim) m): 1 : 1 – – 6 6

 Sele

lectivity y

 from 1

m 1/16di

dim t

to 1 16di

dim/1

/16di

dim i

in 2 n 2 & & 4 4-a

• ary H

y HD t D tree

 from 1

m 1/8di

dim t

to 8 8di

dim/8

/8di

dim i

in 8 n 8-a

• ary H

y HD t D tree

38 DS-RT 2011 .

SLIDE 39

39

Total N l Nodes In Involv lved / / T Total N l Nodes F Func nctioni ning ng Rang nge Q Query S y Size / / E Ent ntire Da Data S Space

DS-RT 2011 .

SLIDE 40

40

• Avg. #

. # o

• f H

Hops Us Used / / 2 2 * * De Depth h Total B l By-p

• passing

ng R Routing ng / / T Total R l Routing ng

DS-RT 2011 .

SLIDE 41

 HD T

D Tree i is t the he f first d data s structure t tha hat i is a able le t to g give a a c comple lete view o

• f t

the he s sys ystem s m state w whe hen p n processing ng M MDR DRQ a at d different nt le levels ls

• f s

sele lectivity. .

 In a

In a k k-ar ary H HD T D Tree

 Root c

can s n survive ( (k-1

• 1) p

point nt f failu lures

 Int

Interna nal no l nodes c can s n survive ( (2k -1

• 1) p

point nt f failu lures

 Leave no

nodes c can s n survive 1 1 p point nt f failu lure

 Routing

ng i in H n HD T D Tree o

• utperforms

ms r routing ng i in t n the he e equivale lent nt t tree structure structure

 MDR

DRQ c can b n be o

• ptimi

mized i in t n the he H HD T D Tree ne network b k because i it i is

 direct ma

mapping ng i in na n nature

 greedy a

y and nd o

• ptima

mal r l routing ng s strategies

 10% o

• f r

routing ng no nodes’ f ’ failu lures d do no not c cause s signi nificant nt performa manc nce v variations ns i in H n HD T D Tree. .

 Furthe

her i improveme ment nt i is p possible le i if a all p ll possible le r routing ng o

• ptions

ns c can n be e explo lored i in t n the he b by-p

• passing

ng r routing ng mo mode.

41 DS-RT 2011 .

SLIDE 42

42

[1] J.L. Bentley, “Multidimensional Binary Search Trees Used for Associative Searching” Commun. of ACM, Vol18, No. 9, (Sep 1975), pp 509-517 [2]R.A. Finkel, and J.L. Bentley, "Quad trees: a data structure for retrieval on composite key." Acta lnformatica 4, 1 (1974), pp 1-9. [3] Hanan, “The Quadtree and Related Hierarchical Data Structures ”, ACM Computing Surveys (CSUR), Volume 16 , Issue 2, (June 1984) pp 187-260 [4] G.M. Morton, “A Computer Oriented Geodetic Data Base and a New Technique in File Sequencing,” technical report, Ottawa, Canada, IBM Ltd. 1966. [5] H. Jagadish. “Linear clustering of objects with multiple Attributes”. In Proc. SIGMOD, Vol 19, Issue 2,1990, pp 332-342 [6] J. Orenstein and T. Merrett. “A class of data structures for associative searching” In Proc. PODS, 1984, pp 181-190 [7] James Aspnes, Gauri Shah “Skip Graphs” Fourteenth Annual ACM-SIAM Symposium on Discrete Algorithms (SODA), January 2003, pp. 384-393.. [8] S. Ratnasamy et al.,”A scalable content addressable network”. Proc. ACM SIGCOMM, 2001, Vol. 31, No. 4. (Oct. 2001) pp. 161-172. [9] I. Stoica, R. Morris et al.,”Chord: a scalable peer-to-peer lookup protocol for Internet applications”.IEEE/ACM Trans. Vol. 11, No. (Feb, 2003), pp. 17-32. [10] P.Ganesan, B. Yang, H. Garcia-molina, “One torus to rule them all: multi-dimensional queries in P2P systems” In WebDB ’04: Proceedings of the 7th International Workshop on the Web and Databases (2004) . pp: 19 – 24 [12] Yanfeng Shu, Beng Chin Ooi, Kian-lee Tan “Supporting Multi-dimensional Range Queries in Peer-to-Peer Systems” In Fifth IEEE International Conference on Peer-to-Peer Computing, 2005. P2P 2005. pp 173 – 180 [13] Chi Zhang, Arvind Krishnamurthy, Randolph Y Wang “SkipIndex: Towards a Scalable Peer-to-Peer Index Service for High Dimensional Data” Vol. TR-703-04 (May 2004). Princeton University Computer Science Department [14] Cristina Schmidt, Manish Parashar “Flexible information discovery in decentralized distributed systems” Proceedings of the 12th IEEE International Symposium on High Performance Distributed Computing (HPDC’03) pp 226 - 235 [15] N. Harvey, M.B. Jones, S. Saroiu, M. Theimer, A. Wolman, “SkipNet: a scalable overlay network with practical locality properties” in: Proceedings

• f the 4th USENIX Symposium on Internet Technologies and Systems USITS’03, March 2003.

[16] K. Aberer, P. Cudre-Mauroux, A. Datta, Z. Despotovic, M. Hauswirth, M. Punceva, R. Schmidt, “P-Grid: a self-organizing structured P2P system” ACM SIGMOD Record 32 (3) (2003) 29–33. [17] A. Bharambe, M. Agrawal, S. Seshan, “Mercury: supporting scalable multi-attribute range queries” SIGCOMM’04, August 30–September 3, 2004. [18] J. Risson, T. Moors, “Survey of research towards robust peer-to-peer networks: search methods” Computer Networks, Vol50 No17, (Dec 2006), pp 3485-3521

DS-RT 2011 .