1 ,1'!-,%%!% .!2 - - PDF document

1

• ! !! !" #! !!$%!&%

'&()!&*%*!$%!&+

Proceedings of IEEE High Performance Distributed Computing conference ( HPDC-8 ), August 1999

,!!!-'&!.! %

'/! )!

• !

!

• (

0 ) 0 ,1'!-,%%!%!&!2 0 !#$+-3 4!#$+#!5 0 %

)

• 0!%.!&!

.-!-!%3! % %%! !- !%6!-* %!- !!- & .!% 0.!&,.& 7!!8 !!!)!)%!*-.'! !!-!%! '! !'%!%! !* %%'

)

(! %-!)!% ..3!!--! 1!)!+% 2 ,)-!!&--#&*- !

• '&-'%!-'/!

** !%&!..3!!-#-!!!#$ 1!!-'!4!#$#!52 %- .! !1..3!!- .!9)%-$2 6!- -.!!* !*.!!#$

, 1'!-,%%!%!&!2

, +! %..3!* !#-!%&-'!-..%& %'%! .! !* .%!*-

Designed by LBL as a part of MAGIC project and US. Departmet of Energy’s high-speed distributed computing program.

Lawrence Berkeley National Laboratory (LBL) Defense Advanced Project Research Agency (DAPRA) Distributed Parallel Storage System (DPSS) www.didc.lbl.gov/DPSS MAGIC: The MAGIC Gigabit Network. www.magic.net

2

,1'!-,%%!% .!2

.! )-!!.*!.& %% %-$- !& :%-$ !'!%!%!-3 !'% $3!!-9

• !) !

%!% *!&!&! + %!'**!'!#!!%#!&!-..3 !* !!#$ !

!

% $!)! !!)! %!

; !<!*'% $*-!!* %!,! !)! = !!)!-!!!#. .,'% $!)!.)!.!'% $ > !!)!*#-!<!!!)! ? !)!!---! %&.! %! Low cost workstations as DPSS block servers (several disks and controllers)

!%+ .! !

From: www.liris.cnrs.fr/~jpierson

,87%!

From: www.liris.cnrs.fr/~jpierson/

,!!

6!-*!%)!%&%%'!*%!*%!

!)!%.-*%!!!.@A ,!* ! %!B? %!1!!)!* ! !2 , %!#..!C*'% $!)! % $!)! %!%!%&#..!C*%%!%-$9 %%! 1!#$%2

%'%!

!!!* !'& !C*%%!%-$

,!!

%.!-!- %!%'&1!.!-*!)!&!)!2 +% ! * !* !!7!!%&%! !* % %'% $ %!%'& %-!*%!7'%!-!% '%& ,'% $!)%'%!.!&!% !- .!

• *%!*%!9*!&!

D.)!#*%!*! %!! & %%&! *'%! +--!)!!)!-$.!*%&1.-#!** ,!)!+,!)!92

3

,8+,

E%!&!3%$!+,

• !
• !
• +%%
• !+%%
• !!$
• !!$
• !
• +& !
• !-
• +& !-
• %!-
• %!
• E%.

FFFFFFF

%!'&+,

dpss_Open dpss_Close dpss_OpenURLRead Connection Routines dpssPrepareDataSets dpssConnectAllRcvStreams dpssConnectAllRequestStreams dpssConnectAllSendStreams dpssReady Request Blocks dpssInitRequestList dpssBuildRequestList dpssAddToRequestList dpssSendRequestList …………………….

(G%$!9! %!+,%'& 1-!-!--!-!!$-%!2

,8(%!

! .! !-**!! !*.!, "# $% *!,!1*%!2% %!

• H! 73-899?%'%)9 3;33= !!399!*%!

! *!*%!*-$.!,

• H! 399!*%! 373-899?%'%)9 3;33= !!

,8(%!

Data layout on the disks is up to the application.

Eg: for sequential reading applications, striping data across servers (round-robin) For Image tiles, the program loaddata is used instead. An example configuration file

# Tile Set Name TSM URL tile_set: file://localhost/home/data3/DPSS/tile-data/Pyramids/ntc- 1-utm-2scene.33.dem tile_set: file://localhost/home/data3/DPSS/tile-data/Pyramids/ntc- 1-utm- 2scene.128.oi servers: lbl-server1 lbl-server2 lbl-server3

!

%' 1!! !$!2 ,1.,!* !!&!2

.!!.'!!33*.!,)!.!, !)!

From: www.didc.lbl.gov/DPSS

4!#$+#!5 !#$+-

'!!**! )!#-!!!#$!)!8 ** !$#%!-!*.!!#$ ** !-'%&'!'%!-& %%& ! *!!%*

• +1+:++!*-!!2

+:+ &!9!#$%1!!*)!! 2 ,'%.!%+,-'!(-!!% !)%9!)!&*!#! - !)!&.1 %- %!2

. !6.!!#$#.%!)%! '!%!-I !#$*)!-& %%& !%!-#$8 1!#$!.!!) !2 # !-

. %*- !- *!!* !

4

,! !)!**!

**!6!J'-#-.7-!%& , %!%'& !,'**!%)%!*! . ! ,!)!

Related work: Automatic TCP Buffer Tuning for NetBSD

Pittsburgh SuperComputing Center

From: www.liris.cnrs.fr/~jpierson/

• %
• % 8*-'% $!!% !-

%8 . %!'!!-!!!)! #.&!)!'!)!%- ,! 6!.!!*'% $!<! !)! E%#9=(developed by Andrew Goldberg and Boris Cherkassky) **%! '*%.

!.! %!<!*.! ! %!-!!.! %!D '% $!<!

!!!#.*.!'% $#%%'!-!%)!!-'& ! .!)!

E%#

• !
• !%&!-!)!K-!%&K%! &*

.!.1* %!!!!)!!)! %!2

• ! &
• #-.*.!!-!

%& & 7'-#-.)%'%!&! %!

8 !)!%!-!&.!.!!%$.!!%!#$ ! -!*!)!..!-!1!-!!!* !% 2

s1 s2 c2 c1 server/source client/sink

E%#

!!!*.!!#$

s1 s3 s2 c2 c1 s4 S´ C´

8 8 8 8 8 8

supersource supersink

c3

8

client/sink server/source

,!* !!%

!!)!

Server A to Client A 112Mbps Client A to Server B 80 Mbps Server B to Client B 11 Mbps Server C to Client A 107 Mbps Client B to Server A 56 Mbps Server D to Client C 85 Mbps File 150MB block size 64KB each client requests 64 blocks

From: www.liris.cnrs.fr/~jpierson/

,!* !!%

!% -%- '% !! ..

3 servers A,C,D were used 3x2 : there are 3 servers and blocks are replicated in 2 servers Client A does the best, client C (slow connection) does the worst Server A is used in the single client test

From: B. Tierney et al., "A Network-Aware Distributed Storage Cache for Data Intensive Environments", HPDC’99

5

,!* !!%

!!-& !%! .! ! #..!..! '-#-.

Throughput for 3 server,2-way replication (3x2)

From: B. Tierney et al., "A Network-Aware Distributed Storage Cache for Data Intensive Environments", HPDC’99

%

!#$+#!! !%&)!.!!* ! .%-'!!% -! .!)!'!!* !

E*.!*8

,*!)%6 Brian et al. “Distributed Parallel Data Storage Systems: A Scalable Approach to High Speed Image Servers,” ACM Multimedia ‘94 .! !*, Brian et al. “System Issues in Implementing High Speed Distributed Parallel Storage Systems,” Proceedings of the USENIX Symposium on High Speed Networking, 1994. !'%!.!, Brian et al. “Performance Analysis in High-Speed Wide-Area ATM Networks: Top-to-Bottom End-to-End Monitoring,” IEEE Network, 1996.