pNFS Parallel Network File System Thijs Stuurman - - PowerPoint PPT Presentation

▶

Mar 13, 2023 614 likes •780 views

pNFS Parallel Network File System Thijs Stuurman Thijs.Stuurman@os3.nl July 2, 2008 Masters program System and Network Engineering 1 / 14 Index Introduction Research question Approach Research pNFS? How does it work?

SLIDE 1

pNFS

Parallel Network File System Thijs Stuurman Thijs.Stuurman@os3.nl July 2, 2008 Masters program System and Network Engineering

1 / 14

SLIDE 2

Index

◮ Introduction ◮ Research question ◮ Approach ◮ Research

◮ pNFS? ◮ How does it work? ◮ pNFS behaviour. . . ◮ pNFS availability & status ◮ Usage scenario ◮ NorStore

◮ Conclusion ◮ Questions

2 / 14

SLIDE 3

NorStore

UNINETT, UNINETT Sigma, UiO (University of Oslo), NTNU (Norwegian University of Science and Technology, UiB (University

f Bergen), UiT (University of Tromsø)

3 / 14

SLIDE 4

Research question

Main research question: ” Is pNFS capable of transferring the large amounts

f data required in the NorStore context?”

Thesis: ” pNFS will be capable of transferring large amounts

f data but is currently still missing good integration with

underlying file systems and under heavy development. Therefore it is currently not ready for production use.”

4 / 14

SLIDE 5

Approach

Approach:

◮ Reading, setting up PoC ◮ Analyse ◮ Analyse some more... ◮ Making sense of it all

5 / 14

SLIDE 6

pNFS?

Example current solutions:

◮ Panasas PanFS ◮ IBM GPFS ◮ Lustre ◮ PVFS2 ◮ EMC MPFS

Working on pNFS:

◮ NetApp ◮ Sun Microsystems ◮ EMC ◮ Panasas ◮ CITI/UMICH

6 / 14

SLIDE 7

pNFS?

Figure: pNFS network overview

7 / 14

SLIDE 8

How does it work?

1. Client ⇒ MDS OPEN
2. Client ⇐ MDS Reply OPEN
3. Client ⇒ MDS LAYOUTGET
4. Client ⇐ MDS Reply LAYOUTGET
5. Client ⇒ DSs Parallel I/O
6. Client ⇒ MDS LAYOUTCOMMIT
7. Client ⇐ MDS Reply LAYOUTCOMMIT
8. Client ⇒ MDS CLOSE
9. Client ⇐ MDS Reply CLOSE

8 / 14

SLIDE 9

Behaviour

◮ Layout size & file size ◮ Small files ◮ Big files

9 / 14

SLIDE 10

pNFS availability & status

◮ File based: Sun, IBM, DESY, NetApp, Red Hat, CITI ◮ Object based: Panasas ◮ Block/Volume based: EMC

Pnfs != pNFS

◮ pNFS file system?

2009. . .

10 / 14

SLIDE 11

Usage scenario’s

◮ Any decent storage platform! ◮ Combined with existing tools as GridFTP

11 / 14

SLIDE 12

NorStore

◮ pNFS is usefull for the NorStore platform and is able to

transfer large amounts of data.

12 / 14

SLIDE 13

Conclusion and future work

Conclusions: ” Is pNFS capable of transferring the large amounts

f data required in the NorStore context?”

Thesis: ” pNFS will be capable of transferring large amounts

f data but is currently still missing good integration with

underlying file systems and under heavy development. Therefore it is currently not ready for production use. Future work:

◮ Wait until pNFS (near)ready! ◮ Use a real 10Gbit environment, these things are tricky! ◮ Use two different storage back-ends, test interoperability.

13 / 14

SLIDE 14

Questions

Questions?

14 / 14