[PPT] - Remote File System Suite Softwarepraktikum f ur Fortgeschrittene PowerPoint Presentation

SLIDE 1

Introduction rfsd librfs grfs Evaluation Conclusion

Remote File System Suite

Softwarepraktikum f¨ ur Fortgeschrittene Michael Kuhn

Parallele und Verteilte Systeme Institut f¨ ur Informatik Ruprecht-Karls-Universit¨ at Heidelberg

2009-07-07

1 / 22

SLIDE 2

Introduction rfsd librfs grfs Evaluation Conclusion

1

Introduction Introduction

2

Remote File System Daemon

3

Remote File System Library

4

Global Remote File System

5

Evaluation

6

Conclusion

2 / 22

SLIDE 3

Introduction rfsd librfs grfs Evaluation Conclusion Introduction

FUSE

Goal was to implement a global network file system

Needed to implement the underlying network file system first

Should be implemented as a FUSE file system

Runs in user space Relatively easy to implement Relatively easy to maintain

High performance

Microscope delivers 1 GB/s

Transparent

Use existing file systems as storage No striping of files

3 / 22

SLIDE 4

Introduction rfsd librfs grfs Evaluation Conclusion Introduction

Overview

rfsd – Remote File System Daemon

Low-level network file system

librfs – Remote File System Library

Abstracts protocol implementation

rfsc – Remote File System Client

Basically a simple throughput and metadata benchmark

grfs – Global Remote File System

High-level global network file system

4 / 22

SLIDE 5

Introduction rfsd librfs grfs Evaluation Conclusion

1

Introduction

2

Remote File System Daemon Motivation Overview Features

3

Remote File System Library

4

Global Remote File System

5

Evaluation

6

Conclusion

5 / 22

SLIDE 6

Introduction rfsd librfs grfs Evaluation Conclusion Motivation

A separate protocol was designed Existing protocols did not meet the requirements SSH

Does not support unencrypted data channels Data encryption makes transfers too slow

Not possible to deactivate the encryption

FTP

Only possible to write a complete file or append data to it File listings are hard to parse, because their format is not well-defined

6 / 22

SLIDE 7

Introduction rfsd librfs grfs Evaluation Conclusion Overview

Implement our own protocol Separate control and data channels

No encryption Control channel can be encrypted via SSH forwarding

Should be as fast as possible

Microscope pumps out 1 GB/s 6 · 2 servers ⇒ 100-200 MB/s

Should be as transparent as possible

Use underlying local file system Do not stripe files across servers

Should be as safe as possible

Support replication

7 / 22

SLIDE 8

Introduction rfsd librfs grfs Evaluation Conclusion Features

Basically provide remote access to the local file system

Protocol very similar to POSIX

pread(), pwrite(), . . .

Plus some fancy features, of course :-)

Fully multi-threaded

Each connection handled in its own thread Long-running operations do not block other connections

Replication

Master-slave concept One master, multiple slaves All operations are replicated in a background thread

Pushed into the thread when the operation begins Check whether the thread finished when the operation ends

8 / 22

SLIDE 9

Introduction rfsd librfs grfs Evaluation Conclusion Features

Logging

All operations are logged when a slave is offline Kept in memory and written to log file Replayed when slave comes online

chroot-like restrictions

All accesses can be restricted to a sub-tree of the file system

9 / 22

SLIDE 10

Introduction rfsd librfs grfs Evaluation Conclusion

1

Introduction

2

Remote File System Daemon

3

Remote File System Library Motivation

4

Global Remote File System

5

Evaluation

6

Conclusion

10 / 22

SLIDE 11

Introduction rfsd librfs grfs Evaluation Conclusion Motivation

Hide all the “ugly” implementation details :-) Good error reporting via GError

Part of GLib

Some operations require multiple steps

For example: rfs read() (“open”), rfs read do() (“pread”), rfs read end() (“close”)

11 / 22

SLIDE 12

Introduction rfsd librfs grfs Evaluation Conclusion

1

Introduction

2

Remote File System Daemon

3

Remote File System Library

4

Global Remote File System Overview Features

5

Evaluation

6

Conclusion

12 / 22

SLIDE 13

Introduction rfsd librfs grfs Evaluation Conclusion Overview

Merge multiple file systems into one global namespace Example:

serv1 has directory /foo, serv2 has directory /bar $ grfs serv1:6666 serv2:6666 /grfs $ ls /grfs > foo bar

13 / 22

SLIDE 14

Introduction rfsd librfs grfs Evaluation Conclusion Features

High Availability

Continues to work when servers go offline Provides a partial view of the file system

setuid-like functionality

Supports FUSE’s allow other option

Fast reads and writes

Keeps the state of read and write operations Accesses to the same file are handled with very little overhead

14 / 22

SLIDE 15

Introduction rfsd librfs grfs Evaluation Conclusion

1

Introduction

2

Remote File System Daemon

3

Remote File System Library

4

Global Remote File System

5

Evaluation Configurations Evaluation

6

Conclusion

15 / 22

SLIDE 16

Introduction rfsd librfs grfs Evaluation Conclusion Configurations

12 Daemons & 12 Clients

One daemon and one client on each machine

6 Daemons & 6 Clients

One daemon on each of the first six machines One client on each of the last six machines

6 Daemons & 12 Clients

One daemon on each of the first six machines Two clients on each of the last six machines

All numbers are per-client

16 / 22

SLIDE 17

Introduction rfsd librfs grfs Evaluation Conclusion Evaluation

write read 20 40 60 80 100 120

Remote File System Performance

12 Daemons & 12 Clients (without Replication) 12 Daemons & 12 Clients 6 Daemons & 6 Clients Configurations MB/s

Σ=968 Σ=437 Σ=216 Σ=1067 Σ=1106 Σ=640

Replication: Significant performance drop

17 / 22

SLIDE 18

Introduction rfsd librfs grfs Evaluation Conclusion Evaluation

create stat access chmod chown unlink 1000 2000 3000 4000 5000 6000

Remote File System Metadata Performance

12 Daemons & 12 Clients (without Replication) 12 Daemons & 12 Clients 6 Daemons & 6 Clients Configurations Operations/s

18 / 22

SLIDE 19

Introduction rfsd librfs grfs Evaluation Conclusion Evaluation

6 Servers/Clients 6 Servers/Clients (Direct I/O) 1 Server/Client 1 Server/Client (Direct I/O) 10 20 30 40 50 60 70 80

Global Remote File System Performance

Write Read Configurations MB/s

Σ=48 Σ=114 Σ=114 Σ=108

1 GBit/s maximum throughput – Read: 1.7 GB/s (cache)

19 / 22

SLIDE 20

Introduction rfsd librfs grfs Evaluation Conclusion Evaluation

6 Servers/Clients 6 Servers/Clients (Direct I/O) 1 Server/Client 1 Server/Client (Direct I/O) 200 400 600 800 1000 1200 1400 1600 1800 2000

Global Remote File System Metadata Performance

create unlink Configurations Operations/s

20 / 22

SLIDE 21

Introduction rfsd librfs grfs Evaluation Conclusion

1

Introduction

2

Remote File System Daemon

3

Remote File System Library

4

Global Remote File System

5

Evaluation

6

Conclusion Conclusion

21 / 22

SLIDE 22

Introduction rfsd librfs grfs Evaluation Conclusion Conclusion

Global Remote File System performance is limited by FUSE

Read and write buffers are at most 128 KB in size FUSE 2.8.0 pre-release supports up to 512 KB

22 / 22