Designing an NFS-based Mobile Distributed File System for Ephemeral - - PowerPoint PPT Presentation

1

ASWN’04, August 11, 2004

Designing an NFS-based Mobile Distributed File System for Ephemeral Sharing in Proximity Networks

Dimitris Kalofonos

Pervasive Computing Group (PCG) Nokia Research Center, Boston, MA

Nikolaos Michalakis

Computer Science Department New York University, New York, NY

2

ASWN’04, August 11, 2004

Outline

• Ephemeral vs. Persistent file sharing
• Motivation for using NFS as a basis for Ephemeral M-DFS
• Related Approaches
• NFS Performance in Proximity Networks
• Problems with Mobility
• Experimental and Simulation Setup
• Performance Results
• Designing an M-DFS for Ephemeral Sharing
• M-DFS Design Topics and Suggestions
• An Implementation Proposal
• Future Directions

3

ASWN’04, August 11, 2004

Ephemeral vs. Persistent File Sharing

>> cd /sophia/pictures >> ./netscape /tom/cool_page.html >> /sophia/mplayer /mary/music/cool_song.mp3 >> /tom/xview /jack/cool_pic.jpg >> ./excel /office/not_so_cool_spreadsheet.xls >> ls /home/my_files/ HERMES’ PHONE

PROXIMITY NETWORK

INTERNET

TOM JOHN HERMES SOPHIA JACK

HOME

HERMES

OFFICE

HERMES

Example of persistent sharing Example of ephemeral sharing

4

ASWN’04, August 11, 2004

Ephemeral vs. Persistent File Sharing (cont.)

• Ephemeral file sharing: a remote file system is discovered and mounted to

enable short-lived collaboration among users

• Intended mainly for read-only file sharing
• Client caching takes place mainly to improve performance (e.g. in low

bandwidth-links). It does not guarantee disconnected operation

• Deals with short disconnections (order of seconds-minutes) due to

intermittent connectivity at the link level. No state is expected to survive long (as defined by the user) disconnections

• If files are to remain accessible in the long-term, user “saves” local

copies

• Persistent file sharing: enables users to keep a personal distributed file

repository among their devices (e.g. phones, PC’s)

• Intended mainly for read-write file sharing
• Client performs (Coda-like) hoarding when fully connected, can operate
• n files when disconnected, reintegration-conflict resolution when

reconnected

• Disconnections can be arbitrarily long (order of hours-days)
• Files are accessible in the long-term without “saving” local copies

5

ASWN’04, August 11, 2004

Why Use NFS as Basis for Ephemeral M-DFS?

• NFS perhaps the most successful DFS to date. Widely used,

well tested

• Native support for NFS in Linux kernel. Appealing for Linux-

based mobile devices and experimentation

• NFS interface has made it attractive as a basis to build DFS

with additional features. Tools available (e.g. SFS toolkit) for user–level implementation. Makes portability across platforms possible

• Light mounting and unmounting process allows discovering

exported files systems and mounting on demand

• Allows RPC-based access to remote files, without requiring the

transfer of whole files

• Not as complex as other popular DFS for mobility (e.g. CODA).

More suitable as a basis to build M-DFS for resource- constraint devices

6

ASWN’04, August 11, 2004

Related Approaches

• CODA/Intermezzo
• Supports true disconnected operation
• Client performs aggressive caching (hoarding) when

connected

• Optimizations available for weak connectivity
• Intended for Persistent sharing
• Some design aspects applicable to Ephemeral sharing.

However, too complex to be used as is. Removing functionality tricky

• NFS/M
• NFS modified to support (CODA-like) disconnected operation
• Requires Kernel modifications
• Intended for Persistent sharing. Redundant functionality for us
• SFS/LBFS
• User-level DFS on top of NFS to add security, low BW

enhancements

• SFS toolkit available that deals with issues of loopback NFS

servers

7

ASWN’04, August 11, 2004

NFS Performance in Proximity Networks: Experimental Setup

• NFS v3 client and server on two laptops running Linux 2.4.18
• Mobility: IP address changes by attaching to different AP
• Intermittent connectivity: disconnections by releasing IP

address

• 802.11b measurements
• Transfers with both client and server on same AP
• Data rate selection: 1Mbps, 2Mbps, 11Mbps
• Bluetooth measurements
• BlueZ stack v2.2 with PAN profile 1.0
• Bluetooth v1.1 USB adapters
• Simple piconet with a Master (NFS Server) and a Slave

(NFS Client)

• Increasing distance increased the RTT because of link-level

re-Tx

• Data rate selection: DH1, DH3, DH5

8

ASWN’04, August 11, 2004

NFS Performance in Proximity Networks: Simulation

Client1 Client2 Server1 Server2 Link11 Link12 Link21 Link22 Hub1 Hub2 Routing Host eth0 eth1

Network simulator using the ‘Click’ modular router

9

ASWN’04, August 11, 2004

NFS Performance in Proximity Networks: Simulation (cont.)

Delay Module Bandwidth Module Disconnection Module Loss Module Delay Module Bandwidth Module Disconnection Module Loss Module Server to client path Client to server path Control Socket eth0 eth1

The configurable link

10

ASWN’04, August 11, 2004

NFS Problems with Mobility and Disconnections

No errors. As long as the reassignment does not take longer than the timeout, the client only experiences a delay. Server disconnects and reconnects with old IP address: No errors. As long as the reassignment does not take longer than the timeout, the client only experiences a delay. Client disconnects and reconnects with old IP address: Client cannot mount. Mount process gets “can't get address for server” error. If hard mounted client blocks forever. If soft mounted it timeouts and fails (I/O error). Server network interface down: The mounted directory is empty, i.e. not mounted to anything. Network unreachable error, no matter whether data cached or not at the client. Client network interface down: No errors. Client is trying to contact old

• address. If hard mounted then it

blocks forever. If soft mounted it timeouts and fails (I/O error). Server IP address change: Client receives an “access denied” error. The server has already created a list of client addresses that can access the exported file systems. Client receives a “stale filehandle”

• error. The client is using a filehandle

that is exported to another client address. Client IP address change: Client not mounted (server already started) Client has mounted

11

ASWN’04, August 11, 2004

NFS in Proximity Networks: Performance Results

time to read file (s)

Effect of RTT vs Bandwidth (measurements)

2 4 6 8 10 12 50 100 150 200 250 300 350

bandwidth (KB/s)

WLAN BTH

10K file 50K file 100K file

time to read file (s)

Effect of RTT vs Bandwidth (measurements)

2 4 6 8 10 12 50 100 150 200 250 300 350

bandwidth (KB/s)

WLAN BTH

10K file 50K file 100K file

Effect of RTT vs Bandwidth (measurements)

2 4 6 8 10 12 50 100 150 200 250 300 350

bandwidth (KB/s)

WLAN WLAN BTH BTH

10K file 50K file 100K file 10K file 50K file 100K file

) ( bw rpc RTT N T + × ≈

12

ASWN’04, August 11, 2004

NFS in Proximity Networks: Performance Results (cont.)

Effect of RTT vs Bandwidth (simulation model) 10 20 30 40 50 60 70 1 10 100 1000

Bandwidth (KB/s) log scale

Time to read 50K file (s)

RTT=200ms RTT=100ms RTT=20ms Effect of RTT vs Bandwidth (simulation model) 10 20 30 40 50 60 70 1 10 100 1000

Bandwidth (KB/s) log scale

Time to read 50K file (s)

RTT=200ms RTT=100ms RTT=20ms RTT=200ms RTT=100ms RTT=20ms

) ( bw rpc RTT N T + × ≈

13

ASWN’04, August 11, 2004

NFS in Proximity Networks: Performance Results (cont.)

BTH dark zone (measurements)

20 40 60 80 100 120 140 20 40 60 80 100 120 140

Time (s) RPC seq. no

timeout =0.1s timeout =0.5s timeout =1s timeout =10s T C P UDP, timeout =0.1s UDP, timeout =0.5s UDP, timeout =1s UDP, timeout =10s T C P

BTH dark zone (measurements)

20 40 60 80 100 120 140 20 40 60 80 100 120 140

Time (s) RPC seq. no

timeout =0.1s timeout =0.5s timeout =1s timeout =10s T C P timeout =0.1s timeout =0.5s timeout =1s timeout =10s T C P UDP, timeout =0.1s UDP, timeout =0.5s UDP, timeout =1s UDP, timeout =10s T C P

BTH dark zone (measurements)

0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 20 40 60 80 100 120

RPC req. no time to reply (s)

timeout =0.1s timeout =0.5s timeout =1s timeout =10s TCP

BTH dark zone (measurements)

0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 20 40 60 80 100 120

RPC req. no time to reply (s)

timeout =0.1s timeout =0.5s timeout =1s timeout =10s TCP timeout =0.1s timeout =0.5s timeout =1s timeout =10s TCP

14

ASWN’04, August 11, 2004

NFS in Proximity Networks: Performance Results (cont.)

Effect of disconnections (measurements)

500 1000 1500 2000 2500 3000 10 20 30 40 50 60 70 80 90 100 Time (s) RPC seq. no cold no discon. UDP warm no discon. UDP cold with discon. UDP warm with discon. UDP cold with discon. TCP warm with discon. TCP

5.5s 5.5s 5.5s finish when no time wasted (warm) finish when no time wasted (cold) wasted time (warm) wasted time (cold)

Effect of disconnections (measurements)

500 1000 1500 2000 2500 3000 10 20 30 40 50 60 70 80 90 100 Time (s) RPC seq. no cold no discon. UDP warm no discon. UDP cold with discon. UDP warm with discon. UDP cold with discon. TCP warm with discon. TCP

5.5s 5.5s 5.5s finish when no time wasted (warm) finish when no time wasted (cold) wasted time (warm) wasted time (cold)

15

ASWN’04, August 11, 2004

NFS in Proximity Networks: Performance Results (cont.)

Effect of packet loss rate (simulation model)

50 100 150 200 250 300 350 5 10 15 20 25

packet loss rate (%) time to read 100K file (s) timeo=0.1s timeo=0.5s timeo=1s timeo=10s TCP

Effect of packet loss rate (simulation model)

50 100 150 200 250 300 350 5 10 15 20 25

packet loss rate (%) time to read 100K file (s) timeo=0.1s timeo=0.5s timeo=1s timeo=10s TCP

16

ASWN’04, August 11, 2004

M-DFS Design Topics and Suggestions

• More aggressive caching to support temporary disconnections
• In the case of RO ephemeral sharing caching can be simplified, e.g.

callbacks with leases: close-to-open consistency, reduced traffic

• In addition to file and directory contents, client caches metadata such

as picture thumbnails or mp3 samples

• Automatic session recovery to deal with errors because of client/server

mobility and loss of connectivity

• Session support mechanism: client and server have active session

(server exported and client mounted the file system), data in transit. Need to decide whether to resume or drop pending sessions upon reconnection

• Session recovery mechanism: the server has started a session and is

waiting for client to join (server only exported file system). Need to preserve the file system state related to mounting

• Techniques for traffic reduction
• data compression techniques, application specific data reduction

techniques using metadata (e.g. thumbnails, mp3 samples)

17

ASWN’04, August 11, 2004

M-DFS Design Topics and Suggestions (cont.)

• Enhanced metadata to help deal with weak connectivity networks
• Application specific metadata (e.g. thumbnails, mp3 samples) are
• paque to the M-DFS. The M-DFS allocates extra space and an API for

applications to use them

• M-DFS specific metadata are seen only by the FS and the OS. Can be

used to improve performance, provide enhanced capabilities and make the system more adaptive to mobile conditions. Example: estimated file transmission time

• Access control and distributed authentication
• ACL provide better security than Unix access control
• Use “ephemeral” ACLs, i.e. ACLs that are created and updated by the

server owner on the fly. Remove stale entries in ACLs to avoid long ACLs with few active user.

• Distributed client/server authentication an issue without ‘trusted

authorities’. Simplest solution would be to use offline means, e.g. user- to-user communication

• Naming and file system removal
• A global name space not required. Server repository may be identified

explicitly by the device name that exports it. Uniform name space desirable

18

ASWN’04, August 11, 2004

An Implementation Proposal

RO M-DFS Client (loopback server) Cache Manager (loopback client) RO M-DFS Server (loopback client)

App VFS VFS NFS client NFS server NFS server local FS local FS User-level Kernel-level M-DFS server

file_op vnode file_op inode NFS RPC M-DFS RPC

M-DFS client

RO M-DFS Client (loopback server) Cache Manager (loopback client) RO M-DFS Server (loopback client)

App VFS VFS NFS client NFS server NFS server local FS local FS User-level Kernel-level M-DFS server

file_op vnode file_op inode NFS RPC M-DFS RPC file_op vnode file_op inode NFS RPC M-DFS RPC

M-DFS client

19

ASWN’04, August 11, 2004

Future Directions

• Conclude the proposed implementation and experiment with

the design suggestions. Assess performance of different design choices

• Experiment with the M-DFS for ephemeral sharing over ad-hoc

testbed using Bluetooth

• Explore alternatives to NFS to base design of M-DFS, e.g.

HTTP-based approach