SLIDE 1
Background Research question Test approach SURFnet design Conclusion Future work
2
By Jeroen Klaver & Roel van der Jagt Background Research - - PowerPoint PPT Presentation
By Jeroen Klaver & Roel van der Jagt Background Research - - PowerPoint PPT Presentation
By Jeroen Klaver & Roel van der Jagt Background Research question Test approach SURFnet design Conclusion Future work 2 SURFnet is the Dutch NREN Storage explosion 3 Add more storage 4 And more...
SLIDE 2
SLIDE 3
SURFnet is the Dutch NREN Storage explosion
3
SLIDE 4
Add more storage
4
SLIDE 5
And more... But adding includes high costs and new bottlenecks
5
SLIDE 6
SURFnet thinks there is a smarter way
6
SLIDE 7
7
SLIDE 8
8
SLIDE 9
9
SLIDE 10
What infrastructure and open source tools
provide SURFnet or the participants a scalable and distributed any-kind-storage solution?
10
SLIDE 11
Coda Lustre GlusterFS XtreemFS Ceph PVFS MooseFS Hadoop
11
SLIDE 12
First stage
- Null test
Second stage
- Architectures in the lab environment
Third stage
- Architectures in practice
12
SLIDE 13
13
SLIDE 14
10000 20000 30000 40000 50000 60000 70000 80000 90000 256 8192 256 8192 256 8192 256 8192 Sequential read Sequential write Random read Random write Speed (KBytes/sec) Block size (KBytes)
Iozone - Lustre file system benchmark
Lustre avg Null test avg
14
SLIDE 15
10000 20000 30000 40000 50000 60000 70000 80000 90000 256 8192 256 8192 256 8192 256 8192 Sequential read Sequential write Random read Random write Speed (KBytes/sec) Block size (KBytes)
Iozone - Lustre with latency
No added latency avg Latency (20ms RTT) avg
15
SLIDE 16
16
SLIDE 17
10000 20000 30000 40000 50000 60000 70000 256 8192 256 8192 256 8192 256 8192 Sequential read Sequential write Random read Random write Speed (KBytes/sec) Block size (KBytes)
Iozone - Coda file system benchmark
Coda avg Null test avg
17
SLIDE 18
10000 20000 30000 40000 50000 60000 256 8192 256 8192 256 8192 256 8192 Sequential read Sequential write Random read Random write Speed (KBytes/sec) Block size (KBytes)
Iozone - Coda with latency
No added latency avg Latency (20ms RTT) avg
18
SLIDE 19
10000 20000 30000 40000 50000 60000 70000 80000 90000 256 8192 256 8192 256 8192 256 8192 Sequential read Sequential write Random read Random write Speed (KBytes/sec) Block size (KBytes)
Iozone - File system benchmark
Lustre avg Coda avg Null test avg
19
SLIDE 20
100 200 300 400 500 600 700 min avg max Seeks (seeks/sec)
Bonnie++ - Seeks
Lustre Coda Null test
20
SLIDE 21
10000 20000 30000 40000 50000 60000 70000 80000 90000 256 8192 256 8192 256 8192 256 8192 Sequential read Sequential write Random read Random write Speed (KBytes/sec) Block size (KBytes)
Iozone - Coda vs Lustre with latency
Lustre No added latency avg Lustre Latency (20ms RTT) avg Coda No added latency avg Coda Latency (20ms RTT) avg
21
SLIDE 22
Requirements
- Scalable
- Available
- Durable
- Performance
- Dynamic
- Cost effective
- Generic interfaces
- Open protocols
- Geographic dispersion
Deployment \ Maintenance Security features
22
SLIDE 23
Requirements
- Scalable
- Available
- Durable
- Performance
- Dynamic
- Cost effective
- Generic interfaces
- Open protocols
- Geographic dispersion
Deployment \ Maintenance Security features
23
SLIDE 24
Coda:
- Hard to configure
- Can not handle large files
- Promising architecture
- Low latency impact
Lustre:
- DRBD is needed for replication / latency reduction
- Latency has impact
- No multi-tier
- Promising future
24
SLIDE 25
Review of GlusterFS Test environment without VMWare ESXi Further tuning and configuring of Coda Coda and ZFS tests Lustre and DRBD tests
25
SLIDE 26
(Source: mastergoogle.com)
26
SLIDE 27
27
SLIDE 28
28
SLIDE 29
29
SLIDE 30
30
SLIDE 31
31
SLIDE 32
32
SLIDE 33
33
SLIDE 34
Coda Lustre
min avg max No added latency
27,6 28 28,4 Latency (20ms RTT) 9,5 9,8 10
min avg max No added latency 151 153 155 Latency (20ms RTT) 28,7 29 29,2
34