Current Status of CVMFS Fast, scalable, and reliable software - - PowerPoint PPT Presentation

▶
current status of cvmfs
SMART_READER_LITE
LIVE PREVIEW

Current Status of CVMFS Fast, scalable, and reliable software - - PowerPoint PPT Presentation

Nov 22, 2022 279 likes •586 views

Current Status of CVMFS Fast, scalable, and reliable software distribution Jakob Blomer July 3rd, 2009 1 / 20 Outline 1 Overview 2 Recent Development 3 Benchmarks 4 Next Steps 2 / 20 1 Overview 2 Recent Development 3 Benchmarks 4 Next Steps 3

slide-1
SLIDE 1

Current Status of CVMFS

Fast, scalable, and reliable software distribution Jakob Blomer July 3rd, 2009

1 / 20

slide-2
SLIDE 2

Outline

1 Overview 2 Recent Development 3 Benchmarks 4 Next Steps

2 / 20

slide-3
SLIDE 3

1 Overview 2 Recent Development 3 Benchmarks 4 Next Steps

3 / 20

slide-4
SLIDE 4

Software Distribution for Virtual Machines

  • “Ready to run” binaries, i. e.

/mnt/cvmfs mirrors destination of make install

  • Read-only, public files

4 / 20

slide-5
SLIDE 5

Repository Statistics

Latest Release, CernVM x86 Platform

  • Many (many) small files
  • In the repository: significant amount of redundancy

# Files Size Median 10th / 90th Percentile ZIP-Ratio ALICE 38 000 1.6 GB 3.3 KB 330 Byte / 24 KB 2.8 ATLAS 150 000 6.5 GB 2.2 KB 217 Byte / 36 KB 10.7 CMS 110 000 5 GB 2.1 KB 320 Byte / 20 KB 1.9 LHCb 166 000 4.5 GB 1.5 KB 32 Byte / 16 KB 2.6

5 / 20

slide-6
SLIDE 6

Infrastructure

Reference setup: Apache + Squid Key: Set of web caches serving static HTTP content from memory

6 / 20

slide-7
SLIDE 7

1 Overview 2 Recent Development 3 Benchmarks 4 Next Steps

7 / 20

slide-8
SLIDE 8

The CernVM File System

Evolved from GROW-FS Building Blocks Components Fuse Modules

SHA1 MD5 zlib SQLite libcurl libcrypto Catalog Cache Prefetcher Quota / LRU Trace Capturing CVMFS SCVMFS

8 / 20

slide-9
SLIDE 9

Distinctive Features

  • Satisfied with outgoing HTTP(S) connection, i. e.

works with practically every Internet connection

  • Every downloaded file is verified by SHA1
  • Chain of forward proxy servers
  • Pre-built cache packages
  • Offline mode
  • Trace file system operations
  • Nested catalogs
  • Catalogs can be signed by X.509 certificate
  • Catalogs store TTL, thereby stay up to date

9 / 20

slide-10
SLIDE 10

Distinctive Features

  • Satisfied with outgoing HTTP(S) connection, i. e.

works with practically every Internet connection

  • Every downloaded file is verified by SHA1
  • Chain of forward proxy servers
  • Pre-built cache packages
  • Offline mode
  • Trace file system operations
  • Nested catalogs
  • Catalogs can be signed by X.509 certificate
  • Catalogs store TTL, thereby stay up to date

9 / 20

slide-11
SLIDE 11

Distinctive Features

  • Satisfied with outgoing HTTP(S) connection, i. e.

works with practically every Internet connection

  • Every downloaded file is verified by SHA1
  • Chain of forward proxy servers
  • Pre-built cache packages
  • Offline mode
  • Trace file system operations
  • Nested catalogs
  • Catalogs can be signed by X.509 certificate
  • Catalogs store TTL, thereby stay up to date

9 / 20

slide-12
SLIDE 12

Distinctive Features

  • Satisfied with outgoing HTTP(S) connection, i. e.

works with practically every Internet connection

  • Every downloaded file is verified by SHA1
  • Chain of forward proxy servers
  • Pre-built cache packages
  • Offline mode
  • Trace file system operations
  • Nested catalogs
  • Catalogs can be signed by X.509 certificate
  • Catalogs store TTL, thereby stay up to date

9 / 20

slide-13
SLIDE 13

Distinctive Features

  • Satisfied with outgoing HTTP(S) connection, i. e.

works with practically every Internet connection

  • Every downloaded file is verified by SHA1
  • Chain of forward proxy servers
  • Pre-built cache packages
  • Offline mode
  • Trace file system operations
  • Nested catalogs
  • Catalogs can be signed by X.509 certificate
  • Catalogs store TTL, thereby stay up to date

9 / 20

slide-14
SLIDE 14

Distinctive Features

  • Satisfied with outgoing HTTP(S) connection, i. e.

works with practically every Internet connection

  • Every downloaded file is verified by SHA1
  • Chain of forward proxy servers
  • Pre-built cache packages
  • Offline mode
  • Trace file system operations
  • Nested catalogs
  • Catalogs can be signed by X.509 certificate
  • Catalogs store TTL, thereby stay up to date

9 / 20

slide-15
SLIDE 15

Distinctive Features

  • Satisfied with outgoing HTTP(S) connection, i. e.

works with practically every Internet connection

  • Every downloaded file is verified by SHA1
  • Chain of forward proxy servers
  • Pre-built cache packages
  • Offline mode
  • Trace file system operations
  • Nested catalogs
  • Catalogs can be signed by X.509 certificate
  • Catalogs store TTL, thereby stay up to date

9 / 20

slide-16
SLIDE 16

Distinctive Features

  • Satisfied with outgoing HTTP(S) connection, i. e.

works with practically every Internet connection

  • Every downloaded file is verified by SHA1
  • Chain of forward proxy servers
  • Pre-built cache packages
  • Offline mode
  • Trace file system operations
  • Nested catalogs
  • Catalogs can be signed by X.509 certificate
  • Catalogs store TTL, thereby stay up to date

9 / 20

slide-17
SLIDE 17

Distinctive Features

  • Satisfied with outgoing HTTP(S) connection, i. e.

works with practically every Internet connection

  • Every downloaded file is verified by SHA1
  • Chain of forward proxy servers
  • Pre-built cache packages
  • Offline mode
  • Trace file system operations
  • Nested catalogs
  • Catalogs can be signed by X.509 certificate
  • Catalogs store TTL, thereby stay up to date

9 / 20

slide-18
SLIDE 18

SCVMFS

SCVMFS

make install Catalog, Compression

  • SCVMFS provides a frontend to a repository directory
  • Missing part: atomic switch between repository versions

(e. g. with ZFS)

10 / 20

slide-19
SLIDE 19

Performance

  • Locally
  • Multi-Threaded
  • SQLite for catalogs instead of

in-memory tree representation

  • Pure data cache,

decoupled from catalog

SHA1-Cache

  • Network Traffic
  • HTTP Keep-Alive
  • Prefetching
  • Compression

Protocol Overhead

Web Server CernVM S Y N S Y N , A C K SYN H T T P G E T 200 OK F I N , A C K ACK F I N , A C K A C K

70 MB repository = ⇒ 20 MB network traffic

11 / 20

slide-20
SLIDE 20

Performance

Prefetcher

High predictability: ≈ 10-50 shared libraries per binary Construct prefetch hints:

1 Follow ldd-dependencies 2 Automatic construction from traces

Analysis of open() sequence

Automatic Construction (ROOT Extract)

libCore.so.5.22 (7) libCint.so.5.22 (7) libRIO.so.5.22 (7) libNet.so.5.22 (7) libHist.so.5.22 (7) libGraf.so.5.22 (7) libGraf3d.so.5.22 (7) libMinuit.so (11) 18% 18% 100% 100% 100% 100% 100% 100% 100% 100% 100% 100% 100%

12 / 20

slide-21
SLIDE 21

Performance

Prefetcher

High predictability: ≈ 10-50 shared libraries per binary Construct prefetch hints:

1 Follow ldd-dependencies 2 Automatic construction from traces

Analysis of open() sequence

Automatic Construction (ROOT Extract)

libCore.so.5.22 (7) libCint.so.5.22 (7) libRIO.so.5.22 (7) libNet.so.5.22 (7) libHist.so.5.22 (7) libGraf.so.5.22 (7) libGraf3d.so.5.22 (7) libMinuit.so (11) 18% 18% 100% 100% 100% 100% 100% 100% 100% 100% 100% 100% 100%

12 / 20

slide-22
SLIDE 22

1 Overview 2 Recent Development 3 Benchmarks 4 Next Steps

13 / 20

slide-23
SLIDE 23

Setup

We measure extra time for application benchmark stressHepix residing on CVMFS / AFS

14 / 20

slide-24
SLIDE 24

High Latency

∆t [s] Round Trip Time [ms] Extra Running Time AFS CVMFS + Keep-Alive + Prefetching + Compression 100 200 300 400 500 600 700 800 LAN 25 50 100 150

15 / 20

slide-25
SLIDE 25

WAN Simulation

∆t [s] iPerf-measured Throughput [mbit/s] Extra Running Time (RTT 100ms) AFS CVMFS + Keep-Alive + Prefetching, Compression 100 200 300 400 500 600 700 800 5 10 15 20 25

16 / 20

slide-26
SLIDE 26

Gbit Ethernet

Running Time [s] 177 178 179 180 181 182 183 184 185 186 187 Local Storage AFS CVMFS Current

17 / 20

slide-27
SLIDE 27

1 Overview 2 Recent Development 3 Benchmarks 4 Next Steps

18 / 20

slide-28
SLIDE 28

Next Steps

  • Testing
  • Zero-Conf, in particular nearby proxy server
  • “Share My Cache”: Replace Squid by co-operative peers

19 / 20

slide-29
SLIDE 29

Current Status of CVMFS

Fast, scalable, and reliable software distribution Jakob Blomer July 3rd, 2009

20 / 20