Algorithms and Methods for Distributed Storage Networks 1. - - PowerPoint PPT Presentation

▶
algorithms and methods for distributed storage networks
SMART_READER_LITE
LIVE PREVIEW

Algorithms and Methods for Distributed Storage Networks 1. - - PowerPoint PPT Presentation

Aug 28, 2023 208 likes •452 views

Algorithms and Methods for Distributed Storage Networks 1. Motivation, Organization, Overview Christian Schindelhauer Albert-Ludwigs-Universitt Freiburg Institut fr Informatik Rechnernetze und Telematik Wintersemester 2007/08

slide-1
SLIDE 1

Albert-Ludwigs-Universität Freiburg Institut für Informatik Rechnernetze und Telematik Wintersemester 2007/08

Algorithms and Methods for Distributed Storage Networks

  • 1. Motivation, Organization, Overview

Christian Schindelhauer

slide-2
SLIDE 2

Distributed Storage Networks Winter 2008/09 Rechnernetze und Telematik Albert-Ludwigs-Universität Freiburg Christian Schindelhauer

Organization

  • Lecture
  • Thursday 11 am - 1pm, room 101/SR 01-009/13
  • Friday 11 am - 12 pm, room 101/SR 01-009/13
  • Exercise (Stefan Rührup)
  • starts Oct 29, 2008
  • Friday 12 - 1 pm, room 101/SR 01-009/13
  • appear every Friday on the web-pages
  • solved voluntarily by students
  • are the bases for the oral exam
  • solutions of the exercises are discussed in the following week

2

slide-3
SLIDE 3

Peer-to-Peer-Networks Summer 2008 Computer Networks and Telematics Albert-Ludwigs-Universität Freiburg Christian Schindelhauer

Web

  • Web page
  • http://cone.informatik.uni-freiburg.de/teaching/

vorlesung/distributed-storage-w08

  • Slides, exercises, link to forum
  • Forum
  • for discussion, links, funnies etc.
  • http://cone.informatik.uni-freiburg.de/forum/

viewforum.php?f=28

3

slide-4
SLIDE 4

Peer-to-Peer-Networks Summer 2008 Computer Networks and Telematics Albert-Ludwigs-Universität Freiburg Christian Schindelhauer

Exam

  • Dates by appointment
  • possible dates
  • Monday, 23.02.2009
  • Thursday, 05.03.2009
  • Tuesday, 07.04.2009
  • Contact me during the lecture or send an E-Mail to
  • Oral exam
  • based on the lecture and the exercises
  • Mandatory registration
  • Students of computer science register at the secretary of

exams (Prüfungssekretariat)

4

slide-5
SLIDE 5

Distributed Storage Networks Winter 2008/09 Rechnernetze und Telematik Albert-Ludwigs-Universität Freiburg Christian Schindelhauer

Motivation Evolution of Disks

Algorithms and Methods for Distributed Storage Networks

5

slide-6
SLIDE 6

Figure 6 Cost of storage at the disk drive and system level

PRICE/MBYTE, DOLLARS 100 10 1 0.1 0.01 0.001 0.0001 1000 1980 1985 1990 1995 2000 2005 2010 PRODUCTION YEAR DESKTOP HDD HIGH PERFOR- MANCE, SERVER HDD INDUSTRY PROJECTION FROM MULTIPLE SOURCES SYSTEMS W/LARGE FF DRIVES SYSTEMS W/SMALL FF DRIVES SMALL FF HDD STORAGE SYSTEM

Figure 7 Cost of storage for disk drive, paper, film, and semiconductor memory

Distributed Storage Networks Winter 2008/09 Rechnernetze und Telematik Albert-Ludwigs-Universität Freiburg Christian Schindelhauer

Fall in Storage Prices

6

Technological impact of magnetic hard disk drives on storage systems, Grochowski, R. D. Halem IBM SYSTEMS JOURNAL, VOL 42, NO 2, 2003

slide-7
SLIDE 7

Figure 7 Cost of storage for disk drive, paper, film, and semiconductor memory

PRICE (DOLLARS PER MEGABYTE) 100 10 1 0.1 0.01 0.001 0.0001 1000 1980 1985 1990 1995 2000 2005 PRODUCTION YEAR PAPER AND FILM 3.5 INCH 2.5 INCH SEMICONDUCTOR MEMORY (DRAM AND FLASH) DISK DRIVES

Distributed Storage Networks Winter 2008/09 Rechnernetze und Telematik Albert-Ludwigs-Universität Freiburg Christian Schindelhauer

Price Fall of RAM and Disk Storage

7

Technological impact of magnetic hard disk drives on storage systems, Grochowski, R. D. Halem IBM SYSTEMS JOURNAL, VOL 42, NO 2, 2003

slide-8
SLIDE 8

Figure 1 Hard disk drive areal density trend

AFC=ANTIFERROMAGNETICALLY COUPLED GMR=GIANT MAGNETORESISTIVE RANGE OF POSSIBLE VALUES AREAL DENSITY MEGABITS/SQUARE INCH 105 104 103 102 10 1 10–1 10–2 10–3 106 1970 1960 1980 1990 2000 2010 PRODUCTION YEAR IBM RAMAC™ (FIRST HARD DISK DRIVE) ~35 MILLION X INCREASE 1st THIN FILM HEAD 25% CGR 60% CGR 100% CGR 1st MR HEAD 1st GMR HEAD 1st AFC MEDIA IBM DISK DRIVE PRODUCTS INDUSTRY LAB DEMOS

Distributed Storage Networks Winter 2008/09 Rechnernetze und Telematik Albert-Ludwigs-Universität Freiburg Christian Schindelhauer

Increase of Density

8

Technological impact of magnetic hard disk drives on storage systems, Grochowski, R. D. Halem IBM SYSTEMS JOURNAL, VOL 42, NO 2, 2003

slide-9
SLIDE 9

Figure 2 Storage floor space utilization trend — IBM storage systems

RAW CAPACITY PER FLOOR SPACE AREA, GBYTES/SQUARE FOOT 103 102 101 100 10–1 10–2 10–3 10–4 104 1970 1960 1980 1990 2000 2010 PRODUCTION YEAR IBM RAMAC (FIRST HARD DISK DRIVE) ESS MOD F 3380 RAMAC 2 IBM ESS MOD 800 25% CGR 60% CGR RANGE OF POSSIBLE VALUES ESS=IBM TOTALSTORAGE™ ENTERPRISE STORAGE SERVER™

Distributed Storage Networks Winter 2008/09 Rechnernetze und Telematik Albert-Ludwigs-Universität Freiburg Christian Schindelhauer

Increase of Density (Floor Space)

9

Technological impact of magnetic hard disk drives on storage systems, Grochowski, R. D. Halem IBM SYSTEMS JOURNAL, VOL 42, NO 2, 2003

slide-10
SLIDE 10

Figure 3 Floor space required to store 1 terabyte

FLOOR SPACE AREA, SQUARE FOOT 106 105 104 103 102 10 1 0.1 107 1970 1960 1980 1990 2000 2010 PRODUCTION YEAR IBM RAMAC (FIRST HARD DISK DRIVE) ESS MOD F 3380 RAMAC 2 ESS MOD 800 RANGE OF POSSIBLE VALUES

Distributed Storage Networks Winter 2008/09 Rechnernetze und Telematik Albert-Ludwigs-Universität Freiburg Christian Schindelhauer

Increase of Density (Floor Space)

10

Technological impact of magnetic hard disk drives on storage systems, Grochowski, R. D. Halem IBM SYSTEMS JOURNAL, VOL 42, NO 2, 2003

slide-11
SLIDE 11

Figure 4 Hard disk drive volumetric density trend

VOLUMETRIC DENSITY, GBITS/CUBIC INCH 103 102 101 100 10–1 10–2 10–3 104 1980 1985 1990 1995 2000 2005 2010 PRODUCTION YEAR RANGE OF POSSIBLE VALUES CGR = 100% CGR = 25% RANGE OF POSSIBLE VALUES HARD DISK DRIVE FORM FACTOR 14/10.8 INCH 5.25 3.5 2.5 1.0

Distributed Storage Networks Winter 2008/09 Rechnernetze und Telematik Albert-Ludwigs-Universität Freiburg Christian Schindelhauer

Increase of Density (Cubic Space)

11

Technological impact of magnetic hard disk drives on storage systems, Grochowski, R. D. Halem IBM SYSTEMS JOURNAL, VOL 42, NO 2, 2003

slide-12
SLIDE 12

Distributed Storage Networks Winter 2008/09 Rechnernetze und Telematik Albert-Ludwigs-Universität Freiburg Christian Schindelhauer

Evolution of Disk Form Factors

12

Technological impact of magnetic hard disk drives on storage systems, Grochowski, R. D. Halem IBM SYSTEMS JOURNAL, VOL 42, NO 2, 2003

Figure 8 The evolution of disk drive form factors 1956–2002

24-INCH FORM FACTOR 5 MB (NOT SHOWN) 10.8-INCH FORM FACTOR 5.7 GB 14-INCH FORM FACTOR 3.8 GB 1.0-INCH FORM FACTOR 1 GB 8-INCH FORM FACTOR 65 MB 2.5-INCH FORM FACTOR 60 GB 3.25-INCH FORM FACTOR 84-MM DISKS/10000 RPM 146 GB 70-MM DISKS/15000 RPM 36 GB 3.25-INCH FORM FACTOR 120 GB

slide-13
SLIDE 13

Distributed Storage Networks Winter 2008/09 Rechnernetze und Telematik Albert-Ludwigs-Universität Freiburg Christian Schindelhauer

Increase of Speed

13

Technological impact of magnetic hard disk drives on storage systems, Grochowski, R. D. Halem IBM SYSTEMS JOURNAL, VOL 42, NO 2, 2003 Figure 10 Hard disk drive maximum internal data rate for enterprise/server drives

MAXIMUM INTERNAL DATA RATE, MBYTES/SECONDS 100 10 1 0.1 1000 1970 1980 1990 2000 AVAILABILITY YEAR DATA RATE = LINEAR x RPM x DISK DENSITY DIAMETER <10% CGR 40% CGR

slide-14
SLIDE 14

Distributed Storage Networks Winter 2008/09 Rechnernetze und Telematik Albert-Ludwigs-Universität Freiburg Christian Schindelhauer

Increase of Speed

14

Technological impact of magnetic hard disk drives on storage systems, Grochowski, R. D. Halem IBM SYSTEMS JOURNAL, VOL 42, NO 2, 2003

Figure 11 Disk drive access/seek times

TIME, MILLISECONDS 10 1 100 1970 1975 1980 1985 1990 1995 2000 2005 AVAILABILITY YEAR SEEKING ACCESSING ULTRASTAR XP ULTRASTAR 2XP ULTRASTAR 18XP ULTRASTAR 9ZX ULTRASTAR 36XP ULTRASTAR 18ZX ULTRASTAR 36ZX ULTRASTAR 73LZX ULTRASTAR 36LZX ULTRASTAR 36Z15 SEEK TIME ~ (ACTUATOR INERTIALPOWER)1/3 x (DATA BAND)2/3 ROTATIONAL TIME (LATENCY) ~ (RPM)-1 ACCESS TIME = SEEK TIME + LATENCY ~ ~

slide-15
SLIDE 15

Distributed Storage Networks Winter 2008/09 Rechnernetze und Telematik Albert-Ludwigs-Universität Freiburg Christian Schindelhauer

Motivation Consumer Behavior

Algorithms and Methods for Distributed Storage Networks

15

slide-16
SLIDE 16

Distributed Storage Networks Winter 2008/09 Rechnernetze und Telematik Albert-Ludwigs-Universität Freiburg Christian Schindelhauer

Consumer Usage

  • Consumer Survey on Digital Storage in Consumer Electronics

2008, Coughlin Associates (Dec. 2007)

  • 51% said that 1 TB disk would be useful
  • Most storage of content was on hard disk
  • 46% backup data less than once per year
  • except pictures most of them do not backup
  • but most think it is important to have backups out of their homes
  • Most people want to store entire TV series, copies of their entire

music collection

  • Projection
  • by 2013 average home has 9 Terabyte
  • by 2015 user content sums up to 650 Exabyte

16

slide-17
SLIDE 17

Distributed Storage Networks Winter 2008/09 Rechnernetze und Telematik Albert-Ludwigs-Universität Freiburg Christian Schindelhauer

Storage Hierarchy

  • Primary storage
  • Processors registers
  • Processor cache
  • RAM
  • Secondary storage
  • Hard disks
  • Solid state disks
  • CD, DVD
  • Tertiary storage
  • tape libraries
  • optical jukeboxes

17

slide-18
SLIDE 18

Distributed Storage Networks Winter 2008/09 Rechnernetze und Telematik Albert-Ludwigs-Universität Freiburg Christian Schindelhauer

Characteristics of Storage

  • Volatile — non-volatile memory
  • non-volatile: dynamic or static
  • Read & write — Read only — Slow write, fast

read

  • Random access – Sequential access
  • Addressability
  • location addressable
  • file addressable
  • content addressable
  • Capacity
  • Performance
  • Latency
  • Throughput

18

slide-19
SLIDE 19

Distributed Storage Networks Winter 2008/09 Rechnernetze und Telematik Albert-Ludwigs-Universität Freiburg Christian Schindelhauer

Non-volatile Storage Technologies

  • Punch cards (Hollerith) 1886-1950s
  • Magnetic tape data storage 1951-today
  • Hard disk drive 1956-today
  • Floppy disks 1970s-1990s
  • EEPROM (Electrically Erasable Programmable

Read-Only Memory) 1980-today

  • Flash memory
  • Optical disc drive (read/write) 1997-today

19

slide-20
SLIDE 20

Distributed Storage Networks Winter 2008/09 Rechnernetze und Telematik Albert-Ludwigs-Universität Freiburg Christian Schindelhauer

Network Storage Types

  • Direct attached storage (DAS)
  • traditional storage
  • Network attached storage (NAS)
  • storage attached to another computer accessible at file level
  • ver LAN or WAN
  • Storage area network (SAN)
  • specialized network providing other computers with storage

capacity with access on block-addressing level

  • File area network (FAN)
  • systematic approach to organize file-related storage systems
  • rganization wide high-level storage network

20

slide-21
SLIDE 21

Distributed Storage Networks Winter 2008/09 Rechnernetze und Telematik Albert-Ludwigs-Universität Freiburg Christian Schindelhauer

Overview

  • Basic Storage Technology
  • Hard disks
  • Flash memory, solid state disks
  • Storage device design
  • File systems
  • Classic file systems
  • Network and distributed file

systems

  • Storage organization
  • SAN, NAS, FAN
  • Storage hierarchies, Tiers
  • Selected topics of Distributed

Algorithms

  • Conflict resolution
  • Cache strategies
  • Redundancy
  • RAID levels
  • Coding techniques
  • Internet and storage
  • TCP/IP

, FTP , Webdav, etc.

  • Distributed Storage Systems
  • Online storage
  • e.g. Amazon S3, Google Shared

Storage

  • Peer-to-peer network storage
  • e.g. Oceanstore

21

slide-22
SLIDE 22

Albert-Ludwigs-Universität Freiburg Institut für Informatik Rechnernetze und Telematik Wintersemester 2007/08

Algorithms and Methods for Distributed Storage Networks

  • 1. Organization & Overview

Christian Schindelhauer