Distributed Storage Networks and Computer Forensics 5 Raid-6 - - PowerPoint PPT Presentation

▶
distributed storage networks and computer forensics
SMART_READER_LITE
LIVE PREVIEW

Distributed Storage Networks and Computer Forensics 5 Raid-6 - - PowerPoint PPT Presentation

Jul 16, 2023 297 likes •709 views

Distributed Storage Networks and Computer Forensics 5 Raid-6 Encoding Christian Schindelhauer University of Freiburg Technical Faculty Computer Networks and Telematics Winter Semester 2011/12 Donnerstag, 3. November 11 RAID Redundant

slide-1
SLIDE 1

University of Freiburg Technical Faculty Computer Networks and Telematics Winter Semester 2011/12

Distributed Storage Networks and Computer Forensics

5 Raid-6 Encoding

Christian Schindelhauer

Donnerstag, 3. November 11

slide-2
SLIDE 2

Distributed Storage Networks and Computer Forensics Winter 2011/12 Computer Networks and Telematics University of Freiburg Christian Schindelhauer

RAID

  • Redundant Array of Independent Disks
  • Patterson, Gibson, Katz, „A Case for Redundant Array of

Inexpensive Disks“, 1987

  • Motivation
  • Redundancy
  • error correction and fault tolerance
  • Performance (transfer rates)
  • Large logical volumes
  • Exchange of hard disks, increase of storage during
  • peration
  • Cost reduction by use of inexpensive hard disks

2

Donnerstag, 3. November 11

slide-3
SLIDE 3

http://en.wikipedia.org/wiki/RAID

Distributed Storage Networks and Computer Forensics Winter 2011/12 Computer Networks and Telematics University of Freiburg Christian Schindelhauer

Raid 1

  • Mirrored set without parity
  • Fragments are stored on all disks
  • Performance
  • if multi-threaded operating system

allows split seeks then

  • faster read performance
  • write performance slightly reduced
  • Error correction or redundancy
  • all but one hard disks can fail without

any data damage

  • Capacity reduced by factor 2

3

Donnerstag, 3. November 11

slide-4
SLIDE 4

http://en.wikipedia.org/wiki/RAID

Distributed Storage Networks and Computer Forensics Winter 2011/12 Computer Networks and Telematics University of Freiburg Christian Schindelhauer

Raid 3

  • Striped set with dedicated parity (byte

level parity)

  • Fragments are distributed on all but one

disks

  • One dedicated disk stores a parity of

corresponding fragments of the other disks

  • Performance
  • improved read performance
  • write performance reduced by

bottleneck parity disk

  • Error correction or redundancy
  • one hard disks can fail without any data

damage

  • Capacity reduced by 1/n

4

Donnerstag, 3. November 11

slide-5
SLIDE 5

http://en.wikipedia.org/wiki/RAID

Distributed Storage Networks and Computer Forensics Winter 2011/12 Computer Networks and Telematics University of Freiburg Christian Schindelhauer

Raid 5

  • Striped set with distributed parity

(interleave parity)

  • Fragments are distributed on all but one

disks

  • Parity blocks are distributed over all disks
  • Performance
  • improved read performance
  • improved write performance
  • Error correction or redundancy
  • one hard disks can fail without any data

damage

  • Capacity reduced by 1/n

5

Donnerstag, 3. November 11

slide-6
SLIDE 6

http://en.wikipedia.org/wiki/RAID

Distributed Storage Networks and Computer Forensics Winter 2011/12 Computer Networks and Telematics University of Freiburg Christian Schindelhauer

Raid 6

  • Striped set with dual distributed parity
  • Fragments are distributed on all but two

disks

  • Parity blocks are distributed over two of

the disks

  • one uses XOR other alternative

method

  • Performance
  • improved read performance
  • improved write performance
  • Error correction or redundancy
  • two hard disks can fail without any data

damage

  • Capacity reduced by 2/n

6

Donnerstag, 3. November 11

slide-7
SLIDE 7

Distributed Storage Networks and Computer Forensics Winter 2011/12 Computer Networks and Telematics University of Freiburg Christian Schindelhauer

RAID 6 - Encodings

Algorithms and Methods for Distributed Storage Networks

7

Donnerstag, 3. November 11

slide-8
SLIDE 8

Distributed Storage Networks and Computer Forensics Winter 2011/12 Computer Networks and Telematics University of Freiburg Christian Schindelhauer

Literature

  • A Tutorial on Reed-Solomon Coding for Fault-

Tolerance in RAID-like Systems, James S. Plank , 1999

  • The RAID-6 Liberation Codes, James S. Plank, FAST

´08, 2008

8

Donnerstag, 3. November 11

slide-9
SLIDE 9

Distributed Storage Networks and Computer Forensics Winter 2011/12 Computer Networks and Telematics University of Freiburg Christian Schindelhauer

Principle of RAID 6

  • Data units D1, ..., Dn
  • w: size of words
  • w=1 bits,
  • w=8 bytes, ...
  • Checksum devices C1,C2,..., Cm
  • computed by functions

Ci=Fi(D1,...,Dn)

  • Any n words from data words and

check words

  • can decode all n data units

9 A Tutorial on Reed-Solomon Coding for Fault-Tolerance in RAID-like Systems, James S. Plank , 1999

Donnerstag, 3. November 11

slide-10
SLIDE 10

Distributed Storage Networks and Computer Forensics Winter 2011/12 Computer Networks and Telematics University of Freiburg Christian Schindelhauer

Principle of RAID 6

10 A Tutorial on Reed-Solomon Coding for Fault-Tolerance in RAID-like Systems, James S. Plank , 1999

Donnerstag, 3. November 11

slide-11
SLIDE 11

Distributed Storage Networks and Computer Forensics Winter 2011/12 Computer Networks and Telematics University of Freiburg Christian Schindelhauer

Operations

  • Encoding
  • Given new data elements, calculate the check sums
  • Modification (update penalty)
  • Recompute the checksums (relevant parts) if one data

element is modified

  • Decoding
  • Recalculate lost data after one or two failures
  • Efficiency
  • speed of operations
  • check disk overhead
  • ease of implementation and transparency

11

Donnerstag, 3. November 11

slide-12
SLIDE 12

Distributed Storage Networks and Computer Forensics Winter 2011/12 Computer Networks and Telematics University of Freiburg Christian Schindelhauer

Reed-Solomon

RAID 6 Encodings

12

Donnerstag, 3. November 11

slide-13
SLIDE 13

Distributed Storage Networks and Computer Forensics Winter 2011/12 Computer Networks and Telematics University of Freiburg Christian Schindelhauer

Vandermonde-Matrix

13 A Tutorial on Reed-Solomon Coding for Fault-Tolerance in RAID-like Systems, James S. Plank , 1999

Donnerstag, 3. November 11

slide-14
SLIDE 14

Distributed Storage Networks and Computer Forensics Winter 2011/12 Computer Networks and Telematics University of Freiburg Christian Schindelhauer

Complete Matrix

14 A Tutorial on Reed-Solomon Coding for Fault-Tolerance in RAID-like Systems, James S. Plank , 1999

Donnerstag, 3. November 11

slide-15
SLIDE 15

Distributed Storage Networks and Computer Forensics Winter 2011/12 Computer Networks and Telematics University of Freiburg Christian Schindelhauer

Galois Fields

15

  • GF(2w) = Finite Field over 2w elements
  • Elements are all binary strings of length w
  • 0 = 0w is the neutral element for addition
  • 1 = 0w-11 is the neutral element for multiplication
  • u + v = bit-wise Xor of the elements
  • e.g. 0101 + 1100 = 1001
  • a b= product of polynomials modulo 2 and modulo an

irreducible polynomial q

  • i.e. (aw-1 ... a1 a0) (bw-1 ... b1 b0) =

Donnerstag, 3. November 11

slide-16
SLIDE 16

Distributed Storage Networks and Computer Forensics Winter 2011/12 Computer Networks and Telematics University of Freiburg Christian Schindelhauer

Example: GF(22)

16

+ 0 = 00 1 = 01 2 = 10 3 = 11 0 =00 1 2 3 1 =01 1 3 2 2 =10 2 3 1 3 =11 3 2 1 * 0 = 1 = 1 2 = x 3 = x+1 0 = 0 1 = 1 1 2 3 2 = x 2 3 1 3 = x+1 3 1 2

q(x) = x2+x+1 2.3 = x(x+1) = x2+x = 1 mod x2+x+1 = 1 2.2 = x2 = x+1 mod x2+x+1 = 3

Donnerstag, 3. November 11

slide-17
SLIDE 17

Distributed Storage Networks and Computer Forensics Winter 2011/12 Computer Networks and Telematics University of Freiburg Christian Schindelhauer

Irreducible Polynomials

  • Irreducible polynomials cannot be factorized
  • counter-example: x2+1 = (x+1)2 mod 2
  • Examples:
  • w=2: x2+x+1
  • w=4: x4+x+1
  • w=8: x8+x4+x3+x2+1
  • w=16: x16+x12+x3+x+1
  • w=32: x32+x22+x2+x+1
  • w=64: x64+x4+x3+x+1

17

Donnerstag, 3. November 11

slide-18
SLIDE 18

Distributed Storage Networks and Computer Forensics Winter 2011/12 Computer Networks and Telematics University of Freiburg Christian Schindelhauer

Fast Multiplication

  • Powers laws
  • Consider: {20, 21, 22,...}
  • = {x0, x1, x2, x3, ...
  • = exp(0), exp(1), ...
  • exp(x+y) = exp(x) exp(y)
  • Inverse: log(exp(x)) = x
  • log(x.y) = log(x) + log(y)
  • x y = exp(log(x) + log(y))
  • Warning: integer addition!!!
  • Use tables to compute exponential and logarithm function

18

Donnerstag, 3. November 11

slide-19
SLIDE 19

Distributed Storage Networks and Computer Forensics Winter 2011/12 Computer Networks and Telematics University of Freiburg Christian Schindelhauer

Example: GF(16)

19

q(x)= x4+x+1

  • 5 . 12 = exp(log(5)+log(12)) = exp(8+6) = exp(14) = 9
  • 7 . 9 = exp(log(7)+log(9)) = exp(10+14) = exp(24) = exp(24-15)

= exp(9) = 10

x 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 exp(x) 1 x x2 x3 1+x x+x2 x2+ x3 1+x +x3 1+x2 x+x3 1+x +x2 x +x2+ x3 1+x +x2+ x3 1+x2 +x3 1+x3 1 exp(x) 1 2 4 8 3 6 12 11 5 10 7 14 15 13 9 1 x 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 log(x) 1 4 2 8 5 10 3 14 9 7 6 13 11 12

Donnerstag, 3. November 11

slide-20
SLIDE 20

Distributed Storage Networks and Computer Forensics Winter 2011/12 Computer Networks and Telematics University of Freiburg Christian Schindelhauer

Example: Reed Solomon for GF[24]

  • Compute carry bits for three hard disks by computing
  • F D = C
  • where D is the vector of three data words
  • C is the vector of the three parity words
  • Store D and C on the disks

20

Donnerstag, 3. November 11

slide-21
SLIDE 21

Distributed Storage Networks and Computer Forensics Winter 2011/12 Computer Networks and Telematics University of Freiburg Christian Schindelhauer

Complexity of Reed-Solomon

  • Encoding
  • Time: O(k n) GF[2w]-operations for k check words and n

disks

  • Modification
  • like Encoding
  • Decoding
  • Time: O(n3) for matrix inversion
  • Ease of implementation
  • check disk overhead is minimal
  • complicated decoding

21

Donnerstag, 3. November 11

slide-22
SLIDE 22

Distributed Storage Networks and Computer Forensics Winter 2011/12 Computer Networks and Telematics University of Freiburg Christian Schindelhauer

Cauchy-Reed-Solomon

  • An XOR-Based Erasure-Resilient Coding Scheme, Blömer,

Kalfane, Karp, Karpinski, Luby, Zuckerman, 1995

22

Donnerstag, 3. November 11

slide-23
SLIDE 23

Distributed Storage Networks and Computer Forensics Winter 2011/12 Computer Networks and Telematics University of Freiburg Christian Schindelhauer

Complexity of Cauchy-Reed- Solomon

  • Encoding
  • Time: O(k n) GF[2w]-operations for k check words and n

disks

  • Modification
  • like Encoding
  • Decoding
  • Time: O(n2) for matrix inversion
  • Ease of implementation
  • check disk overhead is minimal
  • less complicated decoding, still not transparent

23

Donnerstag, 3. November 11

slide-24
SLIDE 24

Distributed Storage Networks and Computer Forensics Winter 2011/12 Computer Networks and Telematics University of Freiburg Christian Schindelhauer

Parity Arrays

RAID 6 Encodings

24

Donnerstag, 3. November 11

slide-25
SLIDE 25

Distributed Storage Networks and Computer Forensics Winter 2011/12 Computer Networks and Telematics University of Freiburg Christian Schindelhauer

Parity Arrays

  • Uses Parity of data bits
  • Each check bit collects different

subset of data bits

  • Examples
  • Evenodd
  • RDP

25

Donnerstag, 3. November 11

slide-26
SLIDE 26

Distributed Storage Networks and Computer Forensics Winter 2011/12 Computer Networks and Telematics University of Freiburg Christian Schindelhauer

Hamming Code

  • Use adapted version of Hamming code to compute

check bits

  • Problem: not flexible encoding for various number of

disks or check codes

26

The RAID-6 Liberation Codes James S. Plank

Donnerstag, 3. November 11

slide-27
SLIDE 27

Distributed Storage Networks and Computer Forensics Winter 2011/12 Computer Networks and Telematics University of Freiburg Christian Schindelhauer

One-Dimensional Parity

  • Organize data bits as n/m groups
  • compute parity for each group
  • Results in m check bits
  • Fast and simple computation for
  • Coding, Decoding, Modification
  • Problem
  • tolerates not all combinations of

failures

  • unsafe solution for combined failure of

check disk and data disk

27 A Tutorial on Reed-Solomon Coding for Fault-Tolerance in RAID-like Systems, James S. Plank , 1999

Donnerstag, 3. November 11

slide-28
SLIDE 28

Distributed Storage Networks and Computer Forensics Winter 2011/12 Computer Networks and Telematics University of Freiburg Christian Schindelhauer

Two-Dimensional Parity

  • Organize data disks as a k*k-square
  • compute k parities for all rows
  • compute k parities for all columns
  • Results in 2k check bits
  • Fast computation for
  • Coding, Decoding, Modification
  • Safety
  • tolerate only all combinations for

two failures

  • tolerates not all combinations for

three failures

  • Problem
  • large number of hard disks
  • check disk overhead

28 A Tutorial on Reed-Solomon Coding for Fault-Tolerance in RAID-like Systems, James S. Plank , 1999

Donnerstag, 3. November 11

slide-29
SLIDE 29

Distributed Storage Networks and Computer Forensics Winter 2011/12 Computer Networks and Telematics University of Freiburg Christian Schindelhauer

EVENODD-Encoding

  • Computes exactly two check words
  • P = parity check word
  • Q = parity over the diagonal elements
  • Fast Encoding
  • Decoding
  • O(n2) time for n disks and n data bits
  • Optimal check disk overhead
  • Generalized versions
  • STAR code (Huang, Xu, FAST‘05)
  • Feng, Deng, Bao, Shen, 2005

29

The RAID-6 Liberation Codes James S. Plank

Donnerstag, 3. November 11

slide-30
SLIDE 30

Distributed Storage Networks and Computer Forensics Winter 2011/12 Computer Networks and Telematics University of Freiburg Christian Schindelhauer

EVENODD

30

Donnerstag, 3. November 11

slide-31
SLIDE 31

Distributed Storage Networks and Computer Forensics Winter 2011/12 Computer Networks and Telematics University of Freiburg Christian Schindelhauer

RDP Coding

  • Row Diagonal Parity
  • improved version of EVENODD
  • Two check words
  • Parity over words
  • Use diagonal parities
  • Easier code
  • Creates only two check words

31

The RAID-6 Liberation Codes James S. Plank

Donnerstag, 3. November 11

slide-32
SLIDE 32

Distributed Storage Networks and Computer Forensics Winter 2011/12 Computer Networks and Telematics University of Freiburg Christian Schindelhauer

Liberation Codes

32 The RAID-6 Liberation Codes James S. Plank

Donnerstag, 3. November 11

slide-33
SLIDE 33

Distributed Storage Networks and Computer Forensics Winter 2011/12 Computer Networks and Telematics University of Freiburg Christian Schindelhauer

RDP

33

The RAID-6 Liberation Codes James S. Plank

Donnerstag, 3. November 11

slide-34
SLIDE 34

Distributed Storage Networks and Computer Forensics Winter 2011/12 Computer Networks and Telematics University of Freiburg Christian Schindelhauer

Liberation Codes

34 The RAID-6 Liberation Codes James S. Plank

Donnerstag, 3. November 11

slide-35
SLIDE 35

Distributed Storage Networks and Computer Forensics Winter 2011/12 Computer Networks and Telematics University of Freiburg Christian Schindelhauer

Liberation Codes

35 The RAID-6 Liberation Codes James S. Plank

Donnerstag, 3. November 11

slide-36
SLIDE 36

Distributed Storage Networks and Computer Forensics Winter 2011/12 Computer Networks and Telematics University of Freiburg Christian Schindelhauer

Liberation Codes

36 The RAID-6 Liberation Codes James S. Plank

Donnerstag, 3. November 11

slide-37
SLIDE 37

Distributed Storage Networks and Computer Forensics Winter 2011/12 Computer Networks and Telematics University of Freiburg Christian Schindelhauer

Liberation Codes

37 The RAID-6 Liberation Codes James S. Plank

Donnerstag, 3. November 11

slide-38
SLIDE 38

Distributed Storage Networks and Computer Forensics Winter 2011/12 Computer Networks and Telematics University of Freiburg Christian Schindelhauer

Performance

38 The RAID-6 Liberation Codes James S. Plank

Donnerstag, 3. November 11

slide-39
SLIDE 39

Distributed Storage Networks and Computer Forensics Winter 2011/12 Computer Networks and Telematics University of Freiburg Christian Schindelhauer

Performance

39 The RAID-6 Liberation Codes James S. Plank

Donnerstag, 3. November 11

slide-40
SLIDE 40

Distributed Storage Networks and Computer Forensics Winter 2011/12 Computer Networks and Telematics University of Freiburg Christian Schindelhauer

Performance

40 The RAID-6 Liberation Codes James S. Plank

Donnerstag, 3. November 11

slide-41
SLIDE 41

University of Freiburg Technical Faculty Computer Networks and Telematics Winter Semester 2011/12

Distributed Storage Networks and Computer Forensics

5 Raid-6 Encoding

Christian Schindelhauer

Donnerstag, 3. November 11