Log2fs or how to achieve 150.000 IO/s J orn Engel Lazybastard.org - - PowerPoint PPT Presentation

▶
log2fs or how to achieve 150 000 io s
SMART_READER_LITE
LIVE PREVIEW

Log2fs or how to achieve 150.000 IO/s J orn Engel Lazybastard.org - - PowerPoint PPT Presentation

Aug 19, 2022 569 likes •961 views

Hardware Compression costs Log2 Advanced Credit Log2fs or how to achieve 150.000 IO/s J orn Engel Lazybastard.org September 24, 2010 J orn Engel Log2fs or how to achieve 150.000 IO/s Hardware Compression costs Log2 Advanced Credit

slide-1
SLIDE 1

Hardware Compression costs Log2 Advanced Credit

Log2fs or how to achieve 150.000 IO/s

J¨

  • rn Engel

Lazybastard.org

September 24, 2010

J¨

  • rn Engel

Log2fs or how to achieve 150.000 IO/s

slide-2
SLIDE 2

Hardware Compression costs Log2 Advanced Credit

Just a bunch of random hacks

J¨

  • rn Engel

Lazybastard.org

September 24, 2010

J¨

  • rn Engel

Just a bunch of random hacks

slide-3
SLIDE 3

Hardware Compression costs Log2 Advanced Credit

Flash basics

Fast random reads Fast somethat-random writes Erase before write Large eraseblocks

J¨

  • rn Engel

Log2fs or how to achieve 150.000 IO/s

slide-4
SLIDE 4

Hardware Compression costs Log2 Advanced Credit

Blocks and Pages

256K 256K ... 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K ...

J¨

  • rn Engel

Log2fs or how to achieve 150.000 IO/s

slide-5
SLIDE 5

Hardware Compression costs Log2 Advanced Credit

Drais card

PCIe x4 interface 1 FPGA 64 Flash chips

J¨

  • rn Engel

Log2fs or how to achieve 150.000 IO/s

slide-6
SLIDE 6

Hardware Compression costs Log2 Advanced Credit

Drais card

J¨

  • rn Engel

Log2fs or how to achieve 150.000 IO/s

slide-7
SLIDE 7

Hardware Compression costs Log2 Advanced Credit

Drais driver

Simple MTD driver 64 queues for requests Does error correction Adds FIO interface

J¨

  • rn Engel

Log2fs or how to achieve 150.000 IO/s

slide-8
SLIDE 8

Hardware Compression costs Log2 Advanced Credit

FIO interface

Adds three new methods to struct mtd info fio read fio write fio erase

J¨

  • rn Engel

Log2fs or how to achieve 150.000 IO/s

slide-9
SLIDE 9

Hardware Compression costs Log2 Advanced Credit

FIO interface

fio read reads exactly 1 page, then calls fio->fi end io

J¨

  • rn Engel

Log2fs or how to achieve 150.000 IO/s

slide-10
SLIDE 10

Hardware Compression costs Log2 Advanced Credit

FIO interface

fio write writes exactly 1 page, then calls fio->fi end io

J¨

  • rn Engel

Log2fs or how to achieve 150.000 IO/s

slide-11
SLIDE 11

Hardware Compression costs Log2 Advanced Credit

FIO interface

fio erase erases exactly 1 block, then calls fio->fi end io

J¨

  • rn Engel

Log2fs or how to achieve 150.000 IO/s

slide-12
SLIDE 12

Hardware Compression costs Log2 Advanced Credit

wait multiple

wait multiple waits for N fios to complete

J¨

  • rn Engel

Log2fs or how to achieve 150.000 IO/s

slide-13
SLIDE 13

Hardware Compression costs Log2 Advanced Credit

Read Performance

Single threaded: 6800 IO/s or 27MB/s 4096 threads: 149000 IO/s 610MB/s Scales 22x

J¨

  • rn Engel

Log2fs or how to achieve 150.000 IO/s

slide-14
SLIDE 14

Hardware Compression costs Log2 Advanced Credit

Write Performance

Single threaded: 40 IO/s or 10MB/s 4096 threads: 1859 IO/s or 480MB/s Scales 46x

J¨

  • rn Engel

Log2fs or how to achieve 150.000 IO/s

slide-15
SLIDE 15

Hardware Compression costs Log2 Advanced Credit

Read performance

J¨

  • rn Engel

Log2fs or how to achieve 150.000 IO/s

slide-16
SLIDE 16

Hardware Compression costs Log2 Advanced Credit

Write performance

J¨

  • rn Engel

Log2fs or how to achieve 150.000 IO/s

slide-17
SLIDE 17

Hardware Compression costs Log2 Advanced Credit

Compression in LogFS

byte-precise packing indirect blocks contain pointers block headers contain compressed size

J¨

  • rn Engel

Log2fs or how to achieve 150.000 IO/s

slide-18
SLIDE 18

Hardware Compression costs Log2 Advanced Credit

Alignment

many blocks span a page boundary

J¨

  • rn Engel

Log2fs or how to achieve 150.000 IO/s

slide-19
SLIDE 19

Hardware Compression costs Log2 Advanced Credit

Alignment

J¨

  • rn Engel

Log2fs or how to achieve 150.000 IO/s

slide-20
SLIDE 20

Hardware Compression costs Log2 Advanced Credit

Alignment

uncompressed and compressed blocks are mixed

J¨

  • rn Engel

Log2fs or how to achieve 150.000 IO/s

slide-21
SLIDE 21

Hardware Compression costs Log2 Advanced Credit

Writes

write header and compressed data to buffer

  • ccasionally flush buffer

J¨

  • rn Engel

Log2fs or how to achieve 150.000 IO/s

slide-22
SLIDE 22

Hardware Compression costs Log2 Advanced Credit

Reads

read header plus maximal blocksize to cache uncompress

J¨

  • rn Engel

Log2fs or how to achieve 150.000 IO/s

slide-23
SLIDE 23

Hardware Compression costs Log2 Advanced Credit

Deletions

Read header into cache Use compressed size for accounting

J¨

  • rn Engel

Log2fs or how to achieve 150.000 IO/s

slide-24
SLIDE 24

Hardware Compression costs Log2 Advanced Credit

Cache

Cache has a granularity of (MMU)PAGE SIZE

J¨

  • rn Engel

Log2fs or how to achieve 150.000 IO/s

slide-25
SLIDE 25

Hardware Compression costs Log2 Advanced Credit

Cache

Oops!

J¨

  • rn Engel

Log2fs or how to achieve 150.000 IO/s

slide-26
SLIDE 26

Hardware Compression costs Log2 Advanced Credit

Deletions

J¨

  • rn Engel

Log2fs or how to achieve 150.000 IO/s

slide-27
SLIDE 27

Hardware Compression costs Log2 Advanced Credit

Deletions

J¨

  • rn Engel

Log2fs or how to achieve 150.000 IO/s

slide-28
SLIDE 28

Hardware Compression costs Log2 Advanced Credit

Log2

Don’t mix uncompressed and compressed blocks Align uncompressed blocks

J¨

  • rn Engel

Log2fs or how to achieve 150.000 IO/s

slide-29
SLIDE 29

Hardware Compression costs Log2 Advanced Credit

Log2

Move compressed size to indirect blocks ...and a number of other fields ...and remove (most) direct pointers from inodes

J¨

  • rn Engel

Log2fs or how to achieve 150.000 IO/s

slide-30
SLIDE 30

Hardware Compression costs Log2 Advanced Credit

Venti

Efficient way to store multiple identical copies Ideal for large universities Horrible for personal computers

J¨

  • rn Engel

Log2fs or how to achieve 150.000 IO/s

slide-31
SLIDE 31

Hardware Compression costs Log2 Advanced Credit

VentiLog

Add a block hashtable Check hashtable before writes Increment refcount when possible

J¨

  • rn Engel

Log2fs or how to achieve 150.000 IO/s

slide-32
SLIDE 32

Hardware Compression costs Log2 Advanced Credit

BtrLog

Add reference count to block pointers copyfile() becomes possible clones become possible

J¨

  • rn Engel

Log2fs or how to achieve 150.000 IO/s

slide-33
SLIDE 33

Hardware Compression costs Log2 Advanced Credit

Birthday attack

LogFS stores directory entries in a hash table.

J¨

  • rn Engel

Log2fs or how to achieve 150.000 IO/s

slide-34
SLIDE 34

Hardware Compression costs Log2 Advanced Credit

Birthday attack

Given N random numbers between 1 and M (N ≤ M), what is the probability of having two or more identical numbers?

J¨

  • rn Engel

Log2fs or how to achieve 150.000 IO/s

slide-35
SLIDE 35

Hardware Compression costs Log2 Advanced Credit

Birthday attack

1 − M!−N!

MN

J¨

  • rn Engel

Log2fs or how to achieve 150.000 IO/s

slide-36
SLIDE 36

Hardware Compression costs Log2 Advanced Credit

Birthday attack

J¨

  • rn Engel

Log2fs or how to achieve 150.000 IO/s

slide-37
SLIDE 37

Hardware Compression costs Log2 Advanced Credit

Birthday attack

Given N random numbers between 1 and M (N ≤ M), what is the probability of having O or more identical numbers (O ≤ N ≤ O · M)?

J¨

  • rn Engel

Log2fs or how to achieve 150.000 IO/s

slide-38
SLIDE 38

Hardware Compression costs Log2 Advanced Credit J¨

  • rn Engel

Log2fs or how to achieve 150.000 IO/s