FAT File system Case studies Microsoft, late 70s FAT late 70s; - PowerPoint PPT Presentation

FAT File system Case studies Microsoft, late 70s FAT late 70s; Microsoft File Allocation Table (FAT) key idea: linked list started with MSDOS Today: flash sticks in FAT-32, supports 2 28 blocks and files of 2 32 -1 bytes Unix FFS mid 80’ s FAT Data blocks key idea: tree-based multi-level index 0 Index Structures File Allocation Table (FAT) 1 2 array of 32-bit entries Today: Linux ext2 and ext3 3 file 9 block 3 file represented as a linked list 4 of FAT entries 5 NTFS early 1990s; Microsoft. 6 file # = index of first FAT entry 7 8 Key idea: variable size extents instead of fixed size blocks 9 file 9 block 0 Free space map 10 file 9 block 1 If data block i is free, 11 file 9 block 2 Locality heuristics Today: Windows 7, Linux ext4, Apple HFS then FAT[i] = 0 12 file 12 block 0 As simple as next fit: 13 find free blocks by scan sequentially from 14 scanning FAT ZFS early 2000; open source. last allocated entry and 15 return next free entry 16 file 12 block 1 17 Can be improved through Key idea: copy on write (COW) 18 file 9 block 4 defragmentation 19 20 FAT File system FFS: Fast File System Microsoft, late 70s Unix, 80s Smart index structure File Allocation Table (FAT) multilevel index allows to locate all blocks of a file started with MSDOS efficient for both large and small files [We saw that!] in FAT-32, supports 2 28 blocks and files of 2 32 -1 bytes Smart locality heuristics FAT Data blocks Disadvantages Advantages Poor locality simple! 0 next fit? seriously? 1 used in many free Poor random access 2 inodes data blocks data blocks data blocks USB flash keys lists needs sequential traversal 3 file 9 block 3 used even within S i d I I I I I D D D D D D D D D D D D D D D D D D D D D D D D 4 Limited access control MS Word! 5 0 7 8 15 16 23 24 31 no file owner or group ID metadata 6 data blocks data blocks data blocks data blocks any user can read/write any file 7 No support for hard links 8 D D D D D D D D D D D D D D D D D D D D D D D D D D D D D D D D 9 file 9 block 0 metadata stored in directory entry 32 39 40 47 48 55 56 63 10 file 9 block 1 Volume and file size are limited 11 file 9 block 2 Standard Unix treats disks as if it were RAM FAT entry is 32 bits, but top 4 are 12 file 12 block 0 reserved 13 14 no more than 2 28 blocks lots of seeks 15 with 4kB blocks, at most 1TB volume 16 file 12 block 1 file no bigger than 4GB 17 fragmentation: files just grab first available data block No support for transactional updates 18 file 9 block 4 19 20

Making the FS Locality heuristics: Disk-Aware block group placement Maintain the same interface… Divide disk in block groups Block group 0 sets of nearby tracks open(), close (), read(), write () etc Distribute metadata Block group 1 old design: free space bitmap and inode map in a single contiguous region …but change implementation SB lots of seeks when going from reading metadata to Block group 2 reading data optimize file system layout for how disks work SB Data Data FFS: distribute free space bitmap and inode array Data among block groups. Keep a superblock copy in blocks blocks blocks Free each block group Smart locality heuristics for for for space s File placement files files e files d o Inodes block group placement in in n in when a new regular file is created, FFS looks for /c I b i inodes in the same block as the file’ s directory t /a /b m /d/ q a optimizes placement for when a file data and metadata, and p when a new directory is created, FFS places it in a /d /a/p p a /a/g m different block from the parent’ s directory t i b other files within same directory, are accessed together /b/c space /z Inodes Free Data Placement p a reserved space m first free heuristics t i b SB e c trade short term for long term locality a p s gives up about 10% of storage to allow flexibility needed to Free achieve locality Locality heuristics: Locality heuristics: block group placement block group placement Divide disk in block groups Divide disk in block groups Block group 0 Block group 0 sets of nearby tracks sets of nearby tracks Distribute metadata Distribute metadata Block group 1 Block group 1 old design: free space bitmap and inode map in a old design: free space bitmap and inode map in a single contiguous region single contiguous region SB SB lots of seeks when going from reading metadata to lots of seeks when going from reading metadata to Block group 2 Block group 2 reading data reading data SB SB Data Data Data FFS: distribute free space bitmap and inode array Data FFS: distribute free space bitmap and inode array Data Data among block groups. Keep a superblock copy in among block groups. Keep a superblock copy in blocks blocks blocks blocks Free blocks Free blocks each block group each block group for for for for for for files space s File Placement files space s File Placement files files files e files e d d o Inodes o Inodes in in n in in in n in when a new file is created, FFS looks for inodes in when a new file is created, FFS looks for inodes in I I /c /c b b the same block as the file’ s directory the same block as the file’ s directory i t i t /a m /b /a m /b /d/ q /d/ q a a p when a new directory is created, FFS places it in a p when a new directory is created, FFS places it in a p p /d /a/p /a/g /d /a/p /a/g a a m m different block from the parent’ s directory different block from the parent’ s directory t t i i b b /b/c /b/c space space /z /z Inodes Inodes Free Free Data Placement Data Placement p p a a m m first free heuristics first free heuristics t t i i b b SB SB e e c trade short term for long term locality c trade short term for long term locality a a p p s s Free Free Small file Free In use Start of Start of block group block group

Locality heuristics: Locality heuristics: block group placement reserved space Divide disk in block groups Block group 0 Block group 0 sets of nearby tracks When a disk is full, hard to Distribute metadata Block group 1 Block group 1 old design: free space bitmap and inode map in a optimize locality single contiguous region SB SB lots of seeks when going from reading metadata to file may end up scattered Block group 2 Block group 2 reading data through disk SB SB Data Data FFS: distribute free space bitmap and inode array Data Data Data Data among block groups. Keep a superblock copy in blocks blocks blocks blocks blocks blocks Free Free each block group for for FFS presents applications with for for for for space s File Placement space s files files files files e e files files d d a smaller disk o Inodes o Inodes in in n in in in n in when a new file is created, FFS looks for inodes in /c I /c I b b i the same block as the file’ s directory i t t /a /b /a /b m m about 10%-20% smaller /d/ q /d/ q a a p when a new directory is created, FFS places it in a p /d /a/p p /d /a/p p a /a/g a /a/g m m different block from the parent’ s directory t t i i user write that encroaches on b b /b/c space /b/c space /z /z Inodes Inodes Free Free Data Placement reserved space fails p p a a m m first free heuristics t t i i b b SB SB e e super user still able to allocate c trade short term for long term locality c a a p p s s Free Free inodes to clean things up Large file Start of block group Long File Exception Caching and consistency File systems maintain many data structures Blocks of a huge file not all in the same block group Bitmap of free blocks and inodes Directories 12 blocks in a group (direct index) Inodes others divided in “chunks” Data blocks Locality lost when moving between chunks Data structures cached for performance choose chunk size to amortize cost of seeks works great for read operations... ...but what about writes? Say we want to achieve 90% of peak transfer transfer rate is 40 MB/s positioning time (seek+rotation) is 10ms chuck size = 40MB 1s X 90ms = 3.6 MB X s 1000ms In practice, FFS uses 4 MB chunks