Goals for Today Learning Objective: Understand the challenges of - PowerPoint PPT Presentation

Goals for Today • Learning Objective: • Understand the challenges of file system design • Announcements, etc: • C4: All remaining submissions open on Compass now • MP3 is out! Due April 18th . • Summary of IEF on next slide Reminder : Please put away devices at the start of class 1 CS 423: Operating Systems Design

Informal Early Feedback The lectures help me to understand opera1ng systems concepts Lecture usefulness… pretty 16 14 good! 12 10 8 Piazza/Text Usefulness…. less 6 4 2 good. 0 1 2 3 4 5 The textbook helps me to understand Piazza discussion helps me to understand opera3ng systems concepts opera3ng systems concepts 20 12 10 15 8 10 6 4 5 2 0 0 1 2 3 4 5 1 2 3 4 5 6 Never 1---2---3---4---5 Always CS 423: Operating Systems Design 2

Informal Early Feedback The pace of the course is... The midterm was ... 25 18 16 20 14 12 15 10 8 10 6 4 5 2 0 0 1 2 3 4 5 1 2 3 4 5 The MPs are... The reading assignments are... 30 2.5 25 2 20 1.5 15 1 10 0.5 5 0 0 1 2 3 4 5 1 2 3 4 5 Too Fast 1---2---3---4---5 Too Easy CS 423: Operating Systems Design 3

Informal Early Feedback • Highlights from Comments • Most Popular Topics: Scheduling, Virtual Memory • Practical Exposure is popular, MPs considered useful • Concerns about Participation Points • Class Discussion • More practice (e.g., HW, Quiz) to help w/ final prep • Increase MPs and/or add a team project?? CS 423: Operating Systems Design 4

CS 423 
 Operating System Design: File System Design Professor Adam Bates Spring 2018 CS 423: Operating Systems Design

Data Structures for a FS Data structures in a typical file system: Process Open file Memory Inode control table block (systemwide) Disk Open inode file pointer . . array . CS 423: Operating Systems Design 6

Disk Layout for a FS Disk layout in a typical file system: Boot Super File metadata File data blocks block block (i-node in Unix) ■ Data Structures: ■ File data blocks: File contents ■ File metadata: How to find file data blocks ■ Directories: File names pointing to file metadata ■ Free map: List of free disk blocks CS 423: Operating Systems Design 7

Disk Layout for a FS Disk layout in a typical file system: Boot Super File metadata File data blocks block block (i-node in Unix) ■ Superblock defines a file system size of the file system ■ size of the file descriptor area ■ free list pointer, or pointer to bitmap ■ location of the file descriptor of the root directory ■ other meta-data such as permission and various times ■ ■ For reliability, replicate the superblock CS 423: Operating Systems Design 8

Design Constraints • How can we allocate files efficiently? • For small files: • Small blocks for storage efficiency • Files used together should be stored together • For large files: • Contiguous allocation for sequential access • Efficient lookup for random access • Challenge: May not know at file creation where our file will be small or large!! CS 423: Operating Systems Design 9

Design Challenges • Index structure • How do we locate the blocks of a file? • Index granularity • How much data per each index (i.e., block size)? • Free space • How do we find unused blocks on disk? • Locality • How do we preserve spatial locality? • Reliability • What if machine crashes in middle of a file system op? CS 423: Operating Systems Design 10

File Allocation ■ Contiguous ■ Non-contiguous (linked) ■ Tradeoffs? CS 423: Operating Systems Design 11

Contiguous Allocation Request in advance for the size of the file ■ Search free map to locate a space ■ File header ■ first sector in file ■ number of sectors ■ Pros ■ Fast sequential access ■ Easy random access ■ Cons ■ External fragmentation ■ Hard to grow files ■ CS 423: Operating Systems Design 12

Linked Files File header File header points to 1st ■ block on disk Each block points to next ■ Pros ■ Can grow files dynamically ■ Free list is similar to a file ■ . . . Cons ■ random access: horrible ■ unreliable: losing a block ■ means losing the rest null CS 423: Operating Systems Design 13

Linked Allocation CS 423: Operating Systems Design 14

Indexed File Allocation Link full index blocks together using last entry. CS 423: Operating Systems Design 15

Multilevel Indexed Files Multiple levels of index blocks CS 423: Operating Systems Design 16

File Systems In Practice FAT Berkeley FFS NTFS (Unix FS) Index Linked list Tree Tree structure (fixed, assym) (dynamic) granularity block block extent free space FAT array Bitmap Bitmap allocaCon (fixed (file) locaCon) Locality defragmentaCon Block groups Extents + reserve Best fit space defrag CS 423: Operating Systems Design 17

MS File Allocation Table (FAT) ■ Linked list index structure ■ Simple, easy to implement ■ Still widely used (e.g., thumb drives) ■ File table: ■ Linear map of all blocks on disk ■ Each file a linked list of blocks CS 423: Operating Systems Design 18

MS File Allocation Table (FAT) MFT Data Blocks 0 1 2 3 fj le 9 block 3 4 5 6 7 8 9 fj le 9 block 0 10 fj le 9 block 1 11 fj le 9 block 2 fj le 12 block 0 12 13 14 15 16 fj le 12 block 1 17 18 fj le 9 block 4 19 20 CS 423: Operating Systems Design 19

MS File Allocation Table (FAT) ■ Pros: ■ Easy to find free block ■ Easy to append to a file ■ Easy to delete a file ■ Cons: ■ Small file access is slow ■ Random access is very slow ■ Fragmentation ■ File blocks for a given file may be scattered ■ Files in the same directory may be scattered ■ Problem becomes worse as disk fills CS 423: Operating Systems Design 20

Berkeley FFS / UNIX FS ■ “Fast File System” ■ inode table ■ Analogous to FAT table ■ inode ■ Metadata ■ File owner, access permissions, access times, … ■ Set of 12 data pointers ■ With 4KB blocks => max size of 48KB files ■ Indirect block pointers ■ pointer to disk block of data pointers ■ w/ indirect blocks, we can point to 1K data blocks => 4MB (+48KB) ■ … but why stop there?? CS 423: Operating Systems Design 21

Berkeley FFS / UNIX FS ■ Doubly indirect block pointer ■ w/ doubly indirect blocks, we can point to 1K indirect blocks ■ => 4GB (+ 4MB + 48KB) ■ Triply indirect block pointer ■ w/ triply indirect blocks, we can point to 1K doubly indirect blocks ■ 4TB (+ 4GB + 4MB + 48KB) CS 423: Operating Systems Design 22

Berkeley FFS / UNIX FS Open file description inode Parent File descriptor Mode File position table R/W Link Count Pointer to inode UID File position GID R/W Child Pointer to inode File File size descri Times ptor Address of table first 10 disk blocks Single Indirect Double Indirect Unrelated process Triple Indirect File descriptor table CS 423: Operating Systems Design 23 23

Berkeley FFS / UNIX FS Alternate figure, same basic idea Inode Array Triple Double Indirect Indirect Indirect Data Inode Blocks Blocks Blocks Blocks File Metadata Direct Pointer DP DP DP DP DP DP DP DP DP DP Direct Pointer Indirect Pointer Dbl. Indirect Ptr. Tripl. Indirect Ptr. CS 423: Operating Systems Design 24

Berkeley FFS Asym. Trees ■ Indirection has a cost. Only use if needed! ■ Small files: shallow tree ■ Efficient storage for small files ■ Large files: deep tree ■ Efficient lookup for random access in large files ■ Sparse files: only fill pointers if needed CS 423: Operating Systems Design 25

Berkeley FFS Locality ■ How does FFS provide locality? ■ Block group allocation ■ Block group is a set of nearby cylinders ■ Files in same directory located in same group ■ Subdirectories located in different block groups ■ inode table spread throughout disk ■ inodes, bitmap near file blocks ■ First fit allocation ■ Property: Small files may be a little fragmented, but large files will be contiguous CS 423: Operating Systems Design 26

Berkeley FFS Locality Block Group 0 Block Group 1 s e d Block Group 2 o n I p / a / d n p a a , c c m / / b D t , i / q B a d / t d e F a n r / c a B e a s l e p , e o d i S S c r / o p k e , t a s a e c / c f e r o F e r s i r e s d B e fj i n r i d l i o t e s D m o e t s a l fj t c a n r a i o B n l f e o c k s I p r d i d i r e n c t i o r s i e e s l / fj b , / a / r g , / z o I n f o s d k e s c o l B p a a t m a D t i B e c a p S e e r F CS 423: Operating Systems Design 27

Berkeley FFS Locality “First Fit” Block Allocation: In-Use Free Block Block Start of ... Block Group CS 423: Operating Systems Design 28

Berkeley FFS Locality “First Fit” Block Allocation: Write Two Block File Start of ... Block Group CS 423: Operating Systems Design 29

Berkeley FFS Locality “First Fit” Block Allocation: Write Large File Start of ... Block Group CS 423: Operating Systems Design 30