Goals for Today Learning Objective: Understand the challenges of - - PowerPoint PPT Presentation

CS 423: Operating Systems Design 1

Goals for Today

Reminder: Please put away devices at the start of class

• Learning Objective:
• Understand the challenges of file system design
• Announcements, etc:
• C4: All remaining submissions open on Compass now
• MP3 is out! Due April 18th.
• Summary of IEF on next slide

CS 423: Operating Systems Design

Informal Early Feedback

2

2 4 6 8 10 12 14 16 1 2 3 4 5

The lectures help me to understand

• pera1ng systems concepts

2 4 6 8 10 12 1 2 3 4 5

The textbook helps me to understand

• pera3ng systems concepts

5 10 15 20 1 2 3 4 5 6

Piazza discussion helps me to understand

• pera3ng systems concepts

Never 1---2---3---4---5 Always

Lecture usefulness… pretty good! Piazza/Text Usefulness…. less good.

CS 423: Operating Systems Design 3

Too Fast 1---2---3---4---5 Too Easy

5 10 15 20 25 1 2 3 4 5

The pace of the course is...

2 4 6 8 10 12 14 16 18 1 2 3 4 5

The midterm was ...

5 10 15 20 25 30 1 2 3 4 5

The MPs are...

0.5 1 1.5 2 2.5 1 2 3 4 5

The reading assignments are...

Informal Early Feedback

CS 423: Operating Systems Design 4

• Highlights from Comments
• Most Popular Topics: Scheduling,

Virtual Memory

• Practical Exposure is popular, MPs considered useful
• Concerns about Participation Points
• Class Discussion
• More practice (e.g., HW, Quiz) to help w/ final prep
• Increase MPs and/or add a team project??

Informal Early Feedback

CS 423: Operating Systems Design

Professor Adam Bates Spring 2018

CS 423
 Operating System Design: File System Design

CS 423: Operating Systems Design

Data Structures for a FS

6

Process control block

. . .

Open file pointer array Open file table (systemwide) Memory Inode

Disk inode

Data structures in a typical file system:

CS 423: Operating Systems Design

Disk Layout for a FS

7

■ Data Structures: ■ File data blocks: File contents ■ File metadata: How to find file data blocks ■ Directories: File names pointing to file metadata ■ Free map: List of free disk blocks

Super block File metadata (i-node in Unix) File data blocks Boot block

Disk layout in a typical file system:

CS 423: Operating Systems Design

Disk Layout for a FS

8

■ Superblock defines a file system

■

size of the file system

■

size of the file descriptor area

■

free list pointer, or pointer to bitmap

■

location of the file descriptor of the root directory

■

• ther meta-data such as permission and various times

■ For reliability, replicate the superblock

Super block File metadata (i-node in Unix) File data blocks Boot block

Disk layout in a typical file system:

CS 423: Operating Systems Design

Design Constraints

9

• How can we allocate files efficiently?
• For small files:
• Small blocks for storage efficiency
• Files used together should be stored together
• For large files:
• Contiguous allocation for sequential access
• Efficient lookup for random access
• Challenge: May not know at file creation where our file

will be small or large!!

CS 423: Operating Systems Design

Design Challenges

10

• Index structure
• How do we locate the blocks of a file?
• Index granularity
• How much data per each index (i.e., block size)?
• Free space
• How do we find unused blocks on disk?
• Locality
• How do we preserve spatial locality?
• Reliability
• What if machine crashes in middle of a file system op?

CS 423: Operating Systems Design

File Allocation

11

■ Contiguous ■ Non-contiguous (linked) ■ Tradeoffs?

CS 423: Operating Systems Design

Contiguous Allocation

12

■

Request in advance for the size of the file

■

Search free map to locate a space

■

File header

■

first sector in file

■

number of sectors

■

Pros

■

Fast sequential access

■

Easy random access

■

Cons

■

External fragmentation

■

Hard to grow files

CS 423: Operating Systems Design

Linked Files

13

■

File header points to 1st block on disk

■

Each block points to next

■

Pros

■

Can grow files dynamically

■

Free list is similar to a file

■

Cons

■

random access: horrible

■

unreliable: losing a block means losing the rest File header null

. . .

CS 423: Operating Systems Design

Linked Allocation

14

CS 423: Operating Systems Design

Indexed File Allocation

15

Link full index blocks together using last entry.

CS 423: Operating Systems Design

Multilevel Indexed Files

16

Multiple levels of index blocks

CS 423: Operating Systems Design

File Systems In Practice

17

FAT Berkeley FFS (Unix FS) NTFS Index structure Linked list Tree (fixed, assym) Tree (dynamic) granularity block block extent free space allocaCon FAT array Bitmap (fixed locaCon) Bitmap (file) Locality defragmentaCon Block groups + reserve space Extents Best fit defrag

CS 423: Operating Systems Design

MS File Allocation Table (FAT)

18

■ Linked list index structure ■ Simple, easy to implement ■ Still widely used (e.g., thumb drives) ■ File table: ■ Linear map of all blocks on disk ■ Each file a linked list of blocks

CS 423: Operating Systems Design

MS File Allocation Table (FAT)

19

fjle 9 block 3 fjle 9 block 0 fjle 9 block 1 fjle 9 block 2 fjle 12 block 0 fjle 12 block 1 fjle 9 block 4 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

MFT Data Blocks

CS 423: Operating Systems Design

MS File Allocation Table (FAT)

20

■ Pros: ■ Easy to find free block ■ Easy to append to a file ■ Easy to delete a file ■ Cons: ■ Small file access is slow ■ Random access is very slow ■ Fragmentation ■ File blocks for a given file may be scattered ■ Files in the same directory may be scattered ■ Problem becomes worse as disk fills

CS 423: Operating Systems Design

Berkeley FFS / UNIX FS

21

■ “Fast File System” ■ inode table ■ Analogous to FAT table ■ inode ■ Metadata ■ File owner, access permissions, access times, … ■ Set of 12 data pointers ■ With 4KB blocks => max size of 48KB files ■ Indirect block pointers ■ pointer to disk block of data pointers ■ w/ indirect blocks, we can point to 1K data blocks => 4MB (+48KB) ■ … but why stop there??

CS 423: Operating Systems Design

Berkeley FFS / UNIX FS

22

■ Doubly indirect block pointer ■ w/ doubly indirect blocks, we can point to

1K indirect blocks

■ => 4GB (+ 4MB + 48KB) ■ Triply indirect block pointer ■ w/ triply indirect blocks, we can point to 1K

doubly indirect blocks

■ 4TB (+ 4GB + 4MB + 48KB)

CS 423: Operating Systems Design 23 File position R/W Pointer to inode File position R/W Pointer to inode

Mode Link Count UID GID File size Times Address of first 10 disk blocks

Single Indirect Double Indirect

Triple Indirect inode Open file description Parent File descriptor table Child File descri ptor table Unrelated process File descriptor table

23

Berkeley FFS / UNIX FS

CS 423: Operating Systems Design 24 Inode Array

File Metadata Indirect Pointer

• Dbl. Indirect Ptr.
• Tripl. Indirect Ptr.

Inode Data Blocks Indirect Blocks Double Indirect Blocks Triple Indirect Blocks

DP Direct Pointer DP DP DP DP DP DP DP DP DP Direct Pointer

Alternate figure, same basic idea

Berkeley FFS / UNIX FS

CS 423: Operating Systems Design

Berkeley FFS Asym. Trees

25

■ Indirection has a cost. Only use if needed! ■ Small files: shallow tree ■ Efficient storage for small files ■ Large files: deep tree ■ Efficient lookup for random access in

large files

■ Sparse files: only fill pointers if needed

CS 423: Operating Systems Design

Berkeley FFS Locality

26

■ How does FFS provide locality? ■ Block group allocation ■ Block group is a set of nearby cylinders ■ Files in same directory located in same group ■ Subdirectories located in different block groups ■ inode table spread throughout disk ■ inodes, bitmap near file blocks ■ First fit allocation ■ Property: Small files may be a little fragmented, but large

files will be contiguous

CS 423: Operating Systems Design

Berkeley FFS Locality

27

F r e e S p a c e B i t m a p F r e e S p a c e B i t m a p I n

• d

e s D a t a B l

• c

k s f

• r

fj l e s i n d i r e c t

• r

i e s / b , / a / g , / z F r e e S p a c e B i t m a p I n

• d

e s D a t a B l

• c

k s f

• r

fj l e s i n d i r e c t

• r

i e s / d / q , / c , a n d / a / p

Block Group 0 Block Group 1 Block Group 2

I n

• d

e s D a t a B l

• c

k s f

• r

fj l e s i n d i r e c t

• r

i e s / a , / d , a n d / b / c

CS 423: Operating Systems Design

Berkeley FFS Locality

28

“First Fit” Block Allocation:

...

In-Use Block Start of Block Group Free Block

CS 423: Operating Systems Design

Berkeley FFS Locality

29

“First Fit” Block Allocation:

...

Start of Block Group Write Two Block File

CS 423: Operating Systems Design

Berkeley FFS Locality

30

“First Fit” Block Allocation:

...

Start of Block Group Write Large File

CS 423: Operating Systems Design

Berkeley FFS / UNIX FS

31 ■ Pros ■ Efficient storage for both small and large files ■ Locality for both small and large files ■ Locality for metadata and data ■ Cons ■ Inefficient for tiny files (a 1 byte file requires both an inode

and a data block)

■ Inefficient encoding when file is mostly contiguous on disk

(no equivalent to superpages)

■ Need to reserve 10-20% of free space to prevent

fragmentation

CS 423: Operating Systems Design

NTFS

32

■ Master File Table ■ Flexible 1KB storage for metadata and data ■ Extents ■ Block pointers cover runs of blocks ■ Similar approach in linux (ext4) ■ File create can provide hint as to size of file ■ Journalling for reliability

CS 423: Operating Systems Design 33

NTFS

• Std. Info.

File Name Data (resident) (free)

MFT Record (small fjle) Master File Table

CS 423: Operating Systems Design 34

NTFS

MFT MFT Record

Start Length Start Length

• Std. Info.

File Name (free) Data (nonresident) Data Extent Data Extent

CS 423: Operating Systems Design 35

NTFS Indirect Block

MFT MFT Record (part 2)

• Std. Info.

(free) Data (nonresident) MFT Record (part 1)

• Std. Info.

Attr.list Data (nonresident) File Name

Data Extent Data Extent Data Extent Data Extent Data Extent

CS 423: Operating Systems Design 36

NTFS

MFT Record (normal file) MFT

• Std. Info.

Data (nonresident)

MFT Record (small file)

• Std. Info.

Data (resident)

MFT Record (big/fragmented file)

• Std. Info.

Attr.list Data (nonresident) Data (nonresident) Data (nonresident) Data (nonresident)

MFT MFT Record (huge/badly-fragmented file)

• Std. Info.

Attr.list (nonresident) Data (nonresident) Data (nonresident) Data (nonresident) Data (nonresident) Extent with part of attribute list Extent with part of attribute list Data (nonresident)