The need for File Systems Need to store data and programs in files - - PowerPoint PPT Presentation

The need for File Systems

Need to store data and programs in files Must be able to store lots of data Must be nonvolatile and survive crashes and power outages Must allow multiple processes concurrent access Store on disks OS manages files in a file system

Different views on file systems

User view of file systems

• file names, protections, operations

File system designers views

• how to keep track of free blocks
• how disk blocks are grouped and

managed to form files First look at files from a users perspective.

User views of file systems

File naming

• give users useful names for files
• MS-DOS limitations, 8.3
• file extensions
• NTFS
• Unix -256 chars
• extensions may have meaning to

programs (e.g. cc) OS viewing files as a sequence of bytes gives most flexibility.

• leave up to app what to do

File Types

Regular files – ascii or binary Directories Character special files Block special files Symbolic links Sockets, Named pipe, Doors The file command, /etc/magic compressed files and archives Sequential files – on tape random access files

File attributes - metadata

Returned with stat(2) family mostly

• filename (not from stat(2))
• size
• mtime, ctime, atime
• mode (permissions and type)
• nlinks
• uid, gid

File operations

Create, Delete Open, Close Read, Write Append, Seek Get attributes, Set attributes Rename File descriptors – open files memory-mapped files – map files into process address space

Directories

Root directory Could have just root directory with all files in it

• not useful on multiuser systems
• may be useful for flash, etc.

Have directories in the root directory Hierarchical directories – tree Path names

• absolute and relative to cwd

Directory ops: Create, Delete, Opendir, Closedir, Readdir, Rename, link, unlink

Directories and Files

Contents of a directory Unix . and .. directories (dot and dot-dot) Disk partitions Slices Filesystems Disk labels How to store files and directories on disk? Want efficiency and reliability

Contiguous Allocation of files

Advantages

• simple to implement (address of

first block + number of blocks)

• very good read performance

Disadvantages

• Disk becomes fragmented after awhile

(need to compact, keep track of holes)

• Files change in size

CD-ROMS are a good use for this

• write once

Linked List allocation of files

Keep a linked list of disk blocks no external fragmentation, little internal Need to store first block in directory Random access is slow Pointer takes some disk block space.

Linked List allocation using a table in memory

Table has a pointer to each disk block FAT – File Allocation Table Entire block can be used for data Random access works well Problem is entire table needs to be in memory (proportionate to disk size) “Implementing pointers using arrays”

I-nodes

Needs to be in memory only when file is open. Point to a bunch of disk blocks. Point to a block that points to more disk blocks.

File Attributes

10 or so direct block ptrs

More block ptrs

Directories

Need to map filenames to disk blocks

• n disk (inode number)

file attributes can be in dir too or elsewhere like in the inode typical filename max length is 255 linear search of a directory can be slow Use a hash table or a tree to speed up lookups and/or cache the searches - dnlc

File storage

Storing files on disk have many of the same issues as memory allocation.

• contiguous
• noncontiguous – with fixed size blocks

Block size

• too small and too slow
• seek time and rotational delay
• too big wastes space (internal frag)

½ kB, 1kB, 4kB commonly used Need to keep track of free blocks

• use a linked list or a bitmap

Other file issues

Disk Quotas

• hard and soft limit or just hard limit
• time based or not
• number of files or just space used

Backups – Importance of data

• equipment can be replaced
• but losing data is unacceptable
• Backups to tape or other media
• full and incremental, restores
• physical security of tapes
• offsite copies, encryption

Filesystem consistency

System crashes can leave filesystem in inconsistent state.

• need for scandisk and fsck
• check blocks and files
• missing blocks, duplicate blocks
• lost+found

sync – write out modified blocks

• done every 30 seconds

fsck can take a long time on large filesystems with lots of files

• can make booting up slow

Logging or Journaling

A filesystem that logs changes to on disk data in a separate sequential rolling log.

• maintain accurate picture of filesystem
• speeds up booting greatly
• more reliable in case of power failure

Records each disk transaction Filesystem can be checked with the log instead of a full scan

Logging or Journaling

• log update at start
• modify filesystem
• marked done

When filesystem is checked if intent to change is marked, but not completed file structure for that block is checked. UFS logging, ext3, reisorfs Disksuite, Veritas separate log

Unix Inodes

Structure contains metadata (stuff returned by stat(2))

• mode (permissions and type)
• nlinks, size
• uid, gid
• atime, ctime, mtime
• device file is on

10 or 12 direct disk block addresses single, double, triple indirect blocks

(picture of inodes)

Unix Inodes

Small files can be accessed quickly directly from inode. Larger files use the indirect indexing. Capacity of unix files using inodes.

• example with 4 byte (32bit)

addressing

• block size of 1kB (1024bytes)

Unix Inodes

4 byte addresses and 1k block size Level # of blocks # of bytes Direct 12 12kB Single Ind 256 256kB Double “ 256*256=65k 65MB Triple Ind 256*65k=16M 16GB Max size file is 16GB + 65MB +268kB

Unix Inodes

newfs, mkfs superblock (found in block 1 and other backup copies)

• info about filesystem layout
• # of inodes
• # of disk blocks
• free list for disk blocks

Directory entry needs filenames and inode number

Filesystems

Berkeley Fast Filesystem, ufs

• file names up to 255 chars
• cylinder groups – keep data blocks
• f file close together

Linux – ext2, ext3, ext4 XFS, Reisorfs VxFS WAFL (slide to come) ZFS (more on this later) NFS, CIFS Virtual filesystems, /proc

WAFL File System

Write Anywhere File Layout from Network Appliance

• ptimized for random writes

Used on file servers from NetApp provide files using NFS, CIFS, ftp, http servers have NVRAM cache for writes Ease of use is a principle of WAFL Similar to Berkeley Fast File System, but with several changes. Uses inodes – 16 pointers to blocks (or indirect blocks)

WAFL Snapshots

Each filesystem has a root inode A snapshot duplicates a root inode A snapshot is a read only version of the filesystem at the point of time it is taken. Existing blocks are not overwritten. Space used by snapshot is blocks modified since snapshot was taken. Need to keep track of how many snapshots reference a block. When gets to zero the block can be freed.

See Figure 11.17 on page 446