CSE 513 I ntroduction to Operating Systems Class 8 - I nput/ - - PowerPoint PPT Presentation

CSE 513 I ntroduction to Operating Systems Class 8 - I nput/ Output File Systems

Jonathan Walpole

• Dept. of Comp. Sci. and Eng.

Oregon Health and Science University

I / O devices

Device (mechanical hardware) Device controller (electrical hardware) Device driver (sof tware)

Devices and their controllers

Components of a simple personal computer

Monitor

Bus

How to communicate with a device?

Hardware supports I / O ports or memory

mapped I / O f or accessing device controller registers and buf f ers

Wide perf ormance range f or I / O

Perf ormance challenges: I / O hardware

How to prevent slow devices f rom slowing down

memory

How to identif y I / O addresses without

interf ering with memory perf ormance

Single vs. Dual Memory Bus Architecture

(a) A single- bus architecture (b) A dual- bus memory architecture

Hardware view of Pentium

Structure of a large Pentium system

Perf ormance challenges: I / O sof tware

How to prevent CPU throughput f rom being

limited by I / O device speed (f or slow devices)

How to prevent I / O throughput f rom being

limited by CPU speed (f or f ast devices)

How to achieve good utilization of CPU and I / O

devices

How to meet the real- time requirements of

devices

Programmed I / O

Steps in printing a string

Programmed I / O

Polling/ busy- waiting approach

copy_from_user(buffer,p,count); for(i=0;i<count;i++){ while (*p_stat_reg != READY); *p_data_reg = p[i]; } return();

I nterrupt- driven I / O

Asynchronous approach

give device dat a, do somet hing else! r esume when device int er r upt s

copy_from_user(buffer,p,count); enable_interrupts(); while (*p_stat_reg != READY); *p_data_reg=p[0]; scheduler(); if (count==0){ unblock_user(); } else { *p_data_reg = p[i]; count--; i++; } ack_interrupt(); return_from_interrupt();

I nterrupt driven I / O

(b)

Hardware Support f or I nterrupts

How interrupts happens. Connections between devices and interrupt controller actually use interrupt lines on the bus rather than dedicated wires

DMA

Of f load all work to a DMA controller

avoids using t he CPU t o do t he t r ansf er r educes number of int er r upt s DMA cont r oller is like a co-pr ocessor doing

pr ogr ammed I / O

copy_from_user(buffer,p,count); set_up_DMA_controller(); scheduler(); ack_interrupt(); unblock_user(); return_from_interrupt();

DMA

Sof tware engineering- related challenges

How to remove the complexities of I / O handling

f rom application programs

st andar d I / O API s (libr ar ies and syst em calls) gener ic cat egor ies acr oss dif f er ent device t ypes

How to support a wide range of device types on

a wide range of operating systems

st andar d int er f aces f or device dr iver s st andar d/ published int er f aces f or access t o ker nel

f acilit ies

I / O Sof tware Layers

Layers of the I / O Sof tware System

I nterrupt Handlers

• I nterrupt handlers are best hidden
• have dr iver st ar t ing an I / O oper at ion block unt il

int er r upt not if ies of complet ion

• I nterrupt procedure does its task
• t hen unblocks dr iver t hat st ar t ed it
• Steps must be perf ormed in sof tware af ter

interrupt completed

−

Save regs not alr eady saved by int er r upt har dwar e

−

Set up cont ext f or int er r upt ser vice pr ocedur e

I nterrupt Handlers

• Set up st ack f or int errupt service procedure
• Ack int errupt cont roller, reenable int errupt s
• Copy regist ers f rom where saved
• Run service procedure
• Set up MMU cont ext f or process t o run next
• Load new process' regist ers
• St art running t he new process

Device Drivers

• Communications between drivers and device controllers goes over

the bus

I / O Sof tware: Device Drivers

Device drivers “connect” devices with the

• perating system

Typically a nast y assembly-level j ob

• Must deal with hardware changes
• Must deal with O.S. changes

Hide as many device-specif ic det ails as possible

Device drivers are typically given kernel

privileges f or ef f iciency

Can br ing down O.S.! How t o pr ovide ef f iciency and saf et y???

Device- I ndependent I / O Sof tware Functions

Functions of the device- independent I / O sof tware

Providing a deice- independent block size Allocating and releasing dedicate devices Error reporting Buf f ering Unif orm interf acing f or device drivers

Device- I ndependent I / O Sof tware I nterf ace

(a) Without a standard driver interf ace (b) With a standard driver interf ace

Device- I ndependent I / O Sof tware Buf f ering

(a) Unbuf f ered input (b) Buf f ering in user space (c) Buf f ering in the kernel f ollowed by copying to user space (d) Double buf f ering in the kernel

Copying Overhead in Network I / O

Networking may involve many copies

User- Space I / O Sof tware

Layers of the I / O system and the main f unctions

• f each layer

Devices as f iles

• Bef ore mounting,

f iles on f loppy are inaccessible

• Af ter mounting f loppy on b,

f iles on f loppy are part of f ile hierarchy

Disk Geometry

Disk head, platters, surf aces

cylinder Track Sector

Physical vs. logical disk geometry

Constant Angular Velocity vs. Constant Linear

Velocity

Disks

Disk parameters f or the original I BM PC f loppy disk and a Western Digital WD 18300 hard disk

CD- ROMs

CD- ROM data layout

Logical data layout on a CD- ROM

CD- R Structure

• Cross section of a CD- R disk and laser

not t o scale

• Silver CD- ROM has similar structure

wit hout dye layer wit h pit t ed aluminum layer inst ead of gold

Double sided dual layer DVD

Plastic technology

CDs

Appr oximat ely 650 Mbyt es of dat a Appr oximat ely 74 minut es of audio

DVDs

Many t ypes of f or mat s

• DVD- R, DVD- ROM, DVD- Video

Single layer vs. mult i-layer Single sided vs. double sided Aut hor ing vs. non-aut hor ing

Disk Formatting

A disk sector

Disk f ormatting with cylinder skew

Disk f ormatting with interleaving

No interleaving Single interleaving Double interleaving

Disk scheduling algorithms

Time required to read or write a disk block

determined by 3 f actors

Seek t ime Rot at ional delay Act ual t r ansf er t ime

Seek time dominates Error checking is done by controllers

Disk scheduling algorithms

First- come f irst serve Shortest seek time f irst Scan back and f orth to ends of disk C- Scan only one direction Look back and f orth to last request C- Look only one direction

Disk scheduling algorithms

Error handling complicates scheduling

A disk track with a bad sector Substituting a spare f or the bad sector Shif ting all the sectors to bypass the bad one

RAI D levels 0 to 2

RAI D levels 3 to 5

Part B File Systems

Long- term I nf ormation Storage

• Must store large amounts of data
• I nf ormation stored must survive the termination
• f the process using it
• Multiple processes must be able to access the

inf ormation concurrently

File naming and f ile extensions

Typical f ile extensions.

File Structure

Three kinds of f ile structure

byt e sequence r ecor d sequence t r ee

File Types

(a) An executable f ile (b) An archive

File access

Sequential access

r ead all byt es/ r ecor ds f r om t he beginning cannot j ump ar ound, could r ewind or back up convenient when medium was mag t ape

Random access

byt es/ r ecor ds r ead in any or der essent ial f or dat a base syst ems r ead can be …

• move f ile marker (seek), then read or …
• read and then move f ile marker

File attributes

File operations

Cr eat e Delet e Open Close Read Wr it e Append Seek Get at t r ibut es Set At t r ibut es Rename

Exmple Program Using File System Calls (1/ 2)

Example Program Using File System Calls (2/ 2)

Memory- mapped f iles

(a) Segmented process bef ore mapping f iles into its address space (b) Process af ter mapping

exist ing f ile abc int o one segment cr eat ing new segment f or xyz

Directories - single level

A single level directory system

cont ains 4 f iles

• wned by 3 dif f er ent people, A, B, and C

Directories - two- level

Letters indicate owners of the directories and f iles

Hierarchical directory systems

Path names in Unix

Directory operations

Create Delete Opendir Closedir Readdir Rename Link Unlink

File system implementation on disk

A possible f ile system disk layout

I mplementing f iles - contiguous allocation

(a) Contiguous allocation of disk space f or 7 f iles (b) State of the disk af ter f iles D and E have been removed

I mplementing f iles - linked allocation

Storing a f ile as a linked list of disk blocks

I mplementing Files - FAT

Linked list allocation using a f ile allocation table (FAT) in RAM

I mplementing f iles - index nodes

An example i- node

I mplementing directories

(a) A simple directory

f ixed size ent r ies disk addr esses and at t r ibut es in dir ect or y ent r y

(b) Directory in which each entry just ref ers to an i- node

I mplementing directories

• Two ways of handling long f ile names in directory

(a) I n-line, (b) I n a heap

Shared f iles - hard links

File system containing a shared f ile

Problems with shared f iles

(a) Situation prior to linking (b) Af ter the link is created (c)Af ter the original owner removes the f ile

Disk space management and perf ormance

• Dark line (lef t hand scale) gives data rate of a disk
• Dotted line (right hand scale) gives disk space ef f iciency
• All f iles 2KB

Block size

Disk space management

(a) Storing the f ree list on a linked list (b) A bit map

Disk Space Management (3)

(a) Almost- f ull block of pointers to f ree disk blocks in RAM

• t hree blocks of point ers on disk

(b) Result of f reeing a 3- block f ile (c) Alternative strategy f or handling 3 f ree blocks

• shaded ent ries are point ers t o f ree disk blocks

Disk space management - quotas

Quotas f or keeping track of each user’s disk use

Maintaining File System Consistency

Crashes can cause f ile system to have

corrupted data

First: bitmap vs. linked storage maps f sck / scandisk

Block-level ensur e t hat all blocks on disk ar e

account ed f or

• traverse inodes counting allocated blocks
• compare result to f ree list

File-level ensur e t hat all f iles ar e account ed f or

• traverse directory system counting f iles

File system consistency

File system states

(a) consist ent (b) missing block (c) duplicat e block in f ree list (d) duplicat e dat a block

Maintaining File System Consistency

File level consistency

Have dir ect or ies wit h i-node # ’s of f iles Have link st at us in t he i-nodes t hemselves Compar e!

Perf ormance issues

Buf f er cache

A buf f er in bet ween t he applicat ion and t he FS

• Hold buf f er of accessed data
• Allows data to be reused by other apps
• Can implement pref etch strategies to minimize

latency

• Serves a buf f er f or writing out application data

– Delayed writes allow the writes to happen when convenient – Trade- of f between consistency and perf ormance

Hash t able indexed t o minimize t he sear ching

File system perf ormance - buf f er cache

The buf f er cache data structures

File system perf ormance - data placement

• I - nodes placed at the start of the disk
• Disk divided into cylinder groups

each wit h it s own blocks and i-nodes

Log- Structured File Systems

With CPUs f aster, memory larger

disk caches can also be lar ger incr easing number of r ead r equest s can come f r om cache t hus, most disk accesses will be wr it es

LFS Strategy structures entire disk as a log

have all wr it es init ially buf f er ed in memor y per iodically wr it e t hese t o t he end of t he disk log

• long contiguous writes to disk
• requires large contiguous f ree space!
• data not updated in place

when f ile opened, locat e i-node, t hen f ind blocks

Example: The Unix f ile system

• File systems structure
• The f ile system in UNI X is an integral part of the
• perating system. Files are used f or:

Terminal handling

/ dev/ t t y***

P

ipes “ls | more”

Direct ories Socket s (f or net working)

• The design of the UNI X f ile system allows the user to

write code that has a unif orm interf ace.

Boot block superblock i-nodes data block data block

…

Unix f ile system structures

Superblock - tells the UNI X O. S. what the

f ormat of the disk is, the # of blocks, the number of i- nodes, etc. . .

I - nodes - are the metadata data structures

f or f iles and directories

Files - holds inf or mat ion such as owner , per missions,

dat e of modif icat ion, wher e t he dat a is

Dir ect or ies ar e j ust a special case of f iles t hat have

t he pair s < f ile-name, i-node # > st or ed in t he f ile

The Unix name space

Files

Repr esent ed wit h an I -node and disk blocks t hat

hold t he dat a

File names ar e not st or ed wit h t he i-node (t hey’r e in

t he dir ect or ies t hat r ef er t o t hem)

• This is why there is a link count in the inode

Directories

Dir ect or ies ar e also f iles Ent r ies ar e disblocks wit h <

f ilename, i-node # > pair s

Special dir ect or y mar king in i-node

Unix directory entries

A UNI X V7 directory entry

The Unix name space

/ inode #22

. 22 .. 22 usr 24 var 35 home 26 txt 23

/ inode #24

. 24 .. 22 src 38 conf 53

/ inode #53

## config.. ## Frame rate 33 Frame size ..

Pathname translation in Unix

The steps in looking up / usr/ ast/ mbox

Unix I - node structure

UNI X f iles consist of i- node and data blocks UNI X uses an indexed allocation scheme with

10 dir ect point er s t o blocks 1 indir ect point er t o blocks 1 double indir ect

point er t o blocks

1 t r iple indir ect

point er t o blocks

i-node

... ... ... ... ...

triple indirect pointer

Unix I - node structrure

A UNI X i- node

Maximum f ile size in Unix

• Several parameters determine how many (and of what

size) f iles can be represented

• No. of bit s in a disk address
• No. of bit s in a virt ual memory ref erence

Disk block size

• Example:

10 direct , 1 indirect , 1 double indirect , 1 t riple indirect

• No. bit s in disk address -->

16-bit s

• No. of bit s in virt ual memory ref erence -->

32-bit s

Disk block size -->

1024 kbyt es

What is t he maximum f ile size in t his syst em?

The “mount” command in Unix

Commands

“mount ” allows f ile syst ems t o be added t o t he

names space

“umount ” t akes a f ile syst ems out of t he name space

Mount points

Mount point s can be any dir ect or y in t he name space Once a f ile syst em is mount ed, t he ent ir e subt r ee

at t he mount point is no longer accessible (unt il t he f ile syst em is unmount ed)

Some UNI X f ile system f eatures

mounting - allows other f ile systems (disks),

whether local or remote to be “mounted” into a unif orm f ile system name space

/ mnt usr var home X11 bin

Associating f iles with processes

• To provide unif orm access to data, UNI X has a level of

indirection that is used f or opening f iles

process 1 process n ... Open File Table

... ... ...... ... ... ... ...... ...

... Files Each open file entry holds, permissions (R/W), file offset

File system vs. virtual memory

Process Paging FS Disk Swap Open File Table

A f ile- centric view of I PC

All input and output in UNI X are handled

through the open f ile table structure

This is how UNI X can pr ovide a single unif or m

int er f ace f or local or r emot e dat a access

Pipes in UNI X are nothing more than

A wr it ing pr ocess t hat has it s “st dout ” linked t o a

“pipe” f ile (inst ead of a / dev/ t t y*** f ile)

A r eading pr ocess t hat has it s “st din” linked t o a

“pipe” f ile (inst ead of a / dev/ t t y*** f ile)