Computer Systems Lab Matteo Corti Informatikdienste, ETH Z urich - - PowerPoint PPT Presentation

▶
computer systems lab
SMART_READER_LITE
LIVE PREVIEW

Computer Systems Lab Matteo Corti Informatikdienste, ETH Z urich - - PowerPoint PPT Presentation

Jan 14, 2023 153 likes •388 views

Computer Systems Lab Matteo Corti Informatikdienste, ETH Z urich 2007-04-30 Matteo Corti (Informatikdienste, ETH Z urich) Computer Systems Lab 2007-04-30 1 / 23 A simple disk driver bios.c provides a simple disk-driver to access FAT

slide-1
SLIDE 1

Computer Systems Lab

Matteo Corti

Informatikdienste, ETH Z¨ urich

2007-04-30

Matteo Corti (Informatikdienste, ETH Z¨ urich) Computer Systems Lab 2007-04-30 1 / 23

slide-2
SLIDE 2

A simple disk driver

bios.c provides a simple disk-driver to access FAT disk images using 512-bytes blocks. int bios init(char * name) initializes the disk driver with the given image. void bios shutdown() closes the image file void bios read(int number, char * block) reads a 512 bytes sector from the disk image void bios write(int number, char * block) writes a 512 bytes sector to the disk image.

Matteo Corti (Informatikdienste, ETH Z¨ urich) Computer Systems Lab 2007-04-30 2 / 23

slide-3
SLIDE 3

Goal

The goal of this lab is to implement a minimal FAT driver with the following simplified API: int fs open(char * path)

  • pens a file for reading specified by path for reading and writing and

return a file descriptor int fs read(int fd, char * buff, int len) reads len bytes from fd, puts them into buff and returns the actual number of read bytes int fs write(inf fd, char * buff, int len) writes len bytes from buff to the file specified by the file descriptor fd and returns the actual number of written bytes

Matteo Corti (Informatikdienste, ETH Z¨ urich) Computer Systems Lab 2007-04-30 3 / 23

slide-4
SLIDE 4

Goal

void fs close(int fd) closes the file specified by the file descriptor fd int fs creat(char * path) creates a new file specified by path and opens it for writing Please note that this simple API does not allow to specify the read

  • position. A file must be sequentially read and written. In addition we

do not specify a way to delete a file.

Matteo Corti (Informatikdienste, ETH Z¨ urich) Computer Systems Lab 2007-04-30 4 / 23

slide-5
SLIDE 5

File descriptors

Your filesystem driver will deliver to user a file descriptor: a key to uniquely identify a file. The file descriptor will be then used to identify the kernel (or driver) data structures relative to the given file.

user program fd = open("file"); user space fs driver disk driver ... file pos buffer *internal_data fd

Matteo Corti (Informatikdienste, ETH Z¨ urich) Computer Systems Lab 2007-04-30 5 / 23

slide-6
SLIDE 6

Steps

Divide you work in steps:

1 Understand the filesystem layout 2 Read the boot sector and the structure of the FAT partition you

are handling (i.e., number of clusters, number of FATS and position of the root directory).

3 Read the directory structure: parse the directory entries to

located the starting cluster of the handled file.

4 Access files: read and write the requested clusters

Test your work after each step!

Matteo Corti (Informatikdienste, ETH Z¨ urich) Computer Systems Lab 2007-04-30 6 / 23

slide-7
SLIDE 7

Testing

We provide several images for testing:

  • each image is FAT12 and contains only files and directories

named with the 8.3 schema

  • For each image we provide a tarball with the image content
  • You can inspect the image content with:

> hexdump -C partition.img ... 00008000 74 65 73 74 20 20 20 20 20 20 20 08 00 00 7d 72 |test ...}r| 00008010 b2 34 b2 34 00 00 7d 72 b2 34 00 00 00 00 00 00 |.4.4..}r.4......| 00008020 41 66 00 69 00 6c 00 65 00 00 00 0f 00 bc ff ff |Af.i.l.e........| 00008030 ff ff ff ff ff ff ff ff ff ff 00 00 ff ff ff ff |................| 00008040 46 49 4c 45 20 20 20 20 20 20 20 20 00 64 7d 72 |FILE .d}r| 00008050 b2 34 b2 34 00 00 7d 72 b2 34 03 00 0d 00 00 00 |.4.4..}r.4......| ... Matteo Corti (Informatikdienste, ETH Z¨ urich) Computer Systems Lab 2007-04-30 7 / 23

slide-8
SLIDE 8

Create test images

Example for Linux: # create an empty file dd if=/dev/zero of=/tmp/disk.img \ count=<number of blocks> # format the disk image mkdosfs -f 2 -F 12 -r 512 -S 512 -n disk \ /tmp/disk.img # mount the disk image sudo mount -t vfat -o rw,loop=/dev/loop0 \ test.img /mnt # copy the files sudo cp -vr * /mnt/ # unmount the disk image sudo umount /mnt

Matteo Corti (Informatikdienste, ETH Z¨ urich) Computer Systems Lab 2007-04-30 8 / 23

slide-9
SLIDE 9

Hints

  • The first cluster is cluster 2 (clusters 0 and 1 don’t exist)
  • Names (8+3) are padded with spaces: "EXAMPLE "."TXT"
  • FAT 12: two 12 bit entries are packed into three bytes:

uv.wx.yz = ⇒ xuv, yzw

  • Root dir position: reserved + number of FATs ·

sectors per FAT

  • Root dir length: rootdir−entries·32

sector length

  • Even if the file has a 8.3 name a long name entry is usually
  • created. You can recognize (and skip) this directory entries by

their attribute: 0x0F.

  • FAT type:
  • if there are less than 4085 clusters FAT12
  • if less than 65525 FAT16
  • otherwise FAT32

We provide FAT12 images

Matteo Corti (Informatikdienste, ETH Z¨ urich) Computer Systems Lab 2007-04-30 9 / 23

slide-10
SLIDE 10

Hints: packed structures

Tell the compiler not to word align structures. GCC example:

struct fat_boot_sector { __u8 ignored[3]; /* Boot strap short or near jump */ __u8 system_id[8]; /* Name - can be used to special case * * partition manager volumes */ __u16 sector_size; /* bytes per logical sector */ __u8 sec_per_clus; /* sectors/cluster */ __u16 reserved; /* reserved sectors */ __u8 fats; /* number of FATs */ __u16 dir_entries; /* root directory entries */ __u16 sectors; /* number of sectors */ __u8 media; /* media code */ __u16 fat_length; /* sectors/FAT */ __u16 secs_track; /* sectors per track */ __u16 heads; /* number of heads */ __u32 hidden; /* hidden sectors (unused) */ __u32 total_sect; /* number of sectors (if sectors = 0) */ } __attribute__((packed));

Matteo Corti (Informatikdienste, ETH Z¨ urich) Computer Systems Lab 2007-04-30 10 / 23

slide-11
SLIDE 11

Hints: driver.h

We provide:

  • Definitions for the boot sector and directory entries:
  • struct fat_boot_sector {

__u8 ignored[3]; __u8 system_id[8]; __u16 sector_size; __u8 sec_per_clus;

  • struct dos_dir_entry {

__u8 name[8], ext[3]; __u8 attr; __u8 lcase; __u8 ctime_cs;

  • Constants for attributes: FILE ATTR RONLY,

FILE ATTR HIDDEN, . . .

  • Disk driver (bios.c)

Matteo Corti (Informatikdienste, ETH Z¨ urich) Computer Systems Lab 2007-04-30 11 / 23

slide-12
SLIDE 12

Malloc lab: Sample solutions

Red-black tree: 57 (util) + 40 (thru) = 97/100

  • Best-fit algorithm
  • Free blocks stored in a red-black tree (sorted by size)
  • Freed blocks are temporarily stored in a list (blog) and put in the

tree before a malloc.

  • Blocks have a header and a footer

Buddy (J¨ urg Billeter): 57 (util) + 40 (thru) = 97/100

  • 128 free lists: one unsorted (similar to the blog), one for large

blocks

  • Blocks have a header and a footer
  • Best-fit algorithm

Matteo Corti (Informatikdienste, ETH Z¨ urich) Computer Systems Lab 2007-04-30 12 / 23

slide-13
SLIDE 13

Red-Black Trees (Beyer 1972)

  • Self-balancing binary search tree
  • Search, insert and delete in O(log n)
  • Good worst-case run time
  • Properties:
  • Each node is either red or black
  • The root is black
  • All leaves are black
  • Children of a red node are black
  • All paths from any node to its leaf nodes

contain the same number of black nodes.

nil nil nil nil 8 1 10 42 12 100 150 11

  • The longest path (root–leaf) is no more than twice the shortest

path.

  • R. Bayer

Symmetric binary B-Trees: Data structure and maintenance algorithms Acta Informatica, 1(4):290–306, Springer, December 1972

Matteo Corti (Informatikdienste, ETH Z¨ urich) Computer Systems Lab 2007-04-30 13 / 23

slide-14
SLIDE 14

Red-Black Tree — Algorithm

free(b) if next block is free then coalesce end if if previous block is free then coalesce end if put b in the blob

Matteo Corti (Informatikdienste, ETH Z¨ urich) Computer Systems Lab 2007-04-30 14 / 23

slide-15
SLIDE 15

Red-Black Tree — Algorithm

malloc(size) insert all the nodes of the blob into the tree search a block (b) in the free tree if no free blocks are available then if the heap can be grown then grow the heap and return the newly allocated block else error end if else if the block can be split then split the block put the remainder in the blob end if return b end if

Matteo Corti (Informatikdienste, ETH Z¨ urich) Computer Systems Lab 2007-04-30 15 / 23

slide-16
SLIDE 16

Red-Black Tree — Algorithm

realloc(b, size) if b = NULL then return malloc(size) end if if size = 0 then free(b) return end if if we can expand to the next block1 then coalesce the next block return new block end if b2 = malloc(size) copy b to b2 free(b)

1This includes growing the heap if b is the last block Matteo Corti (Informatikdienste, ETH Z¨ urich) Computer Systems Lab 2007-04-30 16 / 23

slide-17
SLIDE 17

Buddy — Algorithm

free(b) if next block is free then coalesce end if if previous block is free then coalesce end if put b in the unsorted list

Matteo Corti (Informatikdienste, ETH Z¨ urich) Computer Systems Lab 2007-04-30 17 / 23

slide-18
SLIDE 18

Buddy — Algorithm

malloc(size) sort the unsorted list search a block (b) in the free lists if no free blocks are available then if the heap can be grown then grow the heap and return the newly allocated block else error end if else if the block can be split then split the block put the remainder in the corresponding list end if return b end if

Matteo Corti (Informatikdienste, ETH Z¨ urich) Computer Systems Lab 2007-04-30 18 / 23

slide-19
SLIDE 19

Buddy — Algorithm

realloc(b, size) if b = NULL then return malloc(size) end if if size = 0 then free(b) return end if if we can expand to the previous or next block then coalesce the next block return new block end if b2 = malloc(size) copy b to b2 free(b)

Matteo Corti (Informatikdienste, ETH Z¨ urich) Computer Systems Lab 2007-04-30 19 / 23

slide-20
SLIDE 20

Sample solution: review

  • Data structures: does not necessarily need to be complicated

(red-black tree vs. linear lists). Data characteristics (in our case with a certain degree of homogeneity) have to be taken into account.

  • Lazy processing: blocks are reorganized only if needed

Matteo Corti (Informatikdienste, ETH Z¨ urich) Computer Systems Lab 2007-04-30 20 / 23

slide-21
SLIDE 21

Malloc lab: Comments

realloc semantics The newly returned block must preserve the content (up to its size). Pseudocode: new = malloc(size) memcpy(new, old, min(sizeold, sizenew)) free(old) An efficient implementation could check if it is possible to enlarge the current block.

Matteo Corti (Informatikdienste, ETH Z¨ urich) Computer Systems Lab 2007-04-30 21 / 23

slide-22
SLIDE 22

Malloc lab: Comments

Block format

size linking information flags payload payload flags size linking information

Put the linking information after the header: the memory can be used in allocated blocks.

Matteo Corti (Informatikdienste, ETH Z¨ urich) Computer Systems Lab 2007-04-30 22 / 23

slide-23
SLIDE 23

RSDL: Rotating Staircase DeadLine

Characteristics Starvation free, strict fairness, O(1), simple fixed accounting. Principle

  • Priority array (one ready list for priority)
  • Each level has a priority quota: when the priority quota is used all

the remaining processes are moved to the lower priority level.

  • Each process has a priority quota: when used the process is

moved to the lower priority level.

  • When a process quantum is used the process is moved to the

expired array (see the O(1) scheduler by J. Aas). Con Kolivas, RSDL completely fair starvation free interactive cpu scheduler http://thread.gmane.org/gmane.linux.kernel.ck/6462, March 2007

Matteo Corti (Informatikdienste, ETH Z¨ urich) Computer Systems Lab 2007-04-30 23 / 23