Part 1: Getting Started What is next Read mipssim.cc Read - - PDF document

11/19/2010 1

Project 3

Tao Yang

11/19/2010 2

Project 3 Workload

• vm directory for part 1 is empty

– Need to come up the design and code – Leverage Project 2 experience

• Test C programs provided are sample code

and they may contain bugs.

• Amount of work

– Part 1. ~400 lines of code in vm ( few in userprog) – Part 2. ~250 or less in filesys (few in userprog)

• Part 1 and 2 can be done in parallel

11/19/2010 3

Project 3 part 1: Virtual Memory

• Work on vm subdirectory mainly

– And addrspace.cc/.h and exception.cc in userprog

• Create/manage a backing store (a file called SWAP using

the OpenFile class).

• Implement a page fault handler with dirty bit handling and

a page replacement policy (LRU or second chance)

• Design with no implementation for copy-on-write.
• Test under various conditions:

– One process with an address space larger than physical memory. – Concurrent processes with combined address space larger than physical memory.

11/19/2010 4

Project 3 part 2: File system

• Work on filesys subdirectoy mainly.

– And change system call handling code in userprog if needed (to use this extended file system).

• Extend the size of each file from 4K to 100K+

– Few single indirect pointers in i-node.

• Extensible file size (not fixed size).

– Can add more than 10 files to the directory (currently the disk directory file is a fixed size

• f 10 entries)

Part 1: Getting Started

• Read machine/translate.cc:

– In Machine:Translate() for virtual address translation, PageFaultException is detected when the desired page is not in memory. – In Machine:ReadMem, Translate() is called for translating the desired virtual memory address and machine->RaiseException() is called with PageFaultException error.

What is next

• Read mipssim.cc

– Machine->ReadMem() is called in executing each instruction. – If PageFaultException is detected, the exception handler should load the desired page. – The hardware will try again.

• Need to expand exception.cc to handle

PageFaultException.

– Once handled, return to user mode and restart the instruction caused the fault

11/19/2010 2

User Instruction Execution

Machine:Translate() ExceptionHandler() Deal with PageFaultException Cannot find this page? Raise PageFaultException Page writing? Set dirty bit Machine:Run () OneInstruction () ReadMem () WriteMem () Re-execute if Exception is raised

Files to be modified for Part 1

• New files in directory vm

– Virtual memory manager – Swap space manager

• Directory userprog (extension to Project 2)

– exception.cc

• Extension to handle PageFaultException

– Addrspace.cc/.h

• Prepare code/data for SWAP backstore.
• Virtual address translation -> paging if needed

PageFaultException handling

• For a virtual address which is not in

memory,

– Find free space to host the desired page in memory.

• Load the desired page from SWAP to memory.

– If no free space available, execute a replacement policy to find a victim page.

• Swap out the victim page.
• Swap in the desired page.
• Adjust the page table.

Suggested Class in vm

• Write a Swap Manager to facilitate free

sector allocation in Swap File.

– Initialize SWAPSIZE (512) as the total free sector available. – Allocate a sector. – Free a sector.

Suggested Class in vm

• Virtual memory manager

– load a page containing the virtual address (copy from the SWAP file to an allocated memory page). – Find a victim page – Swap out a selected page to SWAP

Modify AddSpace.cc

• In translating a virtual user address for kernel, if it

is not in memory, bring data from SWAP.

• When allocating a new address space,

– Allocate a proper number of sectors from SWAP for this process. – Copy binary data to the allocated SWAP sectors. – Initial page number for this process can be 0.

11/19/2010 3

Synchronization issues

• Two system calls may be processed concurrently

and the synchronization is needed.

• Any time a process sleeps in the kernel, another

process can run – two processes try to initiate I/O on the same page at the same time

• May need a lock (or at least a ``busy'' flag) for

each physical memory page, and possibly also for each virtual memory page.

Design-only task: Copy-on-Write for Process Forking.

• Let a child process share the same page as

the parent process if this page is not modified by the child process or by the parent process.

• During normal process, such a page is read
• nly.
• During write operation, a

ReadOnlyException is generated and a proper action is needed:

• Specify program names/files to be

modified and what to be added.

Part 2: Steps for Nachos file system extension

• 1. Understand how the Nachos file system operates and how

Nachos implements files.

• 2. Modify the file system calls from Project 2 to use Nachos

file system

• 3. Modify Nachos file system so that you can create larger

(100K+)

• 4. Allow the Write system call to extend the size of a file.

5. Support more than 10 files in directory

E.g. expand the total no of directory file entries by 10 every time when a file is added and expansion is needed.

Files to be modified in Part 2

• Directory filesys

– filehdr.cc/.h Support single indirect pointers in i-node and can expand the file length with more sectors allocated. –

• penfile.cc Expand file length/allocate sectors when

writing data at the end or beyond. – directory.cc Add more table entries when the current table is full and a new file is added.

• Directory userprog

– exception.cc

• Make sure that file system calls use extended

Nachos file system.

Comparison: Unix file i-node (4K bytes per block)

Step 3: Extend maximum file size

• Modify the FileHeader i-node to include 4 data sector

pointers, and 26 sectors for single indirect mapping. Each

• f these 26 sectors are a block of pointers

– At most map 4 + 26*32=836 blocks. – Maximum file length = 836*128=107008 bytes.

• Suggest to add a class called IndirectPointerBlock which

takes 1 sector (32 entries) with following operations.

– FetchFrom (int sector). WriteBack(int sector). – AddSector (int sector) – add a new sector. – Deallocate(BitMap *freeMap) – deallocate all sectors. – ByteToSector (int offset). – convert an offset in bytes to a sector.

11/19/2010 4

Step 4: Extensible file

• Allow the Write system call to extend the size of a file.

– Currently, if the user tries to write beyond the end of the file, Nachos returns an error. – Modify the current implementation so that a write beyond the end

• f the file extends the size of the file.

– The exception is that if the file is created to be a given size, your implementation should allocate as many sectors (and block pointers) as required to hold that amount of data.

• Gracefully handle the case of the disk being full or

maximum size is reached - the user process should not

• crash. Instead an error should be returned.

Step 4

• Modify WriteAt() method in openfile.cc :

– Case 1: Write request is within the bounds of the file. Handle like the existing code – Case 2: Write request overwrites some portion

• f existing file and goes beyond by N bytes.

– Case 3: Write request overwrites no data, but goes beyond the end of the file. Here, no data needs to be read in.

• Include a routine in FileHeader to extend a

file by N bytes.

Testing issues

• Clean out all the .c files in the code/test directory. There

might be name conflicts from previous projects

• Get sample test programs in part1 and part2 of the

~cs170/nachos-projtest/proj3 directory.

• Test part 1. Test as you would for project 2. You can run

the ./nachos in the vm/ or userprog/ directories. You can use the unix file system for the test.

• Test part 2. It can be convenient to create another test

directory (e.g. test under test2 directory or fielsys/test)

• Notice file names cannot exceed 9 characters.

Testing issues in Part 2

• May need to load user binaries/data into Nachos file

system during execution

• Example:

csil$ pwd cs170/nachos/code/filesys csil$ ./nachos -f -cp ../test2/Prog1 Prog1 -x Prog1

• Another example with two binary programs

csil$ ./nachos -f -cp ../test2/Prog2 Prog2 -cp ../test2/Prog3/Prog3 -x Prog2

Submission

• Writeup a file HW3_WRITEUP I

– What is completed. What is not. – Design for each functionality (partial credit for those not working or not implemented) – a separate paragraph on the design and code change necessary to implement the copy-on-write feature..

• Provide a list of test program names and explain what to

test with each test program, and your findings for each test.

• Also a short note listing times are available/preferred or

not available in case Maha needs to meet you.