Motivation Laboratory work in TDDI04 Operating systems are nearly - - PDF document

1

Laboratory work in TDDI04 Introduction to Pintos Assignments 00, 0, 1

Viacheslav Izosimov 2009-02-02 viaiz@ida.liu.se

Motivation

Operating systems are nearly everywhere Crucial piece of software Perform concurrent tasks Synchronization C is the main language for real

• perating systems!

Motivation

Nearly the first experience with a relatively large piece of software Direct jump from 1000 to 1000000 can be too difficult!

Your programs so far. Pintos. up to 1000 LOC 5000-10000 LOC 1000000-5000000 LOC Real systems.

Outline

General questions Introduction to Pintos General description of labs Lab rules General algorithm for completing the labs Lab 0

Setting up the program environment Debugging of Pintos

Lab 1

Pintos kernel and command line Execution of user programs in Pintos Argument passing to the user programs

Synchronization

C pointers and address arithmetics

General questions

Answer on the preparatory questions before implementing anything!

Labs:

Work jointly! Groups of two students No cheating! Sign up as soon as possible!

Lab lessons:

More lessons than the last year! Exercises during lessons Have your design ready before implementation! No quizzes

Introduction to Pintos

Shift from Nachos (1990s) to Pintos (2004)

Pintos is the most up-to-date “toy operating” system Can boot as a real operating system Code is not as “messy” as in Nachos Pure C implementation

Pintos

Install on Sun machines during Lab #0 Extend during Lab #1 − #5

2

Introduction to Pintos

Runs on x86 machine QEMU – a computer system emulator is used UNIX environment C implementation (both Kernel and user programs) Not complete…

Your task!

User-exception handler

Introduction to Pintos

Scheduler User programs Threads Memory management Kernel stack User stack User memory File system Interrupt handler Timer Synchronization primitives

General Description of Labs

Lab 00: “Introduction to C Programming” Checks your ability to complete the labs Lab 0: “Introduction and installation”

Introductory lab, where you need to setup your program environment and try out debugging

Lab 1: “Setting Up the Program Stack and Starting the First User Program”

Learn how operating system kernel operates User program & arguments passing Simple synchronization tasks This lab is essential for the work on later labs and understanding of PintOS

General Description of Labs

Lab 2: “System calls”

System calls Single user program Console

Lab 3: “Memory management, securing the system calls”

Memory management & security issues

Lab 4: “Execution, termination and synchronization of user programs”

Handling program arguments Execution of several user programs Termination of a user program Synchronization of shared data structures Wait system call

General Description of Labs

Lab 5: “File system”

Synchronization of read-write operations

But before…

Lab Rules

Every member of the group participate equally and every member should be able to answer questions related to labs Before doing the lab answer on preparatory questions on the lab page Understand first, then implement Try to find ”bugs” yourself at first No copying of source code from any sources, no cheating Pass assignments on time!

3

Lab Rules

Pass assignments on time! Preliminary “self-control” schedule

Lab 00, 0 – Friday! Lab 1 – 13th of February Lab 2 – 25th of February Lab 3 – 4th of March Lab 4 – 1st of April Lab 5 – 8th of May FINAL DEADLINE – 15th of May

General Algorithm to Complete the Labs

• 1. Answer on all preparatory questions!
• 2. Understand what you are supposed to do (read

instructions carefully)

• 3. Discuss all the issues with your lab partner to make sure

that he/she also understands (That’s you future help!)

• 4. Understand the source code, which you are supposed to

understand

• 5. Begin implementation!
• 6. Document carefully what you are doing (put date, name

and short description around each new piece of code)

• 7. Work hard… and not only during lab hours, they are

mostly for asking questions from lab assistants

Lab 00

You will need to debug a small program “debugthis.c” with DDD after compiling it with GCC (not CC!) Create a linked list Learn how to browse the source code

If you are not able to complete the lab, then you should focus on C before you proceed further or even consider to take the course next year. No formal pass/fail or other regulations though…

Lab 0

Set up the program environment (don’t forget!)

module initadd ~TDDI04/labs2008/modules/pintos module add ~TDDI04/labs2008/modules/pintos

Check if you have correct version of GCC and GDB module list GCC should be 3.4.x and you will know if GDB version is correct after completing Lab 00 Copy Pintos to your directory

gzip -cd ~TDDI04/labs2008/pintos_ida.tar.gz | tar xvf -

Lab 0

You will have the following structure of directories pintos/src

devices examples filesys lib misc tests threads userprog utils vm

Lab 0

Compile and build Pintos cd pintos/src/threads gmake Test if Pintos works cd build pintos --qemu -- run alarm-multiple

4

Lab 0

cd ~/pintos/src/ gmake -C examples gmake -C userprog cd userprog/build pintos-mkdisk fs.dsk 2 pintos -v -p ../../examples/sumargv -a sumargv -- -f -q run sumargv

Lab 0

Debugging (from build) To debug pintos you need two terminals. One to run pintos and one to run the debugger:

pintos --qemu --gdb -- run testname ddd --gdb --debugger pintos-gdb kernel.o&

Lab 0

Instead of the normal pintos command you run the command debugpintos: cd ??/build debugpintos -v -p ../../examples/sumargv -a sumargv --

• f -q run sumargv

It will prepare pintos as usual, but stop and wait for the debugger to connect. In the debug terminal, run: cd ??/build pintos-gdb kernel.o It starts gdb configured for pintos. In the debugger, enter the commands: debugpintos break main continue quit

Pintos Disk (1)

Dealing with Pintos disk In “userprog/build”: pintos-mkdisk fs.dsk 2

This will create a file fs.dsk with a 2MB simulated disk in the directory.

Format: pintos --qemu -- -f -q. Copy user programs: pintos --qemu -p programname -- -q with new name

pintos --qemu -p programname -a newname -- -q

Pintos Disk (2)

Dealing with Pintos disk Example:

pintos --qemu -p ../../examples/sumargv

• a severe_student_program -- -q

To copy a file from the disk:

pintos --qemu -g programname -- -q

• r

pintos --qemu -g programname -a newname -- -q * -p = put, -g = get

To run: pintos --qemu -- run programname Example:

pintos --qemu -- run severe_student_program

Lab 1

Learn how operating system kernel operates

Initialization Starting of the user program De-initialization

User program & arguments passing

The first user program

Simple synchronization tasks

Semaphores, Locks, Conditions

This lab is essential for the work on later labs and understanding of PintOS

Synchronization mechanisms

5

Lab 1

Your task is to study Pintos kernel and implement loading of the first user program and argument passing Argument passing is mostly about address arithmetics and learning the general structures used in the kernel There are two alternative implementations of the synchronization during the loading:

with Semaphores with Locks/Conditions

Lab 1

Understand the source code in

threads/init.c threads/synch.[h|c] threads/synchlist.[h|c] threads/thread.[h|c] userprog/process.[h|c]

contains a skeleton of your implementations

Lab 1::Threads (1)

A system contains several threads running in parallel Thread is a “basic unit of CPU utilization” §Thread ID §A program counter §A register set §A stack

Thread 1 Thread 2 Thread execution sequence

Lab 1::Threads (2)

• !"##

$% & & !

• $'

()! $$#*+,$-,.##/

• &

Entering SimpleTest *** thread 0 looped 0 times *** thread 1 looped 0 times *** thread 0 looped 1 times *** thread 1 looped 1 times …

Lab 1::Kernel Lab 1::Kernel

6

Lab 1::Kernel (2) Lab 1::Kernel & User Program (1)

Kernel User Program process_execute(task); // starts the user program

e x e c u t i

• n

e x e c u t i

• n

e x e c u t i

• n

p r

• c

e s s _ e x e c u t e s t a r t _ p r

• c

e s s ( S P ) S P : l

• a

d i n g S P : s t a c k i n i t i a l i z a t i

• n

p r

• c

e s s _ e x e c u t e : g e n e r a t e p i d waiting until completion of start_process

S i g n a l / s e m a _ u p W a i t / s e m a _ d

• w

n

Critical Section Problem

Critical Section: A set of instructions, operating on shared data or resources, that should be executed by a single process without interruption Atomicity of execution Mutual exclusion: At most one process should be allowed to operate inside at any time Consistency: inconsistent intermediate states of shared data not visible to other processes outside General structure, with structured control flow: Entry of critical section C … critical section C: operation on shared data Exit of critical section C

Semaphores

Semaphore S: shared integer variable two atomic operations to modify S: P() and V() P (S): while S <= 0 do Sleep(currentThread) S--; // Critical section V (S): WakeUp (sleepingThread) S++;

Example: At most five passengers in the car

Semaphores

00*

!$! !! $$ 1 $$!2 !34 !$!$2'5!34!#5$34 $2' & !3433 $!$$ &

Semaphores

006

!$! !! $$ 1 $$!2 7!$%5!34! $2'!$%!$$5!34!# ! # !34 $!$$ &

Good for handling simple synchronization problems

7

Locks

Binary semaphore (e.g. “taken” or “not taken”) Only the owner can release the lock Advantage: No other thread can enter the critical section! Essential for security, for example, access to the shared data, arrays, lists, and etc. Problem: deadlock situations

Conditions

Queuing mechanism on top of Locks Helps to manage the Locks Prevents deadlock situations Locks are often used together with Conditions Locks/Conditions are good for handling complex synchronization problems Attention: never use a Condition with an empty Lock!

Lab 1::Part B

Example

Lab 1::Kernel & User Program (2)

Add your implementation of into process_execute() and process_start() in process.c

process_execute() { … tid = thread_create … generate pid from tid; wait until start_process(); return pid or -1 } start_process() { loading – DONE! initialization – DONE! putting program arguments into stack signal to process_execute }

pid = -1, if the program cannot load or run for any reason. Use an array or a list to keep track of pid:s. pid might equal tid, because we have only one thread per process. Limit the number of user programs (t.ex. 64 or 128).

pid = process ID tid = thread ID

Lab 1::Stack (1)

Add command line parsing and stack initialization code here!

Lab 1::Stack (2)

STEP 1. Parse the string: Use strtok_r(), prototyped in lib/string.h Read comments in lib/string.c or man page (run man strtok_r) Limit the number of arguments (for simplicity) STEP 2. Set up the stack: Necessary details about setting up the stack for this task you can find in Program Startup Details section of Pintos documentation.

8

Lab 1::Stack (3) Lab 1::Stack (4)

nasty_program arg1 arg2 arg3 After parsing: nasty_program, arg1, arg2, arg3 Place the words at the top of the stack Align to 4-byte-words, add 0’s Reference the words through the pointers (pointers should point to the addresses of the words in the stack) Put the pointers to the stack (followed with NULL pointer)

Lab 1::Stack (5)

Put the address of the first pointer to the stack Put the number of words to the stack (the number

• f arguments + 1)

Put “faked” return address to the stack (e.g. NULL). This is needed in order to meet x86 conventions for program arguments, even though this return address will not be used. So, what we get if we assume that PHYS_BASE is 0xc0000000 …

void (*) ()

return address

0xbfffffc0 int

4

argc 0xbfffffc4 char ** 0xbfffffd0 argv 0xbfffffc8 char * 0xbfffffe6 argv[0] 0xbfffffcc char * 0xbffffff4 argv[1] 0xbfffffd0 char * 0xbffffff8 argv[2] 0xbfffffd4 char * 0xbffffffc argv[3] 0xbfffffd8 char * argv[4] 0xbfffffdc uint24_t

word-align

0xbfffffe0 char[14]

nasty_program\0

argv[0][...] 0xbfffffe3 char[5]

arg1\0

argv[1][...] 0xbffffff1 char[5]

arg2\0

argv[2][...] 0xbffffff6 char[5]

arg3\0

argv[3][...] 0xbffffffb

Type Data Name Address 29 --> 32

Lab 1::Test

To test your implementation use sumargv. In addition to sumargv the following tests should pass when you run gmake check if your implementation is correct:

tests/userprog/args-none tests/userprog/args-single tests/userprog/args-multiple tests/userprog/args-many tests/userprog/args-dbl-space

Exercise Time!

9

Conclusions

Introduction to Pintos and labs in general Labs 00, 0, and 1 Deadline for Lab #1 is 13th of February