[PPT] - Formally Specifying POSIX File Systems Gian Ntzik , Pedro da Rocha PowerPoint Presentation

SLIDE 1

Formally Specifying POSIX File Systems

Gian Ntzik, Pedro da Rocha Pinto and Philippa Gardner

Imperial College London

July 16, 2015

1/39

SLIDE 2

2/39

POSIX File Systems

◮ POSIX: Portable Operating System Interface ◮ Large part devoted to the file system ◮ English Specification

◮ Underspecified ◮ Ambiguous ◮ Absence of proper memory model

SLIDE 3

3/39

Objectives

◮ Find formalism suitable for specifying POSIX file system

perations

◮ Suitable for clients and implementations ◮ Client reasoning in a program logic

◮ Focus a core fragment:

◮ Structural operations: mkdir, rmdir, link, unlink, rename, ... ◮ IO: open, read, write, lseek, close, ...

SLIDE 4

4/39

File System Example

1 2

tmp usr

3 4 5 7

share bin local lib .X0-lock

0101111011... 8 0110011011... 9

git

SLIDE 5

5/39

First Challenge: Atomicity

SLIDE 6

6/39

Example: unlink

◮ unlink(p) : Atomically remove the file identified by path p. ◮ “Atomic” has an unusual meaning in POSIX.

SLIDE 7

7/39

unlink(/usr/bin/git)

1 2

tmp usr

3 4 5 7

share bin local lib .X0-lock

0101111011... 8 0110011011... 9

git

SLIDE 8

7/39

unlink(/usr/bin/git)

1 2

tmp usr

3 4 5 7

share bin local lib .X0-lock

0101111011... 8 0110011011... 9

git ◮ Only removing git is required to be atomic.

SLIDE 9

7/39

unlink(/usr/bin/git)

1 2

tmp usr

3 4 5 7

share bin local lib .X0-lock

0101111011... 8 0110011011... 9

git ◮ Only removing git is required to be atomic. ◮ Path resolution: a sequence of atomic reads.

SLIDE 10

8/39

Sequence of atomic actions

FS1 FS1 FS2 FS2 FS3 FS4 Environment: mupltiple atomic updates Thread: single atomic read in path traversal Thread: single atomic update

usr bin git

unlink(/usr/bin/git)

SLIDE 11

9/39

Single atomic step specification: Atomic Hoare Triple

From TaDA, IRIS, ... we know how to specify a single atomic step.

P Q

ℂ

P C Q

SLIDE 12

10/39

Multi-atomic specifications

We extend to multiple atomic steps

P1 P2 P3 P4 Pn−1 Pn

...

ℂ

C ⊑

(P1, P2); (P3, P4); . . . ; (Pn−1, Pn)

SLIDE 13

11/39

Multi-atomic Program Logic

◮ Introduce multi-atomics:

C ⊑

(P1, P2); (P3, P4); . . . ; (Pn−1, Pn)
◮ Justified by an encoding in IRIS

◮ Extend reasoning rules for single atomic steps to multiple steps

SLIDE 14

12/39

Single Step Equivalence

SingleAtomic: C ⊑

(P, Q)
⇐

⇒ P C Q

SLIDE 15

13/39

Sequence Rule

C1 ⊑

. . . ; (P1, Q1)
C2 ⊑
(P2, Q2); . . .
C1; C2 ⊑
. . . ; (P1, Q1); (P2, Q2); . . .
Multi-Seq

SLIDE 16

14/39

Stuttering Rule

C ⊑

. . . ; (P, P); (P, Q); . . .
C ⊑
. . . ; (P, Q); . . .
Stutter

SLIDE 17

15/39

unlink: Formal Specification

FS1 FS1 FS2 FS2 FS FS'

usr bin

unlink(/usr/bin/git)

git

unlink(/usr/bin/git) ⊑ resolve(/usr/bin, ι0, r); . . .

SLIDE 18

16/39

unlink: Formal Specification

FS1 FS1 FS2 FS2 FS FS'

usr bin

unlink(/usr/bin/git)

git

unlink(/usr/bin/git) ⊑ resolve(/usr/bin, ι0, r); (fs(FS) , in(FS, r, git) ⇒ rem(FS, r, git) ∗ ret = 0)

SLIDE 19

17/39

unlink: Formal Specification

FS1 FS1 FS2 FS2 FS FS'

usr bin

unlink(/usr/bin/git)

git

unlink(/usr/bin/git) ⊑

resolve(/usr/bin, ι0, r);
fs(FS) ,

in(FS, r, git) ⇒ rem(FS, r, git) ∗ ret = 0 ∧ out(FS, r, git) ⇒ fs(FS) ∗ ret = −1 ∗ errno = ENOENT

SLIDE 20

18/39

unlink: Formal Specification

FS1 FS1 FS2 FS2 FS FS'

usr bin

unlink(/usr/bin/git)

git

unlink(/usr/bin/git) ⊑

resolve(/usr/bin, ι0, r);

 fs(FS) , r ∈ In ⇒ in(FS, r, git) ⇒ rem(FS, r, git) ∗ ret = 0 ∧ out(FS, r, git) ⇒ fs(FS) ∗ ret = −1 ∗ errno = ENOENT

∧ r ∈ Err ⇒ fs(FS) ∗ ret = −1 ∗ errno = r

 

SLIDE 21

19/39

Second Challenge: Unordered actions

SLIDE 22

20/39

Specifying unordered actions

◮ Example: rename(p/a, p′/b) ◮ POSIX does not specify in which order p and p′ are resolved

SLIDE 23

20/39

Specifying unordered actions

◮ Example: rename(p/a, p′/b) ◮ POSIX does not specify in which order p and p′ are resolved ◮ We can’t do:

rename(p/a, p′/b) ⊑

resolve(p, ι0, r1) ; resolve(p′, ι0, r2) ; . . .

SLIDE 24

21/39

Specifying unordered actions

◮ Example: rename(p/a, p′/b) ◮ POSIX does not specify in which order p and p′ are resolved ◮ Solution:

rename(p/a, p′/b) ⊑

resolve(p, ι0, r1) resolve(p′, ι0, r2) ; . . .

SLIDE 25

22/39

Parallel Rule

C1 ⊑

(P1, P2); . . . ; (Pn−1, Pn)
C1 ⊑

(Q1, Q2); . . . ; (Qn−1, Qn) C1 C2 ⊑ (P1, P2); . . . ; (Pn−1, Pn) (Q1, Q2); . . . ; (Qn−1, Qn) Multi-Par

SLIDE 26

23/39

Client Applications

◮ Lock Files ◮ POSIX pipes ◮ POSIX Advisory Locks/Record Locking (on-going) ◮ Persistent Concurrent Queues (future)

SLIDE 27

24/39

Lock Files

◮ lock(path): atomically create a non-existing lock file at path ◮ unlock(path): remove the lock file identified by path ◮ Implemented similarly to spin locks

◮ open(path, O CREAT|O EXCL) to try to lock ◮ unlink to unlock

SLIDE 28

25/39

Lock File Implementation

function lock(path) { do { fd := open(path, O EXCL|O CREAT); } while (fd = −1); close(fd); } function unlock(path) { unlink(path); }

SLIDE 29

26/39

Heap Based Lock Specification

◮ We know how to specify locks on the heap

SLIDE 30

26/39

Heap Based Lock Specification

◮ We know how to specify locks on the heap

emp
makelock()
Lock(ret, 0)
Lock(x, v) lock(x) Lock(x, 1) ∗ v = 0

Lock(x, 1) unlock(x) Lock(x, 0)

SLIDE 31

26/39

Heap Based Lock Specification

◮ We know how to specify locks on the heap

emp
makelock()
Lock(ret, 0)
Lock(x, v) lock(x) Lock(x, 1) ∗ v = 0

Lock(x, 1) unlock(x) Lock(x, 0)

◮ Ideally, we want the same specification for lock files ◮ Using path instead of heap address x

SLIDE 32

27/39

Third Challenge: Ownership

SLIDE 33

28/39

The Heap case

Allocated Heap Allocated Heap

ret

makeLock()

◮ The module owns the newly allocated memory

SLIDE 34

29/39

The Heap case

Allocated Heap Allocated Heap

ret

makeLock() HL

◮ The module enforces the sharing protocol

SLIDE 35

30/39

The File System case

1 2

tmp usr

3 4 5 7

share bin local lib .X0-lock

8 0110011011... 9

git

SLIDE 36

30/39

The File System case

1 2

tmp usr

3 4 5 7

share bin local lib .X0-lock

8 0110011011... 9

git ◮ No operation to extend with fresh path, unknown to

environment

◮ Global path address space

SLIDE 37

31/39

The File System case

1 2

tmp usr

3 4 5 7

share bin local lib .X0-lock

8 0110011011... 9

git

LF

SLIDE 38

31/39

The File System case

1 2

tmp usr

3 4 5 7

share bin local lib .X0-lock

8 0110011011... 9

git

LF

◮ Clients must agree on sharing protocol

SLIDE 39

31/39

The File System case

1 2

tmp usr

3 4 5 7

share bin local lib .X0-lock

8 0110011011... 9

git

LF

◮ Clients must agree on sharing protocol ◮ Cooporative ownership

SLIDE 40

32/39

Lock file specification

If any client’s protocols

1 2

tmp usr

3 4 5 7

share bin local lib .X0-lock

8 0110011011... 9

git

SLIDE 41

33/39

Lock file specification

contains the lock file protocol

1 2

tmp usr

3 4 5 7

share bin local lib .X0-lock

8 0110011011... 9

git

LF

SLIDE 42

34/39

Lock file specification

with other protocols

1 2

tmp usr

3 4 5 7

share bin local lib .X0-lock

8 0110011011... 9

git

SLIDE 43

35/39

Lock File specification

then, we can use the lock file specification: lock(path) ⊑

Lock(path, v) , Lock(path, 1) ∗ v = 0
unlock(path) ⊑
Lock(path, 1) , Lock(path, 0)

SLIDE 44

36/39

Lock File specification

∀path, P . islock

path, P
⇒

lock(path) ⊑

Lock(path, v) , Lock(path, 1) ∗ v = 0
unlock(a) ⊑
Lock(path, 1) , Lock(path, 0)
islock
path, P
∃R. P ⇚

⇛ LF(path) ∗ R

SLIDE 45

37/39

Conclusions

◮ Introduced multi-atomic specifications ◮ Ordered sequences of atomic actions ◮ Unordered parallel atomic actions ◮ Formalised a fragment of POSIX file system operations ◮ Client reasoning ◮ Ownership in file systems is cooperative

SLIDE 46

38/39

Future Work

◮ Link with operational models & testing ◮ Verify implementations ◮ Explore connection with refinement & Hoare’s algebraic laws ◮ Mechanisation