Owen S. Hofmann, Xuan Wang, Emmett Witchel, Donald E. Porter 1 - - PowerPoint PPT Presentation

▶
owen s hofmann xuan wang
SMART_READER_LITE
LIVE PREVIEW

Owen S. Hofmann, Xuan Wang, Emmett Witchel, Donald E. Porter 1 - - PowerPoint PPT Presentation

Aug 06, 2023 185 likes •540 views

Sangman Kim , Michael Z. Lee, Alan M. Dunn, Owen S. Hofmann, Xuan Wang, Emmett Witchel, Donald E. Porter 1 Fine-grained locking - Bug-prone, hard to maintain Parallelism - OS provides poor support Coarse-grained locking - Reduced

slide-1
SLIDE 1

Sangman Kim, Michael Z. Lee, Alan M. Dunn, Owen S. Hofmann, Xuan Wang, Emmett Witchel, Donald E. Porter

1

slide-2
SLIDE 2

2

Parallelism Maintainability

Fine-grained locking

  • Bug-prone, hard to maintain
  • OS provides poor support

Coarse-grained locking

  • Reduced resource utilization
slide-3
SLIDE 3

3

Parallelism

Server Applications working with OS API

System Transaction

Server Applications working with OS API

Maintainability

slide-4
SLIDE 4

Linux TxOS

 TxOS provides operating system transaction

[Porter et al., SOSP 2009]

  • Transaction for OS objects (e.g., files, pipes)

4

System transaction in TxOS

TxOS

Application

JVM

Middleware state sharing with multithreading

TxOS system calls Middleware state sharing

slide-5
SLIDE 5

TxOS

 TxOS provides operating system transaction

[Porter et al., SOSP 2009]

  • Transaction for OS objects (e.g., files , pipes)

5

Synchronization in legacy code

Application

JVM

TxOS system calls Synchronization primitives Middleware state sharing

slide-6
SLIDE 6

TxOS

 TxOS provides operating system transaction

[Porter et al., SOSP 2009]

  • Transaction for OS objects (e.g., files, pipes)

 TxOS+: Improved system transactions

6

Application

JVM

TxOS system calls

TxOS+

TxOS+: pause/resume, commit ordering, and more

Up to 88% throughput improvement At most 40 application line changes

Synchronization primitives Middleware state sharing

slide-7
SLIDE 7

Background: system transaction System transactions in action Challenges for rewriting applications Implementation and evaluation

7

slide-8
SLIDE 8

 Transaction Interface and semantics

  • System calls: xbegin(), xend(), xabort()
  • ACID semantics

▪ Atomic – all or nothing ▪ Consistent – one consistent state to another ▪ Isolated – updates as if only one concurrent transaction ▪ Durable – committed transactions on disk

  • Optimistic concurrency control

 Fix synchronization issues with OS APIs

8

slide-9
SLIDE 9

 Lazy versioning: speculative copy for data  TxOS requires no special hardware

9

xbegin(); write(f, buf); xend();

Commit Abort

Conflict!

inode header

inode i

inum lock … inode data size mode … Copy of inode data

slide-10
SLIDE 10

Background: system transaction System transactions in action Challenges for rewriting applications Implementation and evaluation

10

slide-11
SLIDE 11

 Parallelizing applications that synchronize

  • n OS state

 Example 1: State-machine replication

  • Constraint: Deterministic state update

 Example 2: IMAP Email Server

  • Constraint: Consistent file system operations

11

slide-12
SLIDE 12

 Core component of fault tolerant services

  • e.g., Chubby, Zookeeper, Autopilot

 Replicas execute the same sequence of operations

  • Often single-threaded to avoid non-determinism

 Ordered transaction

  • Makes parallel OS state updates deterministic
  • Applications determine commit order of transactions

12

slide-13
SLIDE 13

 Everyone has concurrent email clients

  • Desktop, laptop, tablets, phones, ....
  • Need concurrent access to stored emails

 Brief history of email storage formats

  • mbox: single file, file locking
  • Lockless Maildir
  • Dovecot Maildir: return of file locking

13

slide-14
SLIDE 14

 mbox

  • Single file mailbox of email messages
  • Synchronization with file-locking

▪ One of fcntl(), flock(), lock file (.mbox.lock) ▪ Very coarse-grained locking ~/.mbox

14

From MAILER-DAEMON Wed Apr 11 09:32:28 2012 From: Sangman Kim <sangmank@cs.utexas.edu> To: EuroSys 2012 audience Subject: mbox needs file lock. Maildir hides message. ….. From MAILER-DAEMON Wed Apr 11 09:34:51 2012 From: Sangman Kim <sangmank@cs.utexas.edu> To: EuroSys 2012 audience Subject: System transactions good, file locks bad! ….

slide-15
SLIDE 15

 Maildir: Lockless alternative to mbox

  • Directories of message files
  • Each file contains a message
  • Directory access with no synchronization (originally)

 Message filenames contain flags

Maildir/cur 00000000.00201.host:2,T 00001000.00305.host:2,R 00002000.02619.host:2,T 00010000.08919.host:2,S 00015000.10019.host:2,S Trashed Replied Trashed Seen Seen

15

slide-16
SLIDE 16

PROCESS 2 (MARKING)

if (access(“043:2,S”)): rename(“043:2,S”, “043:2,R”)

PROCESS 1 (LISTING)

while (f = readdir(“Maildir/cur”)): print f.name

16

018:2,S 021:2,S 052:2,S 061:2,S

Seen

“Maildir/cur” directory

Seen

043:2,S

Seen Seen Seen

slide-17
SLIDE 17

043:2,R

Replied

PROCESS 2 (MARKING)

if (access(“043:2,S”)): rename(“043:2,S”, “043:2,R”)

PROCESS 1 (LISTING)

while (f = readdir(“Maildir/cur”)): print f.name

17

018:2,S 021:2,S 052:2,S 061:2,S

Seen

“Maildir/cur” directory

Seen

043:2,S

Seen Seen Seen Process 1 Result 018:2,S 021:2,S 052:2,S 061:2,S

Message missing!

slide-18
SLIDE 18

 Maildir synchronization

  • Lockless
  • File locks

▪ Per-directory coarse-grained locking ▪ Complexity of Maildir, performance of mbox

  • System transactions

“certain anomalous situations may result” – Courier IMAP manpage

18

slide-19
SLIDE 19

PROCESS 1 (MARKING)

xbegin() if (access(“XXX:2,S”)): rename(“XXX:2,S”, “XXX:2,R”)xend()

PROCESS 2 (MESSAGE LISTING)

xbegin() while (f = readdir(“Maildir/cur”)): print f.name xend() xbegin() xend() xbegin() xend()

Consistent directory accesses with better parallelism

19

slide-20
SLIDE 20

Background: system transaction System transactions in action Challenges for rewriting applications Implementation and evaluation

20

slide-21
SLIDE 21
  • 1. Middleware state sharing
  • 2. Deterministic parallel update for system state
  • 3. Composing with other synchronization primitives

21

slide-22
SLIDE 22

 Problem with memory management

  • Multiple threads share the same heap

Thread 1 Thread 2

In Transaction xbegin(); p1 = malloc(); xabort(); p2 = malloc(); *p2 = 1;

22

Middleware (libc) Kernel mmap() Heap Transactional object for heap p1

slide-23
SLIDE 23

 Problem with memory management

  • Multiple threads share the same heap

Thread 1 Thread 2

In Transaction xbegin(); p1 = malloc(); xabort(); p2 = malloc(); *p2 = 1;

23

Middleware (libc) Kernel Transactional object for heap Heap

FAULT!

Certain middleware actions should not roll back

p1 p2

unmapped

slide-24
SLIDE 24

USER-INITIATED ACTION

 User changes system state

  • Most file accesses
  • Most synchronization

MIDDLEWARE-INITIATED

 System state changed as

side effect of user action

  • malloc() memory mapping
  • Java garbage collection
  • Dynamic linking

 Middleware state shared

among user threads

  • Can’t just roll back!

24

slide-25
SLIDE 25

 Transaction pause/resume

  • Expose state changes by middleware-initiated

actions to other threads

  • Additional system calls

▪ xpause(), xresume()

  • Limited complexity increase

▪ We used pause/resume 8 times in glibc, 4 times in JVM ▪ Only used in application for debugging

25

slide-26
SLIDE 26

SysTransaction.begin(); files = dir.list(); SysTransaction.end();

Java code

xpause() xresume() xbegin(); files = dir.list(); VM operations

(garbage collection)

xend();

JVM Execution

26

slide-27
SLIDE 27

 17,000 lines of kernel changes

  • Transactionalizing file descriptor table
  • Handling page lock for disk I/O
  • Memory protection
  • Optimization with directory caching
  • Reorganizing data structure
  • and more

 Details in the paper

27

slide-28
SLIDE 28

Background: system transaction System transactions in action Challenges for rewriting applications Implementation and evaluation

28

slide-29
SLIDE 29

 Implemented in UpRight BFT library  Fault tolerant routing backend

  • Graph stored in a file
  • Compute shortest path
  • Edge add/remove

 Ordered transactions for deterministic update

29

slide-30
SLIDE 30

Component Total LOC Changed LOC Routing application 1,006 18 (1.8%) Upright Library 22,767 174 (0.7%) JVM 496,305 384 (0.0008%) glibc 1,027,399 826 (0.0008%)

30

slide-31
SLIDE 31

500 1000 1500 2000 2500 3000 3500 4000 10 20 30 40 50 60 70 80 90 100 TxOS, dense Linux,dense TxOS,sparse Linux,sparse Throughput (req/s) Dense graph: 88% tput  Sparse graph: 11% tput  Work to add/delete edges small compared to scheduling overhead

31

Write ratio (%) BFT graph server

slide-32
SLIDE 32

 Dovecot mail server

  • Uses directory lock files for maildir accesses

 Locking is replaced with system transactions

  • Changed LoC: 40 out of 138,723

 Benchmark: Parallel IMAP clients

  • Each client executes operations on a random message

▪ Read: message read ▪ Write: message creation/deletion ▪ 1500 messages total

32

slide-33
SLIDE 33

 Dovecot benchmark with 4 clients

10 20 30 40 50 60 70 80 90

0 10 25 50 100

Write ratio (%) Tput Improvement (%) Better block scheduling enhances write performance

33

slide-34
SLIDE 34

 System transactions parallelize tricky server

applications

  • Parallel Dovecot maildir operations
  • Parallel BFT state update

 System transaction improves throughput with

few application changes

  • Up to 88% throughput improvement
  • At most 40 changed lines of application code

34