ShfmLocks: Scalable and Practjcal Locking for Manycore Systems - - PowerPoint PPT Presentation

ShfmLocks: Scalable and Practjcal Locking for Manycore Systems

Changwoo Min COSMOSS Lab / ECE / Virginia Tech https://cosmoss-vt.github.io/

File system becomes a botuleneck on manycore systems

2

0k 2k 4k 6k 8k 10k 12k 14k 10 20 30 40 50 60 70 80 messages/sec #core Exim mail server on RAMDISK btrfs ext4 F2FS XFS

• 3. Never scales

Embarrassingly parallel application!

• 1. Saturated
• 2. Collapsed

Even in slower storage medium fjle system becomes a botuleneck

0k 2k 4k 6k 8k 10k 12k btrfs ext4 F2FS XFS messages/sec Exim email server at 80 cores RAMDISK SSD HDD

FxMark: File systems are not scalable in manycore systems

• ps/sec

Legend btrfs ext4 ext4NJ F2FS tmpfs XFS

Create fjles on a shared directory Locks are critjcal in performance and scalability

Future hardware further exacerbates the problem

5

0k 2k 4k 6k 8k 10k 12k 14k 10 20 30 40 50 60 70 80 messages/sec #core Exim mail server on RAMDISK btrfs ext4 F2FS XFS

• 3. Never scales

Embarrassingly parallel application!

• 1. Saturated
• 2. Collapsed

Why this happens? : Memory access is NOT scalable

• 1. Read operations are

scalable

Read private Read shared

Why this happens? : Memory access is NOT scalable

• 1. Read operations are

scalable

Read private Read shared Write private Write shared

• 2. Write operations are

NOT scalable

Why this happens? : Memory access is NOT scalable

• 1. Read operations are

scalable

Read private Read shared Write private Write shared

• 2. Write operations are

NOT scalable

• 3. Write operations

interfere read operations

Shared lock variable (fmag) Shared data protected by the lock

Why this happens? : Cache coherence is not scalable

• Cache coherent traffjc dominates!!!
• Writjng a cache line in a popular MESI protocol:

–

Writer’s cache: Shared → Exclusive

–

All readers’ cache line: Shared → Invalidate

Should minimize contended cache lines and core-to-core communication traffic

LLC Memory LLC Memory Socket-1 Socket-2

Linux kernel lock adoptjon / modifjcatjon

Dekker's algorithm (1962) Semaphore (1965) Lamport's bakery algorithm (1974) Backofg lock (1989) Ticket lock (1991) MCS lock (1991) Hierarchical lock – HCLH (2006) Flat combining NUMA lock (2011) Remote Core locking (2012) Cohort lock (2012) RW cohort lock (2013) Malthusian lock (2014) HMCS lock (2015) AHMCS lock(2016) HBO lock (2003)

NUMA- aware locks

Spinlock TTAS → Semaphore TTAS + block → Rwsem TTAS + block → Spinlock ticket → Mutex TTAS + spin + block (3.16) → Rwsem TTAS + spin + block (3.16) → Spinlock ticket (2.6) → Mutex TTAS + block (2.6) → Rwsem TTAS + block → Spinlock qspinlock (4.4) → Mutex TTAS + spin + block → Rwsem TTAS + spin + block →

Lock's research efgorts

1990s 2011 2014 2016

Lock’s research efgorts and their use Adoptjng new locks is necessary but it is not easy

Two dimensions of lock design/goals

11

In high thread count In single thread In oversubscriptjon Minimize lock contentjons No penalty when not contended Avoid bookkeeping overheads

1) High throughput

Memory footprint Scales to millions of locks (e.g., fjle inode)

2) Minimal lock size

Locks performance: Throughput

12

Operations / second # threads

1 socket > 1 socket

Stock

Oversubscribed

• Performance crashes afuer 1 socket.

Due to non-uniform memory access (NUMA). Accessing local socket memory is faster than the remote socket memory.

(e.g., each thread creates a fjle, a serial operatjon, in a shared directory)

LLC Memory LLC Memory Socket-1 Socket-2

Locks performance: Throughput

13

Operations / second # threads

1 socket > 1 socket

Stock

Oversubscribed

• Performance crashes afuer 1 socket.

Due to non-uniform memory access (NUMA). Accessing local socket memory is faster than the remote socket memory.

• NUMA also afgects oversubscriptjon.

(e.g., each thread creates a fjle, a serial operatjon, in a shared directory)

Prevent throughput crash afuer one socket

Existjng research efgorts

• Making locks NUMA-aware:

○ Two level locks: per-socket and global ○ Generally hierarchical

• Problems:

○ Require extra memory allocatjon ○ Do not care about single thread throughput

• Example: CST1

14

Socket-2 Socket-1

Global lock Socket lock

• 1. Scalable NUMA-aware Blocking Synchronizatjon Primitjves. ATC 2017.

Locks performance: Throughput

15

Operations / second # threads

• Maintains throughput:

Stock CST

Oversubscribed > 1 socket 1 socket

Beyond one socket (high thread count). In oversubscribed case (384 threads).

• Poor single thread throughput.

Multjple atomic instructjons.

(e.g., each thread creates a fjle, a serial operatjon, in a shared directory)

Setup: 8-socket 192-core machine

Single thread matuers in non-contended cases

Locks performance: Memory footprint

16

• CST has large memory footprint.

Locks’ memory footprint 140 # threads 18

(e.g., each thread creates a fjle, a serial operatjon, in a shared directory)

Allocate socket structure and global lock. Worst case: ~1 GB footprint out of 32 GB applicatjon’s memory.

Stock CST

Locks performance: Memory footprint

17

• CST has large memory footprint.

Stock CST Locks’ memory footprint 140 # threads 18

(e.g., each thread creates a fjle, a serial operatjon, in a shared directory)

Allocate socket structure and global lock. Worst case: ~1 GB footprint out of 32 GB applicatjon’s memory.

820 Hierarchical lock

Locks performance: Memory footprint

18

• CST has large memory footprint.

Stock CST Locks’ memory footprint 140 # threads 18

(e.g., each thread creates a fjle, a serial operatjon, in a shared directory)

Allocate socket structure and global lock. Worst case: ~1 GB footprint out of 32 GB applicatjon’s memory.

820 Hierarchical lock

Lock’s memory footprint afgect its adoptjon

19

Two goals in our new lock

1) NUMA-aware lock with no memory overhead 2) High throughput in both low/high thread count

Key idea: Sort waiters on the fmy

20

Observatjons:

Hierarchical locks avoid NUMA by passing the lock within a socket Queue-based locks already maintain a set of waiters

Shuffming: Design methodology

21

t1

Representjng a waitjng queue

Socket id (e.g, socket 0)

shuffmer:

Socket ID tail

waiter’s qnode:

Shuffming: Design methodology

22

Another waiter is in a difgerent socket

shuffmer:

Socket ID tail

waiter’s qnode: t1 t2

Shuffming: Design methodology

23

More waiters join

shuffmer:

Socket ID tail

waiter’s qnode: t1 t2 t3 t4

Shuffming: Design methodology

24

Shuffmer (t1) sorts based on socket ID

shuffmer:

Socket ID tail

waiter’s qnode: t1 t2 t3 t4

Shuffming: Design methodology

25

A waiter (shuffmer ) reorders the queue of waiters

• A waiter, otherwise spinning (i.e,. wastjng), amortjses the cost of lock ops

1) By reordering (e.g., lock orders) 2) By modifying waiters’ states (e.g., waking-up/sleeping)

→ Shuffmer computes NUMA-ness on the fmy without using memory unlike others

t1 t2 t3 t4

26

A shuffmer can modify the queue or a waiter’s state with a defjned functjon/policy! Shuffming is generic!

Blocking lock: wake up a nearby sleeping waiter RWlock: Group writers together

Incorporate shuffming in lock design

SHFLLOCKS

27

Minimal footprint locks that handle any thread contentjon

SHFLLOCKS

28

TAS (4B)

(test-and-set lock)

Queue tail (8B)

(waiters list)

• Decouples the lock holder and waiters

○ Lock holder holds the TAS lock ○ Waiters join the queue

Unlock the TAS lock (reset the TAS word to 0)

unlock():

Try acquiring the TAS lock fjrst; join the queue on failure

lock():

SHFLLOCKS

29

TAS (4B)

(test-and-set lock)

Queue tail (8B)

(waiters list)

TAS maintains single thread performance

• Waiters use shuffming to improve applicatjon throughput

○ NUMA-awareness, effjcient wake up strategy ○ Utjlizing Idle/CPU wastjng waiters

• Maintain long-term fairness:

○ Bound the number of shuffming rounds ★ Shuffming is ofg the critjcal path most of the tjme

NUMA-aware SHFLLOCK in actjon

30

t0 (socket 1): lock()

unlocked locked t0 t0 shuffmer:

Socket ID tail

waiter’s qnode:

NUMA-aware SHFLLOCK in actjon

31

Multjple threads join the queue

unlocked locked t0 t1 t2 t3 t4 shuffmer:

Socket ID tail

waiter’s qnode:

NUMA-aware SHFLLOCK in actjon

32

Shuffming in progress

unlocked locked t0 t1 t2 t3 t4

t1 starts the shuffming process

shuffmer:

Socket ID tail

waiter’s qnode:

NUMA-aware SHFLLOCK in actjon

33

Shuffming in progress

unlocked locked t0 t1 t3 t2 t4

t3 now becomes the shuffmer

shuffmer:

Socket ID tail

waiter’s qnode:

NUMA-aware SHFLLOCK in actjon

34

t0: unlock()

unlocked locked t0 t1 t3 t2 t4

t1 acquires the lock via CAS

shuffmer:

Socket ID tail

waiter’s qnode:

NUMA-aware SHFLLOCK in actjon

35

t0: unlock()

unlocked locked t1 t1 t3 t2 t4

t1 notjfjes t3 as a new queue head

shuffmer:

Socket ID tail

waiter’s qnode:

NUMA-aware SHFLLOCK in actjon

36

t0: unlock()

unlocked locked t1 t1 t3 t2 t4 t1 t3 t2 t4 shuffmer:

Socket ID tail

waiter’s qnode:

Other SHFLLOCKS: Blocking SHFLLOCK

37

• NUMA-aware blocking lock.
• Wake up shuffmed waiters based on the socket ID.

○ Avoids the wakeup latency from the critjcal path.

• Lock is always passed to a spinning waiter.

○ Lock stealing: avoid lock-waiter preemptjon problem. ○ Shuffmed waiters are already spinning.

• Guarantees forward progress of the system.

Blocking SHFLLOCK in actjon

38

unlocked locked

shuffmer

t0 t1 t3 t2 t4 ZZ ZZ ZZ scheduled

• ut

t0 t1 t3 t2 t4 ZZ ZZ ZZ

t1 wakes up t3 afuer moving it.

Implementatjon

• Kernel space:

○ Replaced all mutex and rwsem ○ Modifjed slowpath of the qspinlock

• User space:

○ Added to the Litl library

• Please see our paper:

○ Readers-writer lock: Centralized rw-indicator + SHFLLOCK

39

Evaluatjon

• SHFLLOCK performance:

○ Does shuffming maintains applicatjon’s throughput? ○ What is the overall memory footprint? Setup: Eight socket 192-core/8-socket machine

40

Locks performance: Throughput

41

(e.g., each thread creates a fjle, a serial operatjon, in a shared directory)

Operations / second # threads Stock CST

Oversubscribed > 1 socket 1 socket

SHFLLOCK

• SHFLLOCKS maintain performance:

Locks performance: Throughput

42

(e.g., each thread creates a fjle, a serial operatjon, in a shared directory)

Operations / second # threads Stock CST

Oversubscribed > 1 socket 1 socket

SHFLLOCK

• SHFLLOCKS maintain performance:
• Beyond one socket

○ NUMA-aware shuffming

Locks performance: Throughput

43

(e.g., each thread creates a fjle, a serial operatjon, in a shared directory)

Operations / second # threads Stock CST

Oversubscribed > 1 socket 1 socket

SHFLLOCK

• SHFLLOCKS maintain performance:
• Beyond one socket

○ NUMA-aware shuffming

• Core oversubscriptjon

○ NUMA-aware + wakeup shuffming

Locks performance: Throughput

44

(e.g., each thread creates a fjle, a serial operatjon, in a shared directory)

Operations / second # threads Stock CST

Oversubscribed > 1 socket 1 socket

SHFLLOCK

• SHFLLOCKS maintain performance:
• Beyond one socket

○ NUMA-aware shuffming

• Core oversubscriptjon

○ NUMA-aware + wakeup shuffming

• Single thread

○ TAS acquire and release

Locks performance: Memory footprint

45

(e.g., each thread creates a fjle, a serial operatjon, in a shared directory)

Stock CST 140 # threads SHFLLOCK 11 18

• SHFLLOCKS has least memory footprint

Reason: No extra auxiliary data structure ➢ Stock: parking list structure + extra lock ➢ CST: per-socket structure

Locks’ memory footprint

Case study: Exim mail server

46

Stock SHFLLOCK

It is fork intensive and stresses memory subsystem, fjle system and scheduler Improves throughput by up to 1.5x Decreases memory footprint up to 93%

Lock’s memory Messages / second # threads # threads CST

Throughput Memory footprint

Discussion

• Another way to enforce these policies dynamically:

○ Lock holder splits the queue to provide:

• E.g., NUMA-awareness: Compact NUMA-aware lock (CNA).
• E.g., blocking lock: Malthusian lock.
• Shuffming can support other policies:

○ Non-inclusive cache (Skylake architecture). ○ Multj-level NUMA hierarchy (SGI machines).

47

Conclusion

• Locks are critjcal for fjle system and applicatjon performance
• Current lock designs:

○ Do not maintain best throughput with varying threads ○ Have high memory footprint

• Shuffming: Dynamically enforce policies

○ NUMA, blocking, etc

• SHFLLOCKS: Shuffming-based family of lock algorithms

○ NUMA-aware minimal memory footprint locks

48