Introduction to Multithreading and Multiprocessing in the FreeBSD - - PowerPoint PPT Presentation
Introduction to Multithreading and Multiprocessing in the FreeBSD SMPng Network Stack EuroBSDCon 2005 26 November 2005 Robert Watson Ed Maste FreeBSD Core Team FreeBSD Developer rwatson@FreeBSD.org emaste@FreeBSD.org Computer Laboratory
– Symmetric Multi-Processing (SMP) – Strategies for SMP-capable operating systems
– FreeBSD 3.x/4.x SMP – FreeBSD 5.x/6.x SMPng
– Architecture – Synchronization approaches – Optimization approaches
– More than one general purpose processor – Running the same primary system OS – Increase available CPU capacity sharing
– Refers to memory performance and caching – In contrast to NUMA
– In practice, a bit of both
CPU0 Cache CPU3 Cache CPU2 Cache CPU1 Cache
CPU0 Cache CPU3 Cache CPU2 Cache CPU1 Cache
CPU0 Cache CPU3 Cache CPU2 Cache CPU1 Cache
CPU0 CPU1 CPU2 CPU3 Northbridge System Memory
CPU0 Cache CPU3 Cache CPU2 Cache CPU1 Cache
PCI-X Bus Interconnect Card CPU0 CPU1 CPU2 CPU3 Northbridge System Memory
CPU0 Cache CPU3 Cache CPU2 Cache CPU1 Cache
PCI-X Bus Interconnect Card
– Hyper-threading (HTT): physical CPU cores
– Non-Uniform Memory Access (NUMA): different
– This could mean a lot of different things – Usually implies it is able to utilize >1 CPU
– “Look like a single-processor system” – But be faster
– Most carefully select variables to degrade – Weak memory models, message passing, ...
– Only with more CPU power
– Parallelism between processes – Parallelism in thread-enabled processes – Requires minimal changes to applications yet
– Why else buy more CPUs? – However, performance is a nebulous concept
– Systems programming is rife with trade-offs
– Measurement of workload performance as
– Ratio of score on N processors to score on 1
– Don't get in the way of application speed-up – Facilitate application speed-up
– OS + application
– Limits on workload
– Contention on
UP SM P-1 SM P-2 SM P-3 SM P-4 5000 10000 15000 20000 25000 30000 35000 40000
Idealized Predicted linear from Measured
Configuration Transactions/Second
– Make it run – Make it run fast
– Start with the kernel – Then work on the applications – Then repeat until done
– OS must provide primitives to support parallel
– OS may do little, some, or lots of the work
– An MP-capable and MP-optimized thread library
– Wake up processor at boot time – Cause a processor to enter an interrupt handler – Comes with challenges, such as deadlocks
– Kernel memory will generally be mapped
– Memory is therefore shared, and can be read or
– Requires consistency and synchronization model – Atomic operations, higher level primitives, etc.
– Most kernels have a notion of threads similar to
– Multiple execution contexts in a single kernel
– Threads will execute on only one CPU at a time – All execution in a thread is serialized with
– Most systems support migration of threads
– When to migrate is a design choice affecting
– Will become corrupted unless access is
– “Race Conditions”
– From atomic operations and IPIs – To mutexes, semaphores, signals, locks, ... – Lockless queues and other lockless structures
– Affects performance and complexity
– Most straight forward approach to MP OS – User process and thread paralellism – Kernel executes on one processor at a time to
– No in-kernel parallelism
CPU0 CPU1
read() Sleep
read() returns socket() Giant acquired I/O completes Giant acquired CPUs spinning waiting for Giant to be released by the other CPU socket() returns
– User threads performing system calls – Interrupt or timer driver kernel activity
– File system activity – Network activity – Misc. I/O activity – Inter-Process Communication (IPC) – Scheduler and context switches
– Typically over “code” or “data” – E.g., scheduler lock permits user context
– Details vary greatly by OS
– Typically begin with scheduler lock – Dependency locking such as memory allocation – Some high level subsystem locks – Then data-based locking – Drive granularity based on observed contention
CPU0 CPU1
read() Sleep
read() returns socket() I/O completes Socket buffer mutex briefly in contention socket() returns send() send() Wait on mutex Mutex acquired
– Present in FreeBSD 5.x, 6.x
– Adopt more threaded architecture
– Introduce various synchronization primitives
– Iteratively lock subsystems and slide Giant off
– Synchronization, scheduling, memory allocation,
– Memory allocation, VM, VFS, network stack,
– Most major subsystems except VFS and some
– Some network protocols require Giant
– VFS also executes Giant-free, although some file
– Some straggling device drivers require Giant
– Excludes distributed file systems – Excluding device drivers
– “mbuf” memory allocator – Network device drivers, interface abstraction – Protocol-independent routing and event model – Link-layer protocols, network-layer protocols
– Sockets and socket buffers – Netgraph extension framework
kern_send() sosend() sbappend() tcp_send() tcp_output() ip_output() em_start() em_intr() ether_input() ether_output() ip_input() tcp_reass() tcp_input() soreceive() sbappend() kern_recv()
netblast em0 ithread
em_start() send() returns em_intr() preempts sosend() udp_output() ip_output() em_clean_transmit_intr() em_intr() returns send()
netreceive netisr em0 ithread
recv() recv() returns em_intr() preempts schednetisr() swi_net() soreceive() udp_input() netreceive blocks ether_input() em_intr() returns ip_input()
idle
em_process_receive_interrupts() sbappend() sowakeup() netreceive wakes up
– Network stacks may process millions of PPS – Small costs add up quickly if per-packet
– TCP is very sensitive to mis-ordering
– Optimizing for latency may damage throughput,
– Lock orders prevent deadlock – Data passes in various directions through layers
– Don't use finer locks than required by UNIX API
– Lock references to in-flight packets, not packets – Layers have their own locks as objects at
– Driver locks are leaf locks with respect to stack – Protocol drives most inter-layer activity – Acquire protocol locks before driver locks – Acquire protocol locks before socket locks – Avoid lock order issues via deferred dispatch
– 4.x had user threads, netisr, dispatched crypto – 5.x/6.x add ithreads
– Threads involved are typically user threads,
– Work split over many threads for receive – On transmit, work tends to occur in one thread – Opportunities for parallelism in receive are
– Assume a Giant-free network stack – Select an interesting workload – What are remaining source of contention? – Where is CPU-intensive activity serialized in a
– Protocol hand-offs, layer hand-offs, etc – Carefully consider ordering considerations
– Context switches are expensive – Locks are expensive
– Deadlock – Locking Overhead – Event Serialization
– Lock cycles – Any finite resource
– Avoidance – Detect + recover
Thread 2 Thread 1
1 2 3 4
Thread runs holding lock Thread blocked holding lock Thread blocked waiting on lock
– Applies to most mutexes and sx locks – Disallow lock cycles – WITNESS lock verification tool
– Lock order a property of data structures – At any given moment, the lock order is defined – However, it may change as data structure
– Master lock used to serialize simultaenous
– Tracks lock
– Builds graph
– Detects and warns
fd udpinp udp accept so_rcv so_snd
WITNESS will warn about lock cycle
– Avoid multiple lock operations where possible – Amortize cost of locking over multiple packets
– Excessive granularity will increase overhead – Combining across layers can avoid lock
– Execution of threads is serialized
– Use of per-CPU data structures and
– TCP will misinterpret
em0 em1 Link Layer Link Layer Netisr: IP Netisr dispatch em1 Link Layer Netisr: IP Netisr dispatch em1 Link Layer Netisr: IP
– “Source ordering” is used widely in the stack
– Weakening ordering can improve performance – Some forms of parallelism maintain ordering
– Some less mainstream components still need it
– Many workloads show significant improvements:
– Cost of locking hampers per-packet performance
– UP performance sometimes sub-optimal – Of course, 4.x is the gold standard...
– Multi-processor system architectures – Operating system interactions with MP – SMPng architecture and primitives
– The FreeBSD network stack – Changes made to the network stack to the
– Optimization concerns including locking cost and
– Concerns such as packet ordering
– 5.3-RELEASE the first release with Giant off the
– 6.x includes substantial optimizations, MPSAFE