mikro - Multiprocessor Init in Kernel Julien Freche Introduction Interruptions mikro - Multiprocessor Init in Kernel CPU init Percpu variables Conclusion Julien Freche julien.freche@lse.epita.fr http://lse.epita.fr/
Outline I mikro - Introduction 1 Multiprocessor Init in Kernel Interruptions 2 Julien Freche Old PIC Introduction IOAPIC Interruptions LAPIC CPU init IPI Percpu variables Conclusion CPU init 3 BootStrap Processor Application Processor INIT-SIPI-SIPI Percpu variables 4 Usage Implementation Using clang Conclusion 5
Introduction mikro - Multiprocessor Init in Kernel Julien Freche Introduction Interruptions CPU init Percpu variables Introduction Conclusion
Too many questions mikro - Multiprocessor Init in Kernel Julien Freche Introduction I will talk about x86 system with multiple processors. Interruptions CPU init How to handle interruptions ? Percpu variables How to boot all CPUs ? Conclusion What is the state of the system at boot ? How to detect the number of CPU(s) ? Dealing with multiple processors requires to deal with interruptions. Let’s see why.
Interruptions mikro - Multiprocessor Init in Kernel Julien Freche Introduction Interruptions Old PIC IOAPIC LAPIC Interruptions IPI CPU init Percpu variables Conclusion
I interrupt you mikro - Multiprocessor Init in Kernel Julien Freche Introduction Interruptions Interruption Old PIC IOAPIC Signal sent to a processor to report an event that requires LAPIC IPI immediate attention. CPU init Interrupts can be caused by: Percpu variables Conclusion Hardware: event caused by some device Software: system call from userland, debugging purposes, ..
Old PIC mikro - Multiprocessor Init in Kernel Julien Freche Introduction Interruptions Old PIC IOAPIC LAPIC IPI Old PIC CPU init Percpu variables Conclusion
Vintage chip mikro - Multiprocessor Init in Kernel Julien Freche Introduction PIC Interruptions It controls the CPU’s interrupt mechanism, by accepting several Old PIC IOAPIC interrupt requests and feeding them to the processor in order. LAPIC IPI CPU init Limitations: Percpu variables Only 8 pin per PIC (16 IRQ on x86 with two PICs). Conclusion No SMP support, can only send interrupts to one CPU. Programmed with IO ports The PIC is no replaced by the IOAPIC on modern systems.
IOAPIC mikro - Multiprocessor Init in Kernel Julien Freche Introduction Interruptions Old PIC IOAPIC LAPIC IPI IOAPIC CPU init Percpu variables Conclusion
Incoming chip mikro - Multiprocessor Init in Kernel Julien Freche IOAPIC Introduction It collects interrupts from all devices and provide a way to send Interruptions Old PIC it to a CPU or a group of CPU. IOAPIC LAPIC IPI Features: CPU init 24 pin per IOAPIC Percpu variables Conclusion Memory mapped device You can have multiple IOAPICs Special bus to send interruptions: ICC bus Handle global interruptions (not specific to a CPU)
LAPIC mikro - Multiprocessor Init in Kernel Julien Freche Introduction Interruptions Old PIC IOAPIC LAPIC IPI LAPIC CPU init Percpu variables Conclusion
That chip is mine mikro - Multiprocessor Init in Kernel Local APIC Julien Freche It collects local interrupts and provide a way to a CPU to accept Introduction interrupts. Interruptions Old PIC IOAPIC Features: LAPIC IPI 2 pin per local APIC CPU init Percpu variables Memory mapped device Conclusion Each CPU has a local APIC Timer, Performance monitoring, Thermal sensor Use ICC bus to speak with IOAPIC(s) Handle local interruptions Each Lapic has an unique ID
All together mikro - Multiprocessor Init in Kernel Julien Freche Introduction Interruptions Old PIC IOAPIC LAPIC IPI CPU init Percpu variables Conclusion
IPI mikro - Multiprocessor Init in Kernel Julien Freche Introduction Interruptions Old PIC IOAPIC LAPIC IPI IPI CPU init Percpu variables Conclusion
Poke mikro - Multiprocessor Init in Kernel InterProcessor Interrupts Julien Freche IPIs are interrupts issued by one processor and sent to another. Introduction Interruptions Old PIC Issued by the Local Apic IOAPIC LAPIC Destination IPI CPU init All including self All excluding self Percpu variables Self Conclusion One processor Delivery Mode Low priority NMI INIT, STARTUP ...
CPU init mikro - Multiprocessor Init in Kernel Julien Freche Introduction Interruptions CPU init BootStrap Processor Application Processor CPU init INIT-SIPI-SIPI Percpu variables Conclusion
(Gla)DOS: I am still alive mikro - Multiprocessor Init in Kernel Julien Freche Introduction Interruptions For backward compatibility all modern PCs starts in the CPU init BootStrap Processor following state: Application Processor INIT-SIPI-SIPI Only one CPU active Percpu variables Real mode (16 bits) Conclusion PIC available to handle interruptions
BootStrap Processor mikro - Multiprocessor Init in Kernel Julien Freche Introduction Interruptions CPU init BootStrap Processor Application Processor BootStrap Processor INIT-SIPI-SIPI Percpu variables Conclusion
Dibs on the kernel mikro - Multiprocessor Init Bootstrap Processor in Kernel Julien Freche This processor is chosen by the BIOS to start executing the bootloader code. It is the first to execute the kernel code and Introduction Interruptions must wake up the other processors if needed. CPU init BootStrap Processor Duties on mikro: Application Processor INIT-SIPI-SIPI Create a GDT and an IDT Percpu variables Init the paging Conclusion Move the kernel code to its final location Parse MP Tables and / or MADT table Init IOAPIC(s) and its own LAPIC Perform some per-cpu init Wake every processors Start scheduling tasks
Vintage table mikro - Multiprocessor Init in Kernel MultiProcessor Table Julien Freche Created by Intel in 1997. Provides MP related informations to Introduction the OS. Interruptions CPU init Informations: BootStrap Processor Application Processor Processors list INIT-SIPI-SIPI Percpu variables Buses list Conclusion IOAPICs list Interrupts list Limitations: qemu: list only processors, not core nor threads. On some real hardware: list only cores, not threads. On other hardwares: not present
Colona’s favorite toy mikro - Multiprocessor Init in Kernel Julien Freche Introduction MADT: Multiple APIC Description Table Interruptions Part of the ACPI spec. Provides informations about an SMP CPU init system. BootStrap Processor Application Processor INIT-SIPI-SIPI Informations: Percpu variables Conclusion Processors, cores, threads IOAPICs, x2APIC list Interrupt Source Override Kind of the easy part of the ACPI (no AML).
Application Processor mikro - Multiprocessor Init in Kernel Julien Freche Introduction Interruptions CPU init BootStrap Processor Application Processor Application Processor INIT-SIPI-SIPI Percpu variables Conclusion
Dibs on the kernel mikro - Multiprocessor Init in Kernel Julien Freche Application Processor Introduction Processor in the halt state, waiting for a special IPI to start Interruptions executing code. This processor is in real mode. CPU init BootStrap Processor Duties on mikro: Application Processor INIT-SIPI-SIPI Jump to protected mode Percpu variables Conclusion Create its own GDT Load the existing IDT Perform some per-cpu init Init its LAPIC Start scheduling tasks
INIT-SIPI-SIPI mikro - Multiprocessor Init in Kernel Julien Freche Introduction Interruptions CPU init BootStrap Processor Application Processor INIT-SIPI-SIPI INIT-SIPI-SIPI Percpu variables Conclusion
Wake up! mikro - Multiprocessor Init in Kernel Julien Freche Introduction Interruptions A processor will start executing code when receiving this CPU init sequence of IPIs. BootStrap Processor Application Processor Copy some code in low memory INIT-SIPI-SIPI Percpu variables Send an INIT IPI Conclusion Send a StartUP IPI Send another StartUP IPI
Some magic mikro - Multiprocessor Init in Kernel I found this code on the internet. Julien Freche . i n i t : Introduction ; s e t NMI ha n dle r Interruptions mov dword [ 4 ∗ 2 ] , . b o o t CPU init xor eax , eax BootStrap Processor mov ebx , (11 b shl 18) + (0 shl 15) + (1 shl 14) Application Processor + (0 shl 11) + (100 b shl 8) + 2 INIT-SIPI-SIPI Percpu variables ; t r i g g e r i n t e r r u p t s in every p r o c e s s o r Conclusion mov [ dword 0xFEE00300 + 16] , eax mov [ dword 0xFEE00300 + 00] , ebx ret a l i g n 16 . b o o t : It works on qemu and some systems.
Let me explain it mikro - Multiprocessor Init in Kernel Julien Freche Steps: Introduction Interruptions Register boot function as handler into the IDT CPU init Set eax to zero BootStrap Processor Application Processor Set IPI parameters into ebx: INIT-SIPI-SIPI Percpu variables NMI Conclusion Physical, Assert level, Edge trigger mode All excluding self Send the IPI using the LAPIC Kind of the quickest hack to wake every processors !
Recommend
More recommend
