HelenOS in the Year HelenOS in the Year of the Fire Monkey of the - - PowerPoint PPT Presentation
HelenOS in the Year HelenOS in the Year of the Fire Monkey of the Fire Monkey http://www.helenos.org/ http://d3s.mff.cuni.cz Jakub Jerm jakub@jermar.eu Martjn Dck decky@d3s.mff.cuni.cz HelenOS in a Nutshell HelenOS in a Nutshell
Martjn Děcký
decky@d3s.mff.cuni.czJakub Jermář
jakub@jermar.euHelenOS in the Year
HelenOS in the Year
HelenOS in a Nutshell HelenOS in a Nutshell
multjplatgorm microkernel multjserver
implemented from scratch
www.helenos.org HelenOS in a Nutshell (2) HelenOS in a Nutshell (2)
Open source Our own code is BSD, some 3rd party components are GPL General-purpose Not biased towards any single deployment Multjplatgorm amd64, arm32, ia32, ia64, mips32, ppc32, sparc64 Microkernel Obviously :) Multjserver User space built from fjne-grained components (microservices) HelenOS in a Nutshell (3) HelenOS in a Nutshell (3)
Designed and implemented from scratch Architecture based on a set of guiding design principles Natjve API designed to refmect the architecture Custom asynchronous IPC combining kernel preemptjve and user space cooperatjve threads Implementjng concepts akin to futures and promises Compatjbility with legacy APIs a non-goal However, providing API adaptatjon layer Features implemented on-demand binutjls, fdlibm, GCC, Jainja, libgmp, libiconv, libisl, libmpc, libmpfr, libpng, MSIM, PCC, Python 2, zlib HelenOS in a Nutshell (4) HelenOS in a Nutshell (4)
architecture independent shared architecture dependent architecture dependent bootstrap routines CPU mgmt atomics & barriers I/O mgmt platform memory mgmt platform drivers debugging support context switching interrupt handling platform library routines shared platform drivers shared debugging support hierarchical page table support global page hash table support hardware layer abstraction kernel unit tests memory backends memory zones mgmt frame allocator slab allocator address space mgmt memory reservation spinlocks wait queues work queues interrupt & syscall dispatch thread scheduler thread & task mgmt kernel lifecycle mgmt lists, trees, bitmaps concurrent hash table generic resource allocator ELF loader string routines misc routines kernel console IPC kernel log hardware resource mgmt system information cycle & time mgmt tracing support read- copy- update task capabilities cache coherency synchro- nization interface kernel naming service loader task monitor klog location service logger device manager device drivers client session vterm bdsh vfs fjle system drivers FAT exFAT ext4 ISO 9660 UDF MINIX FS TMPFS Location FS init transport layer protocols tcp udp link layer protocols loopip ethip slip inetsrv networking management dnsrsrv dhcp nconfsrv human interface clipboard audio HelenOS in a Nutshell (4) HelenOS in a Nutshell (4)
architecture independent shared architecture dependent architecture dependent bootstrap routines CPU mgmt atomics & barriers I/O mgmt platform memory mgmt platform drivers debugging support context switching interrupt handling platform library routines shared platform drivers shared debugging support hierarchical page table support global page hash table support hardware layer abstraction kernel unit tests memory backends memory zones mgmt frame allocator slab allocator address space mgmt memory reservation spinlocks wait queues work queues interrupt & syscall dispatch thread scheduler thread & task mgmt kernel lifecycle mgmt lists, trees, bitmaps concurrent hash table generic resource allocator ELF loader string routines misc routines kernel console IPC kernel log hardware resource mgmt system information cycle & time mgmt tracing support read- copy- update task capabilities cache coherency synchro- nization interface kernel naming service loader task monitor klog location service logger device manager device drivers client session vterm bdsh vfs fjle system drivers FAT exFAT ext4 ISO 9660 UDF MINIX FS TMPFS Location FS init transport layer protocols tcp udp link layer protocols loopip ethip slip inetsrv networking management dnsrsrv dhcp nconfsrv human interface clipboard audio~ 337,000 physical lines of code ~ 950 person-months worth of efgort (Basic COCOMO Model) 24 master theses, 4 bachelor theses, 3 team projects, 11 GSoC and ESA SOCIS projects, independent contributors
HelenOS Contributors HelenOS Contributors
Sean Bartell Tomáš Benhák Dmitry Bolkhovityanov Sergey Bondari Tobias Börtjtz Zdeněk Bouška Tomáš Brambora Jan Buchar Lubomír Bulej Tomáš Bureš Josef Čejka Aurelio Colosimo Manuele Contj Martjn Děcký Matúš Dekánek Jan Dolejš Andrey Erokhin Matueo Facchinet Beniamino Galvani Matuhieu Gueguen Zbigniew Halas Štepán Henek Vojtěch Horký Adam Hraška Mohammed Hussain Adrian Jamróz Pavel Jančík Martjn Jelen Petr Jerman Jakub Jermář Fan Jinfei Jiří Kavalík Michal Kebrt Jakub Klama Matěj Klonfar Jan Kolárik Michal Konopa Petr Koupý Stanislav Kozina Sandeep Kumar Maurizio Lombardi Peter Majer Jan Mareš Julia Medvedeva Lukáš Mejdrech Vojtěch Mencl Jiří Michalec Ondřej Palkovský Vineeth Pillai Tim Post Vivek Prakash Frantjšek Princ Alexander Prutkov Marin Ramesa Pavel Římský Oleg Romanenko Jefg Rous Thomas Sanchez Ondřej Šerý Ľuboš Slovák Antonín Steinhauser Petr Štěpán Martjn Sucha Jiří Svoboda Agnieszka Tabaka Dominik Táborský Jiří Tlach Lenka Trochtová Petr Tůma Jakub Váňa Radim Vansa Laura-Mihaela Vasilescu Ján Veselý Jan Záloha Jiří Zárevúcky The Case for “Reinventjng the Wheel” The Case for “Reinventjng the Wheel”
Clean-slate design Legacy designs and APIs may be broken, insecure, thread-unsafe, morally obsolete However, not saying that all legacy is broken Thinking out of the box No glue code, mandatory adaptatjon layers, franken-components The Case for “Reinventjng the Wheel” The Case for “Reinventjng the Wheel”
Clean-slate design Legacy designs and APIs may be broken, insecure, thread-unsafe, morally obsolete However, not saying that all legacy is broken Thinking out of the box No glue code, mandatory adaptatjon layers, franken-components The Case for “Reinventjng the Wheel” The Case for “Reinventjng the Wheel”
No maintenance burden due to a fork A forked component tends to become a sofuware fossil Security bugs, new features, diverging licenses, toolchain updates, dependencies HelenOS mainline is always fresh It does not age Opposed to HelenOS coastline (a.k.a. the ports tree) design verify implement evaluate Intermezzo: Cross-pollinatjon
Microkernel Devroom in GSoC Microkernel Devroom in GSoC
Proposal for the Microkernel devroom
Apply for Google Summer of Code 2017 as an umbrella organizatjon HelenOS experience Accepted in 2011, 2012, 2014 Not accepted in 2009, 2010, 2013, 2015, 2016 But stjll GSoC is extremely valuable to us We don’t know Google’s criteria, but maybe too many Microkernel Devroom in GSoC (2) Microkernel Devroom in GSoC (2)
Technicalitjes
Applicatjon deadline February 9th 2017 16:00 UTC Who should fjll in the applicatjon? HelenOS team volunteers (we have some experience) Any input is always welcomed Where should the ideas page be hosted? At microkernel.info (a quasi-neutral locatjon) Don’t hesitate to send us your project topics Should we discourage individual projects from applying? HelenOS Status Update HelenOS Status Update
Since last FOSDEM
February 1st 2016 – February 3rd 2017 Year of the fjre monkey: February 8th 2016 – January 27th 2017 General observatjons Less actjvity than in previous years Contributors distracted by other projects Less students (no GSoC, SOCIS, only two theses running) No essentjal subsystem missing (plateauing) HelenOS Camp 2016 HelenOS Camp 2016
A long traditjon of hackatons
Since 2005 OK, not so long as OpenBSD’s, but an independent ideaAugust 27th – September 3rd 2016
Třemošnice, CZ ~ 3 developers 45 change sets Year in a Nutshell Year in a Nutshell
0.6.1 release Almost ready (a FOSDEM release?) Improvements of the sun4u (sparc64) support Jointly with QEMU improvements heading towards supportjng HelenOS/sun4u in QEMU Generic serial console in user space Simple installer Non-interactjve, for automated deployments Currently targetjng QEMU for ia32, amd64 Includes reproducible GRUB build Year in a Nutshell (2) Year in a Nutshell (2)
USB 2.0 support Part of USB stack rewrite Simplifjcatjon of various component dependencies Betuer suitable for further extension (USB 3.0, etc.) IRQ notjfjcatjon handling optjmizatjons Lower overhead of the processing of IPC message that represent an IRQ Worker fjbrils are no longer created and disposed for every single IRQ Dynamic linking improvements Betuer support for TLS handling on all platgorms Dynamic linking now enabled by default on ia32 Year in a Nutshell (3) Year in a Nutshell (3)
Removal of SYS_TLS_SET Original dedicated syscall for confjguring the thread local storage base address (selector) on ia32 and amd64 Use of GCC specifjc –mno-tls-direct-seg-refs optjon The %gs and %fs segments are no longer used directly as TLS base addresses Generic solutjon: WRFSBASE / WRGSBASE since Ivy Bridge, emulatjon on older CPUs and on ia32 Contjnuous maintenance Many bugs discovered and fjxed thanks to verifjcatjon tools (memory leaks, use-afuer-free, race conditjons) Contjnuous code refactoring Support for the latest compiler toolchain Detectjng a regression in QEMU 2.7.0 on arm32 Year in a Nutshell (4) Year in a Nutshell (4)
RISC-V support
Stjll unfjnished (mostly due to lack of focus) Shootjng on a moving target Some changes in the drafu of the privileged specifjcatjon and in the Spike reference simulator (a de facto reference platgorm) are not explicitly tracked and documented HelenOS mainline contains a skeleton codeService manager for HelenOS User space pagers
Service Manager for HelenOS Service Manager for HelenOS
Master thesis by Michal Koutný
htup://www.helenos.org/doc/theses/mk-thesis.pdf Unifjcatjon of system-level services (i.e. traditjonal microkernel servers) and logical services (e.g. network servers) Remotely inspired by SMF, systemd, etc. Service Manager for HelenOS (2) Service Manager for HelenOS (2)
sysman service Unit dependency resolver Starts/stop services according to its dependencies Explicit dependencies (confjguratjon fjle) Implicit dependencies (IPC broker requests) Unit types: service, mountpoint, confjguratjon, target taskman service Task creatjon and monitoring API Manages logical parent/child relatjons Prospect of restartjng failed services User Space Pagers User Space Pagers
HelenOS memory management
Traditjonal kernel architecture (frame allocator, kernel heap allocator, virtual address space areas manager) Mostly mechanisms, but also unavoidable minimal policies (single point of failure) Originally three address space area backends backend_phys backend_anon backend_elf New address space area backend backend_user User Space Pagers (2) User Space Pagers (2)
Pagee Pagee Pager Pager Kernel Kernel User Space Pagers (2) User Space Pagers (2)
Pagee Pagee Pager Pager Kernel Kernel as_area_create(..., pager_info) IPC connectjon memory object User Space Pagers (2) User Space Pagers (2)
Pagee Pagee Pager Pager Kernel Kernel ~ ~ blocked in page fault handler IPC_M_PAGE_IN(ofgset, memory object) User Space Pagers (2) User Space Pagers (2)
Pagee Pagee Pager Pager Kernel Kernel ~ ~ ipc_answer_1(EOK, virtual address) fjnd physical address wake up and map page to physical address User Space Pagers (2) User Space Pagers (2)
Pagee Pagee Pager Pager Kernel Kernel Plans for the Year of the Fire Rooster Plans for the Year of the Fire Rooster
Finish research papers in the pipeline Finish the RISC-V port Support for Turris Omnia Switch to bi-annual release cycle More IPC optjmizatjons
Wait-free notjfjcatjons Adaptjve handling of synchronous cases www.helenos.org
Backup slides
Downloads Downloads
Feb 2007 Feb 2009 Feb 2011 Feb 2013 Feb 2015 100 1000 10000 MINIX HelenOS Downloads (source: http://minix3.org/stats/) 19.66 %