RTM ux: A Thi n M ul t i pl exer To Pr ovi de H ar d Real - PowerPoint PPT Presentation

RTM ux: A Thi n M ul t i pl exer To Pr ovi de H ar d Real t i m e A ppl i cat i ons For Li nux m H uang ( 黃敬群 ) Ji <j ser v. t w @ gm ai l . com > O ct 15, 2014 / Em bedded Li nux C onf er ence Eur ope

A genda Mission: Build lightweight real-time environments for Linux/ARM Review of existing technologies RTMux: Resource-Multiplexing Real-time Executive Linux-friendly remote communication mechanisms Full source available: https://github.com/rtmux This work is sponsored by ITRI Taiwan and Delta Electronics

M i ssi on: Bui l d Li ght w ei ght Real -t i m e envi r onm ent s f or Li nux/A RM A ppl i cat i ons

I n shor t w or ds, i t i s LO VER

LO VER = L i nux O pt V i i m i zed f or r t ual i zat i on, E m bedded, and R eal t i m e

U se C ase f or RTM ux Quadcopter with Computer Vision

U se C ase f or RTM ux Quadcopter with Computer Vision Hard real-time Autonomous Flight Modes (Landing/Take-off) altitude control, feedback-loop control, RC Autopilot, autonomous navigation Soft real-time Stream real-time flight data on-screen over video Parallel Tracking and Mapping (PTAM) , and the detected walls are visualized in 3D with mapped textures. Source: https://github.com/nymanjens/ardrone-exploration

Ext ernal A ut onom ous N avi gat i on Various Flight Modes-Stabilize, Alt Hold, Loiter, Auto Mode. For the AUTO mode, GPS is necessary. Waypoints are set in advance.

I nt ernal A ut onom ous N avi gat i on GPS fails in a closed-door environment. Detect a door/window and go out where GPS access is present. Design a controller for navigation of quadcopter from indoor to outdoor environsments. SONAR and Computer vision Source: http://wiki.ros.org/tum_ardrone

Applications Linux Real-time Executive RTM ux: Device RTOS Drivers M ul t i pl exer f or Li nux-based Real -t i m e A ppl i cat i ons De-privileged Privileged RTMux

Pow ered by O pen Source St ack POSIX application Linux application Linux application (Real-time) POSIX olibc minilibc glibc Runtime System calls V-Bus VFS Network RTOS Linux Kernel Memory ... RTMux ARM core olibc: http://www.slideshare.net/jserv/olibc Embedded Linux Conference 2013

Revi ew of Exi st i ng Technol ogi es

Real t i m e Perf orm ance 100% source compatibility A ppl i cat i on pr ogr am better binary compatibility API Customized Realtime kernel Real-time Linux (Standard) Linux Response time: < 10 µs Response time: ~1ms One process address space One (physical) with n-threads address space with n-tasks All tasks have All threads of a process have one common API one common API Response time: < 10 µs implemented by implemented by same one common library one common library homogeneity

Real -t i m e A pproaches Two major approaches real time Linux rt-preempt (PREEMPT_RT patch) Allows preemption, so minimize latencies Execute all activities (including IRQ) in “schedulable/thread” context Many of the RT patch have been merged Linux (realtime) extensions Add extra layer between hardware and the Linux kernel to manage real-time tasks separately

Preem pt i ve Kernel non-preemptive system preemptive system A concept linked to that of real time is preemption: the ability of a system to interrupt tasks at many “preemption points”.The longer the non-interruptible program units are, the longer is the waiting time (‘latency’) of a higher priority task before it can be started or resumed. GNU/Linux is “user-space preemptible”: it allows user tasks to be interrupted at any point. The job of real- time extensions is to make system calls preemptible as well.

Part I : Li nux real -t i m e preem pt i on http://www.kernel.org/pub/linux/kernel/projects/rt/ led by kernel developers including Ingo Molnar, Thomas Gleixner, and Steven Rostedt Large testing efforts at RedHat, IBM, OSADL, Linutronix Goal is to improve real time performance Configurable in the Processor type and features ( x86 ), Kernel Features ( arm ) or Platform options ( ppc )...

W rong i deas about real -t i m e preem pt i on It will improve throughput and overall performance Wrong : it will degrade overall performance. It will reduce latency Often wrong . The maximum latency will be reduced. The primary goal is to make the system predictable and deterministic.

PREEM PT_RT: com pl et e RT preem pt i on Replace non-preemptible constructs with preemptible ones Make OS preemptible as much as possible except preempt_disable and interrupt disable Make Threaded (schedulable) IRQs so that it can be scheduled spinlocks converted to mutexes (a.k.a. sleeping spinlocks) Not disabling interrupt and allows preemption Works well with thread interrupts

Tow ard com pl et e RT preem pt i on Most important aspects of Real-time Controlling latency by allowing kernel to be preemptible everywhere

ori gi nal Li nux Kernel Interrupt Handlers Interrupt Context SoftIRQs Kernel N T R t t H a a Space S e i e s s m i ... t t g k k a w p u l l e c e r e o i l k r a o t t r s s s k r Scheduling Process User Context Points Thread Kernel User Space Thread

PREEM PT_RT SO_NODELAY Interrupt Handlers Process N Kernel e Thread t T w T Space a o i s m r k k e l e r S s t s t a Kernel Scheduling c k Threads Points User Space

Threaded I nt errupt s Handle interrupt by interrupt handler thread Interrupt handlers run in normal kernel threads Priorities can be configured Main interrupt handler Do minimal work and wake-up the corresponding thread Thread interrupts allows to use sleeping spinlocks in PREEMPT_RT, all interrupt handlers are switched to threaded interrupt

Threaded I nt errupt s The vanilla kernel Interrupts as threads Real world behavior

Benchm arki ng cyclictest measuring accuracy of sleep and wake operations of highly prioritized realtime threads https://rt.wiki.kernel.org/index.php/Cyclictest vanilla kernel PREEMPT_RT

Part I I : Li nux hard real -t i m e ext ensi ons Three generations A common principle RTLinux Add a extra layer between the hardware and the Linux RTAI kernel, to manage real-time Xenomai tasks separately. real-time real-time Linux tasks tasks kernel Micro/Nano-kernel Hardware

I nt errupt Response Ti m e PREEMPT : standard kernel with XENOMAI : Kernel + Xenomai 2.6.0-rc4 + I-Pipe CONFIG_PREEMPT (“Preemptible Kernel 1.18-03 (Low-Latency Desktop)) enabled cyclictest -n -p99 -t1 -i10000 cyclictest –m -n -p99 -t1 -i10000 -1360000 -1360000 Configuration Avg Max Min Configuration Avg Max Min XENOMAI 43 58 2 XENOMAI 43 58 2 PREEMPT 88 415 27 PREEMPT 88 415 27 Hardware: Freescale i.MX53 ARM Cortex-A8 processor operating at 1GHz. Time in micro second.

Xenom ai proj ect http://www.xenomai.org/ Started in the RTAI project (called RTAI / fusion). Skins mimicking the APIs of traditional RTOS such as VxWorks, pSOS+, and VRTXsa. Initial goals: facilitate the porting of programs from traditional RTOS to RTAI on GNU / Linux. Now an independent project and an alternative to RTAI. Many contributors left RTAI for Xenomai, frustrated by its goals and development style.

Xenom ai archi t ect ure Linux application VxWorks application POSIX application Xenomai Xenomai glibc glibc glibc libvxworks libpthread_rt System calls Xenomai RTOS Pieces added VFS Network (nucleus) by Xenomai Linux Xenomai Memory ... kernel space skins Adeos i-pipe ipipe = interrupt pipeline

O ri gi nal Li nux Runs in User Mode : pthread_create can only address user space Low prio Linux thread main User space readfile(..) , printf(..) Linux scheduler Linux Kernel syscall module Kernel Linux Kernel space module Kernel Kernel module Kernel module filesystem module Communicates with high prio communicates with terminal to display text of harddisk to read the file printf

Xenom ai ( kernel space) Runs in Kernel Mode : can address User and Kernel space Low prio Linux scheduler Shell : # insmod <module> not realtime Linux Kernel module Kernel init_module() scheduler Xenomai Real-time rt_task_start Kernel hard realtime function RT task call high prio User space Kernel space

Xenom ai ( user space) rt_thread_create Runs in User Mode : can only address user space Low prio Linux scheduler Sched main Linux thread other rt_task_create, rt_task_start soft FIFO Xenomai task realtime Linux Linux Linux syscall Kernel syscall xenomai scheduler Real-time syscall Xenomai hard Xenomai task realtime Kernel high prio User space Kernel space

Xenom ai i nt ernal s: i pi pe ipipe = Interrupt pipeline abstraction guest OSes are regarded as prioritized domains. For each event (interrupts, exceptions, syscalls, ...) , the various domains may handle the event or pass it down the pipeline.

i -pi pe: O pt i m i st i c prot ect i on schem e If a real time domain (like Xenomai) has higher priority it is the first in the pipeline It will receive interrupt notification first without delay (or at least with predictable latency) Then it can be decided if interrupts are propagated to low priority domains (like Linux) or not