Legacy Containers and OS Personalities Carsten Weinhold Dresden, - - PDF document

▶
legacy containers and os personalities
SMART_READER_LITE
LIVE PREVIEW

Legacy Containers and OS Personalities Carsten Weinhold Dresden, - - PDF document

Jun 28, 2023 169 likes •357 views

Faculty of Computer Science Institute for System Architecture, Operating Systems Group Legacy Containers and OS Personalities Carsten Weinhold Dresden, 2007-12-18 So far... So far: Basics Concepts: Tasks, Threads, IPC Memory,

slide-1
SLIDE 1

Faculty of Computer Science Institute for System Architecture, Operating Systems Group

Legacy Containers and OS Personalities

Dresden, 2007-12-18 Carsten Weinhold

TU Dresden, 2007-12-18 MOS - Legacy Containers Slide 2 von 34

So far...

So far:

– Basics Concepts:

  • Tasks, Threads, IPC
  • Memory, Synchronization, ...

– Device Drivers – Real-Time – Naming – Virtualization

Today:

– Legacy Containers / OS Personalities

slide-2
SLIDE 2

TU Dresden, 2007-12-18 MOS - Legacy Containers Slide 3 von 34

Motivation

What are legacy containers / OS personalities?

  • Emulation environment
  • Allow to run legacy applications

How is it different from virtualization?

  • Adaptation can be above hardware / OS layer
  • Often requires recompilation of the application

Why?

  • Reuse existing applications on new OS
  • Reuse existing APIs for new applications
  • Flexibility / Configurability
  • Lower resource consumption

TU Dresden, 2007-12-18 MOS - Legacy Containers Slide 4 von 34

Where to Cut?

Adaptation to underlying system can happen at various levels:

  • Hardware level (virtualization):

– Legacy OS and applications are moved to new OS (full + para virtualization, e.g., L4Linux)

  • OS level:

– Legacy OS's interfaces are reimplemented on top of new OS (e.g., Wine on Linux)

  • Application level:

– Toolkits, framework (e.g., Qt)

slide-3
SLIDE 3

TU Dresden, 2007-12-18 MOS - Legacy Containers Slide 5 von 34

Straightforward Way to OS Personality

Single-server approach:

– All (legacy) system services provided by a single server (e.g., L4Linux) – No isolation among services – Provides high level of compatibility, as original implementations are reused – Ideally, applications do not need to be modified

Ext2 VFAT IP Disk Driver NIC Driver App App Syscall Entry Single Server Microkernel

TU Dresden, 2007-12-18 MOS - Legacy Containers Slide 6 von 34

Application Scenarios for Single Server

Hybrid applications (real time):

  • “Enlightened” applications:

– Major parts of applications (e.g., GUIs) do not require real-time guarantees – Small parts may require real- time services running next to legacy container

  • Examples:

– RT network, RT file system, Window with RT widget

App Single Server Microkernel Real-Time Infrastructure Real-Time Server

slide-4
SLIDE 4

TU Dresden, 2007-12-18 MOS - Legacy Containers Slide 7 von 34

Application Scenarios for Single Server

Hybrid applications (security):

  • “Enlightened” applications:

– Most parts are not security critical – Small parts require strong isolation from potentially insecure legacy container

  • Examples:

– Secure file system that encrypts all data

More in following lectures!

App Single Server Microkernel Security Infrastructure Security Server

TU Dresden, 2007-12-18 MOS - Legacy Containers Slide 8 von 34

Multiple Single Servers

  • Multiple instances of

single-server OSes

  • Useful for isolation
  • f applications
  • Not necessarily

identical OS personalities (e.g., have UNIX and Windows servers)

  • “Virtual machines”

UNIX App Single Server UNIX Microkernel UNIX App Single Server UNIX Windows App Single Server Windows

slide-5
SLIDE 5

TU Dresden, 2007-12-18 MOS - Legacy Containers Slide 9 von 34

Single Server to Multi Server

Ext2 VFAT IP Disk Driver NIC Driver App App Ext2 VFAT IP Disk Driver NIC Driver App App Syscall Entry Monolith Microkernel Microkernel

Move from single server to multi server Challenge: Application wants a consistent view in a distributed system

TU Dresden, 2007-12-18 MOS - Legacy Containers Slide 10 von 34

Approach: Central Server

Ext2 VFAT IP Disk Driver NIC Driver Process / VFS / Net / ... App App

  • Central server provides

consistent view for both:

– Applications

  • System resources (files, ...)

– Servers

  • Client state (file tables, ...)
  • Problem:

– Performance bottleneck – Scalability – Complexity

slide-6
SLIDE 6

TU Dresden, 2007-12-18 MOS - Legacy Containers Slide 11 von 34

Approach: Emulation Library

Ext2 VFAT IP Disk Driver NIC Driver App App

  • Emulation library

– Per application – Interacts with servers – Provides consistent view

  • Each servers keeps its own

client state

  • In real world:

– Applications use C library, emulation library hidden below – Emulation provides POSIX API

Emulation Library Emulation Library C Library C Library

TU Dresden, 2007-12-18 MOS - Legacy Containers Slide 12 von 34

Legacy Environment: POSIX

  • POSIX “Portable Operating System Interface”

is a family of standards (POSIX 1003.*)

  • POSIX makes various UNIX-like systems

compatible (even Windows NT and newer)

  • Defines interfaces and properties:

– I/O: files, sockets, terminal, ... – Threads / synchronization: PThread – System tools – ...

  • POSIX API accessible via C library
slide-7
SLIDE 7

TU Dresden, 2007-12-18 MOS - Legacy Containers Slide 13 von 34

C Library: API to POSIX

  • Collection of various common functions
  • Abstraction layer above system calls
  • Functions with different abstraction levels ...

– low level: memcpy(), strlen() – medium: fopen(), fread() – high level: getpwent()

  • ... and dependencies

– none (freestanding): memcpy(), strlen() – small: malloc(): uses mmap() or sbrk() – strong: getpwent(): file access, “/etc/passwd”, name service, ...

TU Dresden, 2007-12-18 MOS - Legacy Containers Slide 14 von 34

Multi-Server Aware C Library

  • Observations:

– C library depends on underlying OS – Specific implementation may differ

Application

libc + syscall bindings

Monolithic / Single-Server OS

System Call Interface

Memory Server

Memory Interface

l4vfs_*(), rtc_*(), dm_phys_*(), ... memcpy(), fopen(), getpwent()

Time BE File BE Mem BE ... libc

  • pen(), read(),

mmap()

File System

File System Interface

Application

+ Backends

slide-8
SLIDE 8

TU Dresden, 2007-12-18 MOS - Legacy Containers Slide 15 von 34

Example: uClibc on L4

  • What is uClibc?

– Reimplementation of C library – Designed for Embedded Linux – Compatible to GNU C library “glibc”

  • Why uClibc?

– Small and portable – Optional C++ support available (uClibc++) – Licensed under LGPL

  • uClibc + backends provide applications with a

consistent view of L4-based multi-server OS

TU Dresden, 2007-12-18 MOS - Legacy Containers Slide 16 von 34

C Library Backends

  • C library functions grouped in backends
  • Simple example: time

unsigned int offset; void l4libc_init_time(void) { int ret; l4_calibrate_tsc(); ret = l4rtc_get_offset_to_realtime(&offset); if (ret != 0) { /* RTC server not found, /* assuming 1/1/1970, 0:00 */

  • ffset = 0;

} } extern unsigned int offset; time_t time(time_t *t) { time_t t_temp; l4_uint32_t s, ns; l4_tsc_to_s_and_ns(l4_rdtsc(), &s, &ns); /* get (cached) system time offset */ t_temp = s + offset; if (t) *t = t_temp; return t_temp; }

  • 1. Initalization

Call time server

  • 2. Usage:

Locally calculate time (no call to time server necessary)

slide-9
SLIDE 9

TU Dresden, 2007-12-18 MOS - Legacy Containers Slide 17 von 34

Flexibility of C Library Backends

Multiple implementations for some backends:

– Memory backends:

  • self_mem
  • sigma0_mem, sigma0_pool_mem
  • simple_mem
  • buddy_slab_mem,
  • TLSF (real-time memory allocator)

– I/O backends:

  • minimal_log_io, io

TU Dresden, 2007-12-18 MOS - Legacy Containers Slide 18 von 34

Example: Minimalist I/O Backend

  • Minimalist I/O backend:

– Simple backend for writing to STDOUT/STDERR – Uses L4/Fiasco kernel debugger

#include <unistd.h> #include <errno.h> #include <l4/sys/kdebug.h> int write(int fd, const void *buf, size_t count) __THROW { /* just accept write to stdout and stderr */ if ((fd == STDOUT_FILENO) || (fd == STDERR_FILENO)) { l4kdb_outnstring((const char*)buf, count); return count; } /* writes to other fds shall fail fast */ errno = EBADF; return -1; }

slide-10
SLIDE 10

TU Dresden, 2007-12-18 MOS - Legacy Containers Slide 19 von 34

Example: Complex I/O Backend

  • Complex L4VFS I/O backend:

1)App test1's libc calls open() 2)Resolve name at name server a)Backend calls name server b)Name server calls server con_term, gets back reference to tty1 3)Backend opens tty1, server con_term updates client state 4)Backend updates local file table 5)Backend returns file handle

Application

(“test1”, 1c.2)

Libc I/O Back- end

fd table

(12.2):23 (12.2):24 1 2

con_term

(12.2)

client_state

’vc2’:1c.2:’r’:0 ’vc5’:1e.4:’rw’:0 23 24 25

name_server

(11.0)

1 5 3 2a 2b 4

(12.2):24 ’vc2’:1c.2:’w’:0 TU Dresden, 2007-12-18 MOS - Legacy Containers Slide 20 von 34

Example: POSIX signals on L4

  • Signals used to deliver asynchronous event

notifications

– CPU exceptions (SIGFPE, ...) – Kernel exceptions (SIGSEGV, SIGCHLD, ...) – Issued by applications (SIGUSR1, SIGUSR2, SIGTERM, ...)

  • Signals on Linux

– built-in kernel mechanism – delivered upon return from kernel

  • Signals on L4

– implement in user space – map signals to L4 IPC? – Problem: IPC is synchronous!

slide-11
SLIDE 11

TU Dresden, 2007-12-18 MOS - Legacy Containers Slide 21 von 34

POSIX signals on L4: startup

signal server Application A Application B libl4sig signal thread libl4sig signal thread register_handler() register_handler() A -> A.3 B -> B.7 setup signal handlers setup signal handlers setup signal handlers setup signal handlers

TU Dresden, 2007-12-18 MOS - Legacy Containers Slide 22 von 34

POSIX signals on L4: sending a signal

signal server Application A Application B libl4sig signal thread libl4sig signal thread signal(A, SIGUSR1) A -> A.3 B -> B.7 notify( SIGUSR1)

slide-12
SLIDE 12

TU Dresden, 2007-12-18 MOS - Legacy Containers Slide 23 von 34

POSIX signals on L4: dispatching

  • Signal handler thread dispatches signal:

– chose arbitrary application thread Ex (except for L4Env threads) – store Ex's thread state (EIP, ESP) – force Ex to run configured signal handler using a dedicated stack – after Ex returns from handler: restore orignal EIP and ESP

  • Specialties:

– SIGSTOP: set all threads to l4sleep() – SIGALRM: dedicated timer provided by the signal server – SIGKILL: signal server kills application at task server

TU Dresden, 2007-12-18 MOS - Legacy Containers Slide 24 von 34

Configurability of C Library Backends

Unnecessary backends can be omitted:

– Reduced TCB for applications – Potentially better security properties – Smaller memory footprint

Memory Server

Memory Interface Time BE File BE Mem BE ... libc + Backends

File System Server

File System Interface

File-System Application

Time BE Net BE Mem BE ... libc + Backends

Network Application PF_INET Server

Network Interface

PF_LOCAL Server

Network Interface Mem BE libc + Backends

Simple Application

slide-13
SLIDE 13

TU Dresden, 2007-12-18 MOS - Legacy Containers Slide 25 von 34

Summary: POSIX Emulation on L4Env

  • Provides a significant part of POSIX APIs:

– C library: all platform independent parts

  • strcpy(), ...

– Dynamic memory allocation

  • Various backends

– I/O support (files, network, terminal, time):

  • L4VFS + L4VFS servers
  • RTC server

– Signal handling:

  • signal

– No PThread support

TU Dresden, 2007-12-18 MOS - Legacy Containers Slide 26 von 34

More Powerful Legacy Containers

  • POSIX is limited to basic OS abstractions:

– No graphics / GUI support – No audio support

  • Examples for more powerful APIs:

– SDL “Simple Direct Layer” (focuses on Multimedia) – Qt toolkit (focuses on rich GUIs and complete OS abstraction)

slide-14
SLIDE 14

TU Dresden, 2007-12-18 MOS - Legacy Containers Slide 27 von 34

Qt Toolkit on L4

  • What is Qt?

– Multi-platform toolkit library – Provides abstraction of underlying OS – Available for:

  • UNIX/X11 (Linux, *BSD, ...)
  • Windows
  • Mac OS X
  • ... and L4 (Bastei port in progress)
  • L4 port based on Qt's Linux/embedded version

– Has its own “Qt Windowing System” (QWS) – Relies on POSIX compliant OS

TU Dresden, 2007-12-18 MOS - Legacy Containers Slide 28 von 34

Qt Architecture

Qt API File Access Application Operating System Network Threads / Synch GUI ...

POSIX / L4VFS POSIX / L4VFS L4Env / thread, semaphore QWS

Platform Independent Platform Specific

slide-15
SLIDE 15

TU Dresden, 2007-12-18 MOS - Legacy Containers Slide 29 von 34

Graphical Console libinput (Hardware Access)

Qt/L4: Qt Windowing System

  • QWS is core part of Qt/L4:

– Displays and manages windows from multiple Qt applications – Operates on shared framebuffer – Events distributed via shared memory / sockets

QWS Server QWS Client Qt library QWS Client Qt library Qt library FB Driver Keyb. Driver Mouse Driver

TU Dresden, 2007-12-18 MOS - Legacy Containers Slide 30 von 34

Qt/L4: Features

  • Complete support of GUI framework
  • Threads, Synchronization supported
  • Support for file access

– Depends on available L4VFS file servers

  • Support for network access

– Basic functionality with L4VFS server “FLIPS”

slide-16
SLIDE 16

TU Dresden, 2007-12-18 MOS - Legacy Containers Slide 31 von 34

Qt/L4: Easy Development

  • Applications (or parts thereof) can be

developed and tested on Linux

  • Rapid Application Development:

– GUIs can be created with Qt Designer – Other tools (e.g, UI compiler) can be reused

TU Dresden, 2007-12-18 MOS - Legacy Containers Slide 32 von 34

DEMO

slide-17
SLIDE 17

TU Dresden, 2007-12-18 MOS - Legacy Containers Slide 33 von 34

References

  • Carsten Weinhold: 'Portierung von Qt auf DROPS'

Großer Beleg 2005

http://os.inf.tu-dresden.de/paper_ps/weinhold-beleg.pdf

  • Resources on POSIX standard:

http://standards.ieee.org/regauth/posix/

  • Christian Helmuth, Frank Mehnert, Lars Reuther,

Martin Pohlack, Alexander Warg, Björn Döbel: MOS slides from previous years

TU Dresden, 2007-12-18 MOS - Legacy Containers Slide 34 von 34

Coming soon...

  • This week:
  • Prepare for Christmas :-)
  • Next weeks:
  • Security (08.01. + 15.01.)
  • Practical exercise “Bastei” (09.01.)