How to boot Linux in one second why userland is a waste of time ;) - - PowerPoint PPT Presentation

how to boot linux in one second
SMART_READER_LITE
LIVE PREVIEW

How to boot Linux in one second why userland is a waste of time ;) - - PowerPoint PPT Presentation

How to boot Linux in one second why userland is a waste of time ;) Jan Altenberg Linutronix GmbH Jan Altenberg Linutronix GmbH 1 from zero Jan Altenberg Linutronix GmbH 2 to hero Jan Altenberg Linutronix GmbH 3 Overview


slide-1
SLIDE 1

How to boot Linux in one second

…why userland is a waste of time ;) Jan Altenberg

Linutronix GmbH

Jan Altenberg Linutronix GmbH 1

slide-2
SLIDE 2

from zero…

Jan Altenberg Linutronix GmbH 2

slide-3
SLIDE 3

to hero…

Jan Altenberg Linutronix GmbH 3

slide-4
SLIDE 4

Overview

. . 1 Basics

Motivation Some technical basics

. . 2 Optimizations

Bootloader Kernel Filesystem Application

. . 3 Example

Optimizing an ARMv5 based device Optimizing the test system

Jan Altenberg Linutronix GmbH 4

slide-5
SLIDE 5

Motivation

. . 1 Basics

Motivation Some technical basics

. . 2 Optimizations

Bootloader Kernel Filesystem Application

. . 3 Example

Optimizing an ARMv5 based device Optimizing the test system

Jan Altenberg Linutronix GmbH 5

slide-6
SLIDE 6

Motivation

Motivation

''marketing'' automotive applications energy saving

  • solution: power-off instead of suspending
  • BUT: Users are not used to wait

Jan Altenberg Linutronix GmbH 6

slide-7
SLIDE 7

Motivation

Motivation

''marketing'' automotive applications energy saving

  • solution: power-off instead of suspending
  • BUT: Users are not used to wait

Jan Altenberg Linutronix GmbH 6

slide-8
SLIDE 8

Motivation

Motivation

''marketing'' automotive applications energy saving

  • solution: power-off instead of suspending
  • BUT: Users are not used to wait

Jan Altenberg Linutronix GmbH 6

slide-9
SLIDE 9

Some technical basics

. . 1 Basics

Motivation Some technical basics

. . 2 Optimizations

Bootloader Kernel Filesystem Application

. . 3 Example

Optimizing an ARMv5 based device Optimizing the test system

Jan Altenberg Linutronix GmbH 7

slide-10
SLIDE 10

Some technical basics

First step: Defjne your requirements!!!!

What's the limit for the boot time? Which functionality should be available? Speed vs. fmexibility NOTE: FastBOOT is not a product, it's a concept!!

Jan Altenberg Linutronix GmbH 8

slide-11
SLIDE 11

Some technical basics

Boot process

Jan Altenberg Linutronix GmbH 9

slide-12
SLIDE 12

Some technical basics

Components of the boot process

Hardware reset Bootloader Operating System (drivers, fjlesystem, …)) INIT process, application (userland)

Jan Altenberg Linutronix GmbH 10

slide-13
SLIDE 13

Some technical basics

Critical hardware components

Power supply Reset logic Boot logic / boot order Boot media Peripherals which need to be accessed while booting IMPORTANT: the hardware is a central part of a fastboot concept!!!

Jan Altenberg Linutronix GmbH 11

slide-14
SLIDE 14

Some technical basics

Bootloader

Basic setup of the CPU Preparing and handing over ATAGS / devicetree Flushing the caches Switch off the MMU

Jan Altenberg Linutronix GmbH 12

slide-15
SLIDE 15

Some technical basics

The Linux Kernel

A lot of functions for boot time optimization Very fmexible Confjgurable compression type Can deffer or parallelize initializations 150ms - 250ms from starting the kernel to mounting the RFS

Jan Altenberg Linutronix GmbH 13

slide-16
SLIDE 16

Some technical basics

The application

Usually the biggest target for optimizations Start scripts / INIT process Linking

Jan Altenberg Linutronix GmbH 14

slide-17
SLIDE 17

Bootloader

. . 1 Basics

Motivation Some technical basics

. . 2 Optimizations

Bootloader Kernel Filesystem Application

. . 3 Example

Optimizing an ARMv5 based device Optimizing the test system

Jan Altenberg Linutronix GmbH 15

slide-18
SLIDE 18

Bootloader

Optimizing the bootloader (U-Boot) 1

Remove unused features: /* include / configs /boardname . h */ [ . . . ] #include <config_cmd_default . h> #undef CONFIG_CMD_NET [ . . . ]

Jan Altenberg Linutronix GmbH 16

slide-19
SLIDE 19

Bootloader

Optimizing the bootloader (U-Boot) 2

Verifying the kernel image:

setenv verify n

Switch off the bootloader console:

setenv silent 1

Switch off the boot delay:

setenv bootdelay 0

Jan Altenberg Linutronix GmbH 17

slide-20
SLIDE 20

Bootloader

Optimizing the bootloader: IPL / SPL

Replacing the general purpose bootloader by an optimized IPL …also useful for update concepts U-Boot offers a generic way: The U-Boot SPL (CONFIG_SPL_OS_BOOT)

Jan Altenberg Linutronix GmbH 18

slide-21
SLIDE 21

Kernel

. . 1 Basics

Motivation Some technical basics

. . 2 Optimizations

Bootloader Kernel Filesystem Application

. . 3 Example

Optimizing an ARMv5 based device Optimizing the test system

Jan Altenberg Linutronix GmbH 19

slide-22
SLIDE 22

Kernel

Optimizing the kernel

Confjguration and build Compression method Boot parameters (kernel commandline) Driver init calls Rootfjlesystem (RFS)

Jan Altenberg Linutronix GmbH 20

slide-23
SLIDE 23

Kernel

Optimizing the kernel: Confjguration

General setup

  • -->

Kernel compression mode -->

LZO usually a good choice for embedded system Copy vs. de-compress ''Execute in Place (XIP)'

Jan Altenberg Linutronix GmbH 21

slide-24
SLIDE 24

Kernel

Optimizing the kernel: Kernel commandline

Delay Loop Calibration: ''lpj=''; can save > 100ms on ARMv5 based systems Parameters for boot time analysis: ''initcall_debug'', ''printk_time=1''

Jan Altenberg Linutronix GmbH 22

slide-25
SLIDE 25

Kernel

Optimizing the kernel: printk.time

... [0.000000] VIC @f1140000: id 0x00041190, vendor 0x41 [0.000000] FPGA IRQ chip 0 "SIC" @ f1003000, 21 irqs [0.000000] Console: colour dummy device 80x30 [0.018847] Calibrating delay loop... 626.68 BogoMIPS (lpj=3133440) [0.316717] pid_max: default: 32768 minimum: 301 [0.317552] Mount-cache hash table entries: 512 ...

Jan Altenberg Linutronix GmbH 23

slide-26
SLIDE 26

Kernel

Optimizing the kernel: Delay Loop

... [0.018847] Calibrating delay loop... 626.68 BogoMIPS (lpj=3133440) [0.316717] pid_max: default: 32768 minimum: 301 ...

Jan Altenberg Linutronix GmbH 24

slide-27
SLIDE 27

Kernel

Optimizing the kernel: initcall_debug

[0.452115] calling exceptions_init+0x0/0x90 @ 1 [0.452172] initcall exceptions_init+0x0/0x90 returned 0 after 0 usecs [0.452203] calling versatile_i2c_init+0x0/0x24 @ 1 [0.452321] initcall versatile_i2c_init+0x0/0x24 returned 0 after 0 usecs [0.452352] calling pl011_init+0x0/0x54 @ 1 [0.452382] Serial: AMBA PL011 UART driver [0.453647] dev:f1: ttyAMA0 at MMIO 0x101f1000 (irq = 12) is a PL011 rev1 [0.481540] console [ttyAMA0] enabled ... [0.484427] initcall pl011_init+0x0/0x54 returned 0 after 29296 usecs

Jan Altenberg Linutronix GmbH 25

slide-28
SLIDE 28

Kernel

bootgraph.pl

. . 1 Boot your system with ''initcall_debug loglevel=8 . . 2 On the target:

$ dmesg > bootlog.txt

. . 3 On the host:

$ cd linux-XXX $ cat /path_to_rfs/bootlog.txt | \ perl scripts/bootgraph.pl > bootlog.svg

Jan Altenberg Linutronix GmbH 26

slide-29
SLIDE 29

Kernel

scripts/bootchart.pl

Jan Altenberg Linutronix GmbH 27

slide-30
SLIDE 30

Filesystem

. . 1 Basics

Motivation Some technical basics

. . 2 Optimizations

Bootloader Kernel Filesystem Application

. . 3 Example

Optimizing an ARMv5 based device Optimizing the test system

Jan Altenberg Linutronix GmbH 28

slide-31
SLIDE 31

Filesystem

UbiFS

The best choice for fmash devices Power-Fail safe The underlying UBI layer can be optimized with (FastMAP) …

Jan Altenberg Linutronix GmbH 29

slide-32
SLIDE 32

Filesystem

InitRAMFS

dir /dev 755 0 0 nod /dev/console 644 0 0 c 5 1 nod /dev/loop0 644 0 0 b 7 0 dir /bin 755 1000 1000 slink /bin/sh busybox 777 0 0 file /bin/busybox initfs/busybox 755 0 0 [...] dir /proc 755 0 0 dir /sys 755 0 0 dir /mnt 755 0 0

Jan Altenberg Linutronix GmbH 30

slide-33
SLIDE 33

Filesystem

InitRAMFS: Switch root

The INIT process for the InitRAMFS can be confjgured with rdinit=. For example: rdinit=/etc/init.d/start.sh

Jan Altenberg Linutronix GmbH 31

slide-34
SLIDE 34

Filesystem

InitRAMFS: Switch root

/etc/init.d/start.sh:

#!/bin/sh mount -t proc proc /proc mount -t sysfs sysfs /sys mount -t devtmpfs devtmpfs /dev # Mount RFS / do some critical stuff mount /dev/mmcblk0p1 /media fbsplash -s /media/splash.ppm -d /dev/fb0 mount -o move /proc /media/proc mount -o move /sys /media/sys mount -o move /dev /media/dev # Switch to production system exec switch_root /media /linuxrc

Jan Altenberg Linutronix GmbH 32

slide-35
SLIDE 35

Application

. . 1 Basics

Motivation Some technical basics

. . 2 Optimizations

Bootloader Kernel Filesystem Application

. . 3 Example

Optimizing an ARMv5 based device Optimizing the test system

Jan Altenberg Linutronix GmbH 33

slide-36
SLIDE 36

Application

The INIT process

SystemV SystemD One letter makes a BIG difference ;-)

Jan Altenberg Linutronix GmbH 34

slide-37
SLIDE 37

Application

Optimizing the application

Analyse the INIT process with bootchartd or systemd-analyze Replace the INIT process with your own application (init=) Linking Pre-Linking and function reordering

Jan Altenberg Linutronix GmbH 35

slide-38
SLIDE 38

Application

Moving start script tasks into your application

ret = mount("sysfs", "/sys", "sysfs", 0, NULL); if(ret < 0) perror("Can't mount sysfs\n");

Jan Altenberg Linutronix GmbH 36

slide-39
SLIDE 39

Application

Dynamic linking

. . 1 ELF DT_RPATH section . . 2 LD_LIBRARY_PATH . . 3 ELF DT_RUNPATH section . . 4 Binary fjle /etc/ld.so.cache . . 5 Default paths /lib und /usr/lib

Jan Altenberg Linutronix GmbH 37

slide-40
SLIDE 40

Application

Dynamic linking: Debug and visualize

$ LD_DEBUG=libs ls 3082: find library=librt.so.1 [0]; searching 3082: search cache=/etc/ld.so.cache 3082: trying file=/lib/librt.so.1

Jan Altenberg Linutronix GmbH 38

slide-41
SLIDE 41

Optimizing an ARMv5 based device

. . 1 Basics

Motivation Some technical basics

. . 2 Optimizations

Bootloader Kernel Filesystem Application

. . 3 Example

Optimizing an ARMv5 based device Optimizing the test system

Jan Altenberg Linutronix GmbH 39

slide-42
SLIDE 42

Optimizing an ARMv5 based device

Test system

ARM9 CPU (Atmel AT91 series) Starting point: Busybox based image (Angstrom Distribution) Boot media: NAND-Flash Test application: Toggling a GPIO via SysFS

Jan Altenberg Linutronix GmbH 40

slide-43
SLIDE 43

Optimizing an ARMv5 based device

Boot strategy of the AT91 controller family

Jan Altenberg Linutronix GmbH 41

slide-44
SLIDE 44

Optimizing an ARMv5 based device

Bootmodes of the AT91 controller family

RomBOOT: internal boot logic External bus interface (CS0, e.g. NOR fmash)

Jan Altenberg Linutronix GmbH 42

slide-45
SLIDE 45

Optimizing an ARMv5 based device

AT91 RomBOOT

Jan Altenberg Linutronix GmbH 43

slide-46
SLIDE 46

Optimizing an ARMv5 based device

Power supply

Jan Altenberg Linutronix GmbH 44

slide-47
SLIDE 47

Optimizing an ARMv5 based device

Reset logic

Jan Altenberg Linutronix GmbH 45

slide-48
SLIDE 48

Optimizing an ARMv5 based device

RomBOOT

Jan Altenberg Linutronix GmbH 46

slide-49
SLIDE 49

Optimizing an ARMv5 based device

Possible hardware optimizations

Using the internal oscillator for deriving the slowclock saves > 1s!! booting from CS0 will save 100 - 150ms

Jan Altenberg Linutronix GmbH 47

slide-50
SLIDE 50

Optimizing the test system

. . 1 Basics

Motivation Some technical basics

. . 2 Optimizations

Bootloader Kernel Filesystem Application

. . 3 Example

Optimizing an ARMv5 based device Optimizing the test system

Jan Altenberg Linutronix GmbH 48

slide-51
SLIDE 51

Optimizing the test system

Boot time measurements with a GPIO

Jan Altenberg Linutronix GmbH 49

slide-52
SLIDE 52

Optimizing the test system

Measuring points

Bootstrap - U-Boot U-Boot - Early-Boot-Code of the kernel (incl. relocation and decompression) Kernel - application (incl. mounting the RFS)

Jan Altenberg Linutronix GmbH 50

slide-53
SLIDE 53

Optimizing the test system

Initial boot time

Jan Altenberg Linutronix GmbH 51

slide-54
SLIDE 54

Optimizing the test system

Initial boot time

measuring point time bootstrap - u-boot

  • u-boot - kernel

6,5s kernel - application 4,5s total 11s

Jan Altenberg Linutronix GmbH 52

slide-55
SLIDE 55

Optimizing the test system

Simple optimizations

Jan Altenberg Linutronix GmbH 53

slide-56
SLIDE 56

Optimizing the test system

U-Boot w/o networking support

measuring point time bootstrap - u-boot

  • u-boot - kernel

4,25s kernel - application 4,5s total 8,75s

Jan Altenberg Linutronix GmbH 54

slide-57
SLIDE 57

Optimizing the test system

U-Boot verify=n

measuring point time bootstrap - u-boot

  • u-boot - kernel

3,89s kernel - application 4,5s total 8,39s

Jan Altenberg Linutronix GmbH 55

slide-58
SLIDE 58

Optimizing the test system

Optimizing the kernel confjg

measuring point time bootstrap - u-boot

  • u-boot - kernel

3,77s kernel - application 4,33s total 8,1s

Jan Altenberg Linutronix GmbH 56

slide-59
SLIDE 59

Optimizing the test system

Analyzing the INIT process: Bootchartd

Jan Altenberg Linutronix GmbH 57

slide-60
SLIDE 60

Optimizing the test system

Optimizing the start scripts

measuring point time bootstrap - u-boot

  • u-boot - kernel

3,77s kernel - application 3,61 total 7,38s

Jan Altenberg Linutronix GmbH 58

slide-61
SLIDE 61

Optimizing the test system

Booting an InitRAMFS

Jan Altenberg Linutronix GmbH 59

slide-62
SLIDE 62

Optimizing the test system

LZO compressed InitRAMFS

The test application is used as an INIT process (rdinit=) measuring point time bootstrap - u-boot

  • u-boot - kernel

3,79s kernel - application 0,372s total 4,162s

Jan Altenberg Linutronix GmbH 60

slide-63
SLIDE 63

Optimizing the test system

Modifjed AT91 Bootstrap

Jan Altenberg Linutronix GmbH 61

slide-64
SLIDE 64

Optimizing the test system

Modifjed AT91 Bootstrap

AT91 Bootstrap starts Linux (without U-Boot) measuring point time bootstrap - kernel 676ms kernel - application 584ms total 1,260s

Jan Altenberg Linutronix GmbH 62

slide-65
SLIDE 65

Optimizing the test system

lpj=

measuring point time bootstrap - kernel 676ms kernel - application 384ms total 1,060s

Jan Altenberg Linutronix GmbH 63

slide-66
SLIDE 66

Optimizing the test system

< 1s !!

Jan Altenberg Linutronix GmbH 64

slide-67
SLIDE 67

Optimizing the test system

No (serial) console output (quiet)

measuring point time bootstrap - kernel 524ms kernel - application 212ms total 736ms

Jan Altenberg Linutronix GmbH 65

slide-68
SLIDE 68

Optimizing the test system

LZO compressed kernel image

measuring point time bootstrap - kernel 444ms kernel - application 212ms total 656ms

Jan Altenberg Linutronix GmbH 66

slide-69
SLIDE 69

Optimizing the test system

Final boot behaviour

Jan Altenberg Linutronix GmbH 67

slide-70
SLIDE 70

Optimizing the test system

Conclusion

Linux can combine the advantages of a modern OS with hard boot time requirements Saving boot time with simple optimizations The hardware is an IMPORTANT part of a FastBOOT concept The boot concept is architecture independent!

Jan Altenberg Linutronix GmbH 68

slide-71
SLIDE 71

Optimizing the test system

Questions?

I'll also be around at the technical showcase! :)

Jan Altenberg Linutronix GmbH 69