Buildroot
Making embedded Linux easy? A real-life example.
Introduction Buildroot overview Br2-external Additional infrastructure ConclusionYann E. MORIN <yann.morin@orange.com> Embedded Linux Conference Europe 2017
Buildroot Buildroot overview Br2-external Additional - - PowerPoint PPT Presentation
Introduction Buildroot Buildroot overview Br2-external Additional infrastructure Making embedded Linux easy? Conclusion A real-life example. Yann E. MORIN <yann.morin@orange.com> Embedded Linux Conference Europe 2017
Making embedded Linux easy? A real-life example.
Introduction Buildroot overview Br2-external Additional infrastructure ConclusionYann E. MORIN <yann.morin@orange.com> Embedded Linux Conference Europe 2017
2
3
The other "Yann E. MORIN"
Linux Embedded Realtime Security Network FLOSS
About me
"Yann E. MORIN" <yann.morin@orange.com>
Linux Embedded Security Network FLOSS
4
Team
Three teams, ~30 developers Two geographical locations Application developers Not all Linux experts Not all embedded experts
Context
Targets: STB, STB-like
Difgerent generations Performance R&D vs. production Inherit constraints Part of the fjrmware Not the main part Aggregation of multiple third-parties External integrator Complete rewrite from scratch
5
OpenEmbedded
Distribution generator Very versatile, highly customisable Steep learning curve No in-house knowledge
Buildroot
Firmware generator Flexible, extendable (BR2_EXTERNAL!) Moderate (leaning to easy) learning curve In-house knowledge
Others: OpenWrt, OE-lite, PTXdist...
Much smaller communities (!OpenWrt) Not really investigated, no in-house knowledge
Choosing a development environment – Late 2013
Requirements
Generic buildsystem Target agnostic Run-time agnostic Easy to learn and use No build-time overhead Existing solution Not my choice!
SoftCo SDK
Dedicated to the production devices Very specifjc, very custom
6
7
# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # l p e g # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # L P E G _ V E R S I O N = 1 . . 1
L P E G _ L I C E N S E = M I T $ ( e v a l $ ( l u a r
k s
a c k a g e ) ) # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # f p i n g # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # F P I N G _ V E R S I O N = 4 . F P I N G _ S I T E = h t t p : / / f p i n g .
g / d i s t F P I N G _ L I C E N S E = B S D
i k e F P I N G _ L I C E N S E _ F I L E S = C O P Y I N G $ ( e v a l $ ( a u t
s
a c k a g e ) )
Buildroot overview - Quick Introduction
Buildroot
Is a simple, effjcient and easy-to-use tool to
generate embedded Linux systems through cross-compilation
Is entirely community-driven Uses kconfjg-driven confjguration Uses Makefjles Has a website https://buildroot.org/ Has a manual https://buildroot.org/manual.html Is fun!
8
Buildroot overview – Build sequence
Buildroot overall build sequence
Build packages, in sequence Dependency chain Finalise target Cleanups: .a .la .h man... Stripping Generate the fjlesystem image(s) A tarball is a fjlesystem image ;-)
Extras
target-fjnalize hooks post-build, fakeroot, post-image scripts
... ... ... confjgure, build, install package target-fjnalize hooks cleanups, stripping
post-build scripts fakeroot scripts generate fjlesystems post-image scripts ... ... confjgure, build, install package install toolchain (package)
9
Buildroot overview – Package infrastructure
Buildroot package infrastructure
Download Extract Patch Confjgure Build [Install in staging/] Install in target/
Extras
Local rsync during development Pre- and post-hooks for each step
pre-hook post-hook download pre-hook post-hook extract pre-hook post-hook patch pre-hook post-hook confjgure pre-hook post-hook build pre-hook post-hook staging install pre-hook post-hook target install pre-hook post-hook rsync
10
Extensibility
Defconfjgs Packages Filesystems Bootloaders Board fjles Extra logic
Buildroot overview - BR2_EXTERNAL
Local extensions
Without touching the Buildroot tree For proprietary packages Staging area before upstreaming Access to all of Buildroot infrastructures Stackable
11
12
$ l s
l F
w
w
y m
i n y m
i n S e p 1 2 8 : 4 9 C
f i g . i n
w
w
y m
i n y m
i n S e p 1 2 8 : 4 9 e x t e r n a l . m k
BR2_EXTERNAL - Warming-up
Local customisations
13
$ l s
l F d r w x r w x r
1 3 y m
i n y m
i n 4 . K A u g 2 9 1 5 : 5 7 b u i l d r
/
w
w
y m
i n y m
i n S e p 1 2 8 : 4 9 C
f i g . i n
w
w
y m
i n y m
i n S e p 1 2 8 : 4 9 e x t e r n a l . m k
BR2_EXTERNAL - Buildroot
Local customisations
Buildroot as a git submodule
14
$ l s
l F d r w x r w x r
1 3 y m
i n y m
i n 4 . K A u g 2 9 1 5 : 5 7 b u i l d r
/ d r w x r w x r
2 y m
i n y m
i n 4 . K S e p 1 2 8 : 5 2 c
f i g s /
w
w
y m
i n y m
i n S e p 1 2 8 : 4 9 C
f i g . i n
w
w
y m
i n y m
i n S e p 1 2 8 : 4 9 e x t e r n a l . m k $ l s
l F c
f i g s /
w
w
y m
i n y m
i n 4 . K S e p 1 2 8 : 5 2 b
r d 1 _ d e v _ d e f c
f i g
w
w
y m
i n y m
i n 3 . 2 K S e p 1 2 8 : 5 2 b
r d 1 _ e 2 e _ d e f c
f i g
w
w
y m
i n y m
i n 3 . K S e p 1 2 8 : 5 2 b
r d 1 _ p r
_ d e f c
f i g
w
w
y m
i n y m
i n 4 . 2 K S e p 1 2 8 : 5 2 b
r d 2 _ d e v _ d e f c
f i g
w
w
y m
i n y m
i n 3 . 3 K S e p 1 2 8 : 5 2 b
r d 2 _ e 2 e _ d e f c
f i g
w
w
y m
i n y m
i n 3 . 1 K S e p 1 2 8 : 5 2 b
r d 2 _ p r
_ d e f c
f i g
BR2_EXTERNAL - Defconfjgs
Local customisations
Buildroot as a git submodule New confjgurations
15
$ l s
l F d r w x r w x r
1 3 y m
i n y m
i n 4 . K A u g 2 9 1 5 : 5 7 b u i l d r
/ d r w x r w x r
2 y m
i n y m
i n 4 . K S e p 1 2 8 : 5 2 c
f i g s / d r w x r w x r
2 y m
i n y m
i n 4 . K S e p 1 2 8 : 5 2 p a c k a g e /
w
w
y m
i n y m
i n 6 . 4 K S e p 1 2 8 : 4 9 C
f i g . i n
w
w
y m
i n y m
i n 1 1 3 S e p 1 2 8 : 4 9 e x t e r n a l . m k $ l s
l F p a c k a g e / d r w x r w x r
7 y m
i n y m
i n 4 . K A u g 2 9 1 5 : 4 a p p l a y e r / d r w x r w x r
5 y m
i n y m
i n 4 . K A u g 2 9 1 5 : 4 t h i r d p a r t y / d r w x r w x r
9 y m
i n y m
i n 4 . K A u g 2 9 1 5 : 4 t
s / $ l s
l F p a c k a g e / a p p l a y e r / d r w x r w x r
7 y m
i n y m
i n 4 . K A u g 2 9 1 5 : 4
a n g e
p g / d r w x r w x r
5 y m
i n y m
i n 4 . K A u g 2 9 1 5 : 4
a n g e
t s / d r w x r w x r
5 y m
i n y m
i n 4 . K A u g 2 9 1 5 : 4
a n g e
m a g e s / d r w x r w x r
5 y m
i n y m
i n 4 . K A u g 2 9 1 5 : 4
a n g e
i v e / d r w x r w x r
9 y m
i n y m
i n 4 . K A u g 2 9 1 5 : 4
a n g e
i b d a t a m
e l / d r w x r w x r
9 y m
i n y m
i n 4 . K A u g 2 9 1 5 : 4
a n g e
v r / $ l s
l F p a c k a g e / t
s / d r w x r w x r
7 y m
i n y m
i n 4 . K A u g 2 9 1 5 : 4
a n g e
p
s / d r w x r w x r
7 y m
i n y m
i n 4 . K A u g 2 9 1 5 : 4
a n g e
e v e l
s / $ c a t e x t e r n a l . m k i n c l u d e $ ( s
t $ ( w i l d c a r d $ ( B R 2 _ E X T E R N A L ) / p a c k a g e / * / * / * . m k ) ) $ c a t C
f i g . i n s
r c e " $ B R 2 _ E X T E R N A L / p a c k a g e / a p p l a y e r /
a n g e
n i t / C
f i g . i n " s
r c e " $ B R 2 _ E X T E R N A L / p a c k a g e / a p p l a y e r /
a n g e
i v e / C
f i g . i n " s
r c e " $ B R 2 _ E X T E R N A L / p a c k a g e / a p p l a y e r /
a n g e
p g / C
f i g . i n " s
r c e " $ B R 2 _ E X T E R N A L / p a c k a g e / t
s /
a n g e
e v e l
s / C
f i g . i n " s
r c e " $ B R 2 _ E X T E R N A L / p a c k a g e / t h i r d p a r t y / f
l a b l a / C
f i g . i n " [ . . . ]
BR2_EXTERNAL - Packages
Local customisations
Buildroot as a git submodule New confjgurations New packages
16
$ l s
l F d r w x r w x r
1 3 y m
i n y m
i n 4 . K A u g 2 9 1 5 : 5 7 b u i l d r
/ d r w x r w x r
2 y m
i n y m
i n 4 . K S e p 1 2 8 : 5 2 c
f i g s / d r w x r w x r
2 y m
i n y m
i n 4 . K S e p 1 2 9 : 1 1 f s / d r w x r w x r
2 y m
i n y m
i n 4 . K S e p 1 2 8 : 5 2 p a c k a g e /
w
w
y m
i n y m
i n 6 . 7 K S e p 1 2 8 : 4 9 C
f i g . i n
w
w
y m
i n y m
i n 1 8 3 S e p 1 2 8 : 4 9 e x t e r n a l . m k $ l s
l F f s / d r w x r w x r
2 y m
i n y m
i n 4 . K S e p 7 1 5 : 3 6
a n g e
p / $ c a t f s /
a n g e
p /
a n g e
p . m k R O O T F S _ O R A N G E _ G P _ D E P E N D E N C I E S = r
f s
a r
a n g e
p
s d e f i n e R O O T F S _ O R A N G E _ G P _ C M D m k
p
$ ( B I N A R I E S _ D I R ) / r
f s . t a r
( @ ) e n d e f $ ( e v a l $ ( c a l l R O O T F S _ T A R G E T ,
a n g e
p ) )
BR2_EXTERNAL - Filesystems
Local customisations
Buildroot as a git submodule New confjgurations New packages New fjlesystems
17
$ l s
l F d r w x r w x r
1 3 y m
i n y m
i n 4 . K A u g 2 9 1 5 : 5 7 b u i l d r
/ d r w x r w x r
2 y m
i n y m
i n 4 . K S e p 1 2 8 : 5 9 b
r d s / d r w x r w x r
2 y m
i n y m
i n 4 . K S e p 1 2 8 : 5 2 c
f i g s / d r w x r w x r
2 y m
i n y m
i n 4 . K S e p 1 2 9 : 1 1 f s / d r w x r w x r
2 y m
i n y m
i n 4 . K S e p 1 2 8 : 5 2 p a c k a g e /
w
w
y m
i n y m
i n 6 . 7 K S e p 1 2 8 : 4 9 C
f i g . i n
w
w
y m
i n y m
i n 1 8 3 S e p 1 2 8 : 4 9 e x t e r n a l . m k $ l s
l F b
r d s / c
m
/ d r w x r w x r
2 y m
i n y m
i n 4 . K S e p 1 2 8 : 5 9 s k e l e t
/ d r w x r w x r
2 y m
i n y m
i n 4 . K S e p 1 2 8 : 5 9
e r l a y
e s t s /
w x r w x r
2 y m
i n y m
i n 4 . K S e p 1 2 8 : 5 9 p
t
u i l d . s h
w x r w x r
2 y m
i n y m
i n 4 . K S e p 1 2 8 : 5 9 p
t
u i l d
e s t s . s h $ l s
l F b
r d s / b
r d 1 / d r w x r w x r
2 y m
i n y m
i n 4 . K S e p 1 2 8 : 5 9
e r l a y / $ l s
l F b
r d s / b
r d 2 / d r w x r w x r
2 y m
i n y m
i n 4 . K S e p 1 2 8 : 5 9
e r l a y /
BR2_EXTERNAL - Board fjles
Local customisations
Buildroot as a git submodule New confjgurations New packages New fjlesystems Board fjles
18
$ l s
l F d r w x r w x r
1 3 y m
i n y m
i n 4 . K A u g 2 9 1 5 : 5 7 b u i l d r
/ d r w x r w x r
2 y m
i n y m
i n 4 . K S e p 1 2 8 : 5 9 b
r d s / d r w x r w x r
2 y m
i n y m
i n 4 . K S e p 1 2 8 : 5 2 c
f i g s / d r w x r w x r
2 y m
i n y m
i n 4 . K S e p 1 2 8 : 5 7 c
e / d r w x r w x r
2 y m
i n y m
i n 4 . K S e p 1 2 9 : 1 1 f s / d r w x r w x r
2 y m
i n y m
i n 4 . K S e p 1 2 8 : 5 2 p a c k a g e /
w
w
y m
i n y m
i n 6 . 7 K S e p 1 2 8 : 4 9 C
f i g . i n
w
w
y m
i n y m
i n 2 3 7 S e p 1 2 8 : 4 9 e x t e r n a l . m k $ l s
l F c
e /
w
w
y m
i n y m
i n 5 . 7 K S e p 1 2 8 : 4 9 p k g
a n g e . a p p a r m
. m k
w
w
y m
i n y m
i n 2 . 3 K A u g 2 9 1 5 : 4 p k g
a n g e . c
f i g . m k
w
w
y m
i n y m
i n 2 . 3 K A u g 2 9 1 5 : 4 p k g
a n g e . d b u s . m k
w
w
y m
i n y m
i n 6 1 1 A u g 2 9 1 5 : 4 p k g
a n g e . d
. m k
w
w
y m
i n y m
i n 1 . 4 K S e p 8 1 6 : 2 2 p k g
a n g e . l i n k s
i s t . m k
w
w
y m
i n y m
i n 1 . 6 K S e p 7 1 5 : 3 6 p k g
a n g e . m k
w
w
y m
i n y m
i n 5 5 3 A u g 2 9 1 5 : 4 p k g
a n g e . s e r v i c e s . m k
w
w
y m
i n y m
i n 7 8 3 S e p 1 2 8 : 4 9 p k g
a n g e . t r a n s l a t i
. m k
w
w
y m
i n y m
i n 1 . 2 K A u g 2 9 1 5 : 4 p k g
a n g e . u s e r s . m k
w
w
y m
i n y m
i n 2 . 9 K S e p 1 1 1 4 : 2 5 s y s t e m . m k
w
w
y m
i n y m
i n 7 , 5 K S e p 1 1 1 4 : 4 7 s y s t e m
x t r a . m k $ c a t e x t e r n a l . m k i n c l u d e $ ( B R 2 _ E X T E R N A L ) / c
e / p k g
a n g e . m k i n c l u d e $ ( B R 2 _ E X T E R N A L ) / c
e / s y s t e m . m k i n c l u d e $ ( B R 2 _ E X T E R N A L ) / c
e / s y s t e m
x t r a . m k $ c a t c
e / p k g
a n g e . m k # D e f i n e v a r i a b l e s
t i s r e c u r s i v e l y e x p a n d e d
a n g e
a c k a g e = i n c l u d e $ ( B R 2 _ E X T E R N A L ) / c
e / p k g
a n g e . u s e r s . m k i n c l u d e $ ( B R 2 _ E X T E R N A L ) / c
e / p k g
a n g e . d
. m k i n c l u d e $ ( B R 2 _ E X T E R N A L ) / c
e / p k g
a n g e . d b u s . m k i n c l u d e $ ( B R 2 _ E X T E R N A L ) / c
e / p k g
a n g e . t r a n s l a t i
. m k i n c l u d e $ ( B R 2 _ E X T E R N A L ) / c
e / p k g
a n g e . c
f i g . m k i n c l u d e $ ( B R 2 _ E X T E R N A L ) / c
e / p k g
a n g e . s e r v i c e s . m k i n c l u d e $ ( B R 2 _ E X T E R N A L ) / c
e / p k g
a n g e . a p p a r m
. m k i n c l u d e $ ( B R 2 _ E X T E R N A L ) / c
e / p k g
a n g e . l i n k s
i s t . m k
BR2_EXTERNAL - Extra logic
Local customisations
Buildroot as a git submodule New confjgurations New packages New fjlesystems Board fjles Extra logic - Additional infrastructure
19
20
$ c a t c
e / s y s t e m
x t r a . m k # C h e c k t h a t p a c k a g e s c
r e c t l y d e c l a r e t h e i r d e p e n d e n c i e s i f e q ( $ ( C H E C K _ P A C K A G E S ) , ) C H E C K _ P A C K A G E S : =
a n g e
h
t
a n g e
e n d i f c h e c k
e p s : @ p r i n t f " C h e c k i n g p a c k a g e s
e b y
e ; c a n t a k e m a n y h
r s . . . \ n " @ f
p i n $ ( f i l t e r $ ( C H E C K _ P A C K A G E S ) , $ ( P A C K A G E S ) ) ; d
p r i n t f " C h e c k i n g % s . . . " " $ $ { p } " ; \ m a k e
$ ( C O N F I G _ D I R ) c l e a n > / d e v / n u l l 2 > & 1 ; \ i f m a k e
$ ( C O N F I G _ D I R ) " $ $ { p } " > " $ ( C O N F I G _ D I R ) / $ $ { p } . l
" 2 > & 1 ; t h e n \ i f g r e p
' ^ . * C i r c u l a r ( . + <
+ ) d e p e n d e n c y d r
p e d \ . $ $ ' \ " $ ( C O N F I G _ D I R ) / $ $ { p } . l
" > / d e v / n u l l 2 > & 1 ; \ t h e n \ p r i n t f " F A I L E D : c i r c u l a r d e p e n d e n c y \ n " ; \ e l s e \ p r i n t f " O K \ n " ; \ f i ; \ e l s e \ p r i n t f " F A I L E D : m i s s i n g d e p e n d e n c y ? \ n " ; \ f i ; \ d
e
Additional infrastructure – Raw make rules
Raw Makefjle code
Can be anything As long as it does not clash with existing
infrastructure!
Example: check dependencies Ensure packages defjne a valid dependency
chain
But does not catch inherited dependencies
like A -> C when A -> B and B -> C
21
$ c a t c
e / s y s t e m . m k d e f i n e O R A N G E _ T A R G E T _ F I N A L I Z E _ C L E A N _ E L F _ L I B S $ ( B R 2 _ E X T E R N A L ) / t
s / c l e a n
l f
i b s $ ( T A R G E T _ D I R ) $ ( T A R G E T _ R E A D E L F ) e n d e f T A R G E T _ F I N A L I Z E _ H O O K S + = O R A N G E _ T A R G E T _ F I N A L I Z E _ C L E A N _ E L F _ L I B S
Additional infrastructure – target-fjnalize
Elf sanitising
Target-fjnalize hook Remove symlinks to shared libraries Rename shared libraries to their SONAME Offmoad to a helper script (shell, python, perl,
whatever you fancy)
22
$ c a t e x t e r n a l . m k O R A N G E _ G I T _ B A S E _ U R L = g i t
s e r @ f
g e .
a n g e . c
: t h e
r
e c t $ c a t p a c k a g e / a p p l a y e r /
a n g e
i v e /
a n g e
i v e . m k # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # O r a n g e L I V E # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # O R A N G E _ L I V E _ V E R S I O N = 2 7 f c b 7 2 5 d 6 f 3 c a 7 e a 5 d 6 8 4 c f d a f 4 1 2 4 5 1 5 3 7 9 5 5 2 O R A N G E _ L I V E _ S I T E = $ ( O R A N G E _ G I T _ B A S E _ U R L ) / s r c / a p p l a y e r / a p p s / l i v e . g i t O R A N G E _ L I V E _ S I T E _ M E T H O D = g i t O R A N G E _ L I V E _ L I C E N S E = P r
r i e t a r y O R A N G E _ L I V E _ R E D I S T R I B U T E = N O O R A N G E _ L I V E _ I N S T A L L _ S T A G I N G = Y E S O R A N G E _ L I V E _ D E P E N D E N C I E S = \
a n g e
i b d a t a m
e l \
a n g e
i b t r a c e r \
a n g e
a c k e n d s $ ( e v a l $ (
a n g e
a c k a g e ) ) $ ( e v a l $ ( c m a k e
a c k a g e ) )
Additional infrastructure – Basics
Orange package
Is a standard Buildroot package With a little addition Mimicking the existing infrastructures Using declarative statements
23
$ c a t c
e / p k g
a n g e . d
. m k d e f i n e
a n g e
u i l d
n n e r i f e q ( $ $ ( $ ( 1 ) _ H A S _ D O C ) , Y E S ) d e f i n e $ ( 1 ) _ B U I L D _ D O C $ $ ( M A K E )
$ $ ( @ D ) / d
e n d e f $ ( 1 ) _ P O S T _ B U I L D _ H O O K S + = $ ( 1 ) _ B U I L D _ D O C d e f i n e $ ( 1 ) _ I N S T A L L _ D O C $ $ ( M A K E )
$ $ ( @ D ) / d
i n s t a l l D E S T D I R = " $ $ ( B A S E _ D I R ) / d
" e n d e f $ ( 1 ) _ P O S T _ I N S T A L L _ T A R G E T _ H O O K S + = $ ( 1 ) _ I N S T A L L _ D O C e n d i f e n d e f
a n g e
a c k a g e + = $ ( c a l l
a n g e
u i l d
n n e r , $ ( c a l l U P P E R C A S E , $ ( p k g n a m e ) ) ) $ c a t p a c k a g e / a p p l a y e r /
a n g e
i v e /
a n g e
i v e . m k O R A N G E _ L I V E _ V E R S I O N = 2 7 f c b 7 2 5 d 6 f 3 c a 7 e a 5 d 6 8 4 c f d a f 4 1 2 4 5 1 5 3 7 9 5 5 2 [ . . . ] O R A N G E _ L I V E _ H A S _ D O C = Y E S $ ( e v a l $ (
a n g e
a c k a g e ) ) $ ( e v a l $ ( c m a k e
a c k a g e ) )
Additional infrastructure – Documentation
Documentation
Let packages handle the documentation how
they want: static, or generated with doxygen, docstring...
Build the documentation Post-build hook Install the documentation Not a target content, not a staging content,
not an image either…
Divert to a doc/ sub-directory in the standard
Buildroot output directory
Hijack post-install-target hook One macro for each package (bad!) Too-many dollars problem
24
$ c a t c
e / p k g
a n g e . d b u s . m k d e f i n e O R A N G E _ B U I L D _ T R A N S L A T I O N _ F I L E S $ ( H O S T _ D I R ) / u s r / b i n / l r e l e a s e $ ( @ D ) / t r a n s l a t i
s / * . t s e n d e f d e f i n e O R A N G E _ I N S T A L L _ T R A N S L A T I O N _ F I L E S m k d i r
$ ( T A R G E T _ D I R ) / u s r / s h a r e /
a n g e / t r a n s l a t i
s $ ( I N S T A L L )
6 4 4 $ ( @ D ) / t r a n s l a t i
s / * . q m $ ( T A R G E T _ D I R ) / u s r / s h a r e /
a n g e / t r a n s l a t i
s / e n d e f d e f i n e
a n g e
n s t a l l
r a n s l a t i
n n e r i f e q ( $ $ ( B R 2 _ P A C K A G E _ Q T 5 T O O L S _ L I N G U I S T _ T O O L S ) , y ) i f e q ( $ $ ( $ ( 1 ) _ H A S _ T R A N S L A T I O N S ) , Y E S ) $ ( 1 ) _ P O S T _ B U I L D _ H O O K S + = O R A N G E _ B U I L D _ T R A N S L A T I O N _ F I L E S $ ( 1 ) _ P O S T _ I N S T A L L _ T A R G E T _ H O O K S + = O R A N G E _ I N S T A L L _ T R A N S L A T I O N _ F I L E S $ ( 1 ) _ D E P E N D E N C I E S + = q t 5 t
s e n d i f e n d i f e n d e f
a n g e
a c k a g e + = $ ( c a l l
a n g e
n s t a l l
r a n s l a t i
n n e r , $ ( c a l l U P P E R C A S E , $ ( p k g n a m e ) ) ) $ c a t p a c k a g e / a p p l a y e r /
a n g e
i v e /
a n g e
i v e . m k O R A N G E _ L I V E _ V E R S I O N = 2 7 f c b 7 2 5 d 6 f 3 c a 7 e a 5 d 6 8 4 c f d a f 4 1 2 4 5 1 5 3 7 9 5 5 2 [ . . . ] O R A N G E _ L I V E _ H A S _ T R A N S L A T I O N S = Y E S $ ( e v a l $ (
a n g e
a c k a g e ) ) $ ( e v a l $ ( c m a k e
a c k a g e ) )
Additional infrastructure – Translations
All based on Qt
Build translation as post-build hook Install translations as post-install-target hook Automatically inherit dependency on qt5tools Translations disabled if qt5tools-linguist not
enabled
One macro for all packages (good!)
25
$ c a t c
e / p k g
a n g e . c
f i g . m k d e f i n e O R A N G E _ I N S T A L L _ C O N F I G _ F I L E S $ ( f
e a c h c f g , $ ( $ ( P K G ) _ C O N F I G _ F I L E S ) , \ $ ( I N S T A L L )
6 4 4
$ ( $ ( P K G ) _ P K G D I R ) / c
f i g / $ ( c f g ) \ $ ( T A R G E T _ D I R ) / u s r / s h a r e /
a n g e / $ ( c f g ) $ ( s e p ) ) e n d e f d e f i n e
a n g e
n s t a l l
f i g
n n e r $ ( 1 ) _ P O S T _ I N S T A L L _ T A R G E T _ H O O K S + = O R A N G E _ I N S T A L L _ C O N F I G _ F I L E S e n d e f
a n g e
a c k a g e + = $ ( c a l l
a n g e
n s t a l l
f i g
n n e r , $ ( c a l l U P P E R C A S E , $ ( p k g n a m e ) ) ) $ c a t p a c k a g e / a p p l a y e r /
a n g e
i v e /
a n g e
i v e . m k O R A N G E _ L I V E _ V E R S I O N = 2 7 f c b 7 2 5 d 6 f 3 c a 7 e a 5 d 6 8 4 c f d a f 4 1 2 4 5 1 5 3 7 9 5 5 2 [ . . . ] O R A N G E _ L I V E _ C O N F I G _ F I L E S = \ a p p l i c a t i
m a n a g e r / a p p s / l i v e . j s
\
w i n / l i v e . j s
O R A N G E _ L I V E _ I N I T _ A P P L I C A T I O N S = l i v e $ ( e v a l $ (
a n g e
a c k a g e ) ) $ ( e v a l $ ( c m a k e
a c k a g e ) ) $ c a t p a c k a g e / a p p l a y e r / s e r v i c e s /
a n g e
e c
d /
a n g e
e c
d . m k O R A N G E _ R E C O R D _ V E R S I O N = a 9 5 4 1 f 7 7 6 4 d 4 2 3 9 7 d c f e 3 3 d e c 4 c 2 3 e e 7 3 7 3 4 c [ . . . ] O R A N G E _ R E C O R D _ I N I T _ S E R V I C E S = r e c
d O R A N G E _ R E C O R D _ I N I T _ S Y S T E M = r e c
d
a c k e n d
Additional infrastructure – Confjguration 1/2
Confjguration fjles
Needed at runtime Simple copy as a post-install-target hook
26
$ c a t c
e / p k g
a n g e . s e r v i c e s . m k d e f i n e
a n g e
n i t
e r v i c e s
n n e r O R A N G E _ I N I T _ S Y S T E M + = $ $ ( i f $ $ ( $ $ ( $ ( 1 ) _ K C O N F I G _ V A R ) ) , $ $ ( $ ( 1 ) _ I N I T _ S Y S T E M ) ) O R A N G E _ I N I T _ S E R V I C E S + = $ $ ( i f $ $ ( $ $ ( $ ( 1 ) _ K C O N F I G _ V A R ) ) , $ $ ( $ ( 1 ) _ I N I T _ S E R V I C E S ) ) O R A N G E _ I N I T _ A P P L I C A T I O N S + = $ $ ( i f $ $ ( $ $ ( $ ( 1 ) _ K C O N F I G _ V A R ) ) , $ $ ( $ ( 1 ) _ I N I T _ A P P L I C A T I O N S ) ) e n d e f
a n g e
a c k a g e + = $ ( c a l l
a n g e
n i t
e r v i c e s
n n e r , $ ( c a l l U P P E R C A S E , $ ( p k g n a m e ) ) ) $ c a t p a c k a g e / a p p l a y e r /
a n g e
p p e n v /
a n g e
p p e n v . m k O R A N G E _ A P P E N V _ V E R S I O N = 2 6 6 a 8 2 4 3 e 8 5 d 9 d f 6 b 8 6 9 4 f 7 3 e 4 8 8 b e d e c 4 7 1 9 e 4 9 [ . . . ] d e f i n e O R A N G E _ A P P E N V _ I N I T _ S E R V I C E S _ J S O N ( \ e c h
{ ' ; \ e c h
" s y s t e m " : [ ' ; \ ( : ; $ ( f
e a c h s , $ ( O R A N G E _ I N I T _ S Y S T E M ) , e c h
" $ ( s ) " , ' ; ) ) | s e d
' $ $ s / , $ $ / / ' ; \ e c h
] , ' ; \ e c h
" s e r v i c e s " : [ ' ; \ ( : ; $ ( f
e a c h s , $ ( O R A N G E _ I N I T _ S E R V I C E S ) , e c h
" $ ( s ) " , ' ; ) ) | s e d
' $ $ s / , $ $ / / ' ; \ e c h
] , ' ; \ e c h
" a p p l i c a t i
s " : [ ' ; \ ( : ; $ ( f
e a c h s , $ ( O R A N G E _ I N I T _ A P P L I C A T I O N S ) , e c h
" $ ( s ) " , ' ; ) ) | s e d
' $ $ s / , $ $ / / ' ; \ e c h
] ' ; \ e c h
} ' ; \ ) > $ ( @ D ) / i n i t
e r v i c e s . j s
e n d e f O R A N G E _ A P P E N V _ P O S T _ B U I L D _ H O O K S + = O R A N G E _ A P P E N V _ I N I T _ S E R V I C E S _ J S O N d e f i n e O R A N G E _ A P P E N V _ I N S T A L L _ I N I T _ S E R V I C E S _ J S O N $ ( I N S T A L L )
6 4 4 $ ( @ D ) / i n i t
e r v i c e s . j s
\ $ ( T A R G E T _ D I R ) / u s r / s h a r e /
a n g e / i n i t
e r v i c e s . j s
e n d e f O R A N G E _ A P P E N V _ P O S T _ I N S T A L L _ T A R G E T _ H O O K S + = O R A N G E _ A P P E N V _ I N S T A L L _ I N I T _ S E R V I C E S _ J S O N $ ( e v a l $ (
a n g e
a c k a g e ) ) $ ( e v a l $ ( c m a k e
a c k a g e ) )
Additional infrastructure – Confjguration 2/2
Registering applications
Variables expanded in a post-hook Not all packages already parsed Evaluated later Four types of applications System, started earliest Services, started early Init, started last Standard, started on-demand
27
$ c a t c
e / p k g
a n g e . u s e r s . m k d e f i n e
a n g e
s e r s
n n e r i f n e q ( $ $ ( $ ( 1 ) _ U S E R S ) , ) $ ( 1 ) _ U S E R = $ $ ( w
d 1 , $ $ ( $ ( 1 ) _ U S E R S ) ) $ ( 1 ) _ U I D = $ $ ( w
d 2 , $ $ ( $ ( 1 ) _ U S E R S ) ) $ ( 1 ) _ G I D = $ $ ( w
d 4 , $ $ ( $ ( 1 ) _ U S E R S ) ) i f e q ( $ $ ( $ ( 1 ) _ U I D ) ,
) # a l l
g r
p
l y d e f i n i t i
i f n e q ( $ $ ( $ ( 1 ) _ U S E R ) ,
$ $ ( e r r
U I D f
$ ( p k g n a m e ) m u s t b e e x p l i c i t ) e n d i f e n d i f i f e q ( $ $ ( $ ( 1 ) _ G I D ) ,
) $ $ ( e r r
G I D f
$ ( p k g n a m e ) m u s t b e e x p l i c i t ) e n d i f O R A N G E _ A L L _ U S E R S + = $ $ ( $ ( 1 ) _ U S E R S ) e n d i f e n d e f
a n g e
a c k a g e + = $ ( c a l l
a n g e
s e r s
n n e r , $ ( c a l l U P P E R C A S E , $ ( p k g n a m e ) ) ) $ c a t p a c k a g e / a p p l a y e r /
a n g e
i v e /
a n g e
i v e . m k O R A N G E _ L I V E _ V E R S I O N = 2 7 f c b 7 2 5 d 6 f 3 c a 7 e a 5 d 6 8 4 c f d a f 4 1 2 4 5 1 5 3 7 9 5 5 2 [ . . . ] d e f i n e O R A N G E _ L I V E _ U S E R S l i v e 1 4 2 l i v e 1 4 2 *
a n g e O r a n g e L i v e a p p l i c a t i
e n d e f $ ( e v a l $ (
a n g e
a c k a g e ) ) $ ( e v a l $ ( c m a k e
a c k a g e ) )
Additional infrastructure – Users
Defjning users
Standard Buildroot mechanism With additional checks for validity Work on variables Known at time of parsing
28
$ c a t c
e / p k g
a n g e . d b u s . m k d e f i n e
a n g e
b u s
n n e r # I n c l u d e i n t e r f a c e s d e f i n e d b y
r d e p e n d e n c i e s $ ( 1 ) _ I N H E R I T E D _ D B U S _ I N T F = $ $ ( s t r i p \ $ $ ( f
e a c h d , $ $ ( $ ( 1 ) _ D E P E N D E N C I E S ) , \ $ $ ( $ $ ( c a l l U P P E R C A S E , $ $ ( d ) ) _ E X P O R T _ D B U S _ I N T F ) ) ) i f e q ( $ $ ( $ ( 1 ) _ H A S _ S Y S T E M _ D B U S _ A P I ) , Y E S ) # E v e n i f j u s t f
t h e d b u s c
f i g f i l e s . . . $ ( 1 ) _ I N S T A L L _ S T A G I N G = Y E S $ ( 1 ) _ E X T R A _ D B U S _ I N T F + = $ $ ( $ ( 1 ) _ I N H E R I T E D _ D B U S _ I N T F ) d e f i n e $ ( 1 ) _ G E N _ D B U S _ C O N F I G $ ( B R 2 _ E X T E R N A L ) / t
s / g e n
b u s
f \
$ $ ( @ D ) \
$ $ ( $ ( 1 ) _ U I D ) \ $ $ ( f
e a c h a p i , $ $ ( $ ( 1 ) _ E X T R A _ D B U S _ I N T F ) ,
$ $ ( a p i ) ) \
$ ( @ D ) / s y s t e m
b u s . c
f e n d e f $ ( 1 ) _ P O S T _ B U I L D _ H O O K S + = $ ( 1 ) _ G E N _ D B U S _ C O N F I G d e f i n e $ ( 1 ) _ I N S T A L L _ D B U S _ C O N F I G $ $ ( I N S T A L L )
6 4 4
$ $ ( @ D ) / s y s t e m
b u s . c
f \ $ $ ( S T A G I N G _ D I R ) / e t c / d b u s
/ s y s t e m . d /
a n g e
$ ( $ ( 1 ) _ U S E R ) . c
f e n d e f $ ( 1 ) _ P O S T _ I N S T A L L _ S T A G I N G _ H O O K S + = $ ( 1 ) _ I N S T A L L _ D B U S _ C O N F I G O R A N G E _ S Y S T E M _ D B U S _ P A C K A G E S + = $ $ ( i f $ $ ( $ $ ( $ ( 1 ) _ K C O N F I G _ V A R ) ) , $ ( 1 ) ) e n d i f e n d e f
a n g e
a c k a g e + = $ ( c a l l
a n g e
b u s
n n e r , $ ( c a l l U P P E R C A S E , $ ( p k g n a m e ) ) )
Additional infrastructure – D-Bus 1/2
D-Bus authorisation
D-Bus APIs installed as XML descriptions Generate code with qdbusxml2cpp Use D-Bus via QDBus Well-known #include <ComOrangeAPIxxx.h> Can scan code for those patterns Post-build hook Two buses Post-target-install hook for session bus Post-staging-install hook for system bus
29
$ c a t p a c k a g e / a p p l a y e r /
a n g e
i b d a t a m
e l /
a n g e
i b d a t a m
e l . m k O R A N G E _ L I B D A T A M O D E L _ V E R S I O N = 1 6 5 1 7 9 e 7 a c b e 7 a 5 6 e e 7 f b 8 c 6 6 b 3 9 2 2 8 e d 2 5 7 1 b d [ . . . ] O R A N G E _ L I B D A T A M O D E L _ E X P O R T _ D B U S _ I N T F = c
.
a n g e . d a t a m
e l $ ( e v a l $ (
a n g e
a c k a g e ) ) $ ( e v a l $ ( c m a k e
a c k a g e ) ) $ c a t p a c k a g e / a p p l a y e r /
a n g e
i v e /
a n g e
i v e . m k O R A N G E _ L I V E _ V E R S I O N = 2 7 f c b 7 2 5 d 6 f 3 c a 7 e a 5 d 6 8 4 c f d a f 4 1 2 4 5 1 5 3 7 9 5 5 2 [ . . . ] O R A N G E _ L I V E _ D E P E N D E N C I E S =
a n g e
i b d a t a m
e l O R A N G E _ L I V E _ H A S _ S Y S T E M _ D B U S _ A P I = Y E S O R A N G E _ L I V E _ E X T R A _ D B U S _ I N T F = c
.
a n g e . p v r $ ( e v a l $ (
a n g e
a c k a g e ) ) $ ( e v a l $ ( c m a k e
a c k a g e ) )
Additional infrastructure – D-Bus 2/2
D-Bus authorisation
Inherit interfaces exported by dependencies Provide extra, non-QDbus APIs When two applications want to talk to each
30
$ c a t p a c k a g e / a p p l a y e r /
a n g e
t s /
a n g e
t s . m k O R A N G E _ F O N T S _ V E R S I O N = 8 5 f f 7 5 1 c 5 4 6 1 6 6 8 9 6 e 8 9 9 c 8 8 c 2 2 3 4 a b a a 6 8 4 e b b [ . . . ] O R A N G E _ F O N T S _ A P P A R M O R _ D A T A = Y E S $ c a t p a c k a g e / a p p l a y e r /
a n g e
t s / a p p a r m
/
a n g e
t s . a a / u s r / s h a r e / f
t s / * . t t f r , $ c a t p a c k a g e / a p p l a y e r /
a n g e
i b d a t a m
e l /
a n g e
i b d a t a m
e l . m k O R A N G E _ L I B D A T A M O D E L _ V E R S I O N = 1 6 5 1 7 9 e 7 a c b e 7 a 5 6 e e 7 f b 8 c 6 6 b 3 9 2 2 8 e d 2 5 7 1 b d [ . . . ] O R A N G E _ L I B D A T A M O D E L _ A P P A R M O R _ L I B S = / u s r / l i b / l i b d a t a m
e l . s
c a t p a c k a g e / a p p l a y e r /
a n g e
i b d a t a m
e l / a p p a r m
/ l i b d a t a m
e l . s
a a / e t c /
a n g e / d a t a m
e l . c f g r , / r u n / d a t a m
e l . s
k e t r w , $ c a t p a c k a g e / a p p l a y e r /
a n g e
i v e /
a n g e
i v e . m k O R A N G E _ L I V E _ V E R S I O N = 2 7 f c b 7 2 5 d 6 f 3 c a 7 e a 5 d 6 8 4 c f d a f 4 1 2 4 5 1 5 3 7 9 5 5 2 [ . . . ] O R A N G E _ L I V E _ D E P E N D E N C I E S =
a n g e
i b d a t a m
e l O R A N G E _ L I V E _ D A T A _ D E P E N D E N C I E S =
a n g e
t s O R A N G E _ L I V E _ A P P A R M O R _ E X E C S = / u s r / b i n / l i v e $ c a t p a c k a g e / a p p l a y e r /
a n g e
i v e / a p p a r m
/ l i v e . a a / e t c /
a n g e / l i v e . c f g r , / v a r / d a t a / l i v e . d b r w k ,
Additional infrastructure – AppArmor 1/3
AppArmor profjles
Tedious task Bit-rot But: security! Automatically generate profjles Functionality-centric Developer-provided Build- and data-dependencies tracking Code-scanning
31
$ c a t c
e / p k g
a n g e . a p p a r m
. m k d e f i n e O R A N G E _ A P P A R M O R _ I N S T A L L _ E X E C S _ P R O F I L E S $ ( f
e a c h x , $ ( $ ( P K G ) _ A P P A R M O R _ E X E C S ) , \ $ ( I N S T A L L )
6 4 4 \ $ ( $ ( P K G ) _ P K G D I R ) / a p p a r m
/ $ ( n
d i r $ ( x ) ) . a a \ $ ( T A R G E T _ D I R ) / $ ( x ) . a a ) e n d e f d e f i n e O R A N G E _ A P P A R M O R _ I N S T A L L _ L I B S _ P R O F I L E S $ ( f
e a c h l , $ ( $ ( P K G ) _ A P P A R M O R _ L I B S ) , \ $ ( I N S T A L L )
6 4 4 \ $ ( $ ( P K G ) _ P K G D I R ) / a p p a r m
/ $ ( n
d i r $ ( l ) ) . a a \ $ ( S T A G I N G _ D I R ) / $ ( l ) . a a ) e n d e f d e f i n e O R A N G E _ A P P A R M O R _ I N S T A L L _ D A T A _ P R O F I L E S $ ( I N S T A L L )
6 4 4 $ ( $ ( P K G ) _ P K G D I R ) / a p p a r m
/ $ ( $ ( P K G ) _ R A W N A M E ) . a a \ $ ( S T A G I N G _ D I R ) / u s r / l i b / a p p a r m
a t a . d / $ ( $ ( P K G ) _ R A W N A M E ) . a a e n d e f d e f i n e
a n g e
p p a r m
n n e r $ ( 1 ) _ P O S T _ I N S T A L L _ T A R G E T _ H O O K S + = \ O R A N G E _ A P P A R M O R _ I N S T A L L _ E X E C S _ P R O F I L E S $ ( 1 ) _ P O S T _ I N S T A L L _ S T A G I N G _ H O O K S + = \ O R A N G E _ A P P A R M O R _ I N S T A L L _ L I B S _ P R O F I L E S \ O R A N G E _ A P P A R M O R _ I N S T A L L _ D A T A _ P R O F I L E S O R A N G E _ A P P A R M O R _ E X E C S + = $ $ ( i f $ $ ( $ $ ( $ ( 1 ) _ K C O N F I G _ V A R ) ) , $ $ ( $ ( 1 ) _ A P P A R M O R _ E X E C S ) ) e n d e f
a n g e
a c k a g e + = $ ( c a l l
a n g e
p p a r m
n n e r , $ ( c a l l U P P E R C A S E , $ ( p k g n a m e ) ) , $ ( p k g n a m e ) )
Additional infrastructure – AppArmor 2/3
AppArmor profjles
Post-install-staging hook Install libraries snippets Install data snippets Post-install-target hook Install ELF executable snippets Register all executables Data may not already be installed Can only be scanned at target-fjnalize Handles static data Translations ...
32
$ c a t c
e / p k g
a n g e . a p p a r m
. m k d e f i n e O R A N G E _ A P P A R M O R _ G E N _ E X E C S _ P R O F I L E S $ ( f
e a c h x , $ ( O R A N G E _ A P P A R M O R _ E X E C S ) , \ $ ( Q ) $ ( A A _ S C A N ) \
i r $ ( T A R G E T _ D I R ) \
t a g i n g
i r $ ( S T A G I N G _ D I R ) \
t p u t
i l e $ ( S T A G I N G _ D I R ) / e t c / a p p a r m
. d /
a n g e $ ( s u b s t / , . , $ ( x ) ) \
l f
e l f
e a d \
l f
r
s
r e f i x $ ( T A R G E T _ C R O S S ) \ $ ( x ) ) e n d e f O R A N G E _ P A C K A G E _ T A R G E T _ F I N A L I Z E _ H O O K S + = O R A N G E _ A P P A R M O R _ G E N _ E X E C S _ P R O F I L E S
Additional infrastructure – AppArmor 3/3
AppArmor profjles
Target-fjnalize hook Scan ELF executables What about scripts?
33
$ c a t b
r d s / c
m
/ p
t
u i l d . s h # ! / b i n / b a s h T A R G E T _ D I R = " $ { 1 } " # S t
e g i t i n f
n a n F H S
p l i a n t v e r s i
f i l e V E R S I O N = " $ ( c d " $ { B R 2 _ E X T E R N A L } " ; g i t d e s c r i b e
b b r e v = 6 4
i r t y 2 > / d e v / n u l l ) " V E R _ I D = " $ ( c d " $ { B R 2 _ E X T E R N A L } " ; g i t d e s c r i b e
b b r e v = 2 > / d e v / n u l l ) " c a t < <
E O F _ > " $ { T A R G E T _ D I R } " / e t c /
e l e a s e N A M E = '
a n g e ' V E R S I O N = ' $ { V E R S I O N } ' I D = '
a n g e ' V E R S I O N _ I D = ' $ { V E R _ I D } ' P R E T T Y _ N A M E = ' O r a n g e ' _ E O F _ # D B u s i n t e r f a c e x m l f i l e s u s e l e s s a t r u n t i m e r m
f " $ { T A R G E T _ D I R } " / u s r / s h a r e / d b u s
/ i n t e r f a c e s # C l e a n u p t h e . e m p t y p l a c e h
d e r s f i n d " $ { T A R G E T _ D I R } "
y p e f
a m e . e m p t y
e l e t e $ c a t b
r d s / c
m
/ p
t
u i l d
e s t s . s h # ! / b i n / b a s h T A R G E T _ D I R = " $ { 1 } " # O p e n D B u s
T C P f
t e s t s i f ! g r e p ' , p
t = 5 5 5 5 4 , ' " $ { T A R G E T _ D I R } " / e t c / d b u s
/
a n g e . c
f > / d e v / n u l l ; t h e n s e d _ s u b s t = ' < l i s t e n > ' s e d _ s u b s t + = ' t c p : h
t = l
a l h
t , b i n d = \ * , p
t = 5 5 5 5 4 , f a m i l y = i p v 4 ' s e d _ s u b s t + = ' < \ / l i s t e n > ' s e d
' s / \ ( . * < l i s t e n > . * < \ / l i s t e n > \ ) / ' " $ { s e d _ s u b s t } " ' \ n \ 1 / ' \ " $ { T A R G E T _ D I R } " / e t c / d b u s
/
a n g e . c
f f i
Additional infrastructure – post-build
Post-build scripts
Not part of a package Generate blurbs Remove fjles Conditional tweaks
34
35
Conclusion
Infrastructure
Automatic Systematic Reproducible Maintainable Extendable
Whish-list
?
Yann E. MORIN <yann.morin@orange.com> Embedded Linux Conference Europe 2017
Questions?