Sylvestre Ledru sylvestre@debian.org Lo Cavaill - - PowerPoint PPT Presentation

▶
sylvestre ledru sylvestre debian org l o cavaill leo
SMART_READER_LITE
LIVE PREVIEW

Sylvestre Ledru sylvestre@debian.org Lo Cavaill - - PowerPoint PPT Presentation

Aug 31, 2023 311 likes •571 views

Sylvestre Ledru sylvestre@debian.org Lo Cavaill leo+debian@cavaille.net Debian + 20 000 source packages ~13 architectures 3 kernels The biggest database of FLOSS code (?) The Debile Project January, 19th 2014 Sylvestre Ledru

slide-1
SLIDE 1

Sylvestre Ledru – sylvestre@debian.org Léo Cavaillé – leo+debian@cavaille.net

slide-2
SLIDE 2

January, 19th 2014 The Debile Project Sylvestre Ledru & Léo Cavaillé

Debian + 20 000 source packages ~13 architectures 3 kernels The biggest database of FLOSS code (?)

slide-3
SLIDE 3

January, 19th 2014 The Debile Project Sylvestre Ledru & Léo Cavaillé

How to:

  • Build all these packages with a new compiler
  • Run static analyzers on their codes
  • Provide reports at the same place for both

upstreams and packagers

slide-4
SLIDE 4

January, 19th 2014 The Debile Project Sylvestre Ledru & Léo Cavaillé

For now, good luck with that !

slide-5
SLIDE 5

January, 19th 2014 The Debile Project Sylvestre Ledru & Léo Cavaillé

Started as a requirement for the clang transparent builds and scan-build static analyzers (Léo Cavaillé's GsoC 2013) Not trying to redevelop yet a new build system Also related to the work presented by Zack this afternoon on sources.debian.net

slide-6
SLIDE 6

January, 19th 2014 The Debile Project Sylvestre Ledru & Léo Cavaillé

To build packages in Debian:

  • wanna-build / buildd (the official services)

http://buildd.debian.org/

  • http://www.debian-ports.org/ (also based on

wanna-build / buildd)

  • Rebuildd
  • Open Build Service
  • ...
slide-7
SLIDE 7

January, 19th 2014 The Debile Project Sylvestre Ledru & Léo Cavaillé

Decided to work on debuild.me with Paul Tagliamonte Python + flask + sqlalchemy Goal : easy to hack / lightweight / expendable Pluggable in the fedmsg bus

slide-8
SLIDE 8

January, 19th 2014 The Debile Project Sylvestre Ledru & Léo Cavaillé

Renamed to Debile After propositions by the DPL cabals (Lucas + Zack)

slide-9
SLIDE 9

January, 19th 2014 The Debile Project Sylvestre Ledru & Léo Cavaillé

Architecture Infrastructure

slide-10
SLIDE 10

January, 19th 2014 The Debile Project Sylvestre Ledru & Léo Cavaillé

Basic architecture

PostgreSQL database

Image credits : Database designed by Stefan Parnarov, Server designed by aLf from the Noun Project

debile master

incoming/

debile slave farm debile-web firewoes signed source packages

  • One slave = a docker instance of debile-slave
  • A slave executes one job (build, static analysis…)

at a time on binaries/sources.

  • Build jobs upload their results to incoming/

flask SQLalchemy python python

slide-11
SLIDE 11

January, 19th 2014 The Debile Project Sylvestre Ledru & Léo Cavaillé

Devices : Master node (~ 8 To of HD) M1000e with : 4 blades (8 cores, 16G) 12 blades (4 cores, 16G)

slide-12
SLIDE 12

January, 19th 2014 The Debile Project Sylvestre Ledru & Léo Cavaillé

Plus : 7 various powerful Dell servers Thanks to Google and IRILL Hosted by IRILL on Renater

slide-13
SLIDE 13

January, 19th 2014 The Debile Project Sylvestre Ledru & Léo Cavaillé

  • Build nodes will run Debian jessie
  • Installation done through PXE
  • Configuration done with Salt Stack
slide-14
SLIDE 14

January, 19th 2014 The Debile Project Sylvestre Ledru & Léo Cavaillé

Results

slide-15
SLIDE 15

January, 19th 2014 The Debile Project Sylvestre Ledru & Léo Cavaillé

Provides build workers:

  • Normal (ie gcc) builds
  • Clang
  • gcc snapshot ?

With repository of clang-built packages Uploads binaries to debile's own APT repository

slide-16
SLIDE 16

January, 19th 2014 The Debile Project Sylvestre Ledru & Léo Cavaillé

Provides analyzer workers:

  • Scan-build (C, C++, Objective-C)
  • Lintian (Debian packages)
  • Coccinelle (C, C++)
  • Cppcheck (C++)
  • Findbugs (Java)
  • ...
slide-17
SLIDE 17

January, 19th 2014 The Debile Project Sylvestre Ledru & Léo Cavaillé

slide-18
SLIDE 18

January, 19th 2014 The Debile Project Sylvestre Ledru & Léo Cavaillé

slide-19
SLIDE 19

January, 19th 2014 The Debile Project Sylvestre Ledru & Léo Cavaillé

slide-20
SLIDE 20

January, 19th 2014 The Debile Project Sylvestre Ledru & Léo Cavaillé

Next steps

slide-21
SLIDE 21

January, 19th 2014 The Debile Project Sylvestre Ledru & Léo Cavaillé

Next steps :

  • Finalize the Debian packages
  • Deploy on the new servers
  • Relaunch all the workers on the whole Debian

Archive

  • Send the reports to a visualisation interface

based on Firehose, a Fedora XML format On going work with Zack & Matthieu

slide-22
SLIDE 22

January, 19th 2014 The Debile Project Sylvestre Ledru & Léo Cavaillé

Future

slide-23
SLIDE 23

January, 19th 2014 The Debile Project Sylvestre Ledru & Léo Cavaillé

TODO in Debile v2.0:

  • Debian PPA
  • Support others archs (example : arm)
  • Plug new analyzers (flake8, etc)
  • Ship the slaves as Docker instances
  • Reverse dependencies rebuild
  • Make it an official service
slide-24
SLIDE 24

January, 19th 2014 The Debile Project Sylvestre Ledru & Léo Cavaillé

Want to contribute ?

https://alioth.debian.org/projects/pkg-debile/ http://anonscm.debian.org/gitweb/?p=pkg-debile/debile.git;a=summary http://anonscm.debian.org/gitweb/?p=pkg-debile/debile-web.git;a=summary irc://irc.oftc.net/#debile

slide-25
SLIDE 25

January, 19th 2014 The Debile Project Sylvestre Ledru & Léo Cavaillé

Any questions ? Remarks ?