ALS 2016 - Jan-Simon Mller Contiguous Integration and Testing for - - PowerPoint PPT Presentation

▶
als 2016 jan simon m ller
SMART_READER_LITE
LIVE PREVIEW

ALS 2016 - Jan-Simon Mller Contiguous Integration and Testing for - - PowerPoint PPT Presentation

Aug 16, 2023 32 likes •262 views

ALS 2016 - Jan-Simon Mller Contiguous Integration and Testing for Automotive in a distributed environment AGL development ... AGL development is done - like for many other opensource projects - in a distributed way Developers around

slide-1
SLIDE 1

ALS 2016 - Jan-Simon Möller

Contiguous Integration and Testing for Automotive in a distributed environment

slide-2
SLIDE 2

AGL development ...

  • AGL development is done - like for many other opensource

projects - in a distributed way

  • Developers around the globe contribute to the sources and

recipes that make up AGL

  • Code review through gerrit, CI builds and so on
slide-3
SLIDE 3

Problem statement

  • BUT: we cannot do tests on HW

in the same distributed way ...

  • ... yet ? Why and how ?

Image: public domain

slide-4
SLIDE 4

Tests on HW are hard !?

  • You need the HW
  • You need it on your desk/in your lab
  • You need to deploy firmware/filesystems
  • You need to reboot the board
  • You need to initiate the test
  • You need to collect the results
  • You need to interpret the results

... rinse & repeat

Images: elinux.org & dl9pf

slide-5
SLIDE 5

Where we'll struggle...

  • Board not available on your desk
  • Board cannot be shipped due to xyz
  • Board hardwired on your desk while travelling
  • Juggling SD-cards takes a lot of time
  • It takes a lot of time

vs.

  • We want the info early - for each proposed patch
slide-6
SLIDE 6

Issues:

  • Lots of manual work involved
  • does not scale

– * boards * build-variants * releases

  • does not integrate with code review systems
  • release management and

maintenance depends on testresults

And now press:

slide-7
SLIDE 7

What do we need ... and when ?

  • We won't replace proper manual/long test runs for

the releases!

  • But for our daily work, we need:

– fast results – as much test-coverage as possible in short time – large number of boards tested – remote / distributed operation

  • => we need to keep pace with the development
slide-8
SLIDE 8

What do we need ... and when ?

  • panacea ?
  • jack of all

trades device ?

https://xkcd.com/763/

slide-9
SLIDE 9

Prior art (opensource)

  • Linux

– ktest.pl (S.Rostedt, http://elinux.org/Ktest) – 0-day builder (Fengguang Wu, Intel, https://01.org/lkp)

  • https://kernelci.org
  • lava (https://validation.linaro.org/static/docs )
  • JTA/fuego (http://elinux.org/Fuego)
  • openQA (https://openqa.opensuse.org/)
slide-10
SLIDE 10

AGLs current infrastructure

  • Gerrit for code review

– git.automotivelinux.org

  • Jenkins with gerrit-trigger for CI builds

– build.automotivelinux.org

  • => pretty much standard toolset
slide-11
SLIDE 11

Beta-test for new additions ...

  • AGL-JTA (fork of fuego)

– jta.agl.homelinux.org – = tool to compile and run predefined testcases

  • Lava

– porter.agl.homelinux.org – = board farm management and deployment framework

slide-12
SLIDE 12

Why JTA ?

  • AGL-JTA is AGL's version of fuego (formerly JTA)

– Has large number of Benchmarks and Functional tests – Visualization and reporting – JTA/fuego is part of the LTSI initiative – AGL modified/updated components to fit our needs – Updated to stable jenkins – Dedicated to upstreaming changes

slide-13
SLIDE 13

Demo JTA

  • Instance with up as public beta:
  • https://jta.automotivelinux.org
slide-14
SLIDE 14

Why Lava

  • Lava supports

– multiple boards (of same type / pool) – different boards at the same time – very good automation capabilities – remote RPC API – master/slave model – distributed slave sites in development – some tests built-in through testdefinitions in git

slide-15
SLIDE 15

Demo Lava

  • Instance with porter board up as public beta:
  • https://porter.automotivelinux.org
slide-16
SLIDE 16

CI builds

Workflow / Feedback loop

Gerrit Jenkins (Build) AGL-JTA (jta) Lava (porter)

slide-17
SLIDE 17

A Vision/Plan (short/mid)

Gerrit Jenkins (Build) AGL-JTA (jta) Lava (lava) Remote Lab Remote Lab SPDX Fossology Extended tests, customized tests

slide-18
SLIDE 18

A Vision/Plan (mid/long)

Gerrit Jenkins (Build) AGL-JTA (jta) Lava (lava) Remote Board Lab Remote Board Lab

  • penQA ?

← ← ← ← UI testing ? SDK Test & Debug App

  • n board

remotely SPDX Fossology VM

slide-19
SLIDE 19

Open Issues

  • present testresults in a unified way
  • aggregate testresults (across revisions/variants/boards)
  • visualize testresults (e.g. trends)
  • Thoughts ?
slide-20
SLIDE 20

Todo's for AGL-JTA/fuego (IMHO)

  • Further simplify installation
  • Use latest version of jenkins (→ less UI modifications)
  • Do not hardcode version of jenkins, use release from repository
  • Do not assume JTA and board are „in the same room“ or on the same network
  • Do not hardcode IPs and board specifics – use an abstraction )*
  • Do not rebuild if not necessary
  • )* Maybe abstract the actual board through a slave daemon like WIP in lava
slide-21
SLIDE 21

Todo's for LAVA (IMHO)

  • Installation on debian works meanwhile, but configuration

is still complex

  • Documentation is good, but we desperately need a

„getting-started in 30min“ or guided installation.

  • UI ease-of-use (less clicks)
  • more cmdline tools (like lava-boot) for developers
slide-22
SLIDE 22

A Roadmap

  • For BB:

– public beta of JTA and Lava

  • For CC:

– All reference boards/platforms included – Remote lab support – Fossology/SPDX reports

  • For DD:

– SDK integration ?!

slide-23
SLIDE 23

Questions ?

  • Q/A
  • Join

– automotive-discussions ML – bi-weekly AGL EG-CIAT meetings – #automotive on freenode