Keeping it Green: Integrated QA with the Yocto Project Paul - - PowerPoint PPT Presentation

▶
keeping it green integrated qa with the yocto project
SMART_READER_LITE
LIVE PREVIEW

Keeping it Green: Integrated QA with the Yocto Project Paul - - PowerPoint PPT Presentation

Aug 02, 2023 400 likes •689 views

Keeping it Green: Integrated QA with the Yocto Project Paul Eggleton Intel Open Source Technology Centre ELCE 2013 Edinburgh 24 Oct 2013 2/27 Yocto Project | The Linux Foundation 3/27 Yocto Project | The Linux Foundation 4/27 Yocto

slide-1
SLIDE 1

Keeping it Green: Integrated QA with the Yocto Project

Paul Eggleton Intel Open Source Technology Centre ELCE 2013 ● Edinburgh ● 24 Oct 2013

slide-2
SLIDE 2

Yocto Project | The Linux Foundation 2/27

slide-3
SLIDE 3

Yocto Project | The Linux Foundation 3/27

slide-4
SLIDE 4

Yocto Project | The Linux Foundation 4/27

slide-5
SLIDE 5

Yocto Project | The Linux Foundation 5/27

?

slide-6
SLIDE 6

Yocto Project | The Linux Foundation 6/27

slide-7
SLIDE 7

Yocto Project | The Linux Foundation 7/27

Yocto Project

  • Extend and improve the core of OpenEmbedded (OE-Core)
  • Develop additional tools around it
  • Releases on a 6 month cadence
  • Help make building an embedded Linux OS easier
slide-8
SLIDE 8

Yocto Project | The Linux Foundation 8/27

Why automated testing?

  • Confidence that changes haven't introduced regressions
  • Humans don't need to do what a machine could be doing
  • Avoid everyone re-inventing the wheel
  • Developers can run tests on their own machines
slide-9
SLIDE 9

Yocto Project | The Linux Foundation 9/27

Runtime testing (1.5+)

  • New framework for 1.5
  • Replaces old shell-based “imagetest” framework
  • Python-based (unittest)
  • Tests run on the build host, run commands on target over SSH
  • QEMU targets only (for now)
slide-10
SLIDE 10

Yocto Project | The Linux Foundation 10/27

Runtime testing (1.5+)

  • Easy to run
  • Automatically after building an image
  • Explicitly
  • Choose which tests to run
  • Minimal intrusion into CI server setup
  • (and doesn't require one)
slide-11
SLIDE 11

Yocto Project | The Linux Foundation 11/27

Runtime testing (1.5+)

  • Easy to extend
  • Written in Python
  • Unimportant details abstracted away
  • Extensible with standard layer mechanism
slide-12
SLIDE 12

Yocto Project | The Linux Foundation 12/27

Running tests

  • To enable running tests automatically after building any

image:

  • In local.conf:

TEST_IMAGE = “1”

  • To run manually:
  • In local.conf:

INHERIT += “testimage”

  • Each time:

$ bitbake -c testimage <imagename>

slide-13
SLIDE 13

Yocto Project | The Linux Foundation 13/27

Finding tests

  • Core tests located in meta/lib/oeqa/runtime
  • Additional layers can add their own
  • Add python modules in lib/oeqa/runtime under the layer
  • Should not use same names as modules in OE-Core
  • Specify tests to run by setting TEST_SUITES variable
slide-14
SLIDE 14

Yocto Project | The Linux Foundation

Example test cases

meta/lib/oeqa/dmesg.py meta/lib/oeqa/df.py meta/lib/oeqa/smart.py

slide-15
SLIDE 15

Yocto Project | The Linux Foundation

Other automated testing features

slide-16
SLIDE 16

Yocto Project | The Linux Foundation 16/27

ptest

  • Run tests from upstream software package
  • Provides packaging and installation of test suites
  • Runner script for the target (run-ptest)
  • Idea is to coalesce test output to the same format
slide-17
SLIDE 17

Yocto Project | The Linux Foundation 17/27

Autobuilder

  • Continuous Integration server
  • Based on Buildbot
  • Configured to run builds & tests out of the box
slide-18
SLIDE 18

Yocto Project | The Linux Foundation 18/27

bitbake-selftest

  • Simple test script for bitbake itself
  • Helps avoid regressions when working on bitbake
  • Quick to run
slide-19
SLIDE 19

Yocto Project | The Linux Foundation 19/27

Other scripts

  • test-reexec
  • Tests re-execution of tasks
  • test-dependencies.sh
  • Finds “floating” auto-detected dependencies
slide-20
SLIDE 20

Yocto Project | The Linux Foundation

Future plans

slide-21
SLIDE 21

Yocto Project | The Linux Foundation 21/27

Future (1.6+) - Runtime tests

  • Runtime tests on real hardware
  • Simple – one or two machines, directly attached
  • Advanced – number of machines attached to test server (“test

rack”)

  • “push model” vs “pull model”
  • ptest integration
  • Report collection / analysis
  • Additional runtime tests
  • GUI tests
slide-22
SLIDE 22

Yocto Project | The Linux Foundation 22/27

Future (1.6+)

  • Non-runtime tests
  • “oe-selftest”
  • Converting current manual test cases into automated tests
slide-23
SLIDE 23

Yocto Project | The Linux Foundation 23/27

Social effects

  • Improve communication between development & QA
  • Opportunities for QA to become more directly involved
slide-24
SLIDE 24

Yocto Project | The Linux Foundation 24/27

Summary

  • New testing framework
  • More testing-related improvements to come
  • Focus on quality
  • Send us feedback
  • Get involved!
slide-25
SLIDE 25

Yocto Project | The Linux Foundation

Questions?

slide-26
SLIDE 26

Thank you for your participation!

www.yoctoproject.org

slide-27
SLIDE 27

Image credits: "Plateau Forest (2)" by Nicholas_T http://www.flickr.com/photos/nicholas_t/9249259415/ "Stop Fracking!" by greensefa http://www.flickr.com/photos/greensefa/8205936358/ "Money" by aresauburn http://www.flickr.com/photos/aresauburnphotos/2678453389