osstest Xen Project automatic test system Community participation - - PowerPoint PPT Presentation

Ian Jackson <ian.jackson@eu.citrix.com> Xen Developer Summit Chicago 2014 Ian Campbell <ian.campbell@citrix.com>

• sstest

Xen Project automatic test system Community participation Recent developments Future plans

To: <xen−devel@xxxxxxxxxxxxxxxxxxx> Subject: [libvirt test] 29207: regressions − FAIL flight 29207 libvirt real [real] http://www.chiark.greenend.org.uk/~xensrcts/logs/29207/ Regressions :−( Tests which did not succeed and are blocking, including tests which could not be run: build−amd64−libvirt 5 libvirt−build fail REGR. vs. 29190 build−i386−libvirt 5 libvirt−build fail REGR. vs. 29190 build−armhf−libvirt 5 libvirt−build fail REGR. vs. 29190 Tests which did not succeed, but are not blocking: test−amd64−i386−libvirt 1 build−check(1) blocked n/a test−amd64−amd64−libvirt 1 build−check(1) blocked n/a test−armhf−armhf−libvirt 1 build−check(1) blocked n/a version targeted for testing: libvirt ddf6b139cfebee4193bf6f0b13c995a182d7bc35 baseline version: libvirt 72a8453948691b75a1802524837b6543353bf829 −−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− People who touched revisions under test: ....

To: <xen−devel@lists.xensource.com> Subject: [xen−unstable bisection] complete build−armhf−libvirt branch xen−unstable xen branch xen−unstable job build−armhf−libvirt test libvirt−build Tree: gnulib_libvirt git://drall.uk.xensource.com:9419/git://git.sv.gnu.org/gnulib.git%20[fetch=try] Tree: libvirt git://xenbits.xen.org/libvirt.git Tree: qemuu git://xenbits.xen.org/staging/qemu−upstream−unstable.git Tree: xen git://xenbits.xen.org/xen.git *** Found and reproduced problem changeset *** Bug is in tree: xen git://xenbits.xen.org/xen.git Bug introduced: 871b43a309d80ac99458c13c2c3da8d15c482d30 Bug not present: 6cc89d3101d8874e01a69a89a65736a2adfbd199 commit 871b43a309d80ac99458c13c2c3da8d15c482d30 Author: Senatus Populus <senatus.populus@example.com> Date: Fri Jun 20 18:19:12 2014 +0200 libxl: get and set soft affinity Make space a new cpumap in vcpu_info, called cpumap_soft, for retrieving soft affinity, and amend the relevant API accordingly. libxl_set_vcpuaffinity() now takes two cpumaps, one for hard and one for soft affinity (LIBXL_API_VERSION is exploited to retain source level backword compatibility). Either of the two cpumap can be NULL, in which case, only the affinity corresponding to the non−NULL cpumap will be affected. ... For bisection revision−tuple graph see: http://www.chiark.greenend.org.uk/~xensrcts/results/bisect.xen−unstable.build−armhf−libvirt.libvirt−build.html Revision IDs in each graph node refer, respectively, to the Trees above. −−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− Searching for failure / basis pass: 27871 fail [host=army] / 27481 ok. Failure / basis pass flights: 27871 / 27481 Tree: gnulib_libvirt git://drall.uk.xensource.com:9419/git://git.sv.gnu.org/gnulib.git%20[fetch=try] Tree: libvirt git://xenbits.xen.org/libvirt.git Tree: qemuu git://xenbits.xen.org/staging/qemu−upstream−unstable.git Tree: xen git://xenbits.xen.org/xen.git

jobs branches

libvirt linux−3.10 linux−3.4 linux−arm−xen linux−linus linux−mingo−tip−master linux−next

• sstest

qemu−mainline qemu−upstream−4.2−testing qemu−upstream−4.3−testing qemu−upstream−4.4−testing qemu−upstream−unstable rumpuserxen seabios xen−4.0−testing xen−4.1−testing xen−4.2−testing xen−4.3−testing xen−4.4−testing xen−unstable

build−amd64 build−amd64−libvirt build−amd64−oldkern build−amd64−pvops build−amd64−rumpuserxen build−amd64−xend build−armhf build−armhf−libvirt build−armhf−pvops build−i386 build−i386−libvirt build−i386−oldkern build−i386−pvops build−i386−rumpuserxen build−i386−xend test−amd64−amd64−libvirt test−amd64−amd64−pair test−amd64−amd64−pv test−amd64−amd64−rumpuserxen−amd64 test−amd64−amd64−rumpuserxen−i386 test−amd64−amd64−xl test−amd64−amd64−xl−pcipt−intel test−amd64−amd64−xl−qemut−debianhvm−amd64 test−amd64−amd64−xl−qemut−win7−amd64 test−amd64−amd64−xl−qemut−winxpsp3 test−amd64−amd64−xl−qemuu−debianhvm−amd64 test−amd64−amd64−xl−qemuu−ovmf−amd64 test−amd64−amd64−xl−qemuu−win7−amd64 test−amd64−amd64−xl−qemuu−winxpsp3 test−amd64−amd64−xl−sedf test−amd64−amd64−xl−sedf−pin test−amd64−amd64−xl−win7−amd64 test−amd64−amd64−xl−winxpsp3 test−amd64−i386−freebsd10−amd64 test−amd64−i386−freebsd10−i386 test−amd64−i386−libvirt test−amd64−i386−pair test−amd64−i386−pv test−amd64−i386−qemut−rhel6hvm−amd test−amd64−i386−qemut−rhel6hvm−intel test−amd64−i386−qemuu−freebsd10−amd64 test−amd64−i386−qemuu−freebsd10−i386 test−amd64−i386−qemuu−rhel6hvm−amd test−amd64−i386−qemuu−rhel6hvm−intel test−amd64−i386−rhel6hvm−amd test−amd64−i386−rhel6hvm−intel test−amd64−i386−rumpuserxen−i386 test−amd64−i386−xend−qemut−winxpsp3 test−amd64−i386−xend−qemuu−winxpsp3 test−amd64−i386−xend−winxpsp3 test−amd64−i386−xl test−amd64−i386−xl−credit2 test−amd64−i386−xl−multivcpu test−amd64−i386−xl−qemut−debianhvm−amd64 test−amd64−i386−xl−qemut−win7−amd64 test−amd64−i386−xl−qemut−winxpsp3 test−amd64−i386−xl−qemut−winxpsp3−vcpus1 test−amd64−i386−xl−qemuu−debianhvm−amd64 test−amd64−i386−xl−qemuu−ovmf−amd64 test−amd64−i386−xl−qemuu−win7−amd64 test−amd64−i386−xl−qemuu−winxpsp3 test−amd64−i386−xl−qemuu−winxpsp3−vcpus1 test−amd64−i386−xl−win7−amd64 test−amd64−i386−xl−winxpsp3 test−amd64−i386−xl−winxpsp3−vcpus1 test−armhf−armhf−libvirt test−armhf−armhf−xl test−i386−i386−libvirt test−i386−i386−pair test−i386−i386−pv test−i386−i386−rumpuserxen−i386 test−i386−i386−xl test−i386−i386−xl−qemut−winxpsp3 test−i386−i386−xl−qemuu−winxpsp3 test−i386−i386−xl−winxpsp3

Introduction ============ OSStest is the Xen Project automated test infrastructure. ... Terminology =========== "flight": Each run of osstest is referred to as a "flight". Each flight is given a unique ID (a number or name). "job": Each flight consists of one or more "jobs". These are a sequence

• f test steps run in order and correspond to a column in the test

report grid. They have names like "build−amd64" or "test−amd64−amd64−pv". A job can depend on the output of another job in the flight −− e.g. most test−* jobs depend on one or more build−* jobs. ... Standalone Mode =============== To run osstest in standalone mode: − You need to install sqlite3 tcl8.4 tclx8.4 libsqlite3−tcl libdbi−perl libdbd−sqlite3−perl pax rsync curl netcat − Optional: ipmitool − Write a config file ~/.xen−osstest/config See below. − Select the "branch" and job to reproduce. By default the system gives you the "branch" consisting of tests run for the xen−unstable push gate. You need to select a job. The list of available jobs is that shown in the publicly emailed test reports on xen−devel, eg http://lists.xen.org/archives/html/xen−devel/2014−01/msg01614.html

| army earwig spider potato−beetle fire−frog itch−mite rice−weevil | bedbug insider leaf−beetle field−cricket marilith−n4 | hydrazine lace−bug lake−frog marilith−n5 | woodlouse moss−bug chicken−mite grain−weevil | bush−cricket gall−mite −−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−+−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− arch−amd64 | x x x x x x x x x x x x x x x x x x arch−armhf | x x x arch−i386 | x x x x x x x x x x x x x x x x x arch−xen−amd64 | x x x x x x x x x x x x x x x x x x arch−xen−armhf | x x arch−xen−i386 | x x x x x x x x x x x x x x x x x blessed−adhoc | x x x x x x x x x x x x x x x x blessed−play | x x x x x x x x x x x x x x x x blessed−real | x x x x x x x x x x x x x x x x equiv−dell−r310 | x x equiv−marilith | x x equiv−rackservers−q21011 | x x equiv−rackservers−s40663 | x x equiv−rackservers−s40670 | x x equiv−rackservers−s40679 | x x equiv−rackservers−s40680 | x x hvm | x x x x x x x x x x x x x x x x x x hvm−amd | x x x x x x x hvm−intel | x x x x x x x x x x x need−firmware−deb−firmware−bnx2 | x x need−kernel−deb−armmp | x x need−uboot−bootscr | x x no−di−kernel | x x no−reinstall | x pcipassthrough−nic | x x purpose−build | x x x x x x x x x x x x x x x x purpose−test | x x x x x x x x x x x x x x x x x x x spong | x suite−squeeze | x x x x x x x x x x x x x x x x x x x x x suite−wheezy | x x x x x x x x x x x x x x x x x x x x x x

100 200 300 400 500 600 700 800 01/01/12 01/07/12 01/01/13 01/07/13 01/01/14 01/07/14

• sstest loc by author date

2014−06−09 Anthony Perard 2013−11−21 Roger Pau Monné 2013−11−06 Dario Faggioli 2013−09−16 Wei Liu 2013−09−03 Ian Campbell 2009−11−23 Ian Jackson

10 20 30 40 50 60 01/06/13 01/08/13 01/10/13 01/12/13 01/02/14 01/04/14 01/06/14 01/08/14 2 4 6 8 10 12 14 16 jobs in xen-unstable branches ("[BRANCH test]")

References Source repository

http://xenbits.xen.org/gitweb/?p=osstest.git;a=summary

README

http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=HEAD

Output example

http://xenbits.xen.org/docs/osstest−output−example/24438/

Contact xen−devel@lists.xenproject.org Ian.Jackson@eu.citrix.com Ian.Campbell@citrix.com