Helper Programs

This page lists the helper programs used by the Toolchain WG. This page is partly here to list the helpers, and partly to start talking about handing them over to the Infrastructure group.

Most of these programs are web based, use web.py, sit directly on the filesystem, are a bit of a hack, and are available at https://code.launchpad.net/tcwg-web.

Continuous Build

See http://cbuild.validation.linaro.org/build/ and https://code.launchpad.net/~cbuild/+junk/cbuild

gcc-linaro, gdb-linaro, and, in the future, other Toolchain WG outputs are built continuously on i686, x86_64, and armel.

Implementation:

  • The master holds all of the artifacts needed for all builds in the directory cbuild

  • Artifacts include build scripts, original tarball, and sundry benchmark files
  • Each slave has a slave-specific directory under slaves/hostname

  • The slaves rsync down the whole cbuild tree
  • The slave then runs build.mk in the slave-specific directory

build.mk could do anything. What it currently does is:

  • List all of the tarballs in the slave-specific directory
  • Using a Makefile, figure out the product specific steps based on tarball name and run them

For GCC these steps include:

  • Build GCC in a slave-specific configuration (it's too slow to build Java on ARM every time, etc)
  • Run the GCC testsuite
  • Run the libstdc++ testsuite
  • Build and run a set of benchmarks using various source files
  • Build eglibc and run the testsuite

For GDB these steps include:

  • Download and use the most recent Linaro GCC compiler
  • Build GDB using this compiler
  • Run the GDB testsuite

Features:

  • Provides source and binary builds
  • All steps are logged
  • Jobs are resumable
  • Slaves are easy to bootstrap. You need to copy the initial run.sh and the rest comes via rsync.

  • Allocating jobs is manual but easy. Gets rid of any complex scheduling
  • Jobs run as normal Makefiles and are easy to interrupt, change, and experiment with
  • The Makefiles send out status notifications over IRC
  • Making new jobs is easy. Write the Makefile, integrate it
  • Multiple variants of the benchmarks are run, such as optimised for speed, optimised for size, etc.

Notes:

  • We've done 30 builds over three months or about one every three days
  • A typical three architecture build is 550 MB in size
  • A binary build is up to 210 MB

Branch Puller

See http://bazaar.launchpad.net/~cbuild/%2Bjunk/cbuild/files/head%3A/tools/

Updates a local bzr branch or svn repo and, if no snapshot exists, exports and creates a new tarball snapshot.

Makes the snapshots that go into the build system. Not much of a tool but still...

Build Log

See http://cbuild.validation.linaro.org/helpers/buildlog

Provides a grid view of all builds on all hosts.

Features:

  • Links off to the interesting results such as binaries and logs
  • Links off to other tools such as the test result comparison

Uses:

  • Find all builds of a certain revision
  • Find builds for a certain architecture
  • Find builds of the same revision and architecture but on different hosts for finding environment specific problems

Future:

  • Show a summary of the build result, such as red if the tests have regressed

Tickets

See http://cbuild.validation.linaro.org/helpers/tickets

Provides a view onto all Linaro Toolchain tickets and ways of grouping on different fields. Done as Michael couldn't get the reports he wanted out of Launchpad.

Features:

  • Shows bug #, title, status, importance, assignee, milestone, and project
  • Allows grouping on the same fields

Uses:

  • View all tickets against a milestone to see how the release is progressing
  • View all tickets against assignee to see what is in someones queue
  • View all tickets by severity across all of the Linaro toolchain

Benchmarks Report

See http://cbuild.validation.linaro.org/helpers/benchcompare

Shows the results for each benchmark on each build and does a basic comparison against baseline.

Features:

  • Parses the build log files for different benchmark results in different formats
  • Gathers all samples from the same run and picks the best
  • Presents all samples so you can 'eyeball' the validity
  • Group by build to see how a particular build performs
  • Group by test to see how results change with time
  • Easy link off to the raw logs

Test Result Comparison

See http://cbuild.validation.linaro.org/helpers/testcompare/gcc-linaro-4.4-2010.09-1/logs/armv7l-maverick-cbuild3-pavo3/gcc-testsuite.txt

Presents the results of a DejaGnu based testsuite and compares against a baseline version. Used to see how a ARM build compares to the FSF baseline version to see what has regressed and what has improved.

Features:

  • Automatically picks the baseline based on filename
  • Compare against other baselines
  • Parses DejaGnu files

  • Splits into suite (such as gcc, g++, etc)
  • Splits into category (such as FAIL, XPASS, etc)
  • Shows the common entries in each category, those only in baseline, and those only in new
  • Handle baselines or builds that don't test all languages

Patch Tracker

See http://cbuild.validation.linaro.org/helpers/patchtrack

To be written.

WorkingGroups/ToolChain/Helpers (last modified 2012-12-04 22:54:08)