Summary

draft r0

The toolchain group runs builds of trunk, mainline, and all releases to find issues early and to have consistent quality checks on releases. This should be taken over by the validation group and polished.

Rationale

User stories

Andy wants to monitor the quality of the current gcc-linaro development version by checking that each commit builds and doesn't regress on the testsuite.

Barry wants to contribute to mainline by monitoring the quality of the mainline trunk and release branches by building periodically, checking the testsuite, and making those results available to the mainline community.

Charlie is developing a new feature for gcc-linaro and wants to check that it has no side effects on the supported architectures.

Dave reviews new features as they are proposed for gcc-linaro and wants to check that it isn't a step backwards in terms of the testsuite.

Ernie triages bugs as they are reported to gcc-linaro. He wants to check if the fault exists in the latest release of each Linaro series, mainline trunk, and interesting mainline release branches.

Frank does the release process for gcc-linaro. He wants to build each release in a fixed configuration across the supported architectures, check the testsuite on each build, build other variants for wider testing, and run deeper tests than on a normal commit.

Assumptions

Design

A build is a single run of a single variant of a single architecture of a single version. An architecture is a processor architecture such as i686, x86_64, or ARMv7. A variant is a particular configuration of a product such as ARM mode with Soft FP, Thumb-2 mode with NEON, profiled bootstrap, or SMS bootstrap. A version is a unique bundle of source code.

Builds shall be:

  • Reproducable
  • Reliable
  • Interruptable
  • Resumable
  • Premptable
  • Timely
  • Able to be re-run

To meet these, the build process shall be automated. Builds shall run unatended. The build system shall be easily installable and runnable on a developer's machine.

The build farm shall be:

  • Reliable
  • Consitent

Builds shall be able to be re-run to check the consistency of the build farm.

Builds shall be uniquely identified. The build name shall be human readable. The build name shall be machine readable. The build name shall be easily referrable back to the source. Names shall include the product, series, and version. The version shall be sortable. The version shall follow the Debian conventions.

Version examples:

  • gcc-4.7~svn12345 - GCC mainline Subversion revision 12345 leading up to GCC 4.7
  • gcc-linaro-4.5+bzr12345 - Linaro GCC 4.5 Bazaar revno 12345

Versions may be of a main branch, feature branch, or release. All branches shall be stored in a version control system. A release is stored as a tarball. The system shall be able to source from Subversion, Bazaar, and GIT. Mirrors shall not be used unless the mirror corresponds exactly to the primary version control system.

Meta data shall be recorded about a version. This meta data shall be sufficient to:

  • Meet the unique identification requirement above
  • Easily discover the ancestor or predecessor of a branch

Variants shall be uniquely identified. Variants shall be versioned. A build of a variant shall include enough information to determine the variant and variant revision. Variants may affect the build through:

  • Setting configure flags
  • Setting build variables such as CFLAGS
  • Changing the default make target

The following architectures shall be supported:

Builds targeting armv5te should be supported in a reduced configuration. This may involve building a cross-compiler and using SSH to run the testsuite and benchmarks on the ARMv5TE target.

The following variants shall be supported:

  • i686 - built on a i686 host with default configure flags
  • x86_64 - built on a i686 host with default configure flags
  • cortexa9 - built on an ARMv7 host configured for Thumb-2 with NEON and tuned for a Cortex-A9
  • cortexa8 - built on an ARMv7 host configured for Thumb-2 with VFPv3-D16 and tuned for a generic ARMv7
  • armv5te - built on an ARMv7 host configured for ARM with softfp and the ARMv5E architecture

The following artifacts shall be recorded:

  • Build logs
  • Test logs
  • Benchmark results
  • Binary output

Artifacts shall be recorded forever. Artifacts are important but not critical and may be recorded and backed up at a level that reflects that.

The following qualities shall be measured:

  • Testsuite results
  • Benchmark scores

Builds happen within an environment. The environment shall be fixed to ensure the builds are consistent. The environment shall be well-described and easy to install to meet the developer requirement. From time to time the environment will need to be upgraded to pull in changes such as new kernel features.

The environments shall be:

  • The latest Linaro stable release, or if not available
  • The latest Ubuntu stable release, or if not available
  • The latest Debian stable release

As at the time of writing these are Linaro Natty, Ubuntu Natty, and Debian squeeze.

The environment shall be a minimal installation plus the packages needed to build the products. Any build dependencies shall be supplied by the release.

Implementation

Michael's typed enough for now. Here's some examples:

  • Andrew commits to bzr trunk
  • The system sees a new revision
  • A version called gcc-linaro-4.5+bzr12345 is created
  • This is spawned into i686, x86_64, and Cortex-A9 builds
  • The builds run
  • The results are recored and merged together at builds.linaro.org
  • Summary emails are sent to linaro-toolchain-builds
  • GCC trunk is build three times a week
  • A cron job triggers three times a week
  • GCC trunk is checked out from SVN
  • A new version called gcc-4.7~svn12345 is created
  • This is spawned into i686, x86_64, and Cortex-A9 builds
  • The builds run
  • The results are recored and merged together at builds.linaro.org
  • Summary emails are sent to gcc-testresults@gcc.gnu.org

  • Release time comes around
  • Andrew runs the manual release process
  • A tarball called gcc-linaro-4.6-2011.06 is created
  • This is spawned into the i686, x86_64, Cortex-A9, and Cortex-A8 queues
  • The builds run
  • The results are recored and merged together at builds.linaro.org
  • Summary emails are sent to linaro-toolchain-builds
  • Michael runs further tests based on the binaries

Migration

Include:

  • data migration, if any
  • redirects from old URLs to new ones, if any
  • how users will be pointed to the new way of doing things, if necessary.

Test/Demo Plan

It's important that we are able to test new features, and demonstrate them to users. Use this section to describe a short plan that anybody can follow that demonstrates the feature is working. This can then be used during testing, and to show off after release. Please add an entry to http://testcases.qa.ubuntu.com/Coverage/NewFeatures for tracking test coverage.

This need not be added or completed until the specification is nearing beta.

Unresolved issues

WorkingGroups/ToolChain/Specs/ToolchainBuilds (last modified 2011-06-19 23:23:10)