NOTE: THIS PAGE CONTAINS OUTDATED INFORMATION! USE AT YOUR OWN RISK!

Cbuildv2 is now working well enough to build full cross toolchains. Some documentation on Using Cbuildv2 exists.

Toolchain Components

There are multiple components used when building a toolchain. These are (with their optional values) as follows:

  • gcc (gcc, g++, objc, fortran, llvm, clang)
  • gas
  • gdb
  • ld (gold)
  • libc (newlib,eglibc,glibc)
  • dependant libs (gmp, mpc, mpfr)

Requirements

  • Needs to support all native and cross compilation of toolchain
  • Needs to be able to specify which compiler and binutils is used for compiling the programs
  • Needs to be able to specify custom configure options, but use a default set as well
  • Needs to use binary snapshots for some builds
  • Need to bootstrap development environment when not using binary snapshots
  • Needs to work 100% locally without LAVA, or using LAVA locally or remotely
  • Needs to be able to run 3rd party benchmarks
  • Needs to supply data for graphing performance and regressions
  • Build with stock Ubuntu toolchains
  • Linaro GCC Maintenance (currently Linaro GCC 4.7)
  • FSF GCC Previous (currently FSF GCC 4.7)
  • FSF GCC Current (currently FSF GCC 4.8)
  • Build with trunk/master
  • Linaro GCC Development (currently Linaro GCC 4.8)
  • FSF GCC Trunk (currently FSF GCC 4.9)

General Ideas: Cbuidv2 vs Cbuildv1

* Uses shell scripts as config files, ala *.conf and set global variables or custom functions.

* Rather than Makefiles, it should uses Bourne shell scripts for more functionality and maintainability. Both Android and Chromium use this technique.

* Each config file lists it's major build time dependencies and configure options. The default config files are copied into the build tree, where they can be edited, overriding the default options. This makes it easier to integrate manual building nd testing with automation in a way that's consistent.

* Replace crosstool-ng with similar functionality, but better integrated into Linaro build and test processes.

Command Line Options

  • (subject to change at this point)

Usage: cbuild2 [options] snapshot

Currently at this time the snapshot can be a partial file name gcc-linaro-4.8~svn12345, which fetches the source tarball from toolchain64.lab~/var/snapshots. Or it can be a URL to a bzr, svn, or git repository.

  • --build
    • specify architecture for the build machine, defaults to native build.
  • --target
    • specify architecture for the target machine, defaults to native. This is required for cross builds.
  • --libc {newlib, eglibc, glibc}
    • specify C library
  • --list {base, prebuilt, snapshots, infrastructure}
    • list remote source and binary snapshots
  • --binutils XXX
    • specify binutils version to use, default is what in the path
  • --gcc XXX
    • gcc version to use, default is what in the path
  • --clean
    • clean a previous build, default is to start where it left off
  • --dispatch
    • run on LAVA build farm, probably remote
  • --sysroot XXX
    • specify path to alternate sysroot
  • --db-user XXX.
    • specify MySQL user for the database (optional)
  • --db-passwd XXX
    • specify MySQL password) optional)
  • --dump
    • dump the values in the host config file
  • --dostep {fetch,extract,configure,build,test,depend,infrastructure,tag,tarball,release,push}
    • execute a single step in the config/buld/test process. This is primarlly used for debugging cbuidv2 itself
      • fetch - Download a source tarball from toolchain64.lab
      • checkout - Checkout a source tree using svn, git, or bzr
      • extract - Extract the tarball into a source tree
      • configure - Configure the extracted sources using the default (or local) config file into a separate build tree
      • build - Compiler the configured build tree
      • test - Run make check on the built tree

      • depend - Build the dependencies for the build so it can complete
      • infrastructure - Download, build, and install the GCC infrastructure libraries (cloog, isl, ppl, gmp, mpc, mpfr)
      • tag - Tag the source tree, usually used only for releases
      • release - Release the toolchain component from its source tree
  • --release XXX
    • make a release
  • --clobber
    • force files to be downloaded even when they exist
  • --force
    • force make errors to be ignored, answer yes for any prompts
  • --parallel
    • do parallel builds, default is one per cpu core

Databases

  • Part of the shift in using Cbuildv2 is a more data centric focus. Although text files are commonly produced by a number of operations, this makes it difficult to compare more than one or two files. All testing results and benchmarks can now go into a MySQL database to support a better way of analyzing multiple toolchain versions. This enables better comparative analysis, and supports graphing of the data. There are several types of data to keep track of. One database is for regression testing results, the other is for benchmarks.

=== Testing Database ==

  • One table contains all the data on each test run, the other table contains the actual results. A ""testrun" number is used as an index into the test results table.

The top level table ""testruns"" contains the following fields: This stores information parsed from the host.txt file, which documents the platform specific data on the build and test run.

testrun

int(20)

The index

date

datetime

Date/Time of the test run

tool

varchar(72)

The component being tested

version

varchar(72)

The component's version

branch`

varchar(72)

The banch if built from svn or git

status

enum('IDLE','RUNNING','DONE')

The current status of the test run

target_arch

varchar(72)

The architecture of the target

build_arch

varchar(72)

The architecture of the build machine

gcc_version

varchar(72)

The version of GCC used for the build

binutils_version

varchar(72)

The binutils version used for the build

libc_version

varchar(72)

The libc version used for the build

The results table ""test"" contains the following fields:

testrun

int(9)

The index for this test run

input

varchar(128)

Any input supplied to the test case

output

varchar(256)

Any output from the test case

result

enum('PASS','FAIL','XPASS','XFAIL','UNTESTED','UNRESOLVED', 'UNSUPPORTED')

The test state

name

varchar(128)

The name of the testcase

prmsid

int(11)

The Bug ID from the Bug Tracker

The testing database schema needs to support being able to do these types of queries:

  • Plot a test run of a specific version of a component across all supported platforms
  • Plot a test case of a component across all supported platforms
  • Plot a test case of a component across a list of versions
  • Plot a component's PASS percentage across all supported platforms
  • Plot a component's PASS percentage across all supported platforms and several versions.
  • Plot a component's FAIL percentage across all supported platforms
  • Plot a component's FAIL percentage across all supported platforms and several versions.
  • Plot all test states as a percentage of total test results
  • Plot all test states of the actual totals for each

All of the code to manage the testing data lives in DejaGnu, instead of Cbuildv2. This is to enable other people and organizations the ability to also graph testing data. Currently this lives in the mysql branch of DejaGnu, which is located in the contrib/mysql directory in the DejaGnu repository.

Benchmark Database

There is support to import the benckmark testing results from EEMBC, EEMBC-Office, Spec2000, Coremark and Denbench benchmark suites.

The benchrun table contains all the relevant data on each run of the benchmarks.

date

datetime

The date the benchmark was run

benchmark

varchar(30_

The name of the benchmark

base_result

float

The base result

target_arch

varchar(72)

The target architedture

build_arch

varchar(72)

The build architecture

build_machine

varchar(72)

The build machine name

build_os

varchar(72)

The OS used for the build

codename

varchar(72)

The code name of the OS

enabled_cores

int(3)

The number of cores used for the benchrmark run

enabled_chips

int(3)

The number of processors used for the benchmark run

cores_per_chip

int(3)

The number of cores per processor

threads_per_core

int(3)

The number of threads supported by each processor

build_gcc

varchar(72)

The version of GCC used for this run

binutils_version

varchar(72)

The version of binutils used

libc_version

varchar(72)

The version of the C library used for this run

peak_result

int(6)

The Peak result of this benchmark run

ram

int(11)

The amount of RAM on the build machine used for this run

benchrun

int(11)

The index used for the other tables

The tabulate table contains data parsed from the benchmark.txt file.

testname

varchar(72)

The name of the benchmark

version

varchar(72)

The version of the component being tested

klass

varchar(6)

No idea, this is in the benchmark.txt file

variant

varchar(72)

The variant of the processor

subname

varchar(72)

The subname of the processor

nsamples

int(11)

The number of samples for this run

min

float

The minimum result from this run

max

float

The maximum result from this run

best

float

The best result from this run

span

float

The span of the results from this run

mean

float

The mean result from this run

std

float

The std deviation from this run

median

float

The median result from this run

benchrun

int(11)

The index

The current sources for Cbuldv2 are in the Cbuldv2 repository

Cbuild2 (last modified 2015-02-27 15:09:30)