Release Process for Linaro GCC

This process prepares the source-base for public consumption.

The process is mostly derived from FSF GCC's process and maintainer-scripts/gcc_release in the sources.

Some of these steps are part of the greater process and are outside the basic source release process.

Note that there are two different processes for 4.7 and earlier where sources are stored in Bazaar, and 4.8 and later where sources are stored in FSF SVN.

4.8 and later.

Unlike the 4.7 and earlier releases for GCC 4.8 and later the sources are stored in Subversion, and so some of these steps will not be scriptable.

Variables

ID= # gcc.gnu.org Login
BRANCH=gcc-N_M-branch # for example gcc-4_8-branch
SPIN=N # for example 0 for first release, 1 for second
NAME="First Last" # Name as appears in ChangeLogs
EMAIL="$(tr "[:upper:] " "[:lower:]." <<EOF
$NAME
EOF
)@linaro.org" # Email address as appears in ChangeLogs.
REL_DIR=/work/release/gcc-linaro-M.m-YYYY.MM # for instance /work/release/gcc-linaro-4.8-2013.05

Before

  1. Verify that Linaro GCC matches up with the latest GCC point release, i.e. are the following two files the same:
    • svn://gcc.gnu.org/svn/gcc/branches/$BRANCH/gcc/BASE-VER; and
    • svn://gcc.gnu.org/svn/gcc/branches/linaro/$BRANCH/gcc/BASE-VER; and

Prepare Sources

Note that [-X] below covers the implicit respin number. It is omitted for the first release in a month.

  1. Check out a fresh source tree:
    ssh-add
    mkdir -p $REL_DIR
    svn checkout svn+ssh://[email protected]/svn/gcc/branches/linaro/$BRANCH $REL_DIR/svn
  2. Update every ChangeLog.linaro file.

    date=$(date +%Y-%m-%d)
    release=$(cut -f1,2 -d. $REL_DIR/svn/gcc/BASE-VER)-$(date +%Y.%m)
    if test $SPIN != "0"; then
      release="${release}-$SPIN"
    fi
    
    # Note tabs in following.
    for cl in $(find $REL_DIR/svn -name 'ChangeLog.linaro'); do
    
      if test $cl = "$REL_DIR/svn/gcc/ChangeLog.linaro"; then
        cat - $cl > $cl.new <<EOF
    $date  $NAME  <$EMAIL>
    
            GCC Linaro $release released.
            * LINARO-VERSION: Update.
    
    EOF
      else
        cat - $cl > $cl.new <<EOF
    $date  $NAME  <$EMAIL>
    
            GCC Linaro $release released.
    
    EOF
      fi
    
      mv $cl.new $cl
    done
  3. Update LINARO-VERSION
    echo $release > $REL_DIR/svn/gcc/LINARO-VERSION

Create the tarballs

  1. Export the sources

    We're going to make some changes to the source tree that should not be checked in.

    svn export $REL_DIR/svn $REL_DIR/gcc-linaro-${release}
  2. Switch to the new source tree
    cd $REL_DIR/gcc-linaro-${release}
  3. Generate the INSTALL documentation.
    env SOURCEDIR=`pwd`/gcc/doc DESTDIR=`pwd`/INSTALL ./gcc/doc/install.texi2html
  4. Build the generated files. First ensure that all the build dependencies are installed (mpfr, gmp, etc.).
    contrib/gcc_build -d $REL_DIR/gcc-linaro-${release} -o $REL_DIR/objdir -c "--enable-generated-files-in-srcdir --disable-multilib" build       
    Make sure the build succeeds. If not, we've got problems.
  5. Move the generated files to their proper place.
    mv -v $REL_DIR/objdir/gcc/po/*.gmo $REL_DIR/gcc-linaro-${release}/gcc/po/

    If this command fails then you probably need to install gettext.

  6. Regenerate the checksums file.
    • cd $REL_DIR/gcc-linaro-${release}
      cat - >MD5SUMS <<EOF
      # This file contains the MD5 checksums of the files in the 
      # gcc-linaro-${release}.tar.xz tarball.
      #
      # Besides verifying that all files in the tarball were correctly expanded,
      # it also can be used to determine if any files have changed since the
      # tarball was expanded or to verify that a patchfile was correctly applied.
      #
      # Suggested usage:
      # md5sum -c MD5SUMS | grep -v "OK$"
      EOF
      
      find . -type f | sed -e 's:^\./::' -e '/MD5SUMS/d' | LC_ALL=C sort | xargs md5sum >>MD5SUMS
  7. Create the tarfile:
    cd $REL_DIR
    tar cfJ gcc-linaro-${release}.tar.xz gcc-linaro-${release}

Verify the release

See Testing.

  1. Upload to cbuild.validation.linaro.org test system:
    scp $REL_DIR/gcc-linaro-${release}.tar.xz \
      [email protected]:/home/cbuild/var/snapshots
  2. We also sign the release at this point:
    ssh -t [email protected] \
      gpg --no-use-agent -q --yes --passphrase-file /home/cbuild/.config/cbuild/password \
      --armor --sign --detach-sig --default-key cbuild \
      "/home/cbuild/var/snapshots/gcc-linaro-${release}.tar.xz"
    scp [email protected]:/home/cbuild/var/snapshots/gcc-linaro-${release}.tar.xz.asc $REL_DIR
  3. Notify OpenEmbedded and Android Platforms team that a proposed Source tarball has been uploaded.

    1. This will be http://cbuild.validation.linaro.org/snapshots/gcc-linaro-${release}.tar.xz

    2. Currently notify: Platform (PoC: Bernhard Rosenkränzer, Riku Voipio, Fathi Boudra)
  4. Spawn the test builds:
    ssh [email protected] ./lib/cbuild-tools/spawn.sh gcc-linaro-${release}
  5. Once the build has finished, spawn the 'ubutest' and 'benchmarks' jobs as follows:

Queue

Tests

Notes

a9hf-builder

ubutest

a9hf-ref

benchmarks

armv5-builder

benchmarks

i686

ubutest

Can't run benchmarks as these are cloud machines.

x86_64

ubutest

Can't run benchmarks as these are cloud machines.

Spawning these jobs consists of ssh'ing into cbuild.validation.linaro.org as the cbuild user and creating the correct job file:

  • ssh [email protected]
    release=gcc-linaro-M.m-YYYY.MMM[-S]
    cd var/scheduler
    for q in a9hf-builder i686 x86_64; do
      cp queue/$q/template.txt queue/$q/ubutest-${release}.job || touch queue/$q/ubutest-${release}.job
    done
    for q in a9hf-ref armv5-builder; do
      cp queue/$q/template.txt queue/$q/benchmarks-${release}.job || touch queue/$q/benchmarks-${release}.job
    done
    Note that the build must be finished first. You can spawn into the individual queues as the architectures finish.
  • Once the job has finished, check the test logs in a directory like http://cbuild.validation.linaro.org/build/gcc-linaro-4.6-2011.11/logs/

    1. Check for *-failed.txt
    2. Review the logs
  • Benchmark build results appear in http://cbuild.validation.linaro.org/benchmarks/. See the toolchain internal page for the password

  • Copy the release checksheet template as 'Linaro GCC 20xx.yy Release Checksheet'

  • Check the buildlog to find the build results

  • On the spreadsheet, fill in the architecture specific sheets
  • Fill in the top summary sheet

See the 4.6-2011.11 sheet for an example.

In the future, consider:

  • Build the Ubuntu gcc package with an updated "linaro" patch, for ARM, x86, and amd64.
  • Rebuild newly fixed FTBFS packages.
  • Anything else .....

Commit the release and tag it

If the tests work and we're happy to release:

  • cd $REL_DIR/svn
    svn commit -m"Make Linaro GCC ${release}."
    svn cp \
      svn+ssh://[email protected]/svn/gcc/branches/linaro/$BRANCH \
      svn+ssh://[email protected]/svn/gcc/tags/gcc-linaro-${release}

Bump the development version number

  1. Return to the Subversion checkout
    cd $REL_DIR/svn
  2. Edit gcc/LINARO-VERSION:

    release=$(cut -f1,2 -d. $REL_DIR/svn/gcc/BASE-VER)-$(date +%Y.%m)-$(($SPIN+1))~dev
    echo "$release" > $REL_DIR/svn/gcc/LINARO-VERSION
    Note that this reads as 'development version leading up to the next respin'.
  3. Add this to the start of gcc/ChangeLog.linaro:

    $date  $NAME  <$EMAIL>
    
            * LINARO-VERSION: Bump version.
  4. Check in the changes.
    svn commit -m "Bump version number, post release."

Push

  1. Create a release against the milestone in Launchpad (such as this)

  2. Fill in the release notes similar to
    Linaro GCC M.m is the Nth release in the M.m series. Based off the latest GCC M.m.n, it includes many ARM-focused performance improvements and bug fixes.
    
    Interesting changes include:
     * Feature 1
     * Feature 2
    ...
  3. Fill in the changelog from all Changelog.linaro files

  4. Upload the tarball and signature to the release
  5. Verify the MD5 sum of the upload
  6. Download the file and check that it extracts correctly

Update others

  1. Create a new milestone for the next release
  2. Change all Fix committed tickets against the milestone to Fix released

  3. Update the current release information at WorkingGroups/ToolChain

  4. When other releases are finished create announcement email (see below).
  5. Update release notes in textile format on http://git.linaro.org/toolchain/release-notes.git

4.7 and earlier.

Before

  1. Verify that Linaro GCC matches up with the latest GCC point release

Prepare Sources

Most of this has been automated by gcc-release.sh in lp:~linaro-toolchain-dev/cbuild/tools. Please use the script and manually check the results. See below for more details on this script. The script cover the first steps below, upto "Verify the release", and includes upload and signature of the tarball.

Note that [-X] below covers the implicit respin number. It is omitted for the first release in a month.

  1. Check out a fresh source tree:
    bzr checkout lp:gcc-linaro/4.X my-bazaar-co
    cd my-bazaar-co
  2. Add this text to the start of ChangeLog.linaro

    20XX-XX-XX  First Last  <[email protected]>
    
            GCC Linaro 4.X-20XX.XX[-X] released.
    
            gcc/
            * LINARO-VERSION: Update.
  3. Edit gcc/LINARO-VERSION:

    4.X-20XX.XX[-X]
  4. Check in the changes, and tag them
    bzr commit -m "Make 4.X-20XX.XX[-X] release."
    bzr tag gcc-linaro-4.X-20XX.XX[-X]

Create the tarballs

  1. Export the sources

    We're going to make some changes to the source tree that should not be checked in.

    bzr export ../gcc-linaro-4.X-20XX.XX[-X]
  2. Switch to the new source tree
    cd ../gcc-linaro-4.X-20XX.XX[-X]
  3. Generate the INSTALL documentation.
    env SOURCEDIR=`pwd`/gcc/doc DESTDIR=`pwd`/INSTALL ./gcc/doc/install.texi2html
  4. Build the generated files. First ensure that all the build dependencies are installed (mpfr, gmp, etc.).
    contrib/gcc_build -d `pwd` -o ../objdir -c "--enable-generated-files-in-srcdir --disable-multilib" build       
    Make sure the build succeeds. If not, we've got problems.
  5. Move the generated files to their proper place.
    mv -v ../objdir/gcc/po/*.gmo gcc/po/

    If this command fails then you probably need to install gettext.

  6. Regenerate the checksums file.
    1. Create a new file: MD5SUMS The new file should contain:

      # This file contains the MD5 checksums of the files in the 
      # gcc-linaro-4.X-20XX.XX[-X].tar.bz2 tarball.
      #
      # Besides verifying that all files in the tarball were correctly expanded,
      # it also can be used to determine if any files have changed since the
      # tarball was expanded or to verify that a patchfile was correctly applied.
      #
      # Suggested usage:
      # md5sum -c MD5SUMS | grep -v "OK$"
    2. Append the checksums:
      find . -type f | sed -e 's:^\./::' -e '/MD5SUMS/d' | LC_ALL=C sort | xargs md5sum >>MD5SUMS
  7. Create the tarfile:
    cd ..
    tar cfj gcc-linaro-4.X-20XX.XX[-X].tar.bz2 gcc-linaro-4.X-20XX.XX[-X]

Bump the development version number

  1. Return to the bazaar checkout
    cd my-bazaar-co
  2. Edit gcc/LINARO-VERSION:

    4.X-20XX.XX-1~dev
    Note that this reads as 'development version leading up to the first respin'.
  3. Add this to the start of ChangeLog.linaro:

    20XX-XX-XX  First Last  <[email protected]>
    
            gcc/
            * LINARO-VERSION: Bump version.
  4. Check in the changes.
    bzr commit -m "Bump version number, post release."

Verify the release

See Testing.

  1. Upload to Michael's test system:
    scp \
        gcc-linaro-4.X-20XX.XX[-X].tar.bz2 \
        [email protected]:~cbuild/var/snapshots

This was the last step covered by the script (plus the signature mentioned below).

  1. Notify OpenEmbedded and Android Platforms team that a proposed Source tarball has been uploaded.

    1. This will be http://cbuild.validation.linaro.org/snapshots/gcc-linaro-4.X-20XX.XX[-X].tar.bz2

    2. Currently notify: Bero (Bernhard Rosenkränzer) for Android and Riku Viopio / Fathi Boudra for OE
  2. Spawn the test builds:
    ssh [email protected] \
        ./lib/cbuild-tools/spawn.sh gcc-linaro-4.X-20XX.XX[-X]
  3. Once the build has finished, spawn the 'ubutest' and 'benchmarks' jobs as follows:

Queue

Tests

Notes

a9hf-builder

ubutest

a9hf-ref

benchmarks

armv5-builder

benchmarks

i686

ubutest

Can't run benchmarks as these are cloud machines.

x86_64

ubutest

Can't run benchmarks as these are cloud machines.

Spawning these jobs consists of ssh'ing into cbuild.validation.linaro.org as the cbuild user and creating the correct job file:

  • ssh [email protected]; cd var/scheduler
    # $q - queue to use, $t - test to run
    # if queue/$q/template.txt exists do:
    # cp queue/$q/template.txt queue/$q/$t-gcc-linaro-4.X-20XX.XX[-X].job
    # otherwise do:
    # touch queue/$q/$t-gcc-linaro-4.X-20XX.XX[-X].jobReview the logs
    Note that the build must be finished first. You can spawn into the individual queues as the architectures finish.
  • Once the job has finished, check the test logs in a directory like http://cbuild.validation.linaro.org/build/gcc-linaro-4.6-2011.11/logs/

    1. Check for *-failed.txt
    2. Review the logs
  • Benchmark build results appear in http://cbuild.validation.linaro.org/benchmarks/. See the toolchain internal page for the password

  • Copy the release checksheet template as 'Linaro GCC 20xx.yy Release Checksheet'

  • Check the buildlog to find the build results

  • On the spreadsheet, fill in the architecture specific sheets
  • Fill in the top summary sheet

See the 4.6-2011.11 sheet for an example.

In the future, consider:

  • Build the Ubuntu gcc package with an updated "linaro" patch, for ARM, x86, and amd64.
  • Rebuild newly fixed FTBFS packages.
  • Anything else .....

Push

  1. Create a release against the milestone in Launchpad (such as this)

  2. Fill in the release notes similar to
    Linaro GCC 4.4 is the second release in the 4.4 series.  Based off the latest GCC 4.4.4, it fixes many of the issues found during building Ubuntu over the last few months.
    
    Linaro GCC 4.5 is the first release in the 4.5 series. Based off the latest GCC 4.5.1, it includes many ARM-focused performance improvements and bug fixes.
    
    Interesting changes include:
     * Feature 1
     * Feature 2
    ...
  3. Fill in the changelog from Changelog.linaro

  4. Sign the tarball using the Linaro Toolchain Builder key (https://launchpad.net/~cbuild) cat password | gpg --passphrase-fd 0 --armor --sign --detach-sig file (this step has been performed by the gcc-release.sh script, if you used it)

  5. Upload the tarball and signature to the release
  6. Verify the MD5 sum of the upload
  7. Download the file and check that it extracts correctly

Update others

  1. Create a new milestone for the next release
  2. Change all Fix committed tickets against the milestone to Fix released

  3. Update the current release information at WorkingGroups/ToolChain

  4. When other releases are finished create announcement email (see below).
  5. Update release notes in textile format on http://git.linaro.org/toolchain/release-notes.git

Announce

Send an e-mail to Linaro Toolchain mailing list <linaro-toolchain AT lists DOT linaro DOT org> similar to:

Subject: [ANNOUNCE] Linaro GCC 4.7 and 4.6 2012.05 released

The Linaro Toolchain Working Group is pleased to announce the release of both Linaro GCC 4.7 and Linaro GCC 4.6.

Linaro GCC 4.4 is the third release in the 4.4 series. Based off the latest GCC 4.4.4, it pulls in the pre-4.4.5 changes made by the FSF over the last six months.

Linaro GCC 4.5 is the second release in the 4.5 series. Based off the latest GCC 4.5.1, it finishes the merge of many ARM-focused performance improvements and bug fixes.

Interesting changes include:
 * Improved performance on the Cortex-A9
 * Backports of a range of performance improvements from mainline
 * New inline versions of the GCC builtin sync primitives

The source tarball is available from:
 https://launchpad.net/gcc-linaro/+milestone/4.6-2010.11-0
 https://launchpad.net/gcc-linaro/+milestone/4.7-2010.11-0

Downloads are available from the Linaro GCC page on Launchpad:
 https://launchpad.net/gcc-linaro

Mailing list:  http://lists.linaro.org/mailman/listinfo/linaro-toolchain

Bugs:  https://bugs.launchpad.net/gcc-linaro/

Questions? http://ask.linaro.org/

Interested in commercial support? inquire at [email protected]

In addition make an announcement on the Launchpad page here and copy in the text of the announcement.

Using the GCC-release.sh script

Most of the above process has been automated by gcc-release.sh in lp:~linaro-toolchain-dev/cbuild/tools.

The following are some extra notes on using the script.

Setup

If you are using Ubuntu 12.04 (and possibly later) you need to work around the multi-arch support:

cd /usr/include
# The following will be different for a 32-bit host.
ln -s x86_64-linux-gnu/asm asm
export LIBRARY_PATH=/usr/lib/x86_64-linux-gnu:$LIBRARY_PATH

Import the `Linaro Toolchain Builder' public key:

gpg --keyserver keyserver.ubuntu.com --recv-keys D47877808F427EAF

Running the script

The script takes two parameters. The first parameter is a full path to a bazaar branch of the appropriate branch of linaro-gcc you want to release. The second, optional, parameter is only used for a respin and gives the respin ID.

An example command line is as follows:

LIBRARY_PATH=/usr/lib/x86_64-linux-gnu \
  .../gcc-release.sh $HOME/devel/sources/gcc-linaro/4.7

What the script doesn't do

The script doesn't spawn the build jobs, or benchmarking jobs. But it does do the upload.

It doesn't do most of the push steps - but it does do the PGP signing step.


CategoryHowTo CategoryProcess

WorkingGroups/ToolChain/GCC/ReleaseProcess (last modified 2016-09-16 13:08:54)