Bzr Tips

Linaro's toolchain components are maintained in bazaar branches.

The GCC branches are derived from SVN imports of the FSF GCC. These were created from the imports created by Vincent Ladeuil (lp:~vila/gcc/4.4 and lp:~vila/gcc/4.5):

    bzr push -r tag:gcc_4_4_4_release lp:~gcc-linaro-dev/gcc-linaro/4.4
    bzr push -r tag:gcc_4_5_0_release lp:~gcc-linaro-dev/gcc-linaro/4.5

GCC's long history and tree size are not trivial to manage in bzr, see for instance https://bugs.launchpad.net/bzr/+bug/593560 which exposed some bzr issues but also some local setup isues. Please file bugs for any serious performance issues!

Day to day use

The best way to user bzr for toolchain development is to make a local shared branch and then create feature branches in the same directory for individual chunks of work. A shared branch reduces the amount of data that goes across the network, reduces the size of a feature branch on disk, and reduces the time to make a branch.

To create the shared branch:

    bzr init-repo gcc-linaro
    cd gcc-linaro
    # this will download all the revisions of the 4.7 branch
    bzr branch lp:gcc-linaro/4.7
    # this will only download the revisions from 4.6 which weren't
    # downloaded as part of the 4.7 branch already
    bzr branch lp:gcc-linaro/4.6

gcc-linaro ends up containing the common data while 4.7 and 4.6 contain the differences.

Periodically pull down any trunk changes by running cd 4.7; bzr pull. If you have a long-lived feature branch, consider merging the trunk out to your feature branch using cd 4.7; bzr pull; cd ../branch-name; bzr merge.

Create a feature branch by running bzr branch 4.7 branch-name such as bzr branch --hardlink 4.7 lp-605080. This takes 58 s on MLH's machine and creates a ~24 MB .bzr directory.

If you make a big mistake and want to throw away the working copy and start again, try rm -rf *; bzr revert.

Once the change is complete, update the ChangeLog.linaro and commit using bzr commit. Use bzr commit --fixes=lp:ticket-number if the commit is related to a Launchpad ticket.

Push the changes up to a user branch using bzr push lp:~user-name/gcc-linaro/branch-name such as bzr push lp:~michaelh1/gcc-linaro/lp-605080. By default bzr pushes against a stacked branch, which uses an already-uploaded branch to reduce the amount of data shifted about. On MLH's machine, bzr pushes 20 MB at about 60 kB/s for even a small change. Note that bzr push doesn't resume interrupted transfers.

Your new branch should be visible at https://code.launchpad.net/gcc-linaro, such as https://code.launchpad.net/~michaelh1/gcc-linaro/lp-605080.

Try bzr lp-open to jump straight there.

Click on 'Propose for merging' and fill in the merge request. Within 30 minutes, the auto builders will snapshot the branch, schedule the build, and post a comment to the merge request. Lauchpad may take a long time to email but the job will probably have started before. Have a look at the scheduler page: http://cbuild.validation.linaro.org/helpers/scheduler.

You will probably received an Internal Error email from launchpad; don't worry, it's caused by GCC being a very large code base.

After approval, the last step is to merge your changes into bzr head. Say you store all of your GCC branches in ~/linaro/gcc, have a branch of lp:gcc-linaro in ~/linaro/gcc/4.7 and that your branch to merge in is ~/linaro/gcc/lp-12345. Then:

  • Run cd 4.7 to change into the lp:gcc-linaro directory

  • Run bzr pull to make sure it is up to date

  • Run bzr status to make sure there are no local changes

  • Run bzr merge ../lp-12345 to merge your branch in

  • Run bzr status to check that the files were updated

  • Run bzr commit -m "Description of what the branch does" to commit the merge

  • bzr push to push the merged changes back up to bzr

It's likely that you'll get a conflict on the ChangeLog.linaro. Add something like:

[/home/michaelh/linaro/gcc]
changelog_merge_files = ChangeLog.linaro

to ~/.bazaar/locations.conf to get bzr to merge automatically. Check the result before committing.

Tips

Here are some random tips on using bzr efficiently:

  • "bzr checkout" and "bzr branch" are essentially the same thing, they will get you a local copy of a branch with all history downloaded locally, the only difference is that whenever you commit from the former type, it will commit to the remote branch, while commits you do from the latter type will be local and you have to push them afterwards; you can switch a branch to a checkout with "bzr bind" and "bzr unbind" does the reverse
  • "bzr checkout --lightweight" should avoid some network traffic by only downloading the latest revision; it should also use less local disk space
  • /!\ bzr repositories allow sharing of branch data (revisions) across sub-directories; this is a crucial piece to speed up operations:

There are gcc-linaro 4.4 and 4.5 series too so that one can checkout lp:gcc-linaro/4.4 and lp:gcc-linaro/4.5.

  • /!\ make sure you setup bzr for Launchpad use; you can check/fix who bzr thinks you are with "bzr whoami" and you can check/fix the Launchpad account which bzr thinks should be used with Launchpad with "bzr lp-login"; setting the Launchpad account will cause bzr to use bzr+ssh:// URLs instead of http:// URLs to download bzr branches, which should be more efficient

Untried tips

Give these a try. If they work for you, update the wiki page!

Have a look at Andrew's email:

I believe hard-linking should be a win too, but I don't use it much:

    bzr branch --hardlink my_1st_branch my_2nd_branch

I find "bzr push" is quite fast, but there's a special gotcha - it
always stacks new feature branches on top of the gcc-linaro/4.5 branch
(more accurately, the "focus of development" branch), and if you're
working with 4.4 or 4.6, that means there quite a lot of difference to
upload.

You can fix this by telling it what branch to stack on:

bzr push
--stacked-on=bzr+ssh://bazaar.launchpad.net/~linaro-toolchain-dev/gcc-linaro/4.6
lp:~<user>/gcc-linaro/<newbranch>

Unfortunately, the "lp:..." form doesn't work with --stacked-on. :( (https://bugs.launchpad.net/bugs/296592)

Have a look at Loic's email:

 bzr has plugins to merge changelog entries for some types of
 changelogs; I wonder whether we could use these here.  Another option
 would be to generate a GNU ChangeLog from the bzr log at release time
 as we do for linaro-image-tools for instance.

Have a look at Martin's email:

For big trees, I would really recommend you try out 'bzr-colo', which
makes it easier to reuse the same working tree across multiple
branches.  'bzr colo-branch lp-foo' should be pretty fast, and won't
need to create a whole new tree.  merge etc may be faster too.

http://doc.bazaar.canonical.com/plugins/en/colo-plugin.html

How to do a manual bzr merge when "bzr merge" fails

According to the nice folks at #bzr, here's how to do the same merge manually:

  • bzr branch lp:gcc-linaro/4.7

  • cd gcc-linaro

  • bzr log | less

  • find the last merge revision (it should be clear from the message)
  • grab the *SVN* revision number
  • bzr log --show-ids lp:gcc/4.7 | less

  • search for the *SVN* revision number
  • (it should appear on the end of a "revision-id" line, not "parent")
  • grab the corresponding *BZR* revision number ("revno")
  • bzr diff -r <bzr-revno> lp:gcc/4.7 > ../patch

  • patch -p0 -i ../patch

  • Handle binary files which have been modified but won't be changed by the patch: grep 'Binary files.*differ' ../patch

  • If merging between two branches which have backports from another branch note that some files may be both destroyed and created in the same patch.
  • resolve conflicts, rejected hunks, etc.
  • remove all .orig and .rej files: find . \( -name "*.rej" -or -name "*.orig" \) -exec rm {} \;

  • bzr add --file-ids-from lp:gcc/4.7 | tee ~/log.txt

  • see messages like: "adding <file> w/ file id from <file>"

  • check for any other added files that may have been left in the tree: grep -v "file id from" ~/log.txt

  • edit Changelog.linaro, as usual
  • bzr ci

  • bzr push lp:~.........gcc-linaro/merge-from....

After all that, a future "bzr merge" should just work (once the bug has been fixed).

WorkingGroups/ToolChain/BzrTips (last modified 2012-12-04 23:11:57)