Stable Branch

draft r0

See also ../StableBranchTrial

Work items:

  • Write up the summit discussion: DONE

  • Discuss the requirements with Ubuntu: DONE

  • Discuss the work and implementation with Linaro techleads: DONE

  • Get Linaro management approval
  • Call for comments on linaro-dev: DONE

  • Finalise plan and costs

The toolchain group are considering a 'stable' branch that starts every six months, is released once a month, and only accepts bug fixes. This is to give projects near the end of their development a lower risk but still reasonably current toolchain.

User Stories

Andy is developing a new ARM based product that takes a year to make. He wants the best performance and wants to minimise the risk near the end of the project. For the first 9 months, he uses the Linaro main toolchain to get the performance improvements, and then switches to the Linaro stable release for the last three months to minimise the risk of side-effects.

Bravo is managing a distribution that releases every six months. She wants the best performance, newest features, and to minimise the risk near the ship date. For the first two months, she picks up Linaro main, reports any issues, and picks up fixes and new features through later releases. She then switches to Linaro stable and continues to report any issues and pick up fixes through later releases. For the last two months she reports any issues, but chooses how to bring them into her distribution.

Charlie produces a ARM development environment which is sold to end developers. He wants good performance, but needs a way of getting bug fixes only for the life of a release. He picks up the stable release, reports any issues, and picks up the fixes through later releases. The branch is kept open so Charlie can choose to do further work in there once the Linaro support ends.

Design

The flow would be:

  • The 20xx.yy-0 normal release occurs. This starts the six month period
  • Each month,
    • The 20xx.zz-0 release is made containing new features and fixes
    • A 20xx.yy-n release is made with fixes only
  • After six months the 20xx.ff release occurs which starts the next six month period

For example, assuming an April branch point:

  • April: 2011.04-0
  • May: 2011.05-0, 2011.04-1
  • June: 2011.06-0, 2011.04-2
  • ...
  • September: 2011.09-0, 2011.04-5
  • October: 2011.10-0 (a new cycle is started)

A release will be skipped if no change was made in that month. This will be noted on the web site and in the announce for the normal release.

The stable branch and the normal branch will track the FSF release branch. Both will be updated to the same, current SVN revision in the middle of the monthly cycle. Linaro may choose to stay at a point release if it happens in this initial two week window.

All fixes will be made first in the normal branch and then backported to the stable branch. When a new stable branch is created, it shall not regress compared to the previous stable branch. The new branch may contain new bugs due to ongoing development.

Each of the two current series shall have a stable branch. One stable branch per series shall be maintained at a time. Past branches shall not be deleted. Community patches for past stable branches shall be reviewed, accepted, and merged in a timely manor. The stable branch may contain fixes for multiple consumers.

Ubuntu

Ubuntu runs a six month cycle. In this cycle:

  • For months 0-2, Ubuntu will pick up the latest normal Linaro release
  • For months 2-4, Ubuntu will pick up the whole stable branch
  • For months 4-6, Ubuntu may pick up the stable branch

Linaro will create the release branch at the end of month two. Ubuntu will attempt to minimise the difference between Ubuntu GCC and Linaro GCC.

Methods

Linaro provides support for ARM in general, and support for x86 problems caused by Linaro changes.

For all bugs:

  • Linaro will work with the reporter to generate a (preferably reduced) test case
  • Linaro will attempt to reproduce the bug with:
    • Mainline trunk
    • The two current mainline release branches
    • The two current Linaro normal branches
    • The two current Linaro stable branches
  • If the fault exists in mainline and is x86 specific, then the reporter should contact mainline and get the problem fixed there
  • If the fault is ARM specific, then Linaro will fix it at the highest point and then backport to the other branches

For exmaple:

  • If the problem is in mainline trunk, mainline 4.6, Linaro 4.6, and Linaro stable 4.6, then Linaro will fix the problem in mainline and backport it to Linaro 4.6 and Linaro stable 4.6 at a minimum
  • If the problem is in Linaro 4.5 and Linaro stable 4.5, then Linaro will fix Linaro 4.5 then backport to Linaro stable 4.5

Discussion

Our goal is to make the Cortex-A9 more competitive by improving the performance of existing code and reducing time to market. In the compiler area this means:

  • Do the work upstream so all can pick it up
  • Backport to Linaro GCC to make the improvements available early
  • Provide support

Assume a product has goes through prototyping, development, release, and maturity phases. Our work should be appropriate for at least the development and release phases. Our work is more likely to be picked up if it is also available in the prototype phase and has a good story for maturity.

Assuming a one-off portable device product with a one year development cycle and a twenty person team, then the characteristics of the different phases are:

Prototype

Development

Release

Maturity

Amount of change

Medium

High

Medium

Low

Performance

Important

Important

Not important

Not important

Bug fixes

Not important

Important

Important

Not important

Support

Not important

Important

Important

Not important

Ease of use

Important

Important

Not important

Not important

Available out-of-the-box

Important

Not important

Not important

Not important

Workarounds

Acceptable

Prefer fix

Acceptable

Typical

Fallback plan

Not important

Not important

Not important

Important

where:

  • Amount of change is how much change there is in the product at that time. More change means more change of having problems

  • Performance is how important the best performance is. During release and later the performance should already be acceptable

  • Bug fixes is how important it is to get bug fixes to a problem. During prototyping it's fine to work around issues. During maturity the change is low and work arounds are fine. It's very important to get soild fixes during development and quick fixes during release

  • Support is how important it is to have support from Linaro at that time

  • Ease of use is how important it is to be able to use the Linaro outputs in the development at that time. Features like binary builds help here.

  • Available out-of-the-box is how important it is to be able to use the Linaro outputs without doing extra work. Making them available as the default or an option in distrobutions helps here.

  • Workarounds is how acceptable it is to have a work-around for a compiler issue. During maturity it's typical to focus a fix on the problem, including working around in the packaging or source and not the toolchain

  • Fallback plan is how important it is to have an alternative if Linaro is no longer available. Doing our work upstream makes FSF GCC an acceptable fallback for a mature product.

A more self contained, less performance critical deeply embedded product is:

Prototype

Development

Release

Maturity

Amount of change

Medium

Medium

Medium

Low

Performance

Important

Not important

Not important

Not important

Bug fixes

Not important

Important

Important

Not important

Support

Not important

Important

Important

Not important

Ease of use

Important

Important

Not important

Not important

Available out-of-the-box

Important

Not important

Not important

Not important

Work-arounds

Acceptable

Prefer fix

Acceptable

Typical

Fall-back plan

Not important

Important

Important

Important

Notice that the amount of change is lower, performance is only important during prototyping, and that having a fall back plan is important at most stages.

Assume that Linaro GCC gives the best performance fo compiled code on ARM today and we want it used in released products. Then:

Risk needs to be minimised during the release phase. This can be acheived by:

  • Doing work-arounds in the packaging or code
    • Work arounds are not always possible
    • Leaves cruft in the code
  • Running their own branch, backporting fixes, and handling integration
    • Requires a 'toolchain guy' with specialised skills
    • Means a big team (30 people+) to justify the overhead
  • Switching to a stable branch, reporting bugs, pulling in stable branch updates, and working around other faults
    • Same model as development
    • Timing is a problem due to release dates
    • Work-arounds may still have to be done to meet dates

Consumers want the same compiler family in development and release. This can be done by:

  • Picking a stable release at the start of development. FSF is an example. You don't get the best performance
  • Using Linaro during development, freezing at the start of release and using work-arounds or running their own branch
  • Using Linaro during development and switching to Linaro stable at the start of release

We want Linaro to be easy to evaluate during prototyping. The tactic here is to make Linaro part of and hopefully the default in distributions and tools such as Ubuntu, OpenEmbedded, and crosstool-ng.

Pros and cons

This section discusses the pros and cons of a stable branch and attempts to answer the questions that came up in the 2011-06-16 techleads meeting.

What is the best use of our time? Given the importance above, is a stable branch needed, what should it be like, and is there a better place for our effort?

We can spend our time in the following areas:

  • More performance
  • More support
  • More bug fixes
  • More documentation
  • More promotion (especially on using features that improve performance)
  • Adding a stable branch
  • Making the toolchain more widely available in distributions
  • Making the toolchain more easily available through binary builds

Other distros

Debian: for many reasons, Linaro GCC isn't a good fit for Debian including:

  • They use FSF only
  • They go for maximum compatibility, so the standard port is ARMv4T based (note that there is a hard float ARMv7 port under way)
  • They support a large number of architectures and would prefer to use one on all
  • The Debian cycle is long and probably too long for our release branch to help. It could help in the final months before the release

Fedora: Michael doesn't know. He suspects Linaro GCC ins't a good fit as:

  • Fedora's primary architecture is x86 only. ARM is a secondary architecture
  • Redhat have their own compiler team who may have influence
  • They go for good compatibility by compiling for ARMv5 (there is a hard float ARMv7 port underway)

MichaelHope/Sandbox/StableBranch (last modified 2011-06-30 22:50:56)