STALE: needs to be rewritten KS20150104

Requirements and the Linaro Roadmap

The main vehicle for tracking high-level (or product) requirements in Linaro is a roadmap maintained as a Kanban board. The board is maintained by a Product Management, who take requests for new requirements from the TSC and management, and work with engineering to understand their sizing and timing implications.

The Roadmap

The Linaro Core Roadmap is a statement of what Linaro plans to create and deliver over time. It has the following properties:

  • It lays out a plan for the next 2+ years.
  • It covers only major initiatives.
  • It is an actual plan of delivery, not a list of possibilities.
  • It is achievable with a current view of most likely resources over the period.
  • It is presented as a timechart and allocates a timeline to each item. An item in the roadmap signals the point at which either a deliverable is ready or a significant milestone in its delivery is reached.
  • Each item is linked to the Cards that go into the next level of detail about the initiative.
  • Each item is represented by its state in the planning process.

Requirements: the Cards

A card represents a high-level requirement. The following are some guidelines for these requirements.

  • High level, substantial work that will typically take multiple man-months to deliver
  • Requirements that meet Linaro's core objectives of working on areas of common interest to Linaro's members. Examples include Linux consolidation, enablement infrastructure, core optimization, standardization of existing and new frameworks and other new open source work of interest to all members.

Cards are used to represent work in the following areas:

  • Working Group activities, which typically will be consolidation-related features and performance optimizations
  • Platform deliverables, including new products and tools
  • Office of the CTO deliverables
  • Key Linaro events, including Linaro Connect and other important summits
  • Key deliverables related to delivery of significant technology, and in particular, milestones in the ARM architectural roadmap

The following information is required in a properly formed requirement:

  • Headline: a short sentence that summarizes what the requirement aims to achieve.

  • Description: a paragraph describing in more details the requirement.

  • Acceptance Criteria: a sentence that describes a test for delivery of the feature.

  • Sponsor: the person who owns the card and is available to answer questions and upon report of delivery, verify its acceptance criteria. While this will usually be the person who initially submitted the requirement (the submitter), others can volunteer to sponsor the requirement, and the submitter can suggest an alternative sponsor.

  • Requested Priority (optional): the priority of the card as regarded by the submitter

  • Requested Iteration (optional): the iteration by which the submitter would like to see this feature delivered. Note that an iteration is the development period spanning between 2 Connect events.

Cards are maintained in our requirements management tool - JIRA. Once a card is created in JIRA, it receives a permanent ID which can be used to refer to it by URL; for instance, card 144 is accessible via the URL

http://cards.linaro.org/browse/CARD-144

The roadmap process with JIRA wiki page has information on what to do when creating a new card. There you can also find a template with the information that is expected to be in the card and the typical questions which should be answered when creating it.

Card States

Expanding on the above, cards have the following states:

New: requirements that have been submitted by the TSC or other source

Drafting

The initial requirement put into card format by Linaro, working with the sponsor

Reviewing

A draft requirement card being reviewed between TSC and WG. Cards which are reviewed can be either moved back to Drafting for refinement, or to Engineering for delivery

Scheduled: if a card is accepted at review then at this point it is scheduled for delivery by the end of a specific iteration (for instance, "2013I1")

Engineering

The card has been reviewed and accepted for engineering work - regarding blueprints they should also be defined at this point

Change-Review

Cards which are being developed for the current iteration can be marked for a change review. A change is requested by email to TSC, indicating which is the card to change, and what is the suggested change, with emphasis to the impact on delivery scope and time, risk and stakeholders involved. Changes are then reviewed by the TSC and either move the card back to Engineering (either accepting or rejecting the changes - comment is needed to indicate the decision), move the card back to drafting (in case of major changes which require replanning and new blueprints), or defer the card altogether

Cards can moved from Engineering on to the following states

Closing-review

At the end of the iteration or when Acceptance Criteria are satisfied (whichever comes first), cards can move to Closing-Review state triggering a review of the deliverables. From there, if a card is deemed incomplete (results are not approved entirely - work is missing) and should be continued then it moves back to Engineering. If, on the other hand, the card needs redefinition then it moves back to Drafting to restart its way through the process.

Delivered

After the closing review of a card's deliverables, if Acceptance Criteria are fully satisfied, then the card can move to Delivered state which is an end state for the corresponding requirements

Deferred

Cards that have, for whatever reason, been deferred to be reconsidered at a later date. Deferred cards can be marked anew as Drafting in order to be reconsidered for the roadmap

The following simplified diagram describes the key state transitions and decision points:

https://wiki.linaro.org/OPSCOM/RoadmapProcessWithJIRA?action=AttachFile&do=get&target=RoadmapProcess-v2.png source RoadmapProcess-v2.graphml

Classification

The cards are primarily classified in the following categories:

Working Groups

Kernel

Kernel WG

KWG

Linaro Enterprise Group

LEG - Linaro Enterprise Group

LEG

Linaro Mobile Group

LMG - Linaro Mobile Group

LMG

Power Management

Power Management WG

PMWG

Toolchain

Toolchain WG

TCWG

Graphics

Graphics WG

GWG

Platform

Android

Android Engineering Team

ANDROID

Builds and Baselines

Builds and Baselines Team

DEVPLAT

LAVA

Test & Validation Team

LAVA

QA Services

QA Services Team

QA Services

Other

Infrastructure

Infrastructure

Infrastructure

OCTO

Office of CTO

OCTO

For requirements shared across units, the classification records the primary owner of the deliverable. A card can be tagged with multiple units contributing to the deliverable (e.g the current UMM project is categorized as Graphics, as Jesse Barker is the primary owner, but will also be tagged with KWG, MMWG and ANDROID as additional contributors)


The Lifecycle of a Card

Principles

  • Iterations start at Linaro Connect
  • At Connect, Engineering establishes how requirements of the iteration will be implemented
  • The TSC has operational oversight over the requirements
  • Requirements can arrive at any time
  • Requirements for the current iteration do not normally change

1. New Cards (Drafting)

New cards can be proposed by:

  • TSC members,
  • Technical Leads and
  • Linaro management

Requirements will normally be raised for discussion at a TSC meeting (conference call or at Connect). We will record new requirements in the TSC meeting minutes with the tag REQUIREMENT. Once a requirement has been presented and discussed and a sponsor identified, it is the responsibility of the Linaro Product Management to formalize it by:

  • creating a new card
  • placing it into the Drafting state
  • emailing tsc@linaro.org with the contents of the card

  • adding a link to the card to the agenda for the next TSC call under the Roadmap section

For simple cases and where time is critical, email can be used to discuss and agree new requirements. However, if there is not general agreement, the requirement will be discussed at a TSC meeting.

The following diagram illustrates the high-level process of creating cards based on elicited needs, as well as preparing new cards for their review.

https://wiki.linaro.org/OPSCOM/RoadmapProcessWithJIRA?action=AttachFile&do=get&target=RequirementsDevelopment-horizontal.png source RequirementsDevelopment-horizontal.graphml

Since cards will be fleshed out by the engineering teams at Connect, all cards for the upcoming iteration must be in the Drafting state 2 weeks before the next Connect.

2. Drafting to Engineering via a Review

Once a card has been sufficiently drafted then an Engineering Review will be scheduled for it. During the review, the Tech Lead responsible for delivery may contact the sponsors for more information, will talk to their team to understand what the impact of the requirement is, and will provide some initial sizing for the task. The review is run as a regular part of the team's monthly cycle.

Metadata is added by the Tech Lead to the card in this step:

  • Original estimate: an estimate of the effort involved to complete the card.

  • Alias Card ID: a string identifying the card in a unique way. The format is <WG><YEAR>-<TITLE in CAPITALS-SEPARATED-BY-DASHES>

Cards should not sit in the Drafting queue for more than one month, and ideally the review should not take more than 2 weeks. The information collected in the review will be recorded in the card, as comments, and once the Tech Lead is satisfied the card moves into the Engineering state. It's expected that the Tech Lead will raise any serious issues with the requirement by emailing the sponsor and/or tsc@linaro.org.

During review, based on the Requested Iteration and the Tech Lead's feedback, the PMC will decide which iteration the card should be implemented, by fixing the card's fixVersion field, emailing the TSC to communicate the decision, and noting the decision in the agenda for the next TSC meeting. If there are issues with the decision the TSC will discuss until an understanding is reached.

Cards are normally not allocated to the iteration currently in development; this means that the earliest the requirement will be added for consideration is the upcoming iteration. The net effect is that even when accepted and scheduled immediately, a card may wait for the next iteration (typically 3 months) before actually being started on.

At each Connect, engineering units go over the requirements represented by cards indicated as ready for Reviewing, running one or more discussion sessions for each requirement, and defining an execution plan, blueprints and candidate assignees. At the point where the card is considered sufficiently ready by the Tech Lead, it is moved to the Reviewing state. In the spirit of agility, the plan does not need to lay out detailed work items for the iteration, but instead note the major steps required to achieve the card's Acceptance Criteria. It's expected the plan will evolve as the monthly releases progress towards the end of the iteration. The move to Engineering state should not take more than 2 weeks after Connect.

If during Connect, or in the two weeks following it, a requirement is deemed unachievable in the iteration, or there are changes which need to be marked, the Tech Lead is required to contact the sponsor and the TSC to resolve the issue. If there are changes needed the Tech Lead should describe the changes in the card (as a comment) and move the card to Change Review state. This will trigger the notifications needed for the sponsor and TSC to look at the proposed changes and act on accepting or rejecting the changes. During the change review it is also possible that a card may be deemed not ready for immediate development - it can then be moved back to drafting or be deferred. Solutions to the issue will usually involve reshaping the Acceptance Criteria, splitting the requirement up, or changing the target iteration.

One potentially common issue is cards that a tech lead classifies as epic. Epic cards must be broken into separate cards before they are moved to Engineering; this may be achieved through phasing the work, or reducing the scope overall. Phasing work presents challenges with regards to producing concise Acceptance Criteria but is an acceptable alternative if it is unclear at the outset in what exact order the components to the solution must be implemented. (e.g. "UEFI - Investigation and Project Plan", "UEFI - Prototype" as two initial cards for a UEFI project).

4. Engineering to Delivered via a Closing-Review

During the engineering cycle, blueprints that implement part of a requirement will be developed and completed. At the point where all the blueprints for the requirement have been implemented, the requirement's Acceptance Criteria are validated. If all criteria pass the tests, the card transitions into the Closing Review state. For each card getting into Closing Review, the Tech Lead should follow the instructions in this wiki section. If there is disagreement about delivery, it must be resolved and the card either moved back to Engineering for completion, or Drafting for redesign or it can be confirmed as Delivered (probably with updates to the acceptance criteria and list of deliverables).

6. End of the Iteration period

At the end of the iteration period cards which do not pass their Acceptance Criteria (and therefore are not in the Delivered state) will be considered incomplete. For each incomplete card, a Tech Lead will email the sponsor and the TSC to describe how far implementation went, what the issues around it were, and how much was actually delivered. Depending on the situation, the best option may be to split the remaining work into a new card, to move the entire card to the next iteration, or to accept partial delivery by updating the Acceptance Criteria. It is expected that the Tech Lead and TSC can consensually work out a path forward.

An Iteration Post-Mortem document will be generated that summarizes what was Delivered, what was considered incomplete, and what decisions were made to address these incompletely delivered items.

7. Deferred Requirements

A deferred requirement is one that the TSC does not believe should be placed on the current roadmap at the present time. A requirement can, by agreement with the TSC, move to Deferred from any other state at any time.

Once a requirement has been deferred, a TSC member can request that it be reconsidered at any time via a formal request to the TSC. Provided that the TSC agrees with the justification, the requirement moves back into the Drafting state and processing of it continues as normal.


Tracking Changes

It is important that the history of each requirement and the roadmap is tracked. To ensure this is done reliably, only the PMC may change card metadata.

State Changes

The JIRA cards record state changes as the card is moved between states. The card's History tab allows viewing the card's progression through its states in a single screen with the timestamps and agent clearly represented.

Managing the Card Metadata

It's expected that card metadata will be updated throughout its lifecycle, with certain restrictions. The Title, Description, Acceptance Criteria may change as understanding of the problem is improved, but once the card is moved to Engineering changes to those fields should be considered exceptions and warrant emailing the sponsor and TSC. Note that the Requested Priority and Requested Iteration should not change, maintaining the submitter's original request.

Each card stores history of its changes, with some limitations; in particular, the diffs for changes to the description and headline are not recorded. This requires that all changes to card metadata are accompanied by an added Comment that describes what actually changed.

Splitting up cards

Cards will be split up in at least two situations:

  • When a card is established to be of Epic size, it will need to be divided into multiple phases, with the initial phase usually being a prototype or investigation.

  • When an incomplete card delivers enough value to be considered partially delivered, the remaining work may be split into a new card.

When a card is split off into one or more new cards:

  1. a comment will be added to it, noting which child cards were created
  2. each child card will receive comments noting the original card

For the case where a card is part of an Epic series, the cards' headlines should all be prefaced with the same title; for instance, "Kernel Consolidation" or "Unified Memory Management".


FAQ

When does an iteration start?

Iterations begin at Linaro Connects and end immediately before. This means that iterations and calendar years are not perfectly aligned; for instance, 2012I4 runs from the October Copenhagen Connect right up to the February 2013 Connect. It also means that iterations may be longer or shorter than 3 months. There should be little practical consequence; for shorter iterations we should just try and schedule slightly less work. One thing to note is that iterations should be aligned with release milestones as this ensures that all necessary deliverables are already in (or close to be in) a release artifact (package, image, document etc).

How do we ensure that cards have all the required information?

A lot of the value of requirement cards comes from complete descriptions, expectations and in particular, from a clear statement of Acceptance Criteria. To ensure that this is filled out before we start the process of sizing and scheduling, it's expected that cards placed in the Drafting state be fully fleshed out before a Review is requested. Only when the card has all its initial data complete and the Tech Lead is happy with the content is it a candidate for moving into Engineering.

In practice, it is likely that the Tech Lead, Sponsor and potentially other TSC members refine the description and Acceptance Criteria before the card is ready for an Engineering Review. We expect each card will go through a number of these refinement iterations, and therefore a reasonable amount of lead time -- in the order of weeks -- should be provided to allow the card to be comfortably sized and scheduled.

Periodically, the TSC should meet to discuss and agree new requirements and changes to existing requirements. Input to these meetings are:

  • Proposed new requirements (shared with the TSC ahead of time through email, as per lifecycle section 1 above)
  • Changes to the requirements (for example, when a requirement is moved to Engineering)

New requirements can be presented using slides built to the slide template that will be provided.

What Happens at Linaro Connect?

Engineering teams should use their face to face time at Connect primarily to blueprint cards in the iteration's Engineering state; this involves discussing implementation, building consensus, documenting designs and noting potential work items to be executed. Tech Leads may opt to also do a review of cards in Draft state, meeting with TSC sponsors to clarify new requirements.

The TSC should:

  • Discuss the previous engineering iteration, in particular reviewing cards which were found to be incomplete during their closing review.
  • Discuss any new requirements
  • As per above, discuss draft requirements with technical leads
  • Discuss the current roadmap, suggesting future directions and overall strategy

What happens if I want to add or alter a requirement in the current iteration?

It largely depends on the size and novelty of the request. New requirements of significance should only be brought up through the process outlined in this document. In other words, the engineering teams don't expect substantial new work to be presented to them without a card and the accompanying delivery-related information.

If there is a need to make small additions or changes to work currently underway, the TSC member interested should interact with the monthly Release Planning call that is run once a month. During that call there is a specific section for additional requests and commentary. The Tech Lead has authority to decline a change for the current iteration if he considers the change excessively intrusive. For more information on the planning call, contact the Linaro Release Managers (see MeetTheTeam for contact details).

How do I visualize blueprints and overall delivery of the requirements in the Roadmap?

The roadmap board itself is only intended to capture requirements and the intended delivery period. Visualization of delivery for each of the cards will be provided as part of an updated http://status.linaro.org/ interface, which is currently under development.

As blueprints for a card are registered in Launchpad, they will include a link back to the Roadmap card. The view on status.linaro.org will allow visualizing the collection of blueprints for a card and how far they have been implemented.

How much work for a team does the Roadmap reflect?

In other words, does the Roadmap reflect /everything/ each team in Linaro is doing?

The short answer is "not everything". In general, the board will reflect only major projects underway in each team, and only new projects at that. We expect each team to have a handful of cards for each iteration, each of those cards being broken down into a number of technical projects that achieve the overall desired requirement.

If there is no card for a team represented in the Roadmap, there is reason for concern. Similarly, if a team has more than 3 cards in each state, it is likely that the cards are not of significant size, or the team is loaded beyond what the Tech Lead will be able to manage.

Many teams have significant maintenance work which will not be reflected on the board. One example is the Toolchain WG's monthly linaro-gcc release; while it is significant and provides significant value, it is assumed that we would only note in the roadmap a major enhancement iteration to it, or the creation of a new line of products, say, an LLVM-based OpenCL compiler. Teams with many ongoing product releases will naturally have less capacity for new requirements.

Finally, smaller, engineering teams may engage in small amounts of tactical work that is not directly connected to a card; if this work does amount to more than a few man-days in a month, however, the Tech Lead should instead go through the process of notifying the TSC and inquiring for a new card.

How many cards will there be in the Engineering state for a iteration?

Given the guideline described in the Requirements section in the document above, our objective is that cards represent significant work. As a rule of thumb, each engineering team should have 3 or less cards to concern itself with each iteration. This means that Linaro overall will have around 20 cards representing requirements in each iteration. The overall objective is to focus on a smaller number of high-impact, well-defined improvements that can be fully delivered within one iteration. By constraining the number of cards, the aim is to make the deliverables Linaro is producing easier to evaluate.

Isn't upstreaming unpredictable, and won't it affect delivery of Roadmap items?

The answer is mostly yes, but for a period as long as an iteration, there should be enough time to evolve an agreed idea into an accepted implementation. The true challenge with upstreaming isn't implementing a change, but instead deciding what that change should be. Cards that represent a significant upstream contribution should therefore be carefully planned to include a prototyping phase where concepts are fleshed out into demonstration code. This phase may last up to an iteration, which underlines the importance of planning ahead and announcing designs as early as possible. Engineering teams that naïvely provide patches without having done the initial investigation and formation of consensus are bound to spend time redesigning their original approach.

How do we describe a long (year to multi-year) project

Related to the question above, when a project is classified as Epic before or during an Engineering Review, the card should be split up into multiple cards. This should obey the workflow outlined in the Tracking section above, with each of the cards using a common title prefix to clearly identify they are part of a multi-iteration project.

It's important that the first in an Epic series of cards be focused on prototyping; in other words, before committing to the long-term plan, an iteration should be spent building a proof of concept implementation to allow stakeholders and upstream to understand the practical impact of the changes suggested. The Epic card may remain in the Drafting state for that period of time and only subdivided once this initial prototype is complete.

Prototype cards can be marked with a postfix "- Prototype" to clearly indicate they are precursors to an actual card series.

In practice, how does scheduling of cards work?

The PMC will take into account the original requester's suggested iteration, which will often be the next iteration upcoming. Based on feedback from the Tech Lead, the PMC will gather an opinion on the load of the team and the impact the new requirement has on existing plans and deliverables. The result of this evaluation will be a set of changes:

  • the simplest change is when a team is not over capacity for a certain iteration; in this case the card can move to Engineering state and be marked for the specific iteration and the TSC notified of the deadline

  • when the team cannot accept a new card for an iteration without changing plans, different possible scenarios will need to be considered by the Tech Lead and the PMC. The PMC is then responsible for communicating the scenarios back to the TSC and reaching a consensus.


CategoryProcess

Process/Roadmap (last modified 2015-01-04 18:12:02)