Introduction

This section describes how to setup and use an OpenEmbedded build environment including the Linaro layer. The meta-linaro layer focuses on toolchain components and employs recipes for building the Linaro GCC the OE way.

The layer also extends the default OE Core kernel recipe (yocto) to build a kernel for the qemuarmv7a MACHINE using a vexpress defconfig in order to get an image that targets ARMv7 and can be booted using QEMU.

For builds with "master" branch of OpenEmbedded Core some more work is needed due to gcc-4.6 being removed from it (described later).

Prepare build environment

prerequisites

This documentation assumes that build system is a x86(_64) machine running Ubuntu 12.04 (Precise). To satisfy requirements those packages needs to be installed:

  • sudo apt-get install sed wget cvs subversion git bzr coreutils unzip bzip2 tar gzip cpio gawk python patch diffstat make build-essential gcc g++ desktop-file-utils chrpath autoconf automake libgl1-mesa-dev libglu1-mesa-dev libsdl1.2-dev texi2html texinfo

fetch the sources

common part

mkdir oe && cd oe
git clone git://git.linaro.org/openembedded/meta-linaro.git
git clone git://git.openembedded.org/openembedded-core
cd openembedded-core
git clone git://git.openembedded.org/bitbake bitbake

master branch builds

git clone git://git.openembedded.org/meta-openembedded

If you do not use "meta-openembedded/meta" layer then you need to remove mysql recipe addon from meta-linaro:

rm meta-linaro/recipes-lamp/mysql/*

denzil branch builds

cd meta-linaro
git checkout -b 4.7-2012.06 4.7-2012.06
cd ..
cd openembedded-core
git checkout -b 363424c 363424c
cd bitbake
git checkout -b f8bf449 f8bf449
cd ..

extra patches

Most of the patches to OE Core that are required for the meta-linaro are upstream already. Currently there is only one optional patch that removes the -no-tree-vectorize GCC option for armv7a machines.

wget -q -O - http://people.linaro.org/~kwerner/oe-core/patches/armv7a-tree-vectorize.patch | patch -p1

setup the environment

The next step is to create the build directory and setting up an approriate configuration.

cd openembedded-core
. ./oe-init-build-env ../build

add the Linaro layer

Add the Linaro meta layer by editing the conf/bblayers.conf. The BBLAYERS variable should look like this:

BBLAYERS = " \
  ${TOPDIR}/../meta-linaro \
  ${TOPDIR}/../openembedded-core/meta"

For 'master' branch builds there is one more layer required so BBLAYERS will look like this:

BBLAYERS = " \
  ${TOPDIR}/../meta-linaro \
  ${TOPDIR}/../meta-openembedded/toolchain-layer"
  ${TOPDIR}/../openembedded-core/meta"

adjust your conf/local.conf

# set the default machine and target
MACHINE = "qemuarmv7a"
DEFAULTTUNE_qemuarmv7a = "armv7athf-neon"

# specify the alignment of the root file system
# this is required when building for qemuarmv7a
IMAGE_ROOTFS_ALIGNMENT = "2048"

Optionally alter the following variables depending on your build machine:

BB_NUMBER_THREADS
PARALLEL_MAKE

Which values? BB_NUMBER_THREADS controls how many tasks are run at same time and "4" is good amount of them. On fast machine higher values may be used (but 8 is usually more then enough). PARALLEL_MAKE is "-j" argument for GNU Make - usually amount of cpu cores are used.

choose a toolchain

By default the toolchain provided by OE Core gets used. In order to build using the Linaro GCC those lines needs to be added to conf/local.conf:

GCCVERSION = "linaro-4.7"
SDKGCCVERSION = "linaro-4.7"

The choices are:

  • '4.6%' - GCC 4.6 based toolchain provided by OE Core (denzil) or OpenEmbedded (master)

  • '4.7%' - GCC 4.7 based toolchain provided by OE Core
  • 'linaro-4.6' - Linaro GCC 4.6 based toolchain provided by meta-linaro
  • 'linaro-4.7' - Linaro GCC 4.7 based toolchain provided by meta-linaro

build the images

First, bitbake will build some native tools like m4, autoconf, libtool and install them into a sysroot. The tools of the sysroot and the binary toolchain are used to compile the sources for the target including a Linaro 3.x based Linux kernel configured for the versatile express board. It all gets packaged up (ipk by default) and installed into a root fs.

# just enough to get a busybox prompt
bitbake core-image-minimal

# Xorg, gtk+2
bitbake core-image-sato

# Qt 4 embedded
bitbake qt4e-demo-image

The images will land in:

# the image output lands at:
ls -l tmp-eglibc/deploy/images/
# kernel:
ls -l tmp-eglibc/deploy/images/zImage-qemuarmv7a.bin
# fs image:
ls -l tmp-eglibc/deploy/images/*-qemuarmv7a.ext3

Using images

use QEMU to boot the images

As part of the OE Core build process qemu will be compiled. There is a wrapper script called runqemu that can be used to run the default OE Core images. Since we added a new qemuarmv7a MACHINE the runqemu script wouldn't work for us. However, we can easily run qemu manually:

  • minimal image
    tmp-eglibc/sysroots/x86_64-linux/usr/bin/qemu-system-arm \
     -M vexpress-a9 -m 1024 -serial stdio -display none \
     -snapshot -no-reboot \
     -kernel tmp-eglibc/deploy/images/zImage-qemuarmv7a.bin \
     -drive file=tmp-eglibc/deploy/images/core-image-minimal-qemuarmv7a.ext3,if=sd,cache=writeback \
     --append "rw console=ttyAMA0,38400n8 console=tty root=/dev/mmcblk0"
  • sato image
    tmp-eglibc/sysroots/x86_64-linux/usr/bin/qemu-system-arm \
     -M vexpress-a9 -m 1024 -serial stdio \
     -snapshot -no-reboot -show-cursor \
     -kernel tmp-eglibc/deploy/images/zImage-qemuarmv7a.bin \
     -drive file=tmp-eglibc/deploy/images/core-image-sato-qemuarmv7a.ext3,if=sd,cache=writeback \
     --append "rw console=ttyAMA0,38400n8 console=tty root=/dev/mmcblk0"
  • Qt4e image
    tmp-eglibc/sysroots/x86_64-linux/usr/bin/qemu-system-arm \
     -M vexpress-a9 -m 1024 -serial stdio \
     -snapshot -no-reboot \
     -kernel tmp-eglibc/deploy/images/zImage-qemuarmv7a.bin \
     -drive file=tmp-eglibc/deploy/images/qt4e-demo-image-qemuarmv7a.ext3,if=sd,cache=writeback \
     --append "rw console=ttyAMA0,38400n8 console=tty root=/dev/mmcblk0"

kernel debugging

It's quite convenient to debug the Linux kernel using QEMU. All you need is to install a GDB that is able to debug ARM code and to add "-s -S" to the qemu command line (-s: starts gdbserver stub on tcp::1234, S: freezes the CPU at startup).

# prereqs:
sudo apt-get install gdb-multiarch

# start qemu (see above) and add the "-s -S" options
tmp-eglibc/sysroots/x86_64-linux/usr/bin/qemu-system-arm <other args> -s -S &

# then start the gdb and attach it to QEMU
gdb-multiarch tmp-eglibc/work/qemuarmv7a-oe-linux-gnueabi/linux-linaro-*/git/vmlinux
(gdb) target remote localhost:1234
(gdb) directory tmp-eglibc/work/qemuarmv7a-oe-linux-gnueabi
(gdb) break do_execve
(gdb) continue

cleanup

bitbake -ccleansstate world

# if you want to remove everything you may type:
rm -rf sstate-cache tmp* pseudodone

denzil release

The latest release of OpenEmbedded Core is called 'denzil' which serves as a base for Yocto 1.2. Please note that the denzil branches of OE Core and meta-linaro do not support GCC 4.7 based toolchains only 4.6. This is how you get denzil instead of master:

git clone git://git.openembedded.org/openembedded-core
git clone git://git.linaro.org/openembedded/meta-linaro.git
cd meta-linaro
git checkout -b denzil origin/denzil
cd ..
cd openembedded-core
git checkout -b denzil origin/denzil
git clone git://git.openembedded.org/bitbake bitbake
cd bitbake
git checkout -b 1.15.2 1.15.2
cd ..
  • You may aply the following patches to remove the -no-tree-vectorize GCC option and to enable Qt4.8 (which is enabled in master already):

wget -q -O - http://people.linaro.org/~kwerner/oe-core/patches/armv7a-tree-vectorize.patch | patch -p1
wget -q -O - http://people.linaro.org/~kwerner/oe-core/patches/enable-Qt4.8.patch | patch -p1

Linaro binary toolchain

If you are targeting ARMv7 you can use the Linaro binary toolchain instead of building a (Linaro) GCC from scratch. You may want to create a local meta-linaro branch using the following tag: binary-4.6-2012.06.

cd oe
wget https://launchpad.net/linaro-toolchain-binaries/trunk/2012.06/+download/gcc-linaro-arm-linux-gnueabihf-2012.06-20120625_linux.tar.bz2
tar xf gcc-linaro-arm-linux-gnueabihf-2012.06-20120625_linux.tar.bz2
cd ../build

Add something like this to your conf/local.conf:

# add the Linaro binary toolchain:
EXTERNAL_TOOLCHAIN = "${TOPDIR}/../gcc-linaro-arm-linux-gnueabihf-2012.06-20120625_linux"
TCMODE = "external-linaro"
# disable localisation
IMAGE_LINGUAS = ""
LIMIT_BUILT_LOCALES = "POSIX"

# Since the toolchain was made for a specific target you are limited to ARMv7 machines
# The following configuration has been tested:
DEFAULTTUNE_qemuarmv7a = "armv7athf-neon"

Results

One of our goals was to use Openembedded as kind of a testsuite for the Linaro binary toolchain. We created several images using different compiler flags in order to check the effects on the binary size and memory consumption. The resulting images can be found at http://people.linaro.org/~kwerner/oe-core. The -Os images are missing due to lp bug #924889 and the Qt -O0 images are missing because of lp bug #924726.

ELF sizes

  • ELF file sizes of the OE Core images compiled with the Linaro binary toolchain:
    • https://docs.google.com/spreadsheet/ccc?key=0AmsCLxCMnnISdDNQSEM2ZHIxd3dVNjZBU1JkdUxNSGc https://docs.google.com/spreadsheet/ccc?key=0AmsCLxCMnnISdDNQSEM2ZHIxd3dVNjZBU1JkdUxNSGc https://docs.google.com/spreadsheet/ccc?key=0AmsCLxCMnnISdDNQSEM2ZHIxd3dVNjZBU1JkdUxNSGc

    Here is the script to extract the sizes from the images.

memory footprint

  • Memory footprint of the OE Core images compiled with the Linaro binary toolchain:
    • https://docs.google.com/spreadsheet/ccc?key=0AmsCLxCMnnISdHFtZll0OWdiTlhpdjZaOUh6YUNwcXc

    The numbers are measured with the a small script around QEMU that uses named pipes to communicate with the serial console of the guest.

Automating the builds

shell script

cbuild

  • The Linaro Toolchain working group is using cbuild for automated testing. For a local build you might start with:

bzr branch lp:~kwerner/cbuild/oecore cbuild
cd cbuild
mkdir -p slaves/`hostname`
cd slaves/`hostname`

# create local.mk
echo 'TOPDIR = ../..' > local.mk
# publish step rsyncs into a directory called 'local' instead of
# trying and failing to push to the cbuild master
echo 'PUBLISH_URL = local' >> local.mk 
echo 'NOTIFY = ken.werner@linaro.org' >> local.mk 
echo 'NOTIFY_FROM = ken.werner@linaro.org' >> local.mk 
echo 'SMTP_HOST = your.mta' >> local.mk
# disable scheduler notification
echo 'PING_STEP = echo PING $(SCHEDULER_API)/update/$(HOST)/running/$(SUITE)-$(SNAME)$(1)' >> local.mk 

# kick off a build
make -f ../../lib/oecore.mk

patches to OE Core

  • List of patches that are required by meta-linaro and have been merged into OpenEmbedded-Core:

  • cmake.patch changes the cmake.bbclass to have cmake find the binaries of the external toolchain

  • mno-thumb-marm.patch Recent versions of GCC for ARM reject the -mno-thumb option and expects developers to use the -marm instead

  • eglibc-locale.inc.patch Add sanity checks for the eglibc locale recipe to support external binary toolchains that don't have localization support

  • IMAGE_ROOTFS_ALIGNMENT.patch adds a new variable to control the alignment of the size of the root file system

  • fix-bdwgc.patch forces the ARM_INSTRUCTION_SET to "arm" for the Hans Boehm Garbage collector because its libatomic won't build in thumb mode (fixed upstream)

  • Qt4.8.1-GCC4.7.patch fixes Qt in case a GCC 4.7 based toolchain gets used.

future work

  • add recipes for the Linaro GDB and QEMU
  • support for building the Linaro GCC from bzr tip
  • external Linaro binary toolchain: figure out how to:
    • build the (e)glibc instead of copying the one from the binary toolchain
    • run certain testsuites (Qt for example)
  • kernel: use fragments rather than a defconfig for qemuarmv7a (vexpress)

pointers

KenWerner/Sandbox/OpenEmbedded-Core (last modified 2012-07-09 16:44:56)