Updating Raspberry Pi boot firmware files using Yocto Project and Mender

Introduction

The Raspberry Pi family of boards are undeniably very popular, both for hobbyists and in professional settings. Regardless of what I think of the latter, it is the reality and for this reason this tutorial was created to look at one of the limitations of the Raspberry Pi family boards and how to work around it.

In the context of supporting robust system level software updates, Raspberry Pi boards are actually very complicated to work with. This primarily relates to the fact that the Raspberry Pi has a set of firmware files that it needs to boot, which are impossible to update robustly and atomically. These firmware files reside on the vfat boot partition and content is something like this:

bcm2708-rpi-b.dtb       bcm2708-rpi-zero-w.dtb  bcm2710-rpi-3-b-plus.dtb  boot.scr       fixup4cd.dat  fixup_cd.dat  issue.txt     kernel.img        start4db.elf  start_db.elf
bcm2708-rpi-b-plus.dtb  bcm2709-rpi-2-b.dtb     bcm2710-rpi-cm3.dtb       cmdline.txt    fixup4.dat    fixup.dat     kernel7.img   LICENCE.broadcom  start4.elf    start.elf
bcm2708-rpi-cm.dtb      bcm2710-rpi-2-b.dtb     bcm2711-rpi-4-b.dtb       config.txt     fixup4db.dat  fixup_db.dat  kernel7l.img  overlays          start4x.elf   start_x.elf
bcm2708-rpi-zero.dtb    bcm2710-rpi-3-b.dtb     bootcode.bin              COPYING.linux  fixup4x.dat   fixup_x.dat   kernel8.img   start4cd.elf      start_cd.elf  uboot-git-log.txt

For more information about the files and how they are using during the boost process, take a look at this post. Note that when you are using Mender, you are booting U-Boot first instead of the Linux kernel directory. This is because Mender integrates with U-Boot to be able to switch between partitions.

What is important to know is that the firmware files have a relationship to the Linux kernel. Even if this is not explicit, history has shown that there have been firmware changes that require you to update the Linux kernel or the other way around. This means that through the lifetime of your device, you should expect that you need to update these firmware files.

Why is updating these files a problem?

When we talk about robust OTA software updates, we mean that it is tolerant to failures, e.g power-loss during the update process, and the device should always rollback to a known working state should something go wrong. For root filesystem updates, this can be solved by having redundancy (A/B setup), meaning you have two root filesystem partitions and you switch between them in an atomic way, ensuring that you always have something to fallback to in case of failure. It is not possible to implement this fail-safe mechanism for the Raspberry Pi firmware files, as you only have one location to store them and this location is hardcoded in the GPU (yes, GPU) boot ROM code on the Raspberry Pi. If these files are corrupted your device will stop booting.

Another problem with updating these files, is that it is not possible to do this atomically. This means that you can end up with 50 % of the files being updated in case you are interrupted by a power-loss or reset of the device. This puts the device in an unknown state, and if you are lucky it might still boot, if not it will not.

As we mentioned earlier you will probably need to update these files through the lifecycle of your device which means that you need to accept this risk upfront. In this tutorial we will explain how you can update these files (non robustly) using the Yocto Project and Mender. We will use a Raspberry Pi 3 as an example, but this is applicable to all boards and SoM’s within the family.

This tutorial assumes that you are already familiar with Mender and the Yocto Project, and you are looking for a way to update the Raspberry Pi firmware.

Prerequisites

  • A Raspberry Pi 3 Model B or B+ with 8 GB of microSD as a storage medium.
  • A supported Linux distribution and dependencies installed on your workstation/laptop as described in the Yocto Mega Manual
    • NOTE. Instructions depend on which Yocto version you intend to use.
  • Google repo tool installed and in your PATH.

Step 1 - Initial setup of the Yocto Project environment

Set the Yocto Project branch you are building for:

export BRANCH="warrior"

Create a directory for your mender-raspberrypi setup to live in and clone the meta information.

mkdir mender-raspberrypi && cd mender-raspberrypi

Initialize repo manifest:

repo init \
    -u https://github.com/mendersoftware/meta-mender-community \
    -m meta-mender-raspberrypi/scripts/manifest-raspberrypi.xml \
    -b ${BRANCH}

Fetch layers in manifest:

repo sync

Setup the build environment:

source setup-environment raspberrypi

Set 2 - Enable update of firmware files

In meta-mender/meta-mender-raspberrypi we already have the necessary functionality that ensures that:

  • boot firmware files are installed into the root filesystem. This is needed if we are to copy them to the vfat partition
  • a state script is added to the Mender Artifact that will copy the firmware files to the vfat partition in the ArtifactInstall_Leave state

You can find more details in the pull-request that enabled this.

To enable mentioned functionality simply add the following to e.g local.conf:

INHERIT += "rpi-update-firmware"

And now run a build as you normally would:

bitbake core-image-base

The output will be a Mender Artifact that will also update the files found on the vfat partition. NOTE! Updating these files is a non-reversible operation and you are risking to brick your devices by doing this, e.g. if you experience a power loss during the update process.

When you have the Artifact make sure to disable this feature by removing the following from e.g local.conf

INHERIT += "rpi-update-firmware"

I would only recommend enabling rpi-update-firmware in situations where it is absolutely necessary, and is not something that you would keep enabled for all of your Artifacts.

Conclusion

For some additional background information on this topic check Updating a Raspberry boot partition.

The limitation of the Raspberry Pi firmware files is there and needs to be taken into account. In this tutorial we have presented a way to update these files, but this a non-robust workaround.

In an ideal world everything would reside in one location, e.g inside the root filesystem. This way we could utilize the available redundancy and fail-safe mechanisms for all parts of the system. Even though not to the same extent, this problem does exist on other ARM boards where the norm is to keep the bootloader in a isolated location on the storage medium, and updating the bootloader is hard to do robustly without support for it in hardware, which is why you would typically avoid updating the bootloader through the lifecycle of your device.

It is very important to keep the bootloader slim, and have very little logic there as you will have a problem updating it later once your devices are in the field and you find issues. The Raspberry Pi is a cautionary example of what can go wrong when you have too much logic in the boot process.


If this tutorial was useful to you, please press like, or leave a thank you note to the contributor who put valuable time into this and made it available to you. It will be much appreciated!

3 Likes