Introduction
The Raspberry Pi family of boards are undeniably very popular, both for hobbyists and in professional settings. For this reason this tutorial was created to look at one of the limitations of the Raspberry Pi family boards and how to work around it.
In the context of supporting robust operating system level software updates, Raspberry Pi boards are actually very complicated to work with. This primarily relates to the fact that the Raspberry Pi has a set of firmware files that it needs to boot, which are impossible to update robustly and atomically. These firmware files reside on the vfat boot partition and content is something like this:
bcm2708-rpi-b.dtb bcm2708-rpi-zero-w.dtb bcm2710-rpi-3-b-plus.dtb boot.scr fixup4cd.dat fixup_cd.dat issue.txt kernel.img start4db.elf start_db.elf
bcm2708-rpi-b-plus.dtb bcm2709-rpi-2-b.dtb bcm2710-rpi-cm3.dtb cmdline.txt fixup4.dat fixup.dat kernel7.img LICENCE.broadcom start4.elf start.elf
bcm2708-rpi-cm.dtb bcm2710-rpi-2-b.dtb bcm2711-rpi-4-b.dtb config.txt fixup4db.dat fixup_db.dat kernel7l.img overlays start4x.elf start_x.elf
bcm2708-rpi-zero.dtb bcm2710-rpi-3-b.dtb bootcode.bin COPYING.linux fixup4x.dat fixup_x.dat kernel8.img start4cd.elf start_cd.elf uboot-git-log.txt
For more information about the files and how they are using during the boost process, take a look at this post. Note that when you are using Mender, you are booting U-Boot first instead of the Linux kernel directory. This is because Mender integrates with U-Boot to be able to switch between partitions.
What is important to know is that the firmware files have a relationship to the Linux kernel. Even if this is not explicit, history has shown that there have been firmware changes that require you to update the Linux kernel or the other way around. This means that through the lifetime of your device, you should expect that you need to update these firmware files.
Why is updating these files a problem?
When we talk about robust OTA software updates, we mean that it is tolerant to failures, e.g power-loss during the update process, and the device should always rollback to a known working state should something go wrong. For root filesystem updates, this can be solved by having redundancy (A/B setup), meaning you have two root filesystem partitions and you switch between them in an atomic way, ensuring that you always have something to fallback to in case of failure. It is not possible to implement this fail-safe mechanism for the Raspberry Pi firmware files, as you only have one location to store them and this location is hardcoded in the GPU (yes, GPU) boot ROM code on the Raspberry Pi. If these files are corrupted your device will stop booting.
Another problem with updating these files, is that it is not possible to do this atomically. This means that you can end up with 50 % of the files being updated in case you are interrupted by a power-loss or reset of the device. This puts the device in an unknown state, and if you are lucky it might still boot, if not it will not.
As we mentioned earlier you will probably need to update these files through the lifecycle of your device which means that you need to accept this risk upfront. In this tutorial we will explain how you can update these files (non robustly) using the Yocto Project and Mender. We will use a Raspberry Pi 3 as an example, but this is applicable to all boards and SoM’s within the family.
This tutorial assumes that you are already familiar with Mender and the Yocto Project, and you are looking for a way to update the Raspberry Pi firmware.
Prerequisites
- A Raspberry Pi 3 Model B or B+ with 8 GB of microSD as a storage medium.
- A supported Linux distribution and dependencies installed on your workstation/laptop as described in the Yocto Mega Manual
- NOTE. Instructions depend on which Yocto version you intend to use.
- Google repo tool installed and in your
PATH.
Step 1 - Initial setup of the Yocto Project environment
Set the Yocto Project branch you are building for:
export BRANCH="warrior"
Create a directory for your mender-raspberrypi setup to live in and clone the meta information.
mkdir mender-raspberrypi && cd mender-raspberrypi
Initialize repo manifest:
repo init \
-u https://github.com/mendersoftware/meta-mender-community \
-m meta-mender-raspberrypi/scripts/manifest-raspberrypi.xml \
-b ${BRANCH}
Fetch layers in manifest:
repo sync
Setup the build environment:
source setup-environment raspberrypi
Set 2 - Enable update of firmware files
In meta-mender/meta-mender-raspberrypi we already have the necessary functionality that ensures that:
- boot firmware files are installed into the root filesystem. This is needed if we are to copy them to the
vfatpartition - a state script is added to the Mender Artifact that will copy the firmware files to the vfat partition in the
ArtifactInstall_Leavestate
You can find more details in the pull-request that enabled this.
To enable mentioned functionality simply add the following to e.g local.conf:
INHERIT += "rpi-update-firmware"
And now run a build as you normally would:
bitbake core-image-base
The output will be a Mender Artifact that will also update the files found on the vfat partition. NOTE! Updating these files is a non-reversible operation and you are risking to brick your devices by doing this, e.g. if you experience a power loss during the update process.
When you have the Artifact make sure to disable this feature by removing the following from e.g local.conf
INHERIT += "rpi-update-firmware"
I would only recommend enabling rpi-update-firmware in situations where it is absolutely necessary, and is not something that you would keep enabled for all of your Artifacts.
Conclusion
For some additional background information on this topic check Updating a Raspberry boot partition.
The limitation of the Raspberry Pi firmware files is there and needs to be taken into account. In this tutorial we have presented a way to update these files, but this a non-robust workaround.
In an ideal world everything would reside in one location, e.g inside the root filesystem. This way we could utilize the available redundancy and fail-safe mechanisms for all parts of the operating system. Even though not to the same extent, this problem does exist on other ARM boards where the norm is to keep the bootloader in a isolated location on the storage medium, and updating the bootloader is hard to do robustly without support for it in hardware, which is why you would typically avoid updating the bootloader through the lifecycle of your device.
It is very important to keep the bootloader slim, and have very little logic there as you will have a problem updating it later once your devices are in the field and you find issues. The Raspberry Pi is a cautionary example of what can go wrong when you have too much logic in the boot process.
If this tutorial was useful to you, please press like, or leave a thank you note to the contributor who put valuable time into this and made it available to you. It will be much appreciated!