Mender-validation: automatic integration checking

Board Integrations
1

Motivation

Validating the boot loader integration of Mender is crucial for a robust product as it ensures seamless and secure firmware updates throughout its entire lifecycle. However, manual testing of this critical component can be time-consuming, error-prone, and impractical for complex systems. A more efficient approach is an automated test procedure that checks the various scenarios which the bootloader integration needs to handle. The mender-validation suite aims to provide an automated test procedure which not only saves time and resources but also provides a scalable solution for continuous testing and verification, making it an optimal choice for ensuring the reliability and integrity of throughout the full maintenance period of the product.

Concept

The testing procedure for the bootloader integration is described in the client integration checklist.

The mender-bootloader-validation.py script tests the following functionality parts of the bootloader integration according to the above mentioned checklist:

  • A/B switch
  • A/B update
  • A/B rollback

Requirements:

  • the device must be able to reboot multiple times
  • a writeable persistent storage area to keep the current state of the process. Currently, the implementation expects this to be /data.
  • both A and B (active and inactive) partitions to be fully populated
  • the script to be executable from both root filesystem partitions
  • a functional python3 installation including full standard libary

Standalone / manual usage

For manual usage, it is recommended to place the script in the persistent storage mounted at /data so it is executable from both root filesystems.

Download the script:

cd /data
wget [https://raw.githubusercontent.com/mendersoftware/mender/master/support/modules-artifact-gen/single-file-artifact-gen](https://raw.githubusercontent.com/TheYoctoJester/mender-validation/refs/heads/main/mender-bootloader-validation.py)

Make the script executable:

chmod a+x mender-bootloader-validation.py

Run the script:

./mender-bootloader-validation.py

The script places its output in /data/validation.log. You need to reinvoke the script after it initiated a reboot, in the success case up to four times. Check the output in /data/validation.log to understand which checks have been carried out and what the results are.

In the success case, you should see the lines

INFO:__main__:rollback test successful
...
INFO:__main__:new step is end

This concludes a successful run of the bootloader integration test. If the log file ends in other messages, inspect the logged messages to identify actionable areas to complete your integraion.

Automated usage (Yocto)

The meta-mender-validation layer provides the bootloader-validation recipe which installs the mender-bootloader-validation.py script alongside a systemd service mender-bootloader-validation.service to automatically run the script upon device start up.

In order to use it, the following modifications of your existing Yocto build are required:

In local.conf:

MENDER_FEATURES_ENABLE:append = " mender-prepopulate-inactive-partition"
IMAGE_INSTALL:append = " bootloader-validation"

The mender-prepopulate-inactive-partition feature requires using a meta-mender revision later than 3d6099368742f1a461fb0aa14eccd2284c98f3b2 or cherry-picking accordingly

In bblayers.conf:

BBLAYERS ?= " \
  ${TOPDIR}/../meta-mender-community/meta-mender-validation \
"

Make sure that the path of the meta-mender-community layer matches your local setup.

The resulting image will reboot up to five times after flashing and powering up. Check the /data/validation.log file for results of the validation script.

Automated usage (mender-convert)

TBD real soon :slight_smile:

Future improvements

  • support for custom bootloader integrations beyond mender-uboot and mender-grub
  • automated installation of a systemd service unit in the standalone/manual flow
  • upload of the resulting log to a defined location
  • … :slight_smile:
2 Likes