Robust delta update rootfs

Description

An Update Module to install binary delta rootfs updates on a device. This Update module is available with a commercial Mender plan. Please refer to the Mender.io product page for more information about commercial plans.

Example use-cases:

  • Perform a rootfs update using far less bandwidth than a traditional full-image update

Specification

Specification
Module name mender-binary-delta
Supports rollback yes
Requires reboot yes
Artifact generation script yes
Full system updater yes
Source code Closed source

Prerequisites

  • An existing Yocto environment with a Mender board integration working to be able to deploy β€œrootfs-image” updates
  • Ability to use read only root file system in your build (how to enable it is shown below, Delta updates will not work without it)
  • mender-binary-delta binary tools. Get can be downloaded using this link (requires credentials, please email contact@mender.io to receive your credentials)
  • mender-artifact version 3.1.0 or newer on you development machine. You can download the pre-built binary from here
  • Mender client version 2.0.0 or newer

You can enable Mender client version 2.x.x in Yocto by adding the following in your local.conf:

PREFERRED_VERSION_pn-mender = "2.%"
PREFERRED_VERSION_pn-mender-artifact = "3.%"
PREFERRED_VERSION_pn-mender-artifact-native = "3.%"
  • mender-binary-delta requires β€œread-only” rootfs

If you do not already have β€œread-only” mode on your rootfs images you can enable this with adding the following in e.g local.conf

IMAGE_FEATURES_append = " read-only-rootfs"

It is a good idea to enable this first and test that your system is still operating as intended, before proceeding in testing mender-binary-delta.

Unpacking mender-binary-delta

You should have received an tar archive named mender-binary-delta-1.0.0b1.tar.xz which contains the binaries needed to generate and apply deltas. The content of the archive should be:

.
β”œβ”€β”€ aarch64
β”‚   β”œβ”€β”€ mender-binary-delta
β”‚   └── mender-binary-delta-generator
β”œβ”€β”€ arm
β”‚   β”œβ”€β”€ mender-binary-delta
β”‚   └── mender-binary-delta-generator
└── x86_64
    β”œβ”€β”€ mender-binary-delta
    └── mender-binary-delta-generator

As you might have guessed, we have prepared binaries for the armhf and x86_64 architectures. If your platform architecture does not match the provided binaries, please contact us and we should be able to compile binaries for your target.

The mender-binary-delta binary is intended to be installed on the device on which you intend to perform delta updates on.

The mender-binary-delta-generator is intended to be used on your development machine to generate deltas between two Mender Artifacts. You can use the x86_64 mender-binary-delta-generator even if your device is armhf, the generator tool is architecture agnostic.

Change directory to $HOME:

cd ${HOME}

Unpack the mender-binary-delta-1.0.0.tar.xz in your home directory:

mkdir mender-binary-delta-1.0.0 && tar xvf mender-binary-delta-1.0.0.tar.xz -C mender-binary-delta-1.0.0

Integrating mender-binary-delta to an existing Yocto environment

It is assumed that you already have a Yocto Project environment configured using meta-mender and that you are in the build directory.

Add meta-mender-commerical layer to your Yocto environment:

bitbake-layers add-layer ../sources/meta-mender/meta-mender-commercial

Add the following your local.conf to include mender-binary-delta in your build:

cat <<EOF >> conf/local.conf
# Customizations for Mender delta-update support

# This will not impact your image if you already have this enabled
IMAGE_FEATURES_append = " read-only-rootfs"

IMAGE_INSTALL_append = " mender-binary-delta"
LICENSE_FLAGS_WHITELIST = "commercial_mender-binary-delta"
FILESEXTRAPATHS_prepend_pn-mender-binary-delta := "${HOME}/mender-binary-delta-1.0.0/:"

EOF

NOTE! The trailing : in FILESEXTRAPATHS_prepend_pn-mender-binary-delta is intentional and important to have in place.

Generating images, artifacts and deltas

For the purpose of this tutorial you can set the MENDER_ARTIFACT_NAME to the following:

MENDER_ARTIFACT_NAME = "release-v.1.0"

Set helper variables (update with the appropriate values used in your environment):

TARGET_IMAGE="core-image-base"
MACHINE="raspberrypi4"

Build an image (target image might differ in your environment):

bitbake ${TARGET_IMAGE}

Use the output of the build to provision your device with e.g with the provided disk image (e.g .sdimg) file. The result should be that you have a device that has booted and has the following file installed:

# ls /usr/share/mender/modules/v3/mender-binary-delta -alh
-rwxr-xr-x    1 root     root       98.2K Aug 19 08:52 /usr/share/mender/modules/v3/mender-binary-delta

Please verify this before proceeding.

Save the generated Mender Artifact for later (will be used to generate a delta):

cp tmp/deploy/images/${MACHINE}/${TARGET_IMAGE}-${MACHINE}.mender ${HOME}/mender-binary-delta-1.0.0/x86_64/release-v.1.0.mender

Generate a second Mender Artifact which can be used to test deploying delta updates. Start of with making a change to the image content, e.g add an application:

cat <<EOF >> conf/local.conf

IMAGE_INSTALL_append = " vim"

EOF

Update the MENDER_ARTIFACT_NAME to the following:

MENDER_ARTIFACT_NAME = "release-v.2.0"

Build the new image (target image might differ in your environment):

bitbake ${TARGET_IMAGE}

Save the generated Mender Artifact for later (will be used to generate a delta):

cp tmp/deploy/images/${MACHINE}/${TARGET_IMAGE}-${MACHINE}.mender ${HOME}/mender-binary-delta-1.0.0/x86_64/release-v.2.0.mender

Change directory mender-binary-delta:

cd ${HOME}/mender-binary-delta-1.0.0/x86_64/

The content of this directory should be the following:

$ ls
mender-binary-delta  mender-binary-delta-generator  release-v.1.0.mender  release-v.2.0.mender

Generate delta for v1.0 -> v2.0 update path:

./mender-binary-delta-generator -n v2.0-deltafrom-v1.0 -o v2.0-deltafrom-v1.0.mender release-v.1.0.mender release-v.2.0.mender

Generate delta for v2.0 -> v1.0 update path:

./mender-binary-delta-generator -n v1.0-deltafrom-v2.0 -o v1.0-deltafrom-v2.0.mender release-v.2.0.mender release-v.1.0.mender

Using the delta Mender Artifacts

We should have now have the Mender Artifacts ready and you can upload them to the Mender server. The uploaded files should be:

  • release-v.1.0.mender (rootfs-image)
  • release-v.2.0.mender (rootfs-image)
  • v1.0-deltafrom-v2.0.mender (mender-binary-delta)
  • v2.0-deltafrom-v1.0.mender (mender-binary-delta)

If you have followed this tutorial your provisioned device should be running release-v1.0 which was provisioned using a disk image (e.g sdimg).

First deploy a full image update (Yocto Project thud or older)

If you are using Yocto Project warrior (2.7) or newer you can skip this section, as it is not required.

If you are using Yocto Project version thud (2.6) or older, you must deploy a rootfs-image update once after you have provisioned your device, e.g using the disk image (sdimg). This is needed because the checksum of the file system image is not the same in an sdimg and in a mender file, even if they are generated during the same build. And because we generated the delta Artifacts from the mender files, we must deploy one of these first before we can proceed in deploying deltas. As mentioned this is only needed once after you have provisioned a device, after that is done you can use delta Artifacts. This requirement may be lifted in the future, see this ticket for more information.

Deploy delta updates

Start of with deploying release-v.2.0.mender to the device, and once that is completed you can deploy v1.0-deltafrom-v2.0.mender and after one is completed you can deploy v2.0-deltafrom-v1.0.mender.

3 Likes