Introduction
Kubernetes as the management tool for containers is seeing an ever increasing interest and demand also in embedded Linux. Taking the next step from there is orchestration aware deployments, for example in a microservice architecture.
In the cloud world, kubernetes itself usually serves as distribution base. For resource constrained edge devices, k3s is a prominent alternative. Transferring cloud native methods to IoT/edge use cases can provide great value, and to facilitate the usage in device fleets, Mender is providing an Application Update Module for k3s/k8s.
In addition to the tutorial on using the module on a Debian-based platform, this article explains how to set up a Yocto build including the required dependencies and the Application Update Module.
The Yocto Project is an open source collaboration project that helps developers create custom Linux-based operating systems regardless of the hardware architecture.
The project provides a flexible set of tools and a space where embedded developers worldwide can share technologies, software stacks, configurations, and best practices that can be used to create tailored Linux images for embedded and IOT devices, or anywhere a customized Linux OS is needed.
Version notes
The tutorial has been verified on Debian 12, as of 2024-03-12
This tutorial uses kirkstone as the primary target, which is the current LTS release by the Yocto Project. You can find more infomation on releases here. Supported releases for following the tutorial are:
| Yocto Project | Tutorial applies | Maintenance |
|---|---|---|
| scarthgap (5.0) |  |
development |
| nanbield (4.3) | |
current stable |
| mickledore (4.2) | |
EOL |
| langdale (4.1) | |
 EOL |
| kirkstone (4.0) | |
LTS |
| honister (3.4) | |
EOL |
| hardknott (3.3) | |
EOL |
| gatesgarth (3.2) | |
EOL |
| dunfell (3.1) | |
LTS |
| zeus (3.0) | |
EOL |
| warrior (2.7) | |
EOL |
| thud (2.6) | |
EOL |
| sumo (2.5) | |
EOL |
| rocko (2.4) | |
EOL |
Please note: a failure in the “tutorial applies” column indicates that the instructions do not work without modification. Depending on the combination of release and host Linux distribution, installing other python versions or dependencies might provide a functional state.
Prerequisites
To follow this tutorial, you will need:
- A build host prepared to build using the
kastool: please follow the tutorial for an example build.
Goal
Creating a Yocto Project style image which can receive a k3s/k8s style Application Update Module artifact, as described in the tutorial.
This example uses the Raspberry Pi 4 as target device. The resulting build output can either be flashed to the SD card (using the sdimg.bz2 file), deployed through Mender (using the mender file).
If you want to use the SD card route, please note that you will either have to manually set the device up through mender setup, or add your MENDER_TENANT_TOKEN and MENDER_SERVER_URL to the build.
Quick build
The meta-mender-community repository provides a kas configuration to create an Application Update Module enabled image for the Raspberry Pi 4. Clone, setup and build like this:
mkdir mender-rpi4-aum
cd mender-rpi4-aum
git clone https://github.com/mendersoftware/meta-mender-community -b kirkstone
cd meta-mender-community
mkdir my-rpi4-aum
cd my-rpi4-aum
kas build ../kas/demos/raspberrypi4-64-app-updates.yml
Done! The image located at build/tmp/deploy/images/raspberrypi4-64 are ready!
Understanding the parts
Dependencies
In order to use k3s or k8s in the Application Update Module, it needs to be installed in the image. It is provided by the meta-virtualization. A more in-depth description on adding it is here. In the context of a real build, there are
As the module obviously needs the Mender Client too, along with the jq and xdelta3 tools, the full RDEPENDS declaration that the Application Update Module needs is:
RDEPENDS:${PN} = "jq mender-client xdelta3"
Installing the Application Update Module
The Application Update Module is architecture agnostic, as it is implemented as a shell script. Therefore it does also not need a configure nor compile step, just being copied and marked as executable. The recipe implements it this way:
inherit allarch
do_configure[noexec] = "1"
do_compile[noexec] = "1"
do_install:class-target() {
# install Application Update Module
install -d ${D}/${datadir}/mender/modules/v3
install -m 755 ${S}/src/app ${D}/${datadir}/mender/modules/v3/app
# install K8s/K3s Module
install -d ${D}/${datadir}/mender/app-modules/v1
install -m 755 ${S}/src/app-modules/k8s ${D}/${datadir}/mender/app-modules/v1/k8s
# install configuration files
install -d ${D}/${sysconfdir}/mender
install -m 755 ${S}/conf/mender-app.conf ${D}/${sysconfdir}/mender/mender-app.conf
install -m 755 ${S}/conf/mender-app-docker-compose.conf ${D}/${sysconfdir}/mender/mender-app-k8s.conf
}
For the module to be picked up in packaging, it also has to be added to the FILES variable. Note that this only applies to the module located under {datadir}, as it is a package specific path. Files under {sysconfdir} are automatically packaged.
FILES:${PN} += "${datadir}/mender/modules/v3/app"
FILES:${PN} += "${datadir}/mender/app-modules/v1/k8s"
Dynamic layer recipes
The Application Update Module recipe only makes sense if the meta-virtualization layer is also enabled. However, a dependency on the layer does not make sense as other Update Modules will work perfectly well without it. The solution is to place the recipe under a dynamic layer structure. The canonical place for this is dynamic-layers/<layer> with <layer> denoting the layer which needs to be present. However, this is just a convention for clarity. The actual functionality resides in conf/layer.conf:
BBFILES_DYNAMIC += " \
virtualization-layer:${LAYERDIR}/dynamic-layers/virtualization/*/*/*.bb \
virtualization-layer:${LAYERDIR}/dynamic-layers/virtualization/*/*/*.bbappend \
"
This declares that the .bb and .bbappend files in the dynamic-layers/virtualization hierarchy are only included in the build if the virtualization-layer string is present in the BBFILE_COLLECTIONS variable.
Putting the Application Update Module recipe together
All of the aforementioned pieces are present in the meta-mender-update-modules layer in the meta-mender-community repository, so the Application Update Module can be installed right away. Note that this also works without having a full board integration! You can use the Application Update Module to manage the container orchestration without any deeper integrations required.
The demo setup
A complete image consists of the recipes to create the installed packages, but also need an image recipe. Based on the core-image-full-cmdline recipe, the custom image recipe mender-app-update-image.bb brings:
IMAGE_INSTALL:append = " \
mender-app-update \
packagegroup-k3s-host \
packagegroup-k3s-node \
ca-certificates \
kernel-modules \
"
This is provided in the meta-mender-demo-app-updates layer, located under kas/demos. It adds the described Application Update Module, along with the k3s packagegroups and the necessary parts to make those functional.
Plus, in order for k3s to be fully functional, the following kernel command line addition is required:
CMDLINE_ROOTFS:append = " cgroup_enable=cpuset cgroup_memory=1 cgroup_enable=memory "
For the Raspberry Pi 4, rpi-cmdline.bbappend which adds the neccesary changes is provided. Again, using the dynamic-layers technique, this applies only if meta-raspberrypi is also enabled in the build.
The final piece is the kas configuration file raspberrypi4-64-app-updates.yml. It can be used directly by kas build raspberrypi4-64-app-updates.yml. For a simple layout, see the quick build section at the beginning of this article.
Conclusion
In this tutorial we covered the installation of the Application Update Module in a Yocto Project based build. Based on this, you can either get started right away on the Raspberry Pi 4, or easily adapt the installation to your board and manage your orchestrated containers with Mender.
If this tutorial was useful to you, please press like, or leave a thank you note to the contributor who put valuable time into this and made it available to you. It will be much appreciated!