Integrate CFEngine configuration management into Mender

Introduction

Operating system Configuration management is an established industry for servers, with typical use cases including:

  • Network configuration
  • SSH daemon configuration
  • User management
  • Service management
  • Package management
  • and many more

There are several different tools for this purpose and the value of more advanced configuration management tools (compared to e.g. pre-provisioned configuration files, or doing simple file-copies or script execution over SSH) becomes more clear once the system complexity increases; you have more software and configuration to maintain on the devices.

CFEngine is one of the most well-established configuration management tools and is written in C, so it is lightweight (a typical installation is around 10 MB) and thus well suited for IoT use cases. It also happens to be developed by the same company as Mender (Northern.tech).

Although there is some overlap between OTA update management and configuration management tools for simpler use cases, this tutorial will give an example on how these types of tools can be combined to play to the strength of both. This is primarily interesting for more complex or “high end” IoT devices, where the configuration is more advanced or needs to be dynamic (such as devices needing to reconfigure their network when they are moved).

In this tutorial we will at the high level:

  • Use Mender, client and server to deploy a an OTA update (like normal)
  • Use Mender to trigger CFEngine to carry out dynamic device configuration

Prerequisites

Install CFEngine on the device

This can be done in several ways, depending on your OS and architecture.

Yocto Project

For Yocto Project users, there is a recipe for CFEngine in meta-oe you can include.

Debian package installation

In Debian-family distributions, the cfengine3 package can be installed from the repositories with:

apt update
apt install cfengine3

Source installation

CFEngine uses the standard autoconf and make tools to build for a wide variety of architectures, and you can download the latest sources from the download page.

Verification

At this point you should have Mender and CFEngine installed on the device:

/usr/bin/mender -version
2.0.1
runtime: go1.11.1
/var/cfengine/bin/cf-agent -V
CFEngine Core 3.15.0

Prepopulate templates and data on the device

When we configure the device below, we assume some configuration templates and data is already on the device for testing purposes. In real scenarios the templates are typically provisioned with the device itself and the device-specific data is obtained from a web server query.

But for simple testing, we first create a network configuration template on the device:

mkdir -p /tmp/templates
cat << EOF > /tmp/templates/net-config.template
[Match]
Name=wl*

[Network]
Address={{ipaddress}}
DNS=10.1.10.1
Gateway=10.1.1.254
EOF

This is a Mustache formatted template, and you can see the ipaddress is dynamically filled from template data.

The data file input to the template (such as the device IP address) is device-specific and in this example it is assumed all configuration to this device exists in a json file based on the hostname of the device (but this can easily be adjusted with the policy in the next section). Create the data for the template with these commands:

mkdir -p /tmp/data
cat << EOF > /tmp/data/device-config-qemux86-64.json
{
    "ipaddress" : "10.1.10.9/24",
}
EOF

Build a Mender Artifact with post-install configuration on your workstation

The device is now fully configured, so we move to our workstation/laptop to create and deploy the Mender Artifact to the device.

Mender supports post-install configuration using state scripts. Normally these are shell scripts, but in our case we will actually use a CFEngine policy as a state script, which is written in the domain specific language of CFEngine for configuration management.

First, create a state script that contains a CFEngine policy to configure the network address based on a template:

cat << EOF > ArtifactCommit_Leave_01_netconfig
#!/var/cfengine/bin/cf-agent -Kf

bundle agent main
{
files:
   "/tmp/net-config"
     create => "true",
     edit_template => "/tmp/templates/net-config.template",
     template_method => "mustache",
     template_data => readjson("/tmp/data/device-config-qemux86-64.json");
}
EOF

For testing purposes, this policy will render a file to /tmp/net-config, based on the template and data created in the previous section.

Now we create a Mender Artifact with this state script inside. We can use any type of update with Mender for this, but the Single File Update is a simple one. In the same directory, install the Single File Update Module generator:

wget https://raw.githubusercontent.com/mendersoftware/mender/master/support/modules-artifact-gen/single-file-artifact-gen
chmod +x single-file-artifact-gen
echo "File created by Mender single-file Update Module!" > my_file.txt

Note that you will also need to install mender-artifact if you do not have it already.

Now we build the Mender Artifact containing the single file update:

ARTIFACT_NAME="my-net-update-1.0"
DEVICE_TYPE="raspberrypi3"   # adjust this to your device type
OUTPUT_PATH="my-net-update-1.0.mender"
DEST_DIR="/opt/installed-by-single-file/"
FILE="my_file.txt"
./single-file-artifact-gen -n ${ARTIFACT_NAME} -t ${DEVICE_TYPE} -d ${DEST_DIR} -o ${OUTPUT_PATH} ${FILE} -- -s ArtifactCommit_Leave_01_netconfig

This should result in the Mender Artifact my-net-update-1.0.mender in the working directory.

The update itself is my_file.txt and will be installed to /opt/installed-by-single-file/. The CFEngine policy is packaged as a state script with the last option above (note the -- are required) and will be run after the update is installed.

Deploy the update and verify

We now have the device fully configured and the Mender Artifact created. Simply upload the my-net-update-1.0.mender Artifact to your Mender server and deploy it to the device. Deployment may take a minute or so, depending on the device and Mender configuration.

After it has been deployed, you can verify that Mender installed the update:

cat /opt/installed-by-single-file/my_file.txt 
File created by Mender single-file Update Module!

Finally, CFEngine has been invoked to render the device configuration file as intended, filling in the Address from our template:

cat /tmp/net-config
[Match]
Name=wl*

[Network]
Address=10.1.10.9/24
DNS=10.1.10.1
Gateway=10.1.1.254

Conclusions

This demonstrates how Mender can be integrated with mature and powerful configuration management systems like CFEngine to do post-configuration of the device after an OTA update.

To use this in real scenarios, a few things should be adjusted:

  • Obtain the device-specific data from a dynamic location, like a web server or Configuration Management Database
  • Adjust the template net-config.template to a real configuration file template
  • Adjust the destination path to the configuration file in the state script ArtifactCommit_Leave_01_netconfig

If this tutorial was useful to you, please press like, or leave a thank you note to the contributor who put valuable time into this and made it available to you. It will be much appreciated!

4 Likes

I like the approach very much! In my recent blog post I combined Mender with Ansible to get GitOps going for embedded systems. However, CFEngine might be more lightweight.

2 Likes

Thanks for the pointer @lueschem, it provides a nice overview.

What are your thoughts on the use cases for configuration (e.g. Ansible, CFEngine) of embedded / IoT devices? I’ve seen cases where some device-specific (network/IP) configuration should be applied, for example, but wanted to see what your thoughts are.

Do the configs actually need to be dynamic (e.g. ansible-pull / use a CFEngine server) or are they quite static (assuming you do the software deployment with Mender)?

I recommend to keep the base functionality and all its involved components as static as possible (only upgradable as a full, robust OS update):

  • Mender client for robust A/B update.
  • Connectivity setup towards vital backend system(s) (e.g. hosted Mender).
  • Linux kernel.
  • Rootfs with base user space functionality.
  • Ansible or CFEngine (it should not update itself dynamically).

The dynamic part could include the following use cases:

  • Configure LAN ports of an IoT gateway (without modifying the WAN setup).
  • Add some software containing intellectual property that can only be shipped to certain countries.
  • Enable extra features that some customers are paying for.
  • Fix issues of the system in an incremental way. Of course such modifications should not have a harmful effect on the base functionality.

So Mender makes sure that the base functionality works while Ansible or CFEngine dynamically modifies the “nice to have” part that actually provides the features requested by a certain installation. The “nice to have” part can be very dynamic and depend upon facts that need to be gathered from the given device instance.

1 Like