How to update a Docker container with a Raspberry Pi 3

Introduction

For this tutorial, we quickly refresh some topics that you will need to have a grasp of such as Mender client, Mender Artifact and Update Modules in order to deploy an application-based update over-the-air (OTA). As part of the prerequisites for this tutorial, it is recommended that you have completed the Mender onboarding to get familiar with the Mender workflow.

Mender Client is a process that runs in user space on top of an embedded Linux operating system which talks to the Mender server and applies the update. The Mender client needs to be integrated with the device so it can install software updates to it. This integration also adds functionality to work with a Mender server such as provisioning, authentication, inventory monitoring and deployment reporting. When integrated, all images built for this device supports robust OTA updates. For a robust update process, Mender defines and uses a file format with .mender suffix, called Mender Artifact to handle additional metadata alongside the raw bits of the image filesystem namely, the Name of the software build, compatibility with device type, and checksum of the root filesystem for controllability during transit or storage.

Mender uses a framework called Update Modules to enable application-based or partial updates and comes with support for these types of updates out-of-the-box:

  • Packages (deb and rpm)
  • Containers (docker)
  • Files (directory copy/sync)

An Update Module is just a piece of user-written code that defines how the Mender client installs an update to a device. They are customizable and can be written in any programming language, so it can be tailored to meet the specific needs of a device or environment.

This tutorial will guide you through how to use Docker Update Module to update a Docker container using a Raspberry Pi 3.

Prerequisites

To follow this tutorial, you will need:

  • A Raspberry Pi 3 board running Raspbian with SSH enabled. You can use any of the methods suggested to install the OS image to your device.

  • A deployment environment using a desktop PC with Ubuntu 18.04 and Google Chrome as a web browser and at least 10 GB free disk space and 2 GB RAM available for Mender.

  • Mender onboarding completed using the latest version of Mender documentation.

Step 1 - Install and Configure Mender Client Properly on Device

We assume that you have familiarized yourself with Mender by doing the latest Mender onboarding as stated in the prerequisites section of this tutorial. To make sure the experience is smoother we will revisit this step since we are doing a partial update and it is important to have installed and configured the Mender client properly in order to do this tutorial.

Installing this way does not offer a full Mender integration in your Raspberry Pi board and only enables Update Modules to do partial updates as intended for this tutorial, in this case a Docker container.

Download a Debian package .deb to install on Debian, Ubuntu or Raspbian:

wget https://d1b0l86ne08fsf.cloudfront.net/2.3.0/dist-packages/debian/armhf/mender-client_2.3.0-1_armhf.deb

Install the package:

sudo dpkg -i mender-client_2.3.0-1_armhf.deb

After successfully installing the Mender client Debian package, some initial setup is required. First, we have to configure the Mender client for the demo setting using the configuration file at /etc/mender/mender.conf. The .mender.conf file resides on the client side.

Copy the demo configuration file:

sudo cp /etc/mender/mender.conf.demo /etc/mender/mender.conf

By viewing the content of the file you should see the following:

{
  "ServerCertificate": "/etc/mender/server.crt",
  "InventoryPollIntervalSeconds": 5,
  "RetryPollIntervalSeconds": 30,
  "ServerURL": "https://docker.mender.io/",
  "UpdatePollIntervalSeconds": 5
}

The demo server uses docker.mender.io and hosted server uses hosted.mender.io. In this tutorial, we are using the demo server so you need to make sure that ServerURL in the configuration file is set to https://docker.mender.io/. These URLs are not valid domain names and for it to be useful you will need an entry in “/etc/hosts” such as an IP address docker.mender.io s3.docker.mender.io, with the IP address being the IP of the PC or any environment that is running the demo server.

Next, you need to configure your device type.The device type is a string that defines your device and is used to ensure software compatibility by comparing the device type set in a Mender Artifact with the string on the device.

Create Mender client state directory:

sudo mkdir -p /var/lib/mender

Create the device type file:

echo "device_type=raspberrypi3" | sudo tee /var/lib/mender/device_type

Now you should have the Mender client installed and properly setup in your device.

Start the Mender client:

sudo systemctl enable mender && sudo systemctl restart mender

At this point you should have a login password and have also accepted your Raspberry Pi 3.

Step 2 - Install Docker Update Module

A Docker container packages applications in containers by virtualizing the operating system and dividing it into compartments to run applications.

Mender enables this workflow through Docker Update Modules. The Mender client (installed in step 1) supports different types of updates, using a framework called Update Modules. Installing the Docker Update Module will enable support for installing Docker containers. When the Mender client downloads a Docker container from the server, it will run the Docker Update Module executable associated with the type of software downloaded. The Update Module is responsible for carrying out the steps needed to install software of this type.

A deployment with this module will stop all currently running Docker containers on the device, and start new containers with the provided list of Docker images defined in the Mender Artifact (step 3).

This approach enables targeted application-level updates for the benefit of lower bandwidth usage, faster updates and more frequent deployments.

In order to use this Update Module you will need to download the mender.artifact.

Make it executable and make sure your location is in your PATH:

chmod +x ~/PATH/mender-artifact

Download the latest version of this Update Module by running:

sudo mkdir -p /usr/share/mender/modules/v3 && sudo wget -P /usr/share/mender/modules/v3 https://raw.githubusercontent.com/mendersoftware/mender/master/support/modules/docker

Step 3 - Create a Mender Artifact

Download docker-artifact-gen, by running the following command:

wget https://raw.githubusercontent.com/mendersoftware/mender/master/support/modules-artifact-gen/docker-artifact-gen

Make it executable:

chmod +x docker-artifact-gen

You will need to set the variables first just like any standard shell scripting.

ARTIFACT_NAME="my-container-update-1.0"
DEVICE_TYPE="raspberrypi3"
OUTPUT_PATH="my-container-update-1.0.mender"
DOCKER_IMAGES="hello-world"

As you note, in this tutorial we have assigned the ‘hello-world’ container tag to the DOCKER_IMAGES variable. This will allow the Mender Artifact generator command below to point to the Docker Hub with the latest hello-world tag digest.

Now after assigning variables, generate a Mender Artifact using the following command:

./docker-artifact-gen -n ${ARTIFACT_NAME} -t ${DEVICE_TYPE} -o ${OUTPUT_PATH} ${DOCKER_IMAGES}

Step 4 - Update a Docker Container

Now you are all set to deploy a Docker container to your Raspberry Pi 3. Login to the Mender web UI and upload the Artifact file you created in Step 3 in the ‘RELEASES’ tab. You should also have accepted your Raspberry Pi 3 in the DEVICES tab (no pending device) and create a deployment in the DEPLOYMENTS tab.

To make sure you have updated your device successfully, run the following command on the Raspberry Pi 3 board:

docker run hello-world

You should see a success message from Docker with steps taken to generate the pul and stream to the client.

Conclusion

In this tutorial we went through a step-by-step guide on how to utilize Mender Update Modules to do a simple ‘hello-world’ Docker container update to a Raspberry Pi 3 board.

For further reading please visit:


If this tutorial was useful to you, please press like, or leave a thank you note to the contributor who put valuable time into this and made it available to you. It will be much appreciated!

2 Likes

Hi @farshadt,
i am following your tutorial for docker update i am able to create the artifact but when i am deploying it is showing no artifact in mender server i am not able to update cam you please provide any solution for this and also i am able to update file in my raspberry but not docker file i will attach my screen shot for better preference


thanks
Avi

Hi @avi992 can you post a screenshot of your releases tab as well as the detailed view of the artifact you are trying to install?

Drew

Hi @drew
i am attaching my screen shot of release server


and i am following same link to create the artifact

the artifact details is
ARTIFACT_NAME=“my-container-update-1.0”
DEVICE_TYPE=“raspberrypi3”
OUTPUT_PATH=“my-container-update-1.0.mender”
DOCKER_IMAGES=“hello-world”
ad this is the command to create artifact
./docker-artifact-gen -n ${ARTIFACT_NAME} -t ${DEVICE_TYPE} -o ${OUTPUT_PATH} ${DOCKER_IMAGES}

My client version is
mender-client_2.2.0
Thanks
Avi

Hi Avi,
What I need to is is the release details for your specific artifact. I’ve attached a screenshot showing what I see when do the above steps.

Also can you share the text output from when you run the docker-artifact-gen command?

Drew

Hi @drewmoseley
first of all i am appreciating your fast response on queries thank you for your response i am attaching my artifact generation logs please check

i also have the same content in the server

Avi

Your device is reporting rpi3 as device type but you put raspberrypi3 in your Artifact. You need to align these.

1 Like

Hi @mirzak
thanks for response i have align the name so after align i am getting fail i am uploading the screen shot please check


mender-client version is
mender-client_2.2.0
i am attaching the log also please check
2020-04-24 14:27:02 +0000 UTC info: Running Mender version 2.2.0
2020-04-24 14:27:02 +0000 UTC debug: handle update fetch state
2020-04-24 14:27:02 +0000 UTC debug: status reported, response 204 No Content
2020-04-24 14:27:04 +0000 UTC debug: Received fetch update response &{200 OK 200 HTTP/1.1 1 1 map[Accept-Ranges:[bytes] Content-Security-Policy:[block-all-mixed-content] X-Amz-Request-Id:[1608C7AF2CC8A730] Date:[Fri, 24 Apr 2020 14:27:03 GMT] Content-Type:[application/vnd.mender-artifact] Content-Length:[5120] Last-Modified:[Fri, 24 Apr 2020 14:20:40 GMT] Vary:[Origin] X-Frame-Options:[DENY] Server:[openresty/1.13.6.2] X-Xss-Protection:[1; mode=block] Strict-Transport-Security:[max-age=0; includeSubdomains; preload] Connection:[keep-alive] Etag:[“f8b3515c1447bcbc74fad3ce6ae9417d”] X-Content-Type-Options:[nosniff]] 0x958f30 5120 [] false false map[] 0x88a900 0x867d40}+
2020-04-24 14:27:04 +0000 UTC info: State transition: update-fetch [Download_Enter] -> update-store [Download_Enter]
2020-04-24 14:27:04 +0000 UTC debug: handle update install state
2020-04-24 14:27:04 +0000 UTC debug: status reported, response 204 No Content
2020-04-24 14:27:04 +0000 UTC debug: Read data from device manifest file: device_type=raspberrypi3
2020-04-24 14:27:04 +0000 UTC debug: Current manifest data: raspberrypi3
2020-04-24 14:27:04 +0000 UTC info: no public key was provided for authenticating the artifact
2020-04-24 14:27:05 +0000 UTC debug: checking if device [raspberrypi3] is on compatible device list: [raspberrypi3]
2020-04-24 14:27:05 +0000 UTC debug: Executing ModuleInstaller.Initialize
2020-04-24 14:27:05 +0000 UTC debug: Returning artifact name from /etc/mender/artifact_info file.
2020-04-24 14:27:05 +0000 UTC debug: Read data from device manifest file: artifact_name=unknown
2020-04-24 14:27:05 +0000 UTC debug: Current manifest data: unknown
2020-04-24 14:27:05 +0000 UTC debug: Read data from device manifest file: artifact_name=unknown
2020-04-24 14:27:05 +0000 UTC debug: Read data from device manifest file: device_type=raspberrypi3
2020-04-24 14:27:05 +0000 UTC debug: Current manifest data: raspberrypi3
2020-04-24 14:27:06 +0000 UTC debug: installer: successfully read artifact [name: my-container-update-1.0; version: 3; compatible devices: [raspberrypi3]]
2020-04-24 14:27:06 +0000 UTC debug: Executing ModuleInstaller.PrepareStoreUpdate
2020-04-24 14:27:06 +0000 UTC debug: Calling module: /usr/share/mender/modules/v3/docker Download /var/lib/mender/modules/v3/payloads/0000/tree
2020-04-24 14:27:06 +0000 UTC debug: Executing ModuleInstaller.FinishStoreUpdate
2020-04-24 14:27:06 +0000 UTC debug: Executing ModuleInstaller.SupportsRollback
2020-04-24 14:27:06 +0000 UTC debug: Calling module: /usr/share/mender/modules/v3/docker SupportsRollback /var/lib/mender/modules/v3/payloads/0000/tree
2020-04-24 14:27:07 +0000 UTC debug: Update module output: Yes
2020-04-24 14:27:07 +0000 UTC debug: Module supports rollback
2020-04-24 14:27:08 +0000 UTC debug: status reported, response 204 No Content
2020-04-24 14:27:08 +0000 UTC info: State transition: update-store [Download_Enter] -> update-after-store [Download_Leave]
2020-04-24 14:27:08 +0000 UTC info: State transition: update-after-store [Download_Leave] -> update-install [ArtifactInstall]
2020-04-24 14:27:08 +0000 UTC debug: statescript: timeout for executing scripts is not defined; using default of 1h0m0s seconds
2020-04-24 14:27:08 +0000 UTC debug: statescript: timeout for executing scripts is not defined; using default of 1h0m0s seconds
2020-04-24 14:27:09 +0000 UTC debug: status reported, response 204 No Content
2020-04-24 14:27:09 +0000 UTC debug: Executing ModuleInstaller.InstallUpdate
2020-04-24 14:27:09 +0000 UTC debug: Calling module: /usr/share/mender/modules/v3/docker ArtifactInstall /var/lib/mender/modules/v3/payloads/0000/tree
2020-04-24 14:27:09 +0000 UTC info: Update module output: /usr/share/mender/modules/v3/docker: line 21: docker: command not found
2020-04-24 14:27:09 +0000 UTC error: Update module terminated abnormally: exit status 127
2020-04-24 14:27:09 +0000 UTC error: transient error: Update module terminated abnormally: exit status 127
2020-04-24 14:27:09 +0000 UTC info: State transition: update-install [ArtifactInstall] -> rollback [ArtifactRollback]
2020-04-24 14:27:09 +0000 UTC debug: transitioning to error state
2020-04-24 14:27:09 +0000 UTC debug: statescript: timeout for executing scripts is not defined; using default of 1h0m0s seconds
2020-04-24 14:27:09 +0000 UTC debug: statescript: timeout for executing scripts is not defined; using default of 1h0m0s seconds
2020-04-24 14:27:09 +0000 UTC info: performing rollback
2020-04-24 14:27:10 +0000 UTC debug: Executing ModuleInstaller.Rollback
2020-04-24 14:27:10 +0000 UTC debug: Calling module: /usr/share/mender/modules/v3/docker ArtifactRollback /var/lib/mender/modules/v3/payloads/0000/tree
2020-04-24 14:27:10 +0000 UTC info: Update module output: /usr/share/mender/modules/v3/docker: line 31: docker: command not found
2020-04-24 14:27:10 +0000 UTC info: State transition: rollback [ArtifactRollback] -> update-error [ArtifactFailure]
2020-04-24 14:27:10 +0000 UTC debug: statescript: timeout for executing scripts is not defined; using default of 1h0m0s seconds
2020-04-24 14:27:10 +0000 UTC debug: statescript: timeout for executing scripts is not defined; using default of 1h0m0s seconds
2020-04-24 14:27:10 +0000 UTC debug: handle update error state
2020-04-24 14:27:10 +0000 UTC debug: Executing ModuleInstaller.Failure
2020-04-24 14:27:10 +0000 UTC debug: Calling module: /usr/share/mender/modules/v3/docker ArtifactFailure /var/lib/mender/modules/v3/payloads/0000/tree
2020-04-24 14:27:11 +0000 UTC info: State transition: update-error [ArtifactFailure] -> cleanup [Error]
2020-04-24 14:27:11 +0000 UTC debug: statescript: timeout for executing scripts is not defined; using default of 1h0m0s seconds
2020-04-24 14:27:11 +0000 UTC debug: statescript: timeout for executing scripts is not defined; using default of 1h0m0s seconds
2020-04-24 14:27:11 +0000 UTC debug: Handling Cleanup state
2020-04-24 14:27:11 +0000 UTC debug: Executing ModuleInstaller.Cleanup
2020-04-24 14:27:11 +0000 UTC debug: Calling module: /usr/share/mender/modules/v3/docker Cleanup /var/lib/mender/modules/v3/payloads/0000/tree
2020-04-24 14:27:11 +0000 UTC info: State transition: cleanup [Error] -> update-status-report [none]
2020-04-24 14:27:11 +0000 UTC debug: statescript: timeout for executing scripts is not defined; using default of 1h0m0s seconds
2020-04-24 14:27:11 +0000 UTC debug: handle update status report state
2020-04-24 14:27:12 +0000 UTC debug: status reported, response 204 No Content
2020-04-24 14:27:12 +0000 UTC debug: attempting to upload deployment logs for failed update

You do not have docker installed into your system. See https://phoenixnap.com/kb/docker-on-raspberry-pi for instructions on this.

Drew

Hi @drew
i have install docker in my system running
pi@raspberrypi:~$ docker --version
Docker version 19.03.8, build afacb8b
but still is getting failure i will attach my screen shot with this please check


and also i an uploading logs please check
2020-04-27 14:13:27 +0000 UTC info: Running Mender version 2.2.0
2020-04-27 14:13:27 +0000 UTC debug: handle update fetch state
2020-04-27 14:13:27 +0000 UTC debug: status reported, response 204 No Content
2020-04-27 14:14:42 +0000 UTC error: Can not fetch update image: Get https://s3.docker.mender.io:9000/mender-artifact-storage/783f4c5b-5334-42cc-ba2b-3b53b72ee5ca?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=minio%2F20200427%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20200427T141326Z&X-Amz-Expires=86400&X-Amz-SignedHeaders=host&response-content-type=application%2Fvnd.mender-artifact&X-Amz-Signature=41de4eaf9d99cb3e89232a6839f1f3ead6b4ac77d6655cbeb90f20375324112c: dial tcp 52.247.205.165:9000: connect: connection refused
2020-04-27 14:14:42 +0000 UTC error: update fetch failed: update fetch request failed: Get https://s3.docker.mender.io:9000/mender-artifact-storage/783f4c5b-5334-42cc-ba2b-3b53b72ee5ca?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=minio%2F20200427%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20200427T141326Z&X-Amz-Expires=86400&X-Amz-SignedHeaders=host&response-content-type=application%2Fvnd.mender-artifact&X-Amz-Signature=41de4eaf9d99cb3e89232a6839f1f3ead6b4ac77d6655cbeb90f20375324112c: dial tcp 52.247.205.165:9000: connect: connection refused

thanks
avinash

Hi @avi992 it looks like something is not right in your Mender demo server. It is failing on the connection to the storage proxy running on port 9000.

Is 52.247.205.165 reachable from your Raspberry Pi device?

Can you post the docker logs from your server instance?
Drew

Is step one of this page describing the installation of the mender client directly onto a device? I have only installed the client via the mender-convert process. Is the mender-convert process considered another way of installing the mender client? I only ask because I have a need to install the docker update module and when I follow along with this, I see step one and thing to myself “Hmmmm I’ve never done this, yet I have a working Mender environment”.

Also, is the Docker Update Module installed as part of the mender-convert process as well? Or must this be installed manually? I would hope it’s part of mender-convert which would make sure everything is installed in both the active and passive partitions. Having to install it manually after mender-convert leaves me wondering how to insure it’s installed in both partitions.
Any clarification would be helpful. Thanks!

In the event that anyone had the same question I did, I’m answering my own question. I navigated to /usr/share/mender/modules/v3 on my device and there is a docker update module there. I didn’t explicitly install it, so I am assuming that mender-convert has put it there.

Yes, that is the purpose of mender-convert. It starts with your golden master image, with no Mender integrated, and add Mender, config files, and generates the multiple partition sdimg/uefiimg file as well as a Mender artifact for deploying OTA to devices that have already been provisioned with sdimg/uefiimg builds.

Drew

1 Like

Hi Team,

From the above post, “A deployment with this module will stop all currently running Docker containers on the device, and start new containers with the provided list of Docker images defined in the Mender Artifact (step 3).”

Whether the target is running with multiple containers or a single container?
If single container, whether it is possible to perform multiple container updations?
Does it require to stop all the docker containers, when only one container has to update?

Thanking you,
Aysha