Single File

lluiscampos · April 16, 2019, 1:55pm

Description

The Single File Update Module installs a user defined file into a given destination directory on the device.

If the file already exists in the destination folder on the device, the Update Module will take a backup copy of it. This allows the Mender client to rollback if something goes wrong.

Example use-cases:

Updating a single-binary application
Updating a operating system-wide configuration file

Specification


Module name	single-file
Supports rollback	yes
Requires restart	no
Artifact generation script	yes
Full operating system updater	no
Source code	Update Module, Artifact Generator
Maintainer	Northern.tech

It is not recommended to install a Mender Artifact on your workstation

Prepare the device

This section describes how to setup your target device, i.e. the device to be updated. This will also be referred to as the device environment.

All commands outlined in this section should be run in the device environment.

Prerequisites

This update module has the following prerequisites for the device environment:

Install the Mender client, version 2.0 or later

Install the Update Module

Download the latest version of this Update Module by running:

mkdir -p /usr/share/mender/modules/v3 && wget -N -P /usr/share/mender/modules/v3 https://raw.githubusercontent.com/mendersoftware/mender/master/support/modules/single-file

Prepare the development environment on your workstation

This section describes how to set up your development environment on your workstation.

All commands outlined in this section should be run in the development environment.

Prerequisites

This Update Modules has the following prerequisites for the development environment:

Install mender-artifact , version 3.1.0 or later

Create Mender Artifacts

For convenience, an Artifact generator tool single-file-artifact-gen is provided with the Update Module. This tool will generate Mender Artifacts in the same format that the Update Module expects them.

Download single-file-artifact-gen, by running the following command:

wget https://raw.githubusercontent.com/mendersoftware/mender/master/support/modules-artifact-gen/single-file-artifact-gen

Make it executable:

chmod +x single-file-artifact-gen

Create example file to deploy:

echo "File created by Mender single-file Update Module!" > my_file.txt

Now generate a Mender Artifact using the following command:

ARTIFACT_NAME="my-update-1.0"
DEVICE_TYPE="my-device-type"
OUTPUT_PATH="my-update-1.0.mender"
DEST_DIR="/opt/installed-by-single-file/"
FILE="my_file.txt"
./single-file-artifact-gen -n ${ARTIFACT_NAME} -t ${DEVICE_TYPE} -d ${DEST_DIR} -o ${OUTPUT_PATH} ${FILE}

ARTIFACT_NAME - The name of the Mender Artifact
DEVICE_TYPE - The compatible device type of this Mender Artifact
OUTPUT_PATH - The path where to place the output Mender Artifact. This should always have a .mender suffix
DEST_DIR - The path on target device where FILE will be installed.
FILE - The path to the file to be sent to the device in the update.

You can either deploy this Artifact in managed mode with the Mender server (upload it under Releases in the server UI) or by using the Mender client only in Standalone deployments.

Artifact technical details

The Mender Artifact used by this Update Module has three payload files: a regular file with the DEST_DIR in plain text, a regular file with the filename of the user file in plain text, and the user file itself.

The Mender Artifact contents will look like:

Updates:
    0:
    Type: single-file
    Provides: Nothing
    Depends: Nothing
    Metadata: Nothing
    Files:
        name:     dest_dir
        size:     39
        modified: 2019-04-16 15:29:40 +0200 CEST
        checksum: 58581d50870f179e80a73844f96ab3ef7dd7335aefb21562521674310fb72856
    Files:
        name:     filename
        size:     12
        modified: 2019-04-16 15:29:40 +0200 CEST
        checksum: 0d6b8085d25a0eb9862ec0642f2d1e1aacbeb85d826cdeb570f19ba2719507a9
    Files:
        name:     my_file.txt
        size:     60
        modified: 2019-04-16 15:29:16 +0200 CEST
        checksum: 5e93fe626d6ff553736b48c343b2d1b258960aab24b1b38496c3cf7a1fe5df51

ajithpv · April 29, 2019, 12:28pm

@lluiscampos Thank you for providing the single file update module.

There is already one file-install script present as below while build and run the mender 2.0.0 beta:

cat /usr/share/mender/modules/v3/file-install
#!/bin/bash

set -e

STATE=“$1”
FILES=“$2”

prev_files_tar=“$FILES”/tmp/prev_files.tar
update_files_tar=“$FILES”/files/update.tar
dest_dir_file=“$FILES”/files/dest_dir

case “$STATE” in
NeedsArtifactReboot)
    echo "No"
;;

SupportsRollback)
    echo "Yes"
;;

ArtifactInstall)
    dest_dir=$(cat $dest_dir_file)
    [[ -d $dest_dir ]] || install -d $dest_dir
    tar -cf ${prev_files_tar} -C ${dest_dir} .
    rm -rf ${dest_dir}
    mkdir -p ${dest_dir}
    tar -xf ${update_files_tar} -C ${dest_dir}
    ;;

ArtifactRollback)
    dest_dir=$(cat $dest_dir_file)
    [[ -f $prev_files_tar ]] || exit 0
    [[ -n "$dest_dir" ]] || exit 1
    rm -rf ${dest_dir}
    mkdir -p ${dest_dir}
    tar -xf ${prev_files_tar} -C ${dest_dir}
    ;;
esac

exit 0

Whether the already present script supports both file and directory update with the same script? Any document for the provided script (file-install) available? How to generate and pass the below identities?

prev_files_tar=“$FILES”/tmp/prev_files.tar
update_files_tar=“$FILES”/files/update.tar
dest_dir_file=“$FILES”/files/dest_dir

May I know which is better (already provided one (i.e. file-install) or single-file/directory update script from mender hub)?

lluiscampos · April 29, 2019, 12:44pm

Hi @ajithpv,

The file-install Update Module from the 2.0.0 beta is going to be deprecated in the final release and instead we are providing two independent ones: single-file and directory.

As you have noticed, the old one supported both use cases, and we thought it we be clearer to just split them in two simpler ones.

Please use the new ones.

speeltronics · May 8, 2019, 10:37pm

Hi, I have an integrated build with u-boot that has worked great since version 1.6 with full updates. I updated my device to 2.0 to test out this new module feature. I did not mess with any u-boot integration this timeso it is still at the changes for 1.6, because I am just trying to do the module updates and didn’t figure I would need to make changes to u-boot for the modules. I have also updated my server to 2.0. I created an update as per this page and my device keeps giving me an error that I am not sure about.

“level=error msg=“Fetching Artifact headers failed: installer: failed to read Artifact: readHeaderV3: handleHeaderReads: Cannot load handler for unknown Payload type ‘single-file’” module=state”

Then it waits and tries again in a min and keeps giving that same error.

Any idea what I am doing wrong?

Thanks!

eystein · May 8, 2019, 10:48pm

Hi @speeltronics and welcome to Mender Hub!

This error indicates that the Single File Update Module is not installed on the device where you are trying to install an Artifact Payload of this type.

To verify, run this command on your device:

mkdir -p /usr/share/mender/modules/v3 && wget -N -P /usr/share/mender/modules/v3 https://raw.githubusercontent.com/mendersoftware/mender/master/support/modules/single-file

Then re-try the deployment.

If this works then you need to simply make sure that this Update Module (single-file script) is present at this location in your image builds.

speeltronics · May 9, 2019, 12:58pm

Oh I see now! Thanks! It would be really great if the modules could be sent with the updates, so that a rootfs update wouldn’t be needed to use the modules on already deployed devices… Anyways, thanks again!

kacf · May 9, 2019, 1:19pm

Unfortunately this can’t be done because it is a chicken and egg problem. The module is needed before the artifact is fully downloaded, but if the module is inside the artifact, then it needs to be downloaded first…

Once the single-file module is installed though, you can use it to install other modules. (*)

(*) Well, almost, except that there is a bug in it right now which doesn’t preserve permissions, like execute permission. I’m in the process of fixing this right now, so this issue should be gone soon.

speeltronics · May 9, 2019, 4:42pm

Understood! So, how do we know what version the modules are at or when it was last updated?

kacf · May 13, 2019, 12:46pm

there is a bug in it right now which doesn’t preserve permissions, like execute permission. I’m in the process of fixing this right now, so this issue should be gone soon.

FYI: Update modules fixes by kacf · Pull Request #396 · mendersoftware/mender · GitHub

Understood! So, how do we know what version the modules are at or when it was last updated?

There is no versioning embedded in the update modules. They are just binaries or scripts, like other binaries on the system.

You can use an inventory script to list the contents of /usr/share/mender/modules/v3, perhaps with a md5sum to figure out the version. This will show up in the UI. See inventory scripts for more information.

ster · July 19, 2019, 3:43pm

Is it possible to include state scripts in the artifacts for update modules?
Is it possible to use LZMA compression for update modules?

kacf · July 23, 2019, 6:51am

Yes and yes!

ster · July 23, 2019, 11:00am

And here how it is possible :

To add state scripts or sign the artifact, you can provide --key and --script as passthrough arguments, for example:

…
KEY=“artifact-sign-key.key”
SCRIPT1=“ArtifactCommit_Enter_01_custom_script”
SCRIPT2=“AntotherScript”
./single-file-artifact-gen -n {ARTIFACT_NAME} -t {DEVICE_TYPE} -d {DEST_DIR} -o {OUTPUT_PATH} {FILE_TREE} -- --key {KEY} --script {SCRIPT1} --script {SCRIPT2}
But --compression lzma can’t be passed like this as this argument should be passed to mender-artifact and not to mender-artifact write module-image. So to use lzma, you have to change the artifact generator script so it calls mender-artifact with this argument:

mender-artifact --compression lzma write module-image \ ...

I’ll try to prepare a PR to improve this.

dYalib · October 7, 2019, 4:01pm

Hi,

thank you for this update module!
It’s possible to perform a device reboot after the new file has been installed?
I like to update a configuration file and a reboot is required to take effect. What is the preferred way to do this?

mEK · October 8, 2019, 6:26am

Hello @dYalib
I’m not quite sure, but there is no reboot activity in the script update module. You can try full system update for this. For more detailed information about the update modules, see here:
https://docs.mender.io/2.1/devices/update-modules

mirzak · October 8, 2019, 7:42am

You will need to modify the module to support reboot.

And the change required would be, changing:

    NeedsArtifactReboot)
        echo "No"
    ;;

to

    NeedsArtifactReboot)
        echo "Automatic"
    ;;

You can reed more about the Update Module API’s here,

github.com

mendersoftware/mender/blob/2.1.0/Documentation/update-modules-v3-file-api.md

Update modules v3 protocol
==========================

Update modules are executables that are placed in `/usr/share/mender/modules/v3`
directory, where `v3` is a reference to the version of the protocol, which is
the same as [the version of the Artifact
format](https://github.com/mendersoftware/mender-artifact/tree/master/Documentation). Mender
will look for update modules in the directory with the same version as the
version of the Artifact being processed.


States and execution flow
-------------------------

Update modules are modelled after the same execution flow as [state
scripts](https://docs.mender.io/artifacts/state-scripts), and consists of the
following states:

* `Download`
* `ArtifactInstall`

This file has been truncated. show original

freibrun · December 16, 2019, 12:57pm

I tried to include a state script in an artifact with the name ArtifactInstall_Leave_00 with this command:

./single-file-artifact-gen -n ${ARTIFACT_NAME} -t ${DEVICE_TYPE} -d ${DEST_DIR} -o ${OUTPUT_PATH} ${FILE} – --script ${SCRIPT}

However, the file seems not to be included in the artifact. This is the error I get on the target:

‘ArtifactInstall_Leave_00’: -1 : fork/exec /var/lib/mender/scripts/ArtifactInstall_Leave_00: no such file or directory

According to the manual Artifact state scripts should be included in the generated artifact: State scripts | Mender documentation

But when I check the content of the artifact file, the state script is not included. I also tried to include the state script with the -f parameter which does indeed include the state script as payload but it’s still not working (since it probably does not put the state script to the above specified path).

What else do I need to specify to successfully include a state script file in an artifact?

Update: I now see that the script is actually correctly installed on the device in the following directory:

However, the script is somehow not executed as we can see in the error log:

2019-12-16 12:41:34 +0000 UTC error: transient error: error executing leave script for update-install state: error running leave state script(s) for ArtifactInstall state: statescript: error executing ‘ArtifactInstall_Leave_00’: -1 : fork/exec /var/lib/mender/scripts/ArtifactInstall_Leave_00: no such file or directory

Any idea why the script is not executed?

Solved: Found the problem. The script was actually correctly executed - but there was a problem INSIDE the script. So the error message was a bit misleading

lluiscampos · December 16, 2019, 3:02pm

Hehe. Good that you figured out yourself

It is quite limited the debug info that Mender client can take from the state script itslef. A good tip (that I always use at least while developing) is adding verbosity to the script itself with set -x. This way if the update fails inside the script, Mender will capture the output and send to the Mender server.

gerardostola · March 23, 2020, 11:18pm

Hi,
I’ve created and deployed the artifact as documented.
I’ve compiled and run the client from an Ubuntu box.
I’ve deployed the artifact on the server, which moved from Active to Finished.
The client was run with the -debug modifier. There I could read
DEBU[0001] Received response:204 No Content
DEBU[0001] No update available
DEBU[0001] no updates available

For there is something missconfigured, I couldn´t figure out.

kacf · March 24, 2020, 7:27am

Are you sure you are not trying to install an Artifact with the same name as the already installed Artifact on the device? Each new Artifact being deployed needs to have a different name.

gerardostola · March 25, 2020, 1:02am

Hi, thank you for your answer. What I had missed was to define the device type on the client, so there was no match.

Now we could download a single file successfully, using the demo server. By the way, it can only send the domain s3.docker.mender.io, we could not change it and that could be nice, given that clients and servers live on different networks. The only workaround we found was to resolve that domain through the client´s /etc/hosts file, which is somewhat dirty.

Topic		Replies	Views
Directory Update Modules	39	10971	September 13, 2024
Rpm packages Update Modules	0	2466	April 23, 2019
Script Update Modules	22	7161	June 24, 2020
IPK packages Update Modules	0	2005	September 23, 2019
Mender-artifact with single-file Update Modules mender-artifact	6	97	July 8, 2024

Single File

Description

Specification

Prepare the device

Prerequisites

Install the Update Module

Prepare the development environment on your workstation

Prerequisites

Create Mender Artifacts

Artifact technical details

Related topics