Single File

Description

The Single File Update Module installs a user defined file into a given destination directory on the device.

If the file already exists in the destination folder on the device, the Update Module will take a backup copy of it. This allows the Mender client to rollback if something goes wrong.

Example use-cases:

  • Updating a single-binary application
  • Updating a operating system-wide configuration file

Specification

Module name single-file
Supports rollback yes
Requires restart no
Artifact generation script yes
Full operating system updater no
Source code Update Module, Artifact Generator
Maintainer Northern.tech

It is not recommended to install a Mender Artifact on your workstation

Prepare the device

This section describes how to setup your target device, i.e. the device to be updated. This will also be referred to as the device environment.

All commands outlined in this section should be run in the device environment.

Prerequisites

This update module has the following prerequisites for the device environment:

Install the Update Module

Download the latest version of this Update Module by running:

mkdir -p /usr/share/mender/modules/v3 && wget -N -P /usr/share/mender/modules/v3 https://raw.githubusercontent.com/mendersoftware/mender/master/support/modules/single-file

Prepare the development environment on your workstation

This section describes how to set up your development environment on your workstation.

All commands outlined in this section should be run in the development environment.

Prerequisites

This Update Modules has the following prerequisites for the development environment:

Create Mender Artifacts

For convenience, an Artifact generator tool single-file-artifact-gen is provided with the Update Module. This tool will generate Mender Artifacts in the same format that the Update Module expects them.

Download single-file-artifact-gen, by running the following command:

wget https://raw.githubusercontent.com/mendersoftware/mender/master/support/modules-artifact-gen/single-file-artifact-gen

Make it executable:

chmod +x single-file-artifact-gen

Create example file to deploy:

echo "File created by Mender single-file Update Module!" > my_file.txt

Now generate a Mender Artifact using the following command:

ARTIFACT_NAME="my-update-1.0"
DEVICE_TYPE="my-device-type"
OUTPUT_PATH="my-update-1.0.mender"
DEST_DIR="/opt/installed-by-single-file/"
FILE="my_file.txt"
./single-file-artifact-gen -n ${ARTIFACT_NAME} -t ${DEVICE_TYPE} -d ${DEST_DIR} -o ${OUTPUT_PATH} ${FILE}
  • ARTIFACT_NAME - The name of the Mender Artifact
  • DEVICE_TYPE - The compatible device type of this Mender Artifact
  • OUTPUT_PATH - The path where to place the output Mender Artifact. This should always have a .mender suffix
  • DEST_DIR - The path on target device where FILE will be installed.
  • FILE - The path to the file to be sent to the device in the update.

You can either deploy this Artifact in managed mode with the Mender server (upload it under Releases in the server UI) or by using the Mender client only in Standalone deployments.

Artifact technical details

The Mender Artifact used by this Update Module has three payload files: a regular file with the DEST_DIR in plain text, a regular file with the filename of the user file in plain text, and the user file itself.

The Mender Artifact contents will look like:

Updates:
    0:
    Type: single-file
    Provides: Nothing
    Depends: Nothing
    Metadata: Nothing
    Files:
        name:     dest_dir
        size:     39
        modified: 2019-04-16 15:29:40 +0200 CEST
        checksum: 58581d50870f179e80a73844f96ab3ef7dd7335aefb21562521674310fb72856
    Files:
        name:     filename
        size:     12
        modified: 2019-04-16 15:29:40 +0200 CEST
        checksum: 0d6b8085d25a0eb9862ec0642f2d1e1aacbeb85d826cdeb570f19ba2719507a9
    Files:
        name:     my_file.txt
        size:     60
        modified: 2019-04-16 15:29:16 +0200 CEST
        checksum: 5e93fe626d6ff553736b48c343b2d1b258960aab24b1b38496c3cf7a1fe5df51
4 Likes

@lluiscampos Thank you for providing the single file update module.

There is already one file-install script present as below while build and run the mender 2.0.0 beta:

cat /usr/share/mender/modules/v3/file-install
#!/bin/bash

set -e

STATE=ā€œ$1ā€
FILES=ā€œ$2ā€

prev_files_tar=ā€œ$FILESā€/tmp/prev_files.tar
update_files_tar=ā€œ$FILESā€/files/update.tar
dest_dir_file=ā€œ$FILESā€/files/dest_dir

case ā€œ$STATEā€ in

NeedsArtifactReboot)
    echo "No"
;;

SupportsRollback)
    echo "Yes"
;;

ArtifactInstall)
    dest_dir=$(cat $dest_dir_file)
    [[ -d $dest_dir ]] || install -d $dest_dir
    tar -cf ${prev_files_tar} -C ${dest_dir} .
    rm -rf ${dest_dir}
    mkdir -p ${dest_dir}
    tar -xf ${update_files_tar} -C ${dest_dir}
    ;;

ArtifactRollback)
    dest_dir=$(cat $dest_dir_file)
    [[ -f $prev_files_tar ]] || exit 0
    [[ -n "$dest_dir" ]] || exit 1
    rm -rf ${dest_dir}
    mkdir -p ${dest_dir}
    tar -xf ${prev_files_tar} -C ${dest_dir}
    ;;

esac

exit 0

Whether the already present script supports both file and directory update with the same script? Any document for the provided script (file-install) available? How to generate and pass the below identities?

prev_files_tar=ā€œ$FILESā€/tmp/prev_files.tar
update_files_tar=ā€œ$FILESā€/files/update.tar
dest_dir_file=ā€œ$FILESā€/files/dest_dir

May I know which is better (already provided one (i.e. file-install) or single-file/directory update script from mender hub)?

Hi @ajithpv,

The file-install Update Module from the 2.0.0 beta is going to be deprecated in the final release and instead we are providing two independent ones: single-file and directory.

As you have noticed, the old one supported both use cases, and we thought it we be clearer to just split them in two simpler ones.

Please use the new ones.

1 Like

Hi, I have an integrated build with u-boot that has worked great since version 1.6 with full updates. I updated my device to 2.0 to test out this new module feature. I did not mess with any u-boot integration this timeso it is still at the changes for 1.6, because I am just trying to do the module updates and didnā€™t figure I would need to make changes to u-boot for the modules. I have also updated my server to 2.0. I created an update as per this page and my device keeps giving me an error that I am not sure about.

ā€œlevel=error msg=ā€œFetching Artifact headers failed: installer: failed to read Artifact: readHeaderV3: handleHeaderReads: Cannot load handler for unknown Payload type ā€˜single-fileā€™ā€ module=stateā€

Then it waits and tries again in a min and keeps giving that same error.

Any idea what I am doing wrong?

Thanks!

Hi @speeltronics and welcome to Mender Hub! :slight_smile:

This error indicates that the Single File Update Module is not installed on the device where you are trying to install an Artifact Payload of this type.

To verify, run this command on your device:

mkdir -p /usr/share/mender/modules/v3 && wget -N -P /usr/share/mender/modules/v3 https://raw.githubusercontent.com/mendersoftware/mender/master/support/modules/single-file

Then re-try the deployment.

If this works then you need to simply make sure that this Update Module (single-file script) is present at this location in your image builds.

Oh I see now! Thanks! It would be really great if the modules could be sent with the updates, so that a rootfs update wouldnā€™t be needed to use the modules on already deployed devicesā€¦ Anyways, thanks again!

Unfortunately this canā€™t be done because it is a chicken and egg problem. The module is needed before the artifact is fully downloaded, but if the module is inside the artifact, then it needs to be downloaded firstā€¦

Once the single-file module is installed though, you can use it to install other modules. (*)

(*) Well, almost, except that there is a bug in it right now which doesnā€™t preserve permissions, like execute permission. Iā€™m in the process of fixing this right now, so this issue should be gone soon.

Understood! So, how do we know what version the modules are at or when it was last updated?

there is a bug in it right now which doesnā€™t preserve permissions, like execute permission. Iā€™m in the process of fixing this right now, so this issue should be gone soon.

FYI: Update modules fixes by kacf Ā· Pull Request #396 Ā· mendersoftware/mender Ā· GitHub

Understood! So, how do we know what version the modules are at or when it was last updated?

There is no versioning embedded in the update modules. They are just binaries or scripts, like other binaries on the system.

You can use an inventory script to list the contents of /usr/share/mender/modules/v3, perhaps with a md5sum to figure out the version. This will show up in the UI. See inventory scripts for more information.

Is it possible to include state scripts in the artifacts for update modules?
Is it possible to use LZMA compression for update modules?

Yes and yes!

And here how it is possible :slight_smile: :

  1. To add state scripts or sign the artifact, you can provide --key and --script as passthrough arguments, for example:

    ā€¦
    KEY=ā€œartifact-sign-key.keyā€
    SCRIPT1=ā€œArtifactCommit_Enter_01_custom_scriptā€
    SCRIPT2=ā€œAntotherScriptā€
    ./single-file-artifact-gen -n {ARTIFACT_NAME} -t {DEVICE_TYPE} -d {DEST_DIR} -o {OUTPUT_PATH} {FILE_TREE} -- --key {KEY} --script {SCRIPT1} --script {SCRIPT2}

  2. But --compression lzma canā€™t be passed like this as this argument should be passed to mender-artifact and not to mender-artifact write module-image. So to use lzma, you have to change the artifact generator script so it calls mender-artifact with this argument:

mender-artifact --compression lzma write module-image \ ...

Iā€™ll try to prepare a PR to improve this.

2 Likes

Hi,

thank you for this update module!
Itā€™s possible to perform a device reboot after the new file has been installed?
I like to update a configuration file and a reboot is required to take effect. What is the preferred way to do this?

Hello @dYalib
Iā€™m not quite sure, but there is no reboot activity in the script update module. You can try full system update for this. For more detailed information about the update modules, see here:
https://docs.mender.io/2.1/devices/update-modules

You will need to modify the module to support reboot.

And the change required would be, changing:

    NeedsArtifactReboot)
        echo "No"
    ;;

to

    NeedsArtifactReboot)
        echo "Automatic"
    ;;

You can reed more about the Update Module APIā€™s here,

1 Like

I tried to include a state script in an artifact with the name ArtifactInstall_Leave_00 with this command:

./single-file-artifact-gen -n ${ARTIFACT_NAME} -t ${DEVICE_TYPE} -d ${DEST_DIR} -o ${OUTPUT_PATH} ${FILE} ā€“ --script ${SCRIPT}

However, the file seems not to be included in the artifact. This is the error I get on the target:

ā€˜ArtifactInstall_Leave_00ā€™: -1 : fork/exec /var/lib/mender/scripts/ArtifactInstall_Leave_00: no such file or directory

According to the manual Artifact state scripts should be included in the generated artifact: State scripts | Mender documentation

But when I check the content of the artifact file, the state script is not included. I also tried to include the state script with the -f parameter which does indeed include the state script as payload but itā€™s still not working (since it probably does not put the state script to the above specified path).

What else do I need to specify to successfully include a state script file in an artifact?

Update: I now see that the script is actually correctly installed on the device in the following directory:
image

However, the script is somehow not executed as we can see in the error log:

2019-12-16 12:41:34 +0000 UTC error: transient error: error executing leave script for update-install state: error running leave state script(s) for ArtifactInstall state: statescript: error executing ā€˜ArtifactInstall_Leave_00ā€™: -1 : fork/exec /var/lib/mender/scripts/ArtifactInstall_Leave_00: no such file or directory

Any idea why the script is not executed?

Solved: Found the problem. The script was actually correctly executed - but there was a problem INSIDE the script. So the error message was a bit misleading :smiley:

Hehe. Good that you figured out yourself :wink:

It is quite limited the debug info that Mender client can take from the state script itslef. A good tip (that I always use at least while developing) is adding verbosity to the script itself with set -x. This way if the update fails inside the script, Mender will capture the output and send to the Mender server.

2 Likes

Hi,
Iā€™ve created and deployed the artifact as documented.
Iā€™ve compiled and run the client from an Ubuntu box.
Iā€™ve deployed the artifact on the server, which moved from Active to Finished.
The client was run with the -debug modifier. There I could read
DEBU[0001] Received response:204 No Content
DEBU[0001] No update available
DEBU[0001] no updates available

For there is something missconfigured, I couldnĀ“t figure out.

Are you sure you are not trying to install an Artifact with the same name as the already installed Artifact on the device? Each new Artifact being deployed needs to have a different name.

Hi, thank you for your answer. What I had missed was to define the device type on the client, so there was no match.

Now we could download a single file successfully, using the demo server. By the way, it can only send the domain s3.docker.mender.io, we could not change it and that could be nice, given that clients and servers live on different networks. The only workaround we found was to resolve that domain through the clientĀ“s /etc/hosts file, which is somewhat dirty.