Single File

ajithpv · April 29, 2019, 12:30pm

@lluiscampos Thank you for providing the single file update module.

There is already one file-install script present as below while build and run the mender 2.0.0 beta:

cat /usr/share/mender/modules/v3/file-install
#!/bin/bash

set -e

STATE="$1"
FILES="$2"

prev_files_tar="$FILES"/tmp/prev_files.tar
update_files_tar="$FILES"/files/update.tar
dest_dir_file="$FILES"/files/dest_dir

case “$STATE” in

NeedsArtifactReboot)
    echo "No"
;;

SupportsRollback)
    echo "Yes"
;;

ArtifactInstall)
    dest_dir=$(cat $dest_dir_file)
    [[ -d $dest_dir ]] || install -d $dest_dir
    tar -cf ${prev_files_tar} -C ${dest_dir} .
    rm -rf ${dest_dir}
    mkdir -p ${dest_dir}
    tar -xf ${update_files_tar} -C ${dest_dir}
    ;;

ArtifactRollback)
    dest_dir=$(cat $dest_dir_file)
    [[ -f $prev_files_tar ]] || exit 0
    [[ -n "$dest_dir" ]] || exit 1
    rm -rf ${dest_dir}
    mkdir -p ${dest_dir}
    tar -xf ${prev_files_tar} -C ${dest_dir}
    ;;

esac

exit 0

Whether the already present script supports both file and directory update with the same script? Any document for the provided script (file-install) available? How to generate and pass the below identities?

prev_files_tar="$FILES"/tmp/prev_files.tar
update_files_tar="$FILES"/files/update.tar
dest_dir_file="$FILES"/files/dest_dir

May I know which is better (already provided one (i.e. file-install) or single-file/directory update script from mender hub)?

lluiscampos · April 29, 2019, 12:44pm

Hi @ajithpv,

The file-install Update Module from the 2.0.0 beta is going to be deprecated in the final release and instead we are providing two independent ones: single-file and directory.

As you have noticed, the old one supported both use cases, and we thought it we be clearer to just split them in two simpler ones.

Please use the new ones.

speeltronics · May 8, 2019, 10:37pm

Hi, I have an integrated build with u-boot that has worked great since version 1.6 with full updates. I updated my device to 2.0 to test out this new module feature. I did not mess with any u-boot integration this timeso it is still at the changes for 1.6, because I am just trying to do the module updates and didn’t figure I would need to make changes to u-boot for the modules. I have also updated my server to 2.0. I created an update as per this page and my device keeps giving me an error that I am not sure about.

“level=error msg=“Fetching Artifact headers failed: installer: failed to read Artifact: readHeaderV3: handleHeaderReads: Cannot load handler for unknown Payload type ‘single-file’” module=state”

Then it waits and tries again in a min and keeps giving that same error.

Any idea what I am doing wrong?

Thanks!

eystein · May 8, 2019, 10:48pm

Hi @speeltronics and welcome to Mender Hub!

This error indicates that the Single File Update Module is not installed on the device where you are trying to install an Artifact Payload of this type.

To verify, run this command on your device:

mkdir -p /usr/share/mender/modules/v3 && wget -N -P /usr/share/mender/modules/v3 https://raw.githubusercontent.com/mendersoftware/mender/master/support/modules/single-file

Then re-try the deployment.

If this works then you need to simply make sure that this Update Module (single-file script) is present at this location in your image builds.

speeltronics · May 9, 2019, 12:58pm

Oh I see now! Thanks! It would be really great if the modules could be sent with the updates, so that a rootfs update wouldn’t be needed to use the modules on already deployed devices… Anyways, thanks again!

kacf · May 9, 2019, 1:19pm

Unfortunately this can’t be done because it is a chicken and egg problem. The module is needed before the artifact is fully downloaded, but if the module is inside the artifact, then it needs to be downloaded first…

Once the single-file module is installed though, you can use it to install other modules. (*)

(*) Well, almost, except that there is a bug in it right now which doesn’t preserve permissions, like execute permission. I’m in the process of fixing this right now, so this issue should be gone soon.

speeltronics · May 9, 2019, 4:42pm

Understood! So, how do we know what version the modules are at or when it was last updated?

kacf · May 13, 2019, 12:46pm

there is a bug in it right now which doesn’t preserve permissions, like execute permission. I’m in the process of fixing this right now, so this issue should be gone soon.

FYI: https://github.com/mendersoftware/mender/pull/396

Understood! So, how do we know what version the modules are at or when it was last updated?

There is no versioning embedded in the update modules. They are just binaries or scripts, like other binaries on the system.

You can use an inventory script to list the contents of /usr/share/mender/modules/v3, perhaps with a md5sum to figure out the version. This will show up in the UI. See inventory scripts for more information.

ster · July 19, 2019, 3:43pm

Is it possible to include state scripts in the artifacts for update modules?
Is it possible to use LZMA compression for update modules?

kacf · July 23, 2019, 6:51am

Yes and yes!

ster · July 23, 2019, 11:01am

And here how it is possible :

To add state scripts or sign the artifact, you can provide --key and --script as passthrough arguments, for example:

…
KEY=“artifact-sign-key.key”
SCRIPT1=“ArtifactCommit_Enter_01_custom_script”
SCRIPT2=“AntotherScript”
./single-file-artifact-gen -n {ARTIFACT_NAME} -t {DEVICE_TYPE} -d {DEST_DIR} -o {OUTPUT_PATH} {FILE_TREE} -- --key {KEY} --script {SCRIPT1} --script {SCRIPT2}
But --compression lzma can’t be passed like this as this argument should be passed to mender-artifact and not to mender-artifact write module-image. So to use lzma, you have to change the artifact generator script so it calls mender-artifact with this argument:

mender-artifact --compression lzma write module-image \ ...

I’ll try to prepare a PR to improve this.

dYalib · October 7, 2019, 4:01pm

Hi,

thank you for this update module!
It’s possible to perform a device reboot after the new file has been installed?
I like to update a configuration file and a reboot is required to take effect. What is the preferred way to do this?

mEK · October 8, 2019, 6:26am

Hello @dYalib
I’m not quite sure, but there is no reboot activity in the script update module. You can try full system update for this. For more detailed information about the update modules, see here:
https://docs.mender.io/2.1/devices/update-modules

mirzak · October 8, 2019, 7:42am

You will need to modify the module to support reboot.

And the change required would be, changing:

    NeedsArtifactReboot)
        echo "No"
    ;;

to

    NeedsArtifactReboot)
        echo "Automatic"
    ;;

You can reed more about the Update Module API’s here,

github.com

mendersoftware/mender/blob/2.1.0/Documentation/update-modules-v3-file-api.md

Update modules v3 protocol
==========================

Update modules are executables that are placed in `/usr/share/mender/modules/v3`
directory, where `v3` is a reference to the version of the protocol, which is
the same as [the version of the Artifact
format](https://github.com/mendersoftware/mender-artifact/tree/master/Documentation). Mender
will look for update modules in the directory with the same version as the
version of the Artifact being processed.


States and execution flow
-------------------------

Update modules are modelled after the same execution flow as [state
scripts](https://docs.mender.io/artifacts/state-scripts), and consists of the
following states:

* `Download`
* `ArtifactInstall`

This file has been truncated. show original

freibrun · December 16, 2019, 2:16pm

I tried to include a state script in an artifact with the name ArtifactInstall_Leave_00 with this command:

./single-file-artifact-gen -n {ARTIFACT_NAME} -t {DEVICE_TYPE} -d {DEST_DIR} -o {OUTPUT_PATH} {FILE} -- --script {SCRIPT}

However, the file seems not to be included in the artifact. This is the error I get on the target:

‘ArtifactInstall_Leave_00’: -1 : fork/exec /var/lib/mender/scripts/ArtifactInstall_Leave_00: no such file or directory

According to the manual Artifact state scripts should be included in the generated artifact: https://docs.mender.io/2.2/artifacts/state-scripts#root-file-system-and-artifact-scripts

But when I check the content of the artifact file, the state script is not included. I also tried to include the state script with the -f parameter which does indeed include the state script as payload but it’s still not working (since it probably does not put the state script to the above specified path).

What else do I need to specify to successfully include a state script file in an artifact?

Update: I now see that the script is actually correctly installed on the device in the following directory:

However, the script is somehow not executed as we can see in the error log:

2019-12-16 12:41:34 +0000 UTC error: transient error: error executing leave script for update-install state: error running leave state script(s) for ArtifactInstall state: statescript: error executing ‘ArtifactInstall_Leave_00’: -1 : fork/exec /var/lib/mender/scripts/ArtifactInstall_Leave_00: no such file or directory

Any idea why the script is not executed?

Solved: Found the problem. The script was actually correctly executed - but there was a problem INSIDE the script. So the error message was a bit misleading

lluiscampos · December 16, 2019, 3:02pm

Hehe. Good that you figured out yourself

It is quite limited the debug info that Mender client can take from the state script itslef. A good tip (that I always use at least while developing) is adding verbosity to the script itself with set -x. This way if the update fails inside the script, Mender will capture the output and send to the Mender server.

gerardostola · March 23, 2020, 11:18pm

Hi,
I’ve created and deployed the artifact as documented.
I’ve compiled and run the client from an Ubuntu box.
I’ve deployed the artifact on the server, which moved from Active to Finished.
The client was run with the -debug modifier. There I could read
DEBU[0001] Received response:204 No Content
DEBU[0001] No update available
DEBU[0001] no updates available

For there is something missconfigured, I couldn´t figure out.

kacf · March 24, 2020, 7:27am

Are you sure you are not trying to install an Artifact with the same name as the already installed Artifact on the device? Each new Artifact being deployed needs to have a different name.

gerardostola · March 25, 2020, 1:02am

Hi, thank you for your answer. What I had missed was to define the device type on the client, so there was no match.

Now we could download a single file successfully, using the demo server. By the way, it can only send the domain s3.docker.mender.io, we could not change it and that could be nice, given that clients and servers live on different networks. The only workaround we found was to resolve that domain through the client´s /etc/hosts file, which is somewhat dirty.

mirzak · March 25, 2020, 7:50am

This is a limitation of the Demo environment, as we provide demo certificates for the docker.mender.io and s3.docker.mender.io domains.

If you want to use a custom domain name you can setup a production environment, https://docs.mender.io/2.3/administration/production-installation