Writing a custom Mender client for an MCU

Hello @mirzak

Thanks for this tutorial How to write a custom client interfacing a Mender server. I’m currently working on an MCU solution to perform OTA using mender.

Several questions please about the download of the artifact, that you don’t really detail here:

  • I realize the expire date of the URI to download the artefact is very short, let say about 3 seconds for me!!! Is it wanted ? Configurable ?
  • My understanding is that I can do a simple GET with the URI, no need for JWT etc, is it correct ?
  • What will be exactly downloaded using this URI ? .mender file uploaded on the server ? Something else ?

Edit: after trying to download, I can answer the two last questions: no need of token the URI serves as an authentication method and mender file is retrieved. So one question comes to me:

  • I have retrieved the mender file but it’s not very suitable because I need to access the bin file inside the data\0000.tar.gz archive. So I need to untar twice now. Clearly will need a lot of memory so I’m thinking another solution: is it possible to retrieve the bin file inside data\0000.tar.gz archive directly from mender ? If this is not a functionality you have I’m thinking to create a small docker container that do the job for me on the fly.

Thanks,
Joel

Answering some question I ask myself:

https://mymender.com/mender-artifact-storage/1d95c890-12d4-442d-8f0f-99bddb38c930?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=mender-deployments0.000000202211170.000000us-east-10.000000s30.000000aws4_request&X-Amz-Date=20221117T223248Z&X-Amz-Expires=86400&X-Amz-SignedHeaders=host&response-content-type=application0.000000vnd.mender-artifact&X-Amz-Signature=88e5f3f62ec596b7d47084293a1bf1f62261ef1c01cc00d416ba35e979800439

becomes:

https://mymender.com/mender-artifact-storage/1d95c890-12d4-442d-8f0f-99bddb38c930/version?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=mender-deployments0.000000202211170.000000us-east-10.000000s30.000000aws4_request&X-Amz-Date=20221117T223248Z&X-Amz-Expires=86400&X-Amz-SignedHeaders=host&response-content-type=application0.000000vnd.mender-artifact&X-Amz-Signature=

Up to this point, is there anybody at mender side able to confirm ?

If this is correct now the URL need to have a new X-Amz-Signature because it depends of the GET path, which is not so easy to compute but should be possible. Is it (on the device side) ?

WIP…

Hello @joelguittet,

By default, the download URL are valid for 900 seconds:

You don’t need a JWT token, because the download URL is presigned. A GET request is enough to download the artifact, which will be the “.mender” file you uploaded to the server. Once you have the mender file, you need to parse it. We use the mender-artifact golang library to parse it, but you can do it manually if you are using a different programming language because a mender file is basically a tarball.

You can find the format specification here:

Hello @tranchitella

Yes I understood that points, parsing the docker configurations and different GO sources (even if it is not my cup of tea!)
Thanks for the location of the expire delay location.
My difficulty is the extraction of the mender file because of the limited ressources on the MCU.

Working on a solution that should fit the need on this… :slight_smile:

Will update here depending of my progress.

Joel

The mender artifact is an uncompressed tarball which you can read as you receive it from the server (stream of bytes). It is designed this way to work on devices with limited (cpu, storage) resources.

Keep us posted!

Hi @joelguittet

If your problem is memory with the Artifact, it is not necessary to download the whole thing up front, and then parse it.

It is a tar format, so it can be handled in a stream. Most tar libraries should be able to do this for you.

Are you writing this in C? Which tar lib are you using?

Also, although Artifacts can be compressed (as you have observed), they don’t have to. It is possible to leave the Artifact uncompressed if you want to.

Hello,
Tar library not chosen yet, if you have a good light-weight one to recommend I m open :slight_smile: yes I m looking for pure C language.
You indicate it is not mandatory to compress artifacts, can you detail this? Is it an option when creating the mender artifact file?
Joel

Information on configuring compression for mender artifact creation can be found here

2 Likes

Hello,
Thanks, currently progressing now on this, will kepp updated.
Joel

1 Like

Progress on this topic:

  • I have created my own tar parser.
  • I retrieve the binary from the data.tar file and it looks okay.

Comments:

  • The documentation do not indicates the mender file compression is optional at mender-artifact/artifact-format-v3.md at master · mendersoftware/mender-artifact · GitHub (it is indicated it’s using tar.gz only)
  • The “–compression” option of mender-artifact is not well documented (even calling “./mender-artifact --help” do not clearly explain this). I have found in the sources that “–compression none” permit no compression, and the mender file is tar only and not tar.gz :slight_smile:

Continue working on the subject now, no more blocked

Joel

1 Like

Ohh, good observation! We will amend this :smile_cat:

PR: docs(artifact-format): Highlight that the compression is optional by oleorhagen · Pull Request #450 · mendersoftware/mender-artifact · GitHub

1 Like