In a lot of cases, transporting metadata is more desirable than handling the actual data. Why? An example.
What is metadata?
You have read a wonderful book. An enlightening, life-changing book. You absolutely want your friend to enjoy the same. So what do you do?
Would you try and replicate all the books content for them? Probably not, you would miss way too much. Hand them your copy of the book? Possible, but complicated if your friend doesn’t live close to you. So again, what do you do?
You send the bibliographic information. Title, Author, maybe year, maybe publisher. Thats all.
As an example, lets take “Ulysses”, by “James Joyce”. At the bare minimum, this requires 3 words, or 20 bytes (payload plus 2 spaces plus null termination) to transfer. However, the book in question consists of 265.222 words, or about 1.5MBytes (UTF-8 encoded, feel free to see for yourself here).
So what is the trick?
Metadata in the OTA context
We only transferred meta data. This term means “data about data”. In our example, the data on the book - consisting of title an author - is perfectly sufficient for the recipient to obtain the full text. There is no need for us to replicate and provide it.
Lets go back to OTA devce management. There are many cases where sending metadata to a device is a superior choice to transporting the full data set.
Again, lets pick an example. Suppose you want to install a specific docker container on a device. Theoretically, you could pack up the whole image and send it over. But it might also be enought to clearly specify the exact image that the device should pull from the registry. Image name and tag might be retroactively changed, but the SHA hash is immutable. So you could specify: “docker pull ubuntu@sha256:26c68657ccce2cb0a31b330cb0be2b5e108d467f641c62e13ab40cbec258c68d”. The metadata consisting of image name and sha256-sum is enough to provide a unique identification of the image.
Leveraging metadata in a Mender artifact
This is exactly the approach that the docker update module for Mender employs. An Artifact is used to marshal an arbitrary number of meta data sets on containers to the Mender Client running on a device, which then iterates over the list to install them.
Hardly surprising, the field in the Mender Artifact structure that is leveraged here is called “meta-data”.
Adding metadata
The metadata property can be used to hold arbitrary data in a subset of JSON format, effectively as key-value pairs. An example file mymeta.json could look like this:
{
"datastring": "foobar",
"datanumber": 1.0
}
During artifact creation, the metadata file can be directly added to the output by passing it as flag:
mender-artifact write ...
-m mymeta.json
...
Retrieving metadata
During installation of a Mender Artifact, the corresponding Update Module is repeatedly invoked, once for each state. The Artifact content is exposed to the called executable through the File API, including the metadata it contains. It can be retrieved by simply opening and reading the file, for example using the jq tool:
FILES="$2" # This parameter exposes the File API
echo $jq -r ".datastring" "$FILES"/header/meta-data)
will yield “foobar” into the logs.
Use case ideas
This form of data being transported through OTA provides an enormous flexibility for custom use cases. Some might be:
- using application specific package manager with a separate repository, such as the snap Update Module
- sending instructions to a specialized Update Module
- attaching information for on-device logging, such as release checksum, version control tag, …
Let us know what interesting things you archieved by adding carefully crafted metadata to your deployments!