# How to write a custom client interfacing a Mender server

**URL:** https://hub.mender.io/t/how-to-write-a-custom-client-interfacing-a-mender-server/1353
**Category:** Mender
**Created:** [December 16, 2019, 7:44pm UTC](https://hub.mender.io/t/how-to-write-a-custom-client-interfacing-a-mender-server/1353 "2019-12-16T19:44:28Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![mirzak](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/mirzak/32/2056_2.png) [@mirzak](https://hub.mender.io/u/mirzak)
#### Post date: [December 16, 2019, 7:44pm UTC](https://hub.mender.io/t/how-to-write-a-custom-client-interfacing-a-mender-server/1353/1 "2019-12-16T19:44:28Z")

</div>

## Introduction

Mender is an end-to-end OTA solution and both the Mender [client](https://github.com/mendersoftware/mender) and [server](https://github.com/mendersoftware/integration) are provided as open-source, additional we provide reference board integrations that bind it all together.

But Mender was designed with interchangeability in mind, especially for the client part of the solution. We provide [documentation](https://docs.mender.io/api/#device-apis) for the device facing API’s of the server which are implemented by the Mender client, but it is also a reference for implementing or integrating thirdparty clients.

These resources can be useful when:

- your connected devices is not running Linux which makes it incompatible with  
the existing Mender client (e.g Android, Windows, MCU’s).

- your want to integrate an existing thirdparty update client, e.g [SWUpdate](https://github.com/sbabic/swupdate), [RAUC](https://rauc.io/) or  
a homegrown solution.

The [API documentation](https://docs.mender.io/api/#device-apis) is a good starting point but it does not cover the expected workflow and one would need to dig trough the Mender client source code to try to figure out how it is using the API’s.

This tutorial aims to provide a overview of the API workflow and how to approach integrating either a custom or a thirdparty client with the Mender server API. For demonstration purposes we will be using standard CLI tools that are available on any major Linux distribution and in the end we should a working Mender client implemented as a `bash` script.

This tutorial will not cover bootloader integrations, flash layout or how you implement the logic to write the payload that you get from the Mender server.

## Prerequisites for this tutorial

It is assumed that you are running a Linux distribution (native or in a VM) with the following packages installed:

- bash - might work with other shell implementations
- jq - parse JSON structures
- curl - does HTTP(s)
- openssl - for keys and signatures

On a Debian based distribution you can run the following to install the dependencies:

```auto
sudo apt-get install bash jq curl openssl

```

Additionally, you will need to [install mender-artifact](https://docs.mender.io/downloads), version 3.1.0 or later

## Prerequisites for Mender client in a micro controller (RTOS)

To implement a custom Mender client your environment must have the equivalent capabilities to the tools listed above (excluding bash).

Your environment must have he following capabilities:

1. Able to communicate over TCP/IP using the HTTP protocol with the SSL extension (HTTPS)
2. Able to encrypt/decrypt/sign/verify data using the RSA or ECDSA256 cryptographic algorithms
3. Able to process JSON structures
4. Able to process [tar](https://en.wikipedia.org/wiki/Tar_(computing)) file archives. [Mender Artifact](https://docs.mender.io/overview/artifact) are uncompressed [tar](https://en.wikipedia.org/wiki/Tar_(computing)) file archives
5. (Optional) Able to decompress gzip or lzma. By default the [Mender Artifact](https://docs.mender.io/overview/artifact) payload is compressed using `gzip` (also supports lzma) but payload compression can be disabled.

If you have an device that is already using connectivity, and specifically HTTPS then you probably have the necessary components in place.

This is a unverified shortlist of stacks that are good candidates to support above requirements:

- [Zephyr Project](https://www.zephyrproject.org/)
- [Mbed OS](https://www.mbed.com/en/platform/mbed-os/)
- [Google Cloud IoT Device SDK for Embedded C](https://github.com/GoogleCloudPlatform/iot-device-sdk-embedded-c)
- and similar

## Overview

A client integrating with a Mender server only needs to implement five API calls:

- [/api/devices/v1/authentication/auth\_requests (POST)](https://docs.mender.io/api/#authenticate-device)
- [/api/devices/v1/inventory/device/attributes (PATCH)](https://docs.mender.io/api/#assign-attributes)
- [/api/devices/v1/deployments/device/deployments/next (GET)](https://docs.mender.io/api/#check-update)
- [/api/devices/v1/deployments/device/deployments/{id}/log (PUT)](https://docs.mender.io/api/#report-deployment-log)
- [/api/devices/v1/deployments//device/deployments/{id}/status (PUT)](https://docs.mender.io/api/#update-deployment-status)

And the workflow can be described with this simple diagram:

 ![mender_client (1)](https://canada1.discourse-cdn.com/flex036/uploads/mender/original/1X/2a2694e0c203f7bcc031ad1d35bc82dfa5383572.png)

## Step 0 - Preparations

As preparation we need to define a couple of variables that we will use trough out this tutorial.

Set Mender server URL:

```bash
MENDER_SERVER_URL="https://hosted.mender.io"

```

Set the tenant token variable (you can get it [here](https://hosted.mender.io/ui/#/settings/my-organization)):

```bash
MENDER_TENANT_TOKEN="< paste your token here >"

```

Set Mender client device type:

```bash
MENDER_DEVICE_TYPE="curl-client"

```

Set Mender Artifact name:

```bash
MENDER_ARTIFACT_NAME="release-v1"

```

## Step 1 - Authentication

The device authentication workflow is probably the most involved process and is something that most people struggle with initially. This is also something that is hard to debug, because you need to cryptographically sign data and if something goes wrong you will only that the signature is wrong, but not indicators on where it went wrong along the way.

You can familiarize your self with the device authorization process by reading the [Device authentication](https://docs.mender.io/overview/device-authentication) section in the official Mender documentation.

Here we need to use the [/api/devices/v1/authentication/auth\_requests (POST)](https://docs.mender.io/api/#authenticate-device) API call.

As the API documentation indicates, we need to send two things:

- `X-MEN-Signature` - Header request signature
- `auth_request` - Request body
  - `device_id` - You can read more about how device identity works in Mender  
[here](https://docs.mender.io/client-installation/identity)
  - `public key` - Used to verify the `X-MEN-Signature`, this proves  
that who ever sent this request also owns the private key.
  - `tenant token` (optional) - Needed to connect to a multitenant server, e.g [https://hosted.mender.io](https://hosted.mender.io)

Lets get to it. To send an authorization request we first must generate a key pair.

Generating a private RSA key can be done by executing the command below:

```bash
openssl genpkey -algorithm RSA -out private.key -pkeyopt rsa_keygen_bits:3072
openssl rsa -in private.key -out private.key

```

Extract a public key from the private key use following command:

```bash
openssl rsa -in private.key -out public.key -pubout

```

Store public key in a variable and replace newlines with `\n`:

```bash
PUBLIC_KEY=$(awk 'NF {sub(/\r/, ""); printf "%s\\n",$0;}' public.key)

```

The `PUBLIC_KEY` content should be:

> ```bash
> -----BEGIN PUBLIC KEY-----\nMIIBojANBgkqhkiG9w0BAQEFAAOCAY8AMIIBigKCAYEA0dIShCaaRf5EyvscTMI+\nXPe5EVrIHUj3kOs4KgOM++dswq+OMh03UJqW73p63Seay8H44s7J/D2SbsxtvAoU\nxH7uCOySquqYtyxi0DkufeYn/MbxPhZAgaK1tIa/jGhGfxqvJkJF1lsWPyboa8+g\nNPxwH3G4/m9IZW5IwhJ99Rbh1q6UWMxhNRi+f4Y1b4/JcFBGPaUxEuO6nkLQbAS4\nBUl+KefzJuliPIenbQYMmO5bkfgD0PZMVXRTTFcQfnpt4fRxLUmqsdnjSMbrloVY\nW/Nu2Z25A6+Dvw1A2eiASLtH5+3B26B5syEGYFcVea3xog5KqkViUbZvXIZQ3uI3\nB/uO8JlUSjXy6La9ZHriRqlbZi3VUfD1OiPfO8qZEfS/8MP5ttvP8rs3nra8Whby\nRAwtfO5R11r1+Mq34wOUM9OZJNDgINHRzSerbvIf6jkneMCE0b1IoW5RBoYapmy5\nv8bEvG4gJf6n62jfIvgzBBCSFn54pwaRCwREXHXNH52XAgMBAAE=\n-----END PUBLIC KEY-----\n
> 
> ```

Generate request body (we use MAC address as identity data here):

```bash
# NOTE! This command will trim newlines, and it is very important to do this,
# otherwise the data will not match what is sent out, and this also means
# that the signature would be invalid
read -r -d '' REQUEST_BODY <<EOF
{
    "id_data": "{ \"mac\": \"00:11:22:33:44:55\"}",
    "pubkey": "${PUBLIC_KEY}",
    "tenant_token": "${MENDER_TENANT_TOKEN}"
}
EOF

```

The `REQUEST_BODY` content should be:

> ```bash
> { "id_data": "{ \"mac\": \"00:11:22:33:44:55\"}", "pubkey": "-----BEGIN PUBLIC KEY-----\nMIIBojANBgkqhkiG9w0BAQEFAAOCAY8AMIIBigKCAYEA0dIShCaaRf5EyvscTMI+\nXPe5EVrIHUj3kOs4KgOM++dswq+OMh03UJqW73p63Seay8H44s7J/D2SbsxtvAoU\nxH7uCOySquqYtyxi0DkufeYn/MbxPhZAgaK1tIa/jGhGfxqvJkJF1lsWPyboa8+g\nNPxwH3G4/m9IZW5IwhJ99Rbh1q6UWMxhNRi+f4Y1b4/JcFBGPaUxEuO6nkLQbAS4\nBUl+KefzJuliPIenbQYMmO5bkfgD0PZMVXRTTFcQfnpt4fRxLUmqsdnjSMbrloVY\nW/Nu2Z25A6+Dvw1A2eiASLtH5+3B26B5syEGYFcVea3xog5KqkViUbZvXIZQ3uI3\nB/uO8JlUSjXy6La9ZHriRqlbZi3VUfD1OiPfO8qZEfS/8MP5ttvP8rs3nra8Whby\nRAwtfO5R11r1+Mq34wOUM9OZJNDgINHRzSerbvIf6jkneMCE0b1IoW5RBoYapmy5\nv8bEvG4gJf6n62jfIvgzBBCSFn54pwaRCwREXHXNH52XAgMBAAE=\n-----END PUBLIC KEY-----\n", "tenant_token": "< paste your token here >" }
> 
> ```

**NOTE!** That the tenant token has the default value here, and instead it should print what you configured.

Generate signature of type [RSASSA-PKCS1-V1\_5-SIGN](https://tools.ietf.org/html/rfc3447#section-8.2):

```bash
X_MEN_SIGNATURE=$(echo -n "${REQUEST_BODY}" | openssl dgst -sha256 -sign private.key | openssl base64 -A)

```

The `X_MEN_SIGNATURE` content should be:

> ```auto
> MXWhOON/w2lQI6uj2Arbxauvl3Quu7hT35yoY3XBG+KxDzJjLiUQDDo45uHDYKIUV6V4OqKdnsz9LQ+4s9X8IlHPKqxPiUKCByAPb5O+OaWfOC1BbkGLYAVULtes1J2p1r4n0benr2E2T3VlTl3x1Zw3lxAJsRiQTPdpgyDvcV46R+lUWCYnCXXwrxAXna5RovgzyhBSkCFIE3bcSm5/M8jKyFNA3iEoSCuJpv4ZcAjpvQTp/wbFC134hHmP03FfbRGYFUNDR5gi5gUYDVfCUoTp739AaVpzt8ASuWTHtQ9PFEVYfhlr7U0L0w6sSNbFCuDSsbGTYgPNBlYHE0d5xAR7zRevgZmUB+++rg07l3FXgimLdd8YJUV7jp2pCfQze3d6DCiIWHpxl4WjuTgBlNDcrIi72MqmlIwzRxAKy9Y5R+A4bLbsFdaVt5JVBAg8QP84O81Xp/dP7TG4wryF25/3VHob3N7W1wF31EKPtVhqVQX1ltQgS7zxT8ttucMf
> 
> ```

**NOTE!** Above value is only valid if all the default variables are used, including the tenant token so your signature might differ.

```bash
curl \
    -H "Content-Type: application/json" \
    -H "X-MEN-Signature: ${X_MEN_SIGNATURE}" \
    --data "${REQUEST_BODY}" \
    ${MENDER_SERVER_URL}/api/devices/v1/authentication/auth_requests

```

The expected response to above is:

> ```bash
> {"error":"dev auth: unauthorized","request_id":"7d637207-6a62-4c16-8e1f-14d37bd0693f"}
> 
> ```

This means that the device is not yet authorized, but we have successfully sent the authentication request.

If you go to the [Pending](https://hosted.mender.io/ui/#/devices/pending) on the server you should see device there with a matching device identity to what we prepare for our authentication request.

 ![Screenshot_2019-12-13_16-52-48](https://canada1.discourse-cdn.com/flex036/uploads/mender/original/1X/c6e5e65131b2bc5a76cca124c54c1f7e5b2256bc.png)

**NOTE!** Make sure to accept the device on the server before proceeding with the next steps.

Once the device is accepted on the server we can run the same command again to get an access token (JSON Web Token):

```bash
JWT=$(curl \
    -H "Content-Type: application/json" \
    -H "X-MEN-Signature: ${X_MEN_SIGNATURE}" \
    --data "${REQUEST_BODY}" \
    ${MENDER_SERVER_URL}/api/devices/v1/authentication/auth_requests)

```

The JSON Web token will be used for all subsequent API calls.

## Step 2 - Publishing inventory data

Each device connected to the Mender server must publish inventory data to populate the necessary fields for the GUI to display in the [Device](https://hosted.mender.io/ui/#/devices) tab.

At a minimum the `device_type` and the `artifact_name` name should be sent. You can read more about inventory data in the official Mender [documentation](https://docs.mender.io/client-installation/inventory)

Inventory data is to be sent as an JSON array, and we can prepare it with the following command:

```bash
read -r -d '' INVENTORY_DATA <<EOF
[
    {
      "name":"device_type",
      "value":"${MENDER_DEVICE_TYPE}"
    },
    {
      "name":"artifact_name",
      "value":"${MENDER_ARTIFACT_NAME}"
    },
    {
      "name":"kernel",
      "value":"$(uname -a)"
    }
]
EOF

```

Now we can publish it to the server:

```bash
curl \
    -H "Content-Type: application/json" \
    -H "Authorization: Bearer $JWT" \
    --data "${INVENTORY_DATA}" \
    -X PATCH \
    ${MENDER_SERVER_URL}/api/devices/v1/inventory/device/attributes

```

You should now see something similar to below in the [Devices](https://hosted.mender.io/ui/#/devices) tab.

 ![Screenshot_2019-12-13_21-21-00](https://canada1.discourse-cdn.com/flex036/uploads/mender/original/1X/72a7236e2dd854a885867053b31d04013340eb4e.png)

## Step 3 - Check for deployments

To check if there are are pending deployments on the Mender server we can run (command will print return code):

```bash
curl -s -o /dev/null -w '%{http_code}' \
    -H "Authorization: Bearer $JWT" \
    -X GET \
    "${MENDER_SERVER_URL}/api/devices/v1/deployments/device/deployments/next?artifact_name=${MENDER_ARTIFACT_NAME}&device_type=${MENDER_DEVICE_TYPE}"

```

The expected result right now is to get return code `400`, meaning that there are no pending deployments. We would keep doing this check periodically until we get return code `200` meaning that there is a pending deployment and we should handle it accordingly.

## Step 4 - Download deployment

For demonstration purposes we need to prepare a Mender Artifact that we can deploy to our `curl-client`.

Create payload data:

```bash
dd if=/dev/urandom of=payload.img bs=1M count=1

```

Create a Mender Artifact:

```bash
mender-artifact write rootfs-image \
    --device-type ${DEVICE_TYPE} \
    --artifact-name release-v2 \
    --output-path release-v2.mender \
    --file payload.img

```

Upload the artifact to the Mender server, under the [Releases](https://hosted.mender.io/ui/#/releases) tab.

You should have something like this:

 ![Screenshot_2019-12-13_21-42-30](https://canada1.discourse-cdn.com/flex036/uploads/mender/original/1X/90f56b6f549909c862e6d32605ab2ac42f0650bd.png)

Press the “CREATE DEPLOYMENT WITH THIS RELEASE” button and click trough the deployment wizard.

Lets check if there are any pending deployments now:

```auto
curl \
    -H "Authorization: Bearer $JWT" \
    -X GET \
    "${MENDER_SERVER_URL}/api/devices/v1/deployments/device/deployments/next?artifact_name=${MENDER_ARTIFACT_NAME}&device_type=${MENDER_DEVICE_TYPE}"

```

Expected result is something similar to below:

> ```bash
> {
> "id": "a4d9c64f-3de5-40f0-9886-c0f77d00baf0",
> "artifact": {
> "artifact_name": "release-v2",
> "source": {
> "uri": "https://s3.amazonaws.com/hosted-mender-artifacts/5b48937d7e71f600014ab529/af0ccfb3-c410-40f8-9b30-649e5dd7878c?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIAQWI25QR6NDTJ7DLD%2F20191213%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20191213T204753Z&X-Amz-Expires=86400&X-Amz-SignedHeaders=host&response-content-type=application%2Fvnd.mender-artifact&X-Amz-Signature=3923b3238890ea98e51c6943805900570f306d1aaa858eedfd6b3ef1417dfd1b",
> "expire": "2019-12-13T20:47:53.869391192Z"
> },
> "device_types_compatible": [
> "curl-client"
> ]
> }
> }
> 
> ```

As you can see that the server has responded that there is a deployment in progress and we have gotten an signed URL from which we can download the Mender Artifact.

## Step 5 - Updating deployment status on the server

The client is responsible of updated the deployment state to the server and in the end marking it either as successful or failure.

Supported states are:

- `downloading`
- `installing`
- `rebooting`
- `success`
- `failure`
- `already-installed`

There is also support for publishing `substate`, which is an optional parameter of the API call. Definition of the state strings are up to the user, and they will be only be displayed by Mender server without further processing. The official Mender client uses `substate` to report execution of [state-scripts](https://docs.mender.io/overview/state-script).

For demonstration purposes lets update the state to `downloading` on our pending deployment that we created in previous steps:

```bash
curl \
    -H "Authorization: Bearer $JWT" \
    -H "Content-Type: application/json" \
    -d "{\"status\":\"downloading\"}" \
    -X PUT \
    "${MENDER_SERVER_URL}/api/devices/v1/deployments/device/deployments/a4d9c64f-3de5-40f0-9886-c0f77d00baf0/status"

```

**NOTE!** We are using the deployment id ( `a4d9c64f-3de5-40f0-9886-c0f77d00baf0` ) that we got from the previous step

You can verify that above command was successful by checking the [Deployments](https://hosted.mender.io/ui/#/deployments) tab on the Mender server.

And finally we can mark the deployment as complete by updating the state to `success`:

```bash
  curl \
    -H "Authorization: Bearer $JWT" \
    -H "Content-Type: application/json" \
    -d "{\"status\":\"success\"}" \
    -X PUT \
    "${MENDER_SERVER_URL}/api/devices/v1/deployments/device/deployments/a4d9c64f-3de5-40f0-9886-c0f77d00baf0/status"

```

**NOTE!** We are using the deployment id (`a4d9c64f-3de5-40f0-9886-c0f77d00baf0`) that we got from the previous step

The last command should have marked the deployment as complete on the server. You can verify this by going to the [Deployments](https://hosted.mender.io/ui/#/deployments/finished) tab.

The is also an possibility to send a [device log](https://docs.mender.io/api/#report-deployment-log) to the Mender server using the API’s, which is something you would do in case of update failure. This way one can troubleshoot deployments using the logs that where pushed to Mender server.

## Conclusion

In this tutorial we have gone trough the Mender server client interface in detail, using common CLI tools on Linux. Hopefully we have demonstrated the simplicity of the interface and that it is helpful in your integration efforts.

Can further recommend tools such as [Postman](https://www.getpostman.com/product/api-client), which are useful to debug HTTP commands. While developing this tutorial. `curl -v` and `curl --trace-ascii` provided very useful which provide very detailed information on how the HTTP commands are sent out, down to byte level.

If you want an overview of the workflow in code, you can take a look at the [mender-client.sh](https://gist.github.com/mirzak/62d8b0574e754f657836ab78b8dea896) script which implements the steps covered in this tutorial.

Additional there are a couple re-implementations of the Mender client:

- [IncludeOS C++ Mender client (archived)](https://github.com/includeos/mender)
- [Rust Mender client (experimental)](https://github.com/olepor/Mender-Rust)

And of course a the source code of the [official Mender client](https://github.com/mendersoftware/mender) can provide a good resource for information.

* * *

_If this tutorial was useful to you, please press like, or leave a thank you note to the contributor who put valuable time into this and made it available to you. It will be much appreciated!_

---

<div class="post-metadata">

### Author: ![mirzak](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/mirzak/32/2056_2.png) [@mirzak](https://hub.mender.io/u/mirzak)
#### Post date: [September 11, 2020, 7:12am UTC](https://hub.mender.io/t/how-to-write-a-custom-client-interfacing-a-mender-server/1353/2 "2020-09-11T07:12:50Z")

</div>

3 posts were split to a new topic: [Use ECC keys for authentication between Mender client server](https://hub.mender.io/t/use-ecc-keys-for-authentication-between-mender-client-server/2431)

---

<div class="post-metadata">

### Author: ![TheYoctoJester](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/theyoctojester/32/1444_2.png) [@TheYoctoJester](https://hub.mender.io/u/TheYoctoJester)
#### Post date: [November 14, 2022, 9:10pm UTC](https://hub.mender.io/t/how-to-write-a-custom-client-interfacing-a-mender-server/1353/3 "2022-11-14T21:10:03Z")

</div>

A post was split to a new topic: [Writing a custom Mender client for an MCU](https://hub.mender.io/t/writing-a-custom-mender-client-for-an-mcu/5404)

---

<div class="post-metadata">

### Author: ![TheYoctoJester](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/theyoctojester/32/1444_2.png) [@TheYoctoJester](https://hub.mender.io/u/TheYoctoJester)
#### Post date: [April 23, 2024, 7:38am UTC](https://hub.mender.io/t/how-to-write-a-custom-client-interfacing-a-mender-server/1353/4 "2024-04-23T07:38:05Z")

</div>

A post was split to a new topic: [Custom Mender client - signed download URL](https://hub.mender.io/t/custom-mender-client-signed-download-url/6750)