Mender management, artifact upload seems slow

General Discussions
1

Hi all,

First of all , I use a free trial version of Mender. I m trying the management API . It works well.

I upload a 232MBytes artifact LZMA compressed. The duration for such upload is about 5 minutes and 30 seconds.
I guess the upload itself is less than one minute according to my bandwidth.

My questions are:

  • Is the duration of about 5 minutes 30 seconds due to the time taken by the Mender server to extract my archive? (I guess it is)
  • Will this time be reduced with the Enterprise version?

here are my logs:

speedtest
==============================================================================

You may only use this Speedtest software and information generated
from it for personal, non-commercial use, through a command line
interface on a personal computer. Your use of this software is subject
to the End User License Agreement, Terms of Use and Privacy Policy at
these URLs:

	https://www.speedtest.net/about/eula
	https://www.speedtest.net/about/terms
	https://www.speedtest.net/about/privacy

==============================================================================

Do you accept the license? [type YES to accept]: yes
License acceptance recorded. Continuing.

==============================================================================

Ookla collects certain data through Speedtest that may be considered
personally identifiable, such as your IP address, unique device
identifiers or location. Ookla believes it has a legitimate interest
to share this data with internet providers, hardware manufacturers and
industry regulators to help them understand and create a better and
faster internet. For further information including how the data may be
shared, where the data may be transferred and Ookla's contact details,
please see our Privacy Policy at:

       http://www.speedtest.net/privacy

==============================================================================

Do you accept the license? [type YES to accept]: YES
License acceptance recorded. Continuing.


   Speedtest by Ookla

      Server: BOUYGUES TELECOM CUBIC - Vélizy (id: 47424)
         ISP: Orange
Idle Latency:    12.12 ms   (jitter: 1.27ms, low: 11.75ms, high: 13.97ms)
    Download:    85.03 Mbps (data used: 108.7 MB)                                                   
                218.89 ms   (jitter: 60.24ms, low: 22.61ms, high: 354.86ms)
      Upload:    97.22 Mbps (data used: 180.2 MB)                                                   
                 35.56 ms   (jitter: 12.14ms, low: 13.14ms, high: 313.14ms)
 Packet Loss:     0.0%
  Result URL: https://www.speedtest.net/result/c/f06cf266-c1dc-4bd9-b2aa-b9e0c33b5bc8
pierr0t@00095:~$ echo $MENDER_URL
https://eu.hosted.mender.io/api/management/v1/deployments/artifacts
pierr0t@00095:~$ echo $DESCRIPTION
Sample artifact for testing direct upload
pierr0t@00095:~$ du -h $FILE_PATH
232M	/home/pierr0t/repo/mbbox-bsp-mbbox_cooker_menu/builds/build-mb-box-debug/tmp/work/raspberrypi4_64-poky-linux/mb-box/1.0-r0/build-wic/partTest-mbb-signed.mender
pierr0t@00095:~$ date;time curl -s -w "%{http_code}" -X POST "$MENDER_URL"     -H "Authorization: Bearer $ACCESS_TOKEN"     -H "Accept: application/json"     -F "description=$DESCRIPTION"     -F "artifact=@$FILE_PATH;type=application/octet-stream"
mar. 29 oct. 2024 11:12:55 CET
Bad Gateway502
real	0m29,036s
user	0m0,187s
sys	0m0,144s
pierr0t@00095:~$ date;time curl -s -w "%{http_code}" -X POST "$MENDER_URL"     -H "Authorization: Bearer $ACCESS_TOKEN"     -H "Accept: application/json"     -F "description=$DESCRIPTION"     -F "artifact=@$FILE_PATH;type=application/octet-stream"
mar. 29 oct. 2024 11:14:00 CET
201
real	5m39,548s
user	0m0,212s
sys	0m0,139s
pierr0t@00095:~$ curl -s -w "%{http_code}" -X GET "$MENDER_URL"     -H "Authorization: Bearer $ACCESS_TOKEN"     -H "Accept: application/json"
[{"id":"631fce4d-b36b-4838-995c-4edc5fb8999b","description":"Sample artifact for testing direct upload","name":"rtc01","device_types_compatible":["mb-box"],"info":{"format":"mender","version":3},"signed":true,"updates":[{"type_info":{"type":"rootfs-image"},"files":[{"name":"mb-box-202410251351-mmcblk0.direct.p3","checksum":"034289f2f7ca37bda7f16fc3eb21ea8ee677a632dd68f99625f2caec09a320f6","size":1610612736,"date":"2024-10-25T13:52:21Z"}]}],"artifact_provides":{"artifact_name":"rtc01","rootfs-image.checksum":"034289f2f7ca37bda7f16fc3eb21ea8ee677a632dd68f99625f2caec09a320f6","rootfs-image.version":"rtc01"},"artifact_depends":{"device_type":["mb-box"]},"clears_artifact_provides":["artifact_group","rootfs_image_checksum","rootfs-image.*"],"size":242371584,"modified":"2024-10-29T10:19:40.323Z"}]200pierr0t@00095:~$ 
pierr0t@00095:~$ $ARTIFACT_ID="631fce4d-b36b-4838-995c-4edc5fb8999b"
=631fce4d-b36b-4838-995c-4edc5fb8999b : commande introuvable
pierr0t@00095:~$ ARTIFACT_ID="631fce4d-b36b-4838-995c-4edc5fb8999b"
pierr0t@00095:~$ curl -s -o /dev/null -w "%{http_code}" -X DELETE "$MENDER_URL/$ARTIFACT_ID" \
    -H "Authorization: Bearer $ACCESS_TOKEN" \
    -H "Accept: application/json"
204pierr0t@00095:curl -s -w "%{http_code}" -X GET "$MENDER_URL"     -H "Authorization: Bearer $ACCESS_TOKEN"     -H "Accept: application/json"
[]200pierr0t@0009date;time curl -s -w "%{http_code}" -X POST "$MENDER_URL"     -H "Authorization: Bearer $ACCESS_TOKEN"     -H "Accept: application/json"     -F "description=$DESCRIPTION"     -F "artifact=@$FILE_PATH;type=application/octet-stream"
mar. 29 oct. 2024 11:22:50 CET
201
real	5m33,785s
user	0m0,210s
sys	0m0,139s
pierr0t@00095:~$ curl -s -w "%{http_code}" -X GET "$MENDER_URL"     -H "Authorization: Bearer $ACCESS_TOKEN"     -H "Accept: application/json"
[{"id":"f8a1bf4d-2d2c-4293-b451-75bea93d0c5c","description":"Sample artifact for testing direct upload","name":"rtc01","device_types_compatible":["mb-box"],"info":{"format":"mender","version":3},"signed":true,"updates":[{"type_info":{"type":"rootfs-image"},"files":[{"name":"mb-box-202410251351-mmcblk0.direct.p3","checksum":"034289f2f7ca37bda7f16fc3eb21ea8ee677a632dd68f99625f2caec09a320f6","size":1610612736,"date":"2024-10-25T13:52:21Z"}]}],"artifact_provides":{"artifact_name":"rtc01","rootfs-image.checksum":"034289f2f7ca37bda7f16fc3eb21ea8ee677a632dd68f99625f2caec09a320f6","rootfs-image.version":"rtc01"},"artifact_depends":{"device_type":["mb-box"]},"clears_artifact_provides":["artifact_group","rootfs_image_checksum","rootfs-image.*"],"size":242371584,"modified":"2024-10-29T10:28:24.682Z"}]200pierr0t@00095:~$ curl -s -w "%{http_code}" -X GET "$MENDE^C
pierr0t@00095:~$ ARTIFACT_ID="f8a1bf4d-2d2c-4293-b451-75bea93d0c5c"
pierr0t@00095:~$ curl -s -o /dev/null -w "%{http_code}" -X DELETE "$MENDER_URL/$ARTIFACT_ID"     -H "Authorization: Bearer $ACCESS_TOKEN"     -H "Accept: application/json"
204pierr0t@00095:~$ 
pierr0t@00095:~$ date;time curl -s -w "%{http_code}" -X POST "$MENDER_URL"     -H "Authorization: Bearer $ACCESS_TOKEN"     -H "Accept: application/json"     -F "description=$DESCRIPTION"     -F "artifact=@$FILE_PATH;type=application/octet-stream"
mar. 29 oct. 2024 11:30:04 CET
201
real	5m29,406s
user	0m0,205s
sys	0m0,141s
pierr0t@00095:~$ curl -s -w "%{http_code}" -X GET "$MENDER_URL"     -H "Authorization: Bearer $ACCESS_TOKEN"     -H "Accept: application/json"
[{"id":"72eccf60-0a2b-48bb-a46a-eda7e3a88b3d","description":"Sample artifact for testing direct upload","name":"rtc01","device_types_compatible":["mb-box"],"info":{"format":"mender","version":3},"signed":true,"updates":[{"type_info":{"type":"rootfs-image"},"files":[{"name":"mb-box-202410251351-mmcblk0.direct.p3","checksum":"034289f2f7ca37bda7f16fc3eb21ea8ee677a632dd68f99625f2caec09a320f6","size":1610612736,"date":"2024-10-25T13:52:21Z"}]}],"artifact_provides":{"artifact_name":"rtc01","rootfs-image.checksum":"034289f2f7ca37bda7f16fc3eb21ea8ee677a632dd68f99625f2caec09a320f6","rootfs-image.version":"rtc01"},"artifact_depends":{"device_type":["mb-box"]},"clears_artifact_provides":["artifact_group","rootfs_image_checksum","rootfs-image.*"],"size":242371584,"modified":"2024-10-29T10:35:33.994Z"}]200pierr0t@00095:~$ 
pierr0t@00095:~$

Regards

2

Hello @LalunePierr0t :wave:

The slow upload is primarily caused by the server validating the artifact as it is uploaded. I believe most of the throttling is caused by the decompression and calucalting the SHA256 sum of the decompressed payload.

3

Hi

thanks for your reply, I confirm by sending the same artifact but not compressed.
And this time, the duration is about 1 minute 30s.

pierr0t@00095:~$ FILE_PATH="/home/pierr0t/repo/mbbox-bsp-mbbox_cooker_menu/builds/build-mb-box-debug/tmp/work/raspberrypi4_64-poky-linux/mb-box/1.0-r0/build-wic/partTest-mbb-signed-nocomp.mender"
pierr0t@00095:~$ date;time curl -s -w "%{http_code}" -X POST "$MENDER_URL"     -H "Authorization: Bearer $ACCESS_TOKEN"     -H "Accept: application/json"     -F "description=$DESCRIPTION"     -F "artifact=@$FILE_PATH;type=application/octet-stream"
mar. 29 oct. 2024 13:46:51 CET
201
real	1m30,695s
user	0m0,328s
sys	0m0,318s
pierr0t@00095:~$ du -h $FILE_PATH
421M	/home/pierr0t/repo/mbbox-bsp-mbbox_cooker_menu/builds/build-mb-box-debug/tmp/work/raspberrypi4_64-poky-linux/mb-box/1.0-r0/build-wic/partTest-mbb-signed-nocomp.mender

My questions:

  • If I send a LZMA compressed artifact does the device download it as a compressed artifact to save its bandwidth ?
  • Or Mender Server will extract it anyway and the device download it as uncompressed ?
  • Or Mender Server will extract it just to check the SHASUM and leave it compressed to preserve the device bandwidth ?
4

The device will receive the artifact exactly as it’s uploaded. The server verifies the artifact as it is streamed to object storage. The reason it needs to decompress the payload to do the verification is because the checksums in the header manifests in the artifact format are computed from the decompressed payload.

5

Thank you
It’s crystal clear

Last question:
Will this time of server verification be reduced with the Enterprise version?

1 Like
6

No, there’s no difference with the Enterprise plan. Unfortunately, there’s no quicker way of performing artifact validation.

7

Thank you for all your answers.

1 Like