Mender-cli artifacts upload `FAILURE: POST EOF` when running in GitHub Actions

General Discussions
,
1

Hi Folks,

I’m trying to setup a CI pipeline for my device OTA updates using hosted Mender and I’m having some difficulty getting artifact uploads working from a GitHub Actions workflow.

The TLDR is that running mender-cli artifacts upload from a GitHub Actions workflow always ends with the following error:

FAILURE: POST /artifacts request failed: Post "https://hosted.mender.io/api/management/v1/deployments/artifacts": EOF

The artifact is 2.1 GiB and uploading from my dev machine or through the browser UI works fine. The issue only occurs when uploading from a GitHub Actions workflow.

I have a script wrapping the invocation of mender-cli but the commands that get run are as follows:

mender-cli login --username '***' --password '***'
mender-cli artifacts upload --description "Company Device box-v0_8_057" "/home/runner/work/company/device/app/mender-convert/deploy/box-v0_8_057-x86_64-mender.mender"

The .mender file is generated using the dockerized version of mender-convert:

./docker-mender-convert --disk-image input/box-v0_8_057.img --config configs/box-config-for-mender-convert --overlay box_rootfs_overlay

And the base image is generated using mkosi.

Running the same set of commands on my dev machine results in the artifact getting uploaded to hosted Mender without any issues so the error is somehow related to the fact that the upload is happening from a GitHub Actions workflow.

The major differences between the GHA workflow and my dev machine as far as I can tell are:

  • My home upload speed maxes out at 4 MiB while the GHA workflow uploads the artifact at 26.68 MiB
  • The CI mender user has the Releases Manager, Deployments Manager roles while my mender account has the Admin role

The GHA workflow runs in a ubuntu-20.04 VM with mender-cli version 1.7.0 installed from https://downloads.mender.io/mender-cli/1.7.0/linux/mender-cli

I looked at some of the topics on mender hub mentioning similar issues but none of them seem to offer a solution to my problem.

I’ve attached the full output of the mender-cli artifacts upload invocation from the GHA workflow:
mender-artifact-upload-gha-error-log.yaml (666.0 KB)

Any input or suggestions on how to get this working are very welcome.

Thank you.

2

@tranchitella @peter any ideas?

3

hello @marekl

thank you for using Mender.
as an additional test, did you try to use curl to upload the artifact?
e.g.:

 jwt=`curl -vvv -k -X POST -H "Content-Type: application/json;" -u 'user:pass' https://hosted.mender.io/api/management/v1/useradm/auth/login`;
 curl -vvv https://hosted.mender.io/api/management/v1/deployments/artifacts -H "Authorization: Bearer ${jwt}" -H "Transfer-Encoding: chunked" -F artifact=@box-v0_8_057-x86_64-mender.mender;

best regards,
peter

4

Hi @peter,

Thanks for the suggestion!

Using curl to upload the artifact as you recommend works without issue. I’ve had multiple successful CI runs now.

I still have no idea why using mender-cli did not work but this solution works for me.

Regards,
Marek

5

I’m having the same overall problem (fail to upload with mender-cli, works with the browser), but this time with a different error. In my case, I’m getting a Entity Too Large message from openresty.

It works with the browser and the curl solution. Fails with mender-cli.

The mender server is self-hosted. I’m running mender-cli on the same host than the server. mender-cli is 1.7.0, mender server 1.6.0