Can't upload signed artifacts compressed with LZMA

I can’t upload signed artifacts via GUI.
Integration is on V2.0.0 and mender-artifact is on v.3.0.1. Artifacts are created using mender-artifact from .ext4. I don’t use --version with mender-artifact, so it should use artifact V3 I suppose.

mender-deplyments container throws these errors:

logs from mender-deplyments
time="2019-07-12T14:51:22Z" level=error msg="reading artifact error: readHeaderV3: handleHeaderReads: readHeader: readNext: Failed to copy from tarReader to the writer: unexpected EOF: Cannot parse artifact file" file=images.go func="controller.(*SoftwareImagesController).NewImage" line=239 request_id=bd965f2e-eb8a-41cb-b0b9-564699d67c8f user_id=5ec43064-1fe1-49bd-92b2-c3a7ee220460
time="2019-07-12T14:51:22Z" level=error msg="reading artifact error: readHeaderV3: handleHeaderReads: readHeader: readNext: Failed to copy from tarReader to the writer: unexpected EOF" file=view.go func="view.(*RESTView).RenderError" line=51 request_id=bd965f2e-eb8a-41cb-b0b9-564699d67c8f user_id=5ec43064-1fe1-49bd-92b2-c3a7ee220460

The problem only exists with signed artifact. This problematic artifact passes mender-artifact validate successfully. If the same artifact is generated without --key argument, it can be uploaded without any issue.

I checked docs about keys and it doesn’t seem that I should somehow provide Server with any artifact keys. What have I missed?

Also, little UX flaw: when this error happens, GUI shows nothing, the upload progress bar just getting stuck at random between 4% and 10%. It should probably notify the user that something went wrong.

Can you share the commands you use to create the Mender Artifact.

I tried to reproduce but was not successful, it uploaded just fine. Though I was using Hosted Mender but should be equivalent to v2.0.0.

Sure:

mender-artifact --compression "lzma" write rootfs-image
      --file "$releaseName-mender.ext4"
      --device-type "raspberrypi3" 
      --artifact-name "$releaseName-mender"
      --output-path "$releaseName-mender.mender"
      --script "$scriptsDir/ArtifactInstall_Enter_01_retain_wpa_supplicant"
      --key "$pathToSignKey"

The command to generate unsigned artifact is the same but without --key. I regenerated signed artifact once again and still have the issue.

Thanks, now I can see it as well. It seems to work if you remove --compression "lzma". Can you confirm this as well?

I created an bug report,

https://tracker.mender.io/browse/MEN-2645

Thanks, now I can see it as well. It seems to work if you remove --compression "lzma" . Can you confirm this as well?

I confirm, everything works fine with gzip

@ster, could you give the version of mender-artifact that you are using?

As mentioned in the first message, I used mender-artifact V3.0.1, precompiled Linux binary downloaded from here

Apologies! That was weak.

However, I am still not able to reproduce this. Downloaded the binary, and ran:

mender-artifact --compression "lzma" write rootfs-image
      --file test.img
      --device-type "raspberrypi3" 
      --artifact-name release-1
      --output-path test.mender
      --key ~/mendersoftware/integration/keys/deviceauth/private.key

That is with hosted mender, and the demo environment locally.
I am not doubting that you are having issues, just to be clear, so bear over with me.
I will have a look at the backend

Maybe the keys are the key? :slight_smile:
I use RSA with length of 3072 bits

also I (as well as @mirzak) am generating artifact from .ext4 and not .img, just a note.

So is the key in the integration repo :confused:
I also, just to be sure, did convert the raw image to an ext4 image, to no avail.
@kacf, perhaps you have some ideas here?

You are also missing a state script in your command, which the original command had to reproduce

I’ve tried both with and without scripts, still works.
Tomorrow I will make a bash script to run the whole thing, so that we can standardize this :slight_smile:

For additional reference, when I performed my test I downloaded this image,

https://d1b0l86ne08fsf.cloudfront.net/2.0.1/raspberrypi3/raspberrypi3_release_1_2.0.1.mender

Unpacked it, and then created a Mender Artifact with the mentioned command using the files from the extracted Artifact

Alright, have a look at this, and if this is the proper workflow, see if it works at yours:
https://pastebin.com/rnmsYheZ

@oleorhagen, I used your script with the only difference that I used GUI instead of mender-cli to upload the artifact and the issue is present, with the same logs from mender-deployments.

One additional thing is that I have relatively low upload speed with my ISP (1-2 Mbit/s), maybe this could play the role?

Here I uploaded test.mender (result of your script) I have issues with, can you try this file?
https://we.tl/t-78hBQlwY3b

And the key pair:

testkey.pub

ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEQyJuskzajlaW3Bd8JSYFF6AjH1CgJvSsOugywQOz21nofi2jEm5qpa/rL9+0OUX+7Vw4rIw+3tGIltL43yNylm49TJCn32oWoMjUGXFMzReifD9/ixBb+9QKDC2s8e8MMqYUSM4R0VQbQZedfj+q3pBAxcEUJEdIL6r+nynxy90GCJm+LAQU+F/UQYtKMP8dKTl+0BcSVifKTy1zFVSs16PEjj/ahQTOSZ4tJAuUZ1k2Tq+gfCifAQ+Ke6xxHKK30NGB+G3VkmjvshiI7pQyipBw+9MisWOE9kjpJYLsAYj5mAXy8BuqfiTOzrOtxN1pAvzUTc0MLxxDOYOH3yhzKpQY4wUkTyBxDjpBqTEBDraXIzHUD1+8T4DTNyH8vkctqzHXuFrm8aDjoTNfWFYoRrZY+MuV2IFn1b92Hboz8vqtKgc9S0znnRboVykiX0LfGHe0jCbeZ0J1GN4EqhiFgzZcst52rflKRJqJhfwX1ijhFqj7689m7qcN5XM5EXU= ster@sterPC

testkey

-----BEGIN RSA PRIVATE KEY-----
MIIG5AIBAAKCAYEAxEMibrJM2o5WltwXfCUmBRegIx9QoCb0rDroMsEDs9tZ6H4t
oxJuaqWv6y/ftDlF/u1cOKyMPt7RiJbS+N8jcpZuPUyQp99qFqDI1BlxTM0Xonw/
f4sQW/vUCgwtrPHvDDKmFEjOEdFUG0GXnX4/qt6QQMXBFCRHSC+q/p8p8cvdBgiZ
viwEFPhf1EGLSjD/HSk5ftAXElYnyk8tcxVUrNejxI4/2oUEzkmeLSQLlGdZNk6v
oHwonwEPinuscRyit9DRgfht1ZJo77IYiO6UMoqQcPvTIrFjhPZI6SWC7AGI+ZgF
8vAbqn4kzs6zrcTdaQL81E3NDC8cQzmDh98ocyqUGOMFJE8gcQ46QakxAQ62lyMx
1A9fvE+A0zch/L5HLasx17ha5vGg46EzX1hWKEa2WPjLldiBZ9W/dh26M/L6rSoH
PUtM550W6FcpIl9C3xh3tIwm3mdCdRjeBKoYhYM2XLLedq35SkSaiYX8F9Yo4Rao
++vPZu6nDeVzORF1AgMBAAECggGBAIJLo69j9fvfJYNQKNNFgmE9FxZaMtewJR22
mfMgCwo75QuhwUCS+vUYfJsiFFz87QhaorAJda5DlU4d+4A3+7uWPSe7Dgo1G6Rc
KBgPsBVxz3ATFsDhHQJERLhK74PytmYdtf6fexd/JlxWBocLis8wpQorf4yUrTIy
W05hcqJRWrInwZHsfOVDI7Oxs/yUXLoKZV5YXgjRY+c++4h8r9Nz8giK+79iMheV
7ko/tORYXHTedmCQLROqy5t/kbMic4a5oHE/p6cB41M3KKM2OLYVx5xypc8uMpyG
Y6CP/1muaK2EyMiW2lOvG2Gqq5lHmHBiXtfHOHKi2XZYZSlJTH1FHlVLzW3ieu3+
YM6EkY83QIMbeOZlBrWpgh9/pdMzRmcu/2jzaUeAvT5sXWF+Gn5wIxmfJsKkL9yb
09m9InoxRkJAvQfULbcPtn3Dla/z5zz2QoFD4pZGD+Ga1lW8HqG0LAIe2i23GDud
gZW2AI6DbyhoDai9qghacF+Z2X8koQKBwQDp9mTT/RADOcGjckFYHacjsRslgP/v
qsT0/79otapgekm0O8T7JD2fxMD7TDwtdK8wG7S0BjzZqHyOvMW6X52NDKo1UtXh
xqZaEbpF/Kd4iDvCrsgLouB3LFnKHnnJYSMuvtZl2r0Hk/BD1JFTzM75xgPG1oLX
1/vsPIQq8jXrY+XDrv+njX8H05hsnjOdpugoEaygC9zp+GoBuUtFD6YXZgenXCm+
t+r2BrZ6Vxmx+6lF9ehUvz1lMuzogZeOix0CgcEA1r+p8WF2pcdnL9ni4rV++bY4
vi8Ioyp6CwGCdAGtaBdjKayXB1f2SrtfVpJmD9MmjkXwi1RrCrDGbOlED+OFqK+c
kqRYIYH1bhC60PSAhUFIFGT/XNgYJVV467/DFzcHBWSkl85FMRkpj0kkX3ynXnPK
hn6EWMJVB9R3qJN3G4sJbfePN0NsSg13ghUdb+fP8gPWO5XOlN6mOmndSxg/SfD5
FNEL/wNG+vrETw9/lmawUznmUpSKBDOltJJvGPg5AoHADyTFrfsJWEwavoucsylo
MiU6jpjk+Axjp0AOkaaAmrIPpzzfKSpVWswPebSbiI2u8Tw53BUzRxtixJJvcfrS
ZdjWA5GTCoQjLF1NY05RQLoAtD8/75DhWpViUnTaiTBiaJotonYgGid3O8y8g2IU
W5J3mB02ZuYyBj58h5MZ7lyO+LWRErVT7ZvCl/U9fgzZpg5O0CU0bWiI5AesWZID
vCv8fta2VQPuj3IryYttvFl/wWmDWkjunmhHcV1CBBX5AoHBANX0qY27rClpLQ6I
yDtV1sdWXNxnIfTm6icy/tUzoqA1xO+Na/++DL3SRx+KxrL3jc9sMUev+OHy1A9I
rS4jYlR5esbDhKcFwxBBDNAI+k9JhlYDdkT7a7sumEZRjT8A7TbbWf2BXoCXcEe4
+MtF977sIJi1TojcPTmK3xkOqzm+4cs1PfQycZ6qIipxfR20WQLn/bbp/6Nk3TsP
pJIf+1ZhBtlNmeUeT8bYt2OJrnWbpAuXgTk/ku6ywmLRDm1TSQKBwDfFl6DCNymt
8oXQKoEvlKcTw74KDOkgwdHa/bmO7Pb7jQmEPvbpOFBF67CMI6IdYXxi6uD8Ash0
7A5tloERvfsb9ik69neqVrydFYosppdfQqNSEiYQJC66fKRjBpuckrd1mjeSlfo7
bXQt35vtSqVQKc9YTKVWFObuqDhruImfQCfHAGANgHMzfx7PIguUR8co4OwOjgbM
isoBpdGFNhwbeqFhSdwSUGAzgaC5kqzvR2ooyiT8xxTjCd0GbN/PAA==
-----END RSA PRIVATE KEY-----

Hmm, that is interesting indeed, cause the artifact you linked uploaded just fine for me.
I guess this calls for some knowledge from @0lmi

I used your script with the only difference that I used GUI instead of mender-cli to upload the artifact

Did you try to use mender-cli? Does it work for you?

And how does you infrastructure look like?