Upload Artifact fails because of size

General Discussions
1

Hello, I am currently testing mender with a self hosted mender Server in demo mode, Version: 2.5.0
I was using raspberry Pi as client, and everything worked very well. Then I tried with x86 debian machine as client. I did a system snapshot and created a artifact from the snapshot.


Then I tried to upload the Artifact, (14.32 GB) what wasn’t possible. The upload started, the upload in progress increases up to 20% (Upload in progress (20%)) and then jumping back to 0%, also the graphical green process bar is jumping back, all this repeating several times. I guess its a space problem on my server, but I can’t figure out how to give the upload the additional space it needs for buffering the bigger Artifact. Upload of “small” artifacts works fine that have 3 GB.
Docker logs say: 2020/11/24 09:45:52 [error] 30#30: *246 client intended to send too large body: 15370870261 bytes, client: xx.xx.xx.xx, server: mender.intern.xxxx.net, request: “POST /api/management/v1/deployments/artifacts HTTP/1.1”, host: “x.x.x.x”, referrer: “https://x.x.x.x/ui/”

Does anybody has a Idea how to enable larger artifact sizes ?

Thanks for your effort to read and your help ! Best regards, Alexander

2

Hello @trailsurfer, the maximum payload size when uploading artifacts is 10GB:

If yo have a 14GB artifact, I believe the only option you have is rebuilding the Docker image for the api gateway increasing the client_max_body_size.

1 Like
3

I’m afraid there is a hard limit on the upload size set to 10GiB in the backend service managing deployments and artifact storage.
One option you can do is to try a better compression algorithm when creating the artifact:

mender-artifact write rootfs-image -f ssh://${USER}@${ADDR} \
                                   -n artifact-name \
                                   -o snapshot-release.1.0.mender \
                                   -t device-type \
                                   --compression lzma

NOTE: lzma compression requires liblzma to be installed on the devices receiving this update.

I have created a ticket to track this issue and make the artifact limit configurable.

1 Like
4

Is there any update on this? We have much larger artifacts, and it’s making it difficult for us to use Mender - we need to recompile it from source.

5

Have you gone through your Golden Image to remove anything you don’t need, as there is normally an awful lot that can be removed to reduce the size of it without affecting functionality. Often there is a lot of caching of packages by the package manger as well as multiple kernels/modules, man page removal, swap file etc that take up a lot of unnecessary space. A 14GB compressed mender artifact does seems a little on the large size. Once you start auditing your root file system, you will often be surprised as to what’s consuming space that you were unaware of.

There’s a more in-depth discussion below on large image sizes.

6

Our fully-stripped binaries alone are ~20GB, without the OS image.
Oh, and to be clear, I’m not the OP, I think my artifacts are going to be much much larger than 14GB - I was just wondering if any progress was made on this, because I’m in the same boat.

7

Unfortunately this ticket has not been prioritized.

Just out of curiosity, if I may ask, what kind of application are you running on your images?