Mender and large disk images

General Discussions
1

We are currently using Yocto to build a distro for a custom board based on the intel-corei7-64 machine arch.
I am looking at Mender to implement an A/B system update feature. I would be using Mender in standalone mode, so no need for the server side or the systemd agent.

Integration has been quite straightforward and I was able to build both the .mender artifact and the .uefiimg initial disk image. However, the SSD we are using is around 1TB, so I ended up with image files between ~500GB and 1000GB large. Most of this would be empty space, as our actual rootfs is only about 10GB.

So, I am currently facing two issues integrating Mender in the production system:

  1. Building the .mender artifact takes very long (>1h). This is regardless of the contents and size of the actual rootfs, Most of the time is spent by the mender-artifact write command, which has to parse a 500GB .ext4 image.
    I managed to somehow work around this by setting the IMAGE_ROOTFS_SIZE variable to something very small and allow Yocto to calculate it dynamically based on the actual rootfs size.
    Otherwise, meta-mender overwrites this with MENDER_IMAGE_ROOTFS_SIZE_DEFAULT, which is calculated based on MENDER_STORAGE_TOTAL_SIZE_MB and ends up being around 500GB in my case.

  2. The.uefiimg is around 1TB, which makes it impractical for an initial deployment using dd. It is true that the compressed version is a lot smaller and easier to transfer around, but actually writing it to disk takes a huge amount of time.

For this I haven’t been able to find a decent work around, other than implementing my own installer that would format the disk and create the corresponding partitions during initial deployment, but I would rather not go that route.

I should mention that I know of the growfs functionality for the data partition, but we keep our data on a separate disk, so ideally I wouldn’t even have a data partition (I set it to 1GB in the config as I didn’t know how to completely remove it).

So I am looking for ideas or insights on how to handle big image files and work around the issues stated above.

Below is an excerpt from my local.conf for reference:

######################################
########### MENDER stuff #############

MENDER_ARTIFACT_NAME = "nx4.0"

INHERIT += "mender-full"

ARTIFACTIMG_FSTYPE = "ext4"

MENDER_STORAGE_DEVICE = "/dev/sdb"
MENDER_STORAGE_TOTAL_SIZE_MB = "915715"
#MENDER_STORAGE_TOTAL_SIZE_MB = "30000"
MENDER_DATA_PART_SIZE_MB = "1000"

IMAGE_ROOTFS_SIZE = "8192"

MENDER_FEATURES_DISABLE:append = " mender-systemd"
IMAGE_ROOTFS_MAXSIZE:forcevariable = "1024000000"
2

Hi @addud,

The usual approach would be to have a relatively small base image and the growfs mechanism for the /data partition. As you say you don’t want to use the /data partition for your application specific persistent data, there are a couple of building blocks that I can suggest. You probably need to mix and match those according to your needs.

  1. use a small MENDER_STORAGE_TOTAL_SIZE. I find even 10GB quite hefty for a root filesystem, but not completely otherworldly.
  2. use the .uefiimg.bmap files. This will just write actual payload instead of useless zeroes.
  3. adjust/create other needed partitions with some first boot-style script.

Concerning the data partition, you can’t remove it completely, as the Mender Client itself needs it (unless you want to go really crazy), but something like 128MB should be fairly enough.

Greets,
Josef

3

Hi Josef,

Thank you for the pointers, especially the one regarding the .bmap file. I missed that initially, but I did try writing it to the disk and it reduced the time considerably, however, still not to acceptable terms (it still takes about 1h to write a 1TB SSD). I thought of reducing MENDER_STORAGE_TOTAL_SIZE
as well, but that reduces rootfs partition size, and I would like to reserve that for any future expansion.

At this point, lacking other ideas, it looks like our best way forward is to forego the initial .uefiimg disk image and do the partitioning and initial deployment in a dedicated initrd installer.

Best regards,
–Adrian