Large persistent data partition without large image file

Hi.

I’m just getting started with mender and try to convert the installation of a x64 host.
For that I followed INTEL NUC x86-64 Ubuntu 18.04 with a few small changes.

My main difference is that I use a 256GB SSD onto which I would like to have the following partition table.
512MB boot/EFI partition
20GB rootfs A partition
20GB rootfs B partition
~200GB data partition

I used the docker-mender-convert script to generate an image with this setup, but was surprised that I actually had to create an image with almost 250GB of data, as the data partition is basically empty at that time.

My next approach was to set MENDER_STORAGE_DEVICE to something like 50GB to get a smaller image file. I was hoping that during the first boot the growing of the partition would allow it to fill the rest of the SSD on its own. This didn’t work however.

So, my question is, can I somehow achieve a small disk image file and automatically resize the /data partition and regrow the filesystem on it afterwards?

I’m especially wondering how I can do this with as low of a risk as possible that a future update would try to rewrite the partition table.

Have a look at mender-growfs-data

1 Like

Yes, like @elipsion says, the data partition is grown on boot, to fill the available space, and this is enabled by default, here
and here
:grinning_face_with_smiling_eyes:

Thank you @elipsion and @oleorhagen .
Yes, I noticed this as well. In the log files I also noticed that the filesystem was grown to the full size of the partition. That was not really the problem.

My concern is that I want to have a data partition that will be resized (partition size and filesystem) to the full size of the SSD I have in the machine.
This is for now 256GB. So I want boot to be 512MB, root A 20GB, root B 20GB and data whatever the rest.

If I set this up like this I get an image of ~256GB of size (for some reason it was not compressed even though it is set to gzip in the convert config file). I feel that having an image with 200+GB of empty space is a waste of time and resources.
So I changed my configuration to have boot to be 512MB, root A 20GB, root B 20GB and data at 10GB. I was hoping that when I flash the resulting image that the date partition would grow beyond the 10GB I set up and would fill the whole available space. And after that update the filesystem with growfs to match the new partition size. This however, is not what I observed.

Should it actually be the case or not?

Check the boot log for errors, as it should expand the data partition if its the last partition and resize the file system to fill it on boot.

However saying that, if you do see errors then i have observed similar issues on my x86 projects and have needed to supplement the systemd growfs feature with our own systemd service and script to fix up any disk issues prior to the growfs feature running as on its own its not enough sometimes depending on the errors encountered.

1 Like

The gpt issue can be fixed up with sgdisk tool, I believe there are other discussions on the topic in the forum,

That’s basically, the reason we have our own systemd service that checks and fixes disk issues, prior to the mender/systemd growfs service

Hi @dellgreen .
sorry it took me so long to get back to you. I found a few more problems blocking me. :sweat:
Mainly EFI not booting into mender’s grub.cfg but another one from the original OS installation. Removing that entry with efibootmgr gave me a booting system.

So, I tried again and made sure to check the logs for growfs being called. What I do find is this line

Sep  6 16:25:47 r25 systemd-growfs[516]: Successfully resized "/data" to 1.0G bytes.

showing that it resized /data to only the size I gave it with
MENDER_DATA_PART_SIZE_MB="1024"
I do not see any other output that looks like it even attempted to resize the actual partition. I mainly checked /var/log/syslog, is there any other log I should be checking?

I have just taken a look back at my projects, and i can see that it clearly wasnt working for me either, so the custom systemd service that I added uses sgdisk to fix GPT issues, grow the last partition, and notify the kernel of the parition table change if it was changed, prior to systemd mounting and growfs running. I cant paste the full service as its highly custom, however the premise is:

to fix the GPT issues is was using the following in a script and customize

/sbin/sgdisk -e "$basedev"

if growpart "$basedev" "$lastpartn";then 
  partprobe
fi

then i created a custrom systemd service file


[Unit]
Description=Grow last parition on rootfs disk
DefaultDependencies=no
Before=data.mount
Before=systemd-growfs@data.service
Before=local-fs-pre.target
ConditionVirtualization=!container
After=systemd-remount-fs.service

[Service]
Type=oneshot
User=root
Group=root
ExecStart=/usr/local/bin/your-growfs-script

[Install]
WantedBy=local-fs-pre.target

then in my mender config i add extra fstabs options

MENDER_DATA_PART_FSTAB_OPTS="${MENDER_DATA_PART_FSTAB_OPTS},x-systemd.after=your-growfs.service"

1 Like