No space left on device

General Discussions
1

Hi,

I’ve just tried updating from Mender 1.6 with Sumo to Mender 1.7 with Thud. I’ve successfully built my image with my own layers (exactly the same layer as before), and I’ve successfully flashed it to a Raspberry Pi. However when I try to push the update out to existing devices (that are running 1.6 with Sumo) they all fail with the following error:

2019-03-06 22:57:38 +0000 UTC error: update (952107008 bytes) is larger than the size of device /dev/mmcblk0p3 (947912704 bytes)
2019-03-06 22:57:38 +0000 UTC error: update image installation failed: no space left on device
2019-03-06 22:57:38 +0000 UTC error: update install failed: installer: failed to read and install update: update: can not install update: &{core-image-base-raspberrypi3.ext4 420 1000 1000 952107008 2019-03-06 11:48:36 +0000 UTC 48    0 0 0001-01-01 00:00:00 +0000 UTC 0001-01-01 00:00:00 +0000 UTC map[]}: update: can not install: no space left on device

I’ve tried removing a bunch of unnecessary packages, but that didn’t seem to reduce the size significantly.

In the Mender GUI it says the old image was 947.9 MB uncompressed, and it says the new one is 952.1 MB uncompressed.

I ran fdisk -l on a Pi running the old version and it says the partitions are:

  • /dev/mmcblk0p1 - 40M
  • /dev/mmcblk0p2 - 904M
  • /dev/mmcblk0p3 - 904M
  • /dev/mmcblk0p4 - 128M

In my layer.conf file I have the following lines:

  • MENDER_BOOT_PART_SIZE_MB = “100”
  • MENDER_STORAGE_TOTAL_SIZE_MB = “2000”

Any ideas on how I can fix this?

I already have about 40 Raspberry Pi’s deployed to remote locations which I can’t get back, so do I need to push an update that has some kind of mechanism to increase the partition sizes? And then later push an update which updates it to Mender 1.7?

We’re using either 8GB or 16GB SD cards, so I have no issue with increasing the partition sizes significantly.

Any suggestions would be greatly appreciated.

Kind Regards,
James

2

It’s probably due to this change. You can try to set it back with:

MENDER_PARTITIONING_OVERHEAD_KB = "${@eval('${MENDER_PARTITION_ALIGNMENT} / 1024 * 4')}"
3

Out of curiousity: has this change been documented somewhere as a (potentially?) breaking change?

4

Sort of, though admittedly not very clearly. It’s in the changelog for thud-v2018.12. In general different meta-mender branches are not guaranteed to be perfectly compatible (Yocto doesn’t guarantee this either). In most cases it comes down to small adjustments though, like this one.

5

@kacf Thanks for this pointer, I’ll try this tomorrow when I’m in the office. For the moment I’ve just changed MENDER_STORAGE_TOTAL_SIZE_MB to be a few MB less and this has worked so far.

6

Kristian, don’t get me wrong, I do understand these kind of changes could come from yocto.

As a product leaning on yocto so heavily as mender does, I’d really value a list of “known issues” (wrong term, but you’ll get my point)

7

@kacf This fix worked, thank you!

8

I’d really value a list of “known issues”

You’re right, and it’s not the first time this has appeared on the forum. I added this troubleshooting section.