Best practices for partition layout on 8GB eMMC: 2.6GB Rootfs A/B + 2GB Data?

General Discussions
1

Hi everyone,

I am working on a project using SolidRun RZ/G2LC boards with an 8GB (7.28 GiB usable) eMMC, and I am trying to lock in the optimal A/B partition layout using mender-convert before moving to production. I want to keep the Rootfs tight while leaving a healthy amount of space for persistent application data.

Here is the exact fdisk layout we are currently generating:

Plaintext

Disk /dev/mmcblk0: 7.28 GiB, 7818182656 bytes, 15269888 sectors
Units: sectors of 1 * 512 = 512 bytes

Device         Boot    Start      End  Sectors  Size Id Type
/dev/mmcblk0p1 * 49152    81919    32768   16M  c W95 FAT32 (LBA)
/dev/mmcblk0p2         81920  5537791  5455872  2.6G 83 Linux
/dev/mmcblk0p3       5537792 10993663  5455872  2.6G 83 Linux
/dev/mmcblk0p4      10993664 15187967  4194304    2G 83 Linux

Based on the sector count, this leaves exactly 40 MiB of unallocated space at the end of the disk, plus 24 MiB at the start for U-boot/overhead.

Before I finalize this, I would love to hear from anyone who has deployed a similar layout. Specifically, I am curious about:

  1. Rootfs Size vs. Artifact Size: Is a 2.6 GiB Rootfs too large for efficient OTA updates in your experience? What is your average .mender artifact size after compression with a similarly sized root filesystem. Most of our devices are connected through 4G.

  2. mender-convert Quirks: Are there any specific things to watch out for when defining MENDER_STORAGE_TOTAL_SIZE_MB to ensure it perfectly fits the physical boundary without truncating the end of the /data partition?

Any insights, gotchas, or alternative layout suggestions would be highly appreciated!

Thanks in advance!

2

Hi @marifante,

Thanks for reaching out! It really depends on your use case and software structure, respectively planning what data you want to put where. So a few thoughts:

  • I would usually suggest to have 1.5 to 2.0 times the size of your initial root filesystem as the root partitions sizes, so you have enough leeway for future expansion and maintenance
  • a 2.6GiB root filesystem is quite massive, so I would ask if all that is really needed. For reference, a Yocto-based basic root filesystem usually comes out somewhere like 100-200MiB, so unless you have a super specific use case and requirements the chances are that you can cut it down substantially.
  • do you have data you need to persist on the /data partition? Then it obviously needs to be large enough (customer data, process data, logs, and so on)
  • you might also look into additional partitions to keep /data from overflowing if something goes rogue
  • 2.6GiB root filesystem size will of course also make your updates slow, unless you can use deltas as provided by the Professional and Enterprise tiers.

Again, without knowing your software structure, for a real, long-lived product to maintain, I would shoot for something like this

  1. ~128MiByte boot
  2. ~2GiByte root A, READONLY
  3. ~2GiByte root B, READONLY
  4. ~256MiByte /data and persistent configuration
  5. ~512MiByte for logs and debug/diagnosis data
  6. rest for your application persistent data.

but again, this really needs tuning against your requirements.

Greetz,
Josef

3

While you’re on this, may I introduce the topic of NAND memory wear and bad cells.

We had this remote Yocto device flashed with a layout similar to yours, occupying all the available disk.
One day we went to install a Mender update, and there was not enough space on the B partition!
Turns out the disk size did shrink, in respect to an identical device that we had on our desk, where the install would progress no problem.

I don’t know if this is the correct explanation, but we though that the eMMC in that device (which was a particularly cheap one, so questionable NAND chip quality…) did find bad blocks with aging, and marked them in the internal bad block table, shrinking the total available space for the partitions.

Since that day, especially on long-life devices, we leave some 10% unallocated disk space, not in any partition, to comply for this.

I understand we may be very wrong on this analysis, but to this day we have two identical devices, both born with the same 8Gb eMMC, one still reporting 8G, one reporting 7.98GB…

4

Interesting point indeed @matteog. So I have never heard of devices where the reported storage size changed, they would “just” throw errors on writing once the available bad block spares are used up. I would strongly argue though that this is an indication of too many writes in operation, possibly even on the root filesystem.

What this showcases though is variance in storage sizes. Especially with consumer grade devices, the usable storage often exhibits variation. A brand and type of SD card which you bought today might yield a number of MiBytes less or more than the exactly same brand and type you bought two weeks ago. In order to be prepared for such situations, you should always construct your images with a couple of 10s of MiBytes reserve at the end, maybe even 100s and dynamically growing on provisioning. But again, it depends.

My hunch would be that this is not so much to be expected on integrated eMMC storage, but I would not place a bet on it to be safe, right.

Greetz,
Josef