Raspberry Pi4 + SD card permanently failing after few days working

General Discussions
1

Hallo,

We’re using Hosted Mender (EU) with mender-convert 5.1.0 on Raspberry Pi 4, running Raspberry Pi OS Lite 64-bit (Debian 13 / Trixie).

We’re hitting a recurring issue where devices run fine for about 2–3 days and then completely fail. Both the SD card and the Pi itself end up unusable. This has already happened on three different Pis, each with a brand-new SD card.

What we do

  • Flash Raspberry Pi OS Lite (64-bit)
  • First boot:
  • do sudo apt update && sudo apt upgrade -y
  • Enable persistent journald logging
  • Power off and treat this SD card as the golden image
  • Convert it using mender-convert (A/B rootfs + /data partition)
  • Deploy the artifact via Hosted Mender

mender setup

  • Mender client installed via mender-convert
  • A/B root filesystem
  • /data partition size: 1 GB
  • Persistent journald logging enabled
  • Custom application running as a systemd service and logging via journald (install via Mender artifact release and deployment afterwards)

what happen

  • Devices boot and operate normally even able to do OTA update of OS as well Application
  • But after 3-4 days, the device fails to boot or the SD card becomes unreadable.
  • led to sd card as well as Pi to be broken and not usable, even Pi is not accessable via new sdcard or with offical pi image anymore.

Questions

  1. Are there any known issues or best practices when using persistent journald logging together with Mender’s A/B rootfs on Raspberry Pi? Does Mender recommend specific filesystem mount options, log handling, or write-reduction measures for SD-card–based devices?
  2. Are there any additional hardening steps you recommend to avoid long-running storage corruption on Raspberry Pi devices managed by Mender?
This is what I mention in my overlays file if is helpful-

# Target 32GB SD cards
MENDER_STORAGE_TOTAL_SIZE_MB=29000

FYI, Currently we are using Sandisk Extreme Pro 32 GB SDCard we also try SanDisk High Endurance 32 GB as well.
We dont have data on write activity, also on one of the running Pi currently the /data partition shows almost no usage (only 144K used out of 2.4G) and very few write operations that we used for some persistent config via simlink.So far, there is bare minimal or no write activity to the rw data partition.

Any idea why this is Happening?

Regards,
Sushil