NXP i.MX8MPlus Yocto Mender integration fails to boot

Board Integrations Yocto Project
, ,
1

Hi everyone,

I’m building Yocto project with mender for i.MX8M Plus, using yocto scarthgap lf-6.6.36. I generated a core-image-minimal-imx8mp-lpddr4-evk.wic.zst, but the booting process failes to load with the following error:

[  OK  ] Finished Virtual Console Setup.
[  *** ] (2 of 4) A start job is running for…ave OS Random Seed (4s / 9min 59s)
[  OK  ] Finished Load/Save OS Random Seed.
[   13.791163] platform sound-bt-sco: deferred probe pending
[  *** ] (3 of 3) A start job is running for /dev/mmcblk0p4 (55s / 1min 30s)
[   60.410460] imx-sdma 30bd0000.dma-controller: Direct firmware load for imx/sdma/sdma-imx7d.bin failed with error -2
[   ***] (2 of 3) A start job is running for /dev/mmcblk0p1 (59s / 1min 30s)
[ TIME ] Timed out waiting for device /dev/mmcblk0p1.
[DEPEND] Dependency failed for File System Check on /dev/mmcblk0p1.
[DEPEND] Dependency failed for /uboot.
[DEPEND] Dependency failed for Local File Systems.
[ TIME ] Timed out waiting for device /dev/mmcblk0p4.
[DEPEND] Dependency failed for /data.
[DEPEND] Dependency failed for File System Check on /dev/mmcblk0p4.
[  OK  ] Stopped Dispatch Password Requests to Console Directory Watch.
[  OK  ] Stopped Forward Password Requests to Wall Directory Watch.
[  OK  ] Reached target Timer Units.
[  OK  ] Closed Syslog Socket.

And it enters emergency mode. These are the boot partition details:

sh-5.2# fdisk -l
Disk /dev/mmcblk2: 29 GB, 31356616704 bytes, 61243392 sectors
478464 cylinders, 4 heads, 32 sectors/track
Units: sectors of 1 * 512 = 512 bytes

Device       Boot StartCHS    EndCHS        StartLBA     EndLBA    Sectors  Size Id Type
/dev/mmcblk2p1 *  128,0,1     1023,3,32        16384     186775     170392 83.1M  c Win95 FAT32 (LBA)
/dev/mmcblk2p2    1023,3,32   1023,3,32       196608   19813169   19616562 9578M 83 Linux
Disk /dev/mmcblk2boot0: 4 MB, 4194304 bytes, 8192 sectors
128 cylinders, 4 heads, 16 sectors/track
Units: sectors of 1 * 512 = 512 bytes

Disk /dev/mmcblk2boot0 doesn't contain a valid partition table
Disk /dev/mmcblk2boot1: 4 MB, 4194304 bytes, 8192 sectors
128 cylinders, 4 heads, 16 sectors/track
Units: sectors of 1 * 512 = 512 bytes

Disk /dev/mmcblk2boot1 doesn't contain a valid partition table
Disk /dev/mmcblk1: 30 GB, 31719424000 bytes, 61952000 sectors
484000 cylinders, 4 heads, 32 sectors/track
Units: sectors of 1 * 512 = 512 bytes

Device       Boot StartCHS    EndCHS        StartLBA     EndLBA    Sectors  Size Id Type
/dev/mmcblk1p1 *  128,0,1     1023,3,32        16384     186775     170392 83.1M  c Win95 FAT32 (LBA)
/dev/mmcblk1p2    1023,3,32   1023,3,32       196608   19813169   19616562 9578M 83 Linux
sh-5.2#


sh-5.2# cat /etc/fstab
# stock fstab - you probably want to override this with a machine specific one

/dev/root            /                    auto       defaults              1  1
proc                 /proc                proc       defaults              0  0
devpts               /dev/pts             devpts     mode=0620,ptmxmode=0666,gid=5      0  0
tmpfs                /run                 tmpfs      mode=0755,nodev,nosuid,strictatime 0  0
tmpfs                /var/volatile        tmpfs      defaults              0  0

# uncomment this if your device has a SD/MMC/Transflash slot
#/dev/mmcblk0p1       /media/card          auto       defaults,sync,noauto  0  0

# Where the U-Boot environment resides; for devices with SD card support ONLY!
/dev/mmcblk0p1       /uboot               auto       defaults,sync         0  2
/dev/mmcblk0p4       /data                auto       defaults              0  2
sh-5.2#

sh-5.2# ls /uboot/ -l
total 0
sh-5.2# ls /data/ -l
total 8
drwxr-xr-x    2 root     root          4096 Jan  1 01:29 mender
drwxr-xr-x    2 root     root          4096 Mar  9  2018 u-boot
sh-5.2#

I checked the integration checklist:

sh-5.2# systemctl is-active mender-authd
inactive
sh-5.2# systemctl is-enabled mender-authd
enabled
sh-5.2# systemctl is-active mender-updated
inactive
sh-5.2# systemctl is-enabled mender-updated
enabled
sh-5.2#

I tried to generate a .sdimg image but it failes to boot, nothing happens while booting. At first it didn’t build because of the “couldn’t find uboot.env” error. I fixed it using u-boot-imx-initial-env-sd instead of uboot.env.

I cannot start mender-authd as a service.

sh-5.2# systemctl start mender-authd
You are in emergency mode. After logging in, type "journalctl -xPress Enter for maintenance
(or press Control-D to continue):

Also, it fails to start the client

sh-5.2# mender-auth daemon
record_id=1 severity=info time="1970-Jan-01 01:58:38.916505" name="Global" msg="Successfully loaded private key from /var/lib/mender/mender-agent.pem"
record_id=2 severity=error time="1970-Jan-01 01:58:38.923184" name="Global" msg="Failed to start the listen loop"
record_id=3 severity=error time="1970-Jan-01 01:58:38.923329" name="Global" msg="DBus connection error: Failed to get connection to system bus: Failed to connect to socket /run/dbus/system_bus_socket: No such file or directory[org.freedesktop.DBus.Error.FileNotFound]"

I am connected to the internet, if it helps with anything.

This is the local.conf file used to build the image:

MACHINE ??= 'imx8mp-lpddr4-evk'
DISTRO ?= 'fsl-imx-wayland'
EXTRA_IMAGE_FEATURES ?= "debug-tweaks"
USER_CLASSES ?= "buildstats"
PATCHRESOLVE = "noop"
BB_DISKMON_DIRS ??= "\
    STOPTASKS,${TMPDIR},1G,100K \
    STOPTASKS,${DL_DIR},1G,100K \
    STOPTASKS,${SSTATE_DIR},1G,100K \
    STOPTASKS,/tmp,100M,100K \
    HALT,${TMPDIR},100M,1K \
    HALT,${DL_DIR},100M,1K \
    HALT,${SSTATE_DIR},100M,1K \
    HALT,/tmp,10M,1K"
PACKAGECONFIG:append:pn-qemu-system-native = " sdl"
CONF_VERSION = "2"

DL_DIR ?= "${BSPDIR}/downloads/"
ACCEPT_FSL_EULA = "1"

# Switch to Debian packaging and include package-management in the image
PACKAGE_CLASSES = "package_deb"
EXTRA_IMAGE_FEATURES += "package-management"

MENDER_ARTIFACT_NAME = "release-1"

INHERIT += "mender-full-ubi"
MENDER_FEATURES_DISABLE:append = " mender-image-ubi mender-ubi"
MENDER_UBOOT_AUTO_CONFIGURE = "1"

MENDER_SERVER_URL = "https://hosted.mender.io"
MENDER_TENANT_TOKEN = "token"

This is the bblayers.conf file used to build the image:

LCONF_VERSION = "7"

BBPATH = "${TOPDIR}"
BSPDIR := "${@os.path.abspath(os.path.dirname(d.getVar('FILE', True)) + '/../..')}"

BBFILES ?= ""
BBLAYERS = " \
  ${BSPDIR}/sources/poky/meta \
  ${BSPDIR}/sources/poky/meta-poky \
  \
  ${BSPDIR}/sources/meta-openembedded/meta-oe \
  ${BSPDIR}/sources/meta-openembedded/meta-multimedia \
  ${BSPDIR}/sources/meta-openembedded/meta-python \
  \
  ${BSPDIR}/sources/meta-freescale \
  ${BSPDIR}/sources/meta-freescale-3rdparty \
  ${BSPDIR}/sources/meta-freescale-distro \
"

# i.MX Yocto Project Release layers
BBLAYERS += "${BSPDIR}/sources/meta-imx/meta-imx-bsp"
BBLAYERS += "${BSPDIR}/sources/meta-imx/meta-imx-sdk"
BBLAYERS += "${BSPDIR}/sources/meta-imx/meta-imx-ml"
BBLAYERS += "${BSPDIR}/sources/meta-imx/meta-imx-v2x"
BBLAYERS += "${BSPDIR}/sources/meta-nxp-demo-experience"
BBLAYERS += "${BSPDIR}/sources/meta-nxp-connectivity/meta-nxp-matter-baseline"
BBLAYERS += "${BSPDIR}/sources/meta-nxp-connectivity/meta-nxp-openthread"

BBLAYERS += "${BSPDIR}/sources/meta-arm/meta-arm"
BBLAYERS += "${BSPDIR}/sources/meta-arm/meta-arm-toolchain"
BBLAYERS += "${BSPDIR}/sources/meta-clang"
BBLAYERS += "${BSPDIR}/sources/meta-openembedded/meta-gnome"
BBLAYERS += "${BSPDIR}/sources/meta-openembedded/meta-networking"
BBLAYERS += "${BSPDIR}/sources/meta-openembedded/meta-filesystems"
BBLAYERS += "${BSPDIR}/sources/meta-qt6"
BBLAYERS += "${BSPDIR}/sources/meta-security/meta-parsec"
BBLAYERS += "${BSPDIR}/sources/meta-security/meta-tpm"
BBLAYERS += "${BSPDIR}/sources/meta-virtualization"

BBLAYERS += "${BSPDIR}/sources/meta-mender/meta-mender-core/"

If anyone can take a look and help me understand what I am missing, that will be much appreciated.

Thank you!

2

Hi @pcarmenp1,

Thanks for reaching out! On a first glance, that build setup looks quite huge, and at the same time missing the parts which tie Mender and NXP together. Have you had a look at meta-mender-nxp already for some inspiration?
You’ll definitely need to figure out the Mender configuration so the generated .sdimg can boot, as the .wic.zst will not have the A/B integration.

Maybe @leon-anavi or @MarekBelisko can share a few pointers, having been the last to touch the NXP stuff?

Greetz,
Josef

3

Having integrated mender on the NXP i.MX8MPlus myself, maybe I can help here.
Though admittedly I am still on Yocto kirkstone, so details may vary here.

As @TheYoctoJester pointed out, you definitely have to figure out how to configure mender so that the .sdimg can boot.
Do you boot from an SD card or from eMMC memory?

On my setup, I flash the bootloader (imx-boot) and the rootfs (.sdimg) to eMMC with NXP’s uuu tool. This is my mender configuration:

MENDER_STORAGE_DEVICE = "/dev/mmcblk2"
MENDER_STORAGE_TOTAL_SIZE_MB = "7000"
MENDER_UBOOT_STORAGE_DEVICE = "2"
MENDER_UBOOT_ENV_STORAGE_DEVICE_OFFSET = "4194304"
MENDER_PARTITION_ALIGNMENT = "4096"
MENDER_RESERVED_SPACE_BOOTLOADER_DATA = "32768"
MENDER_BOOT_PART_SIZE_MB = "64"
MENDER_DATA_PART_SIZE_MB = "2000"
MENDER_DATA_PART_FSTAB_OPTS:append = ",sync"

and the output of fdisk:

sh:~$ sudo fdisk -l
Disk /dev/mmcblk2: 7.28 GiB, 7820083200 bytes, 15273600 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x767191aa

Device         Boot    Start      End Sectors  Size Id Type
/dev/mmcblk2p1 *        8256   139327  131072   64M  c W95 FAT32 (LBA)
/dev/mmcblk2p2        139328  5193751 5054424  2.4G 83 Linux
/dev/mmcblk2p3       5193752 10248175 5054424  2.4G 83 Linux
/dev/mmcblk2p4      10248176 15273599 5025424  2.4G 83 Linux


Disk /dev/mmcblk2boot0: 4 MiB, 4194304 bytes, 8192 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/mmcblk2boot1: 4 MiB, 4194304 bytes, 8192 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/mmcblk1: 1.94 GiB, 2080374784 bytes, 4063232 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x74b199ff

Device         Boot Start     End Sectors  Size Id Type
/dev/mmcblk1p1        239 4063231 4062993  1.9G  e W95 FAT16 (LBA)

I had problems too while integrating (concerning the u-boot environment and uuu), they have already been described and solved here on mender hub: SPL not booting into main u-boot when loading using NXPs UUU tool and mender-uboot is enabled - #6 by Ossian

If your setup is similar to mine, feel free to ask for more information. If you use the SD card though, I will probably not be of much help.

4

Hi,

Thank you for helping me!

I am booting from a SD card.

I succeeded to boot using dd, copying firstly the rootfs (.sdimg) and then the bootloader. I noticed also that .wic.zst doesn’t partition correctly.
The image which was booting correctly doesn’t have mender-uboot, but it hasthe good partitioning (2 rootfs partitions, /boot, /data ). I think that I will need this because I want to do a rootfs update. Also the image doesn’t have mender-authd and mender-updated default active.

When I add mender-uboot in MENDER_FEATURES_ENABLE:append, then the kernel Image couldn’t be loaded due to Bad Linux ARM64 Image magic! I saw in printenv that bootcmd uses kernel_addr_r and fdt_addr_r which are not initialized. In a classic build these values are set, but using MENDER_UBOOT_AUTO_CONFIGURE = “1” delete a lot of environment values.

u-boot=> boot
load - load binary file from a filesystem

Usage:
load <interface> [<dev[:part]> [<addr> [<filename> [bytes [pos]]]]]
    - Load binary file 'filename' from partition 'part' on device
       type 'interface' instance 'dev' to address 'addr' in memory.
      'bytes' gives the size to load in bytes.
      If 'bytes' is 0 or omitted, the file is read until the end.
      'pos' gives the file byte position to start reading from.
      If 'pos' is 0 or omitted, the file is read from the start.
Bad Linux ARM64 Image magic!

Did you enable mender-uboot?

5

Sorry, apparently I don’t get notifications here when someone responds, didn’t saw your post until now :confused:

I did enable mender-uboot and had the problems with kernel_addr_r and fdt_addr_r not being initialized in the u-boot environment too. I simply made a patch file to the u-boot source code that adds the missing env variables.

But if I understood correctly, you managed to flash the SD with .sdimg rootfs and another bootloader (probably grub) and boot from it. Since it has the correct partitioning, I guess mender rootfs updates should work already. You won’t need u-boot.
The Yocto build should have created a .mender in deploy/images/[machine]/. Test whether rootfs updates work by installing it from the command line

touch be-gone-after-update
sudo mender install <image>.mender
sudo reboot

You can tell that the rootfs update succeeded when the be-gone-after-update file is not there after the reboot :slight_smile:

Edit: For mender server integration via mender-updated etc. you better inspect mender documentation, as this is not dependent on your board.

For reference, the u-boot patch file in case you want to use u-boot nevertheless.

From 42b4eb4526811e7e8e0c2d8aa5638596bf09e732 Mon Sep 17 00:00:00 2001
Date: Thu, 28 Nov 2024 13:28:48 +0100
Subject: [PATCH] initialize kernel_addr_r and fdt_addr_r in u-boot environment

---
 include/configs/msc_sm2s_imx8mp.h | 1 +
 include/env_default.h             | 1 +
 2 files changed, 2 insertions(+)

diff --git a/include/configs/msc_sm2s_imx8mp.h b/include/configs/msc_sm2s_imx8mp.h
index d0ef98e479..fd340736f3 100644
--- a/include/configs/msc_sm2s_imx8mp.h
+++ b/include/configs/msc_sm2s_imx8mp.h
@@ -88,6 +88,7 @@
 	"script=boot.scr\0" \
 	"image=Image\0" \
 	"console=ttymxc1,115200\0" \
+	"fdt_addr_r=0x43000000\0" \
 	"fdt_addr=0x43000000\0" \
 	"fdt_addr_ov=0x43100000\0" \
 	"fdt_high=0xffffffffffffffff\0" \
diff --git a/include/env_default.h b/include/env_default.h
index ea31a8eddf..8a889f3c09 100644
--- a/include/env_default.h
+++ b/include/env_default.h
@@ -84,6 +84,7 @@ const uchar default_environment[] = {
 #endif
 #ifdef	CONFIG_LOADADDR
 	"loadaddr="	__stringify(CONFIG_LOADADDR)	"\0"
+	"kernel_addr_r=" __stringify(CONFIG_LOADADDR)"\0"
 #endif
 #if defined(CONFIG_PCI_BOOTDELAY) && (CONFIG_PCI_BOOTDELAY > 0)
 	"pcidelay="	__stringify(CONFIG_PCI_BOOTDELAY)"\0"
-- 
2.17.1
6

Hi,

Sorry for the late reply.
Thank you very much, I solved the issue. Thank you for sharing with me the u-boot patch file, it was very useful. This patch was necessary to enable mender-uboot.