Scarthgap uboot Yocto on BBB - artifact boot fails

I have built a Yocto Scarthgap u-boot version of the BeagleBoneBlack Mender source.
Putting the build on sd-card, the BBB boots OK and appears in Mender UI. I have accepted it.
the boot sequence is:

U-Boot SPL 2024.01 (Jan 08 2024 - 15:37:48 +0000)
Trying to boot from MMC1


U-Boot 2024.01 (Jan 08 2024 - 15:37:48 +0000)

CPU  : AM335X-GP rev 2.1
Model: TI AM335x BeagleBone Black
DRAM:  512 MiB
Core:  160 devices, 18 uclasses, devicetree: separate
WDT:   Started wdt@44e35000 with servicing every 1000ms (60s timeout)
NAND:  0 MiB
MMC:   OMAP SD/MMC: 0, OMAP SD/MMC: 1
Loading Environment from MMC... OK
Net:   eth2: ethernet@4a100000using musb-hdrc, OUT ep1out IN ep1in STATUS ep2in
MAC de:ad:be:ef:00:01
HOST MAC de:ad:be:ef:00:00
RNDIS ready
, eth3: usb_ether
Hit any key to stop autoboot:  0
70112 bytes read in 8 ms (8.4 MiB/s)
8630888 bytes read in 552 ms (14.9 MiB/s)
Kernel image @ 0x82000000 [ 0x000000 - 0x83b268 ]
## Flattened Device Tree blob at 88000000
   Booting using the fdt blob at 0x88000000
Working FDT set to 88000000
   Loading Device Tree to 8ffeb000, end 8ffff1df ... OK
Working FDT set to 8ffeb000

Starting kernel ...

I then renamed the MENDER_ARTIFACT_NAME and rebuilt and uploaded the *.mender output to Mender UI. The deployment downloaded and the BBB reset but failed to boot.

U-Boot SPL 2024.01 (Jan 08 2024 - 15:37:48 +0000)
Trying to boot from MMC1


U-Boot 2024.01 (Jan 08 2024 - 15:37:48 +0000)

CPU  : AM335X-GP rev 2.1
Model: TI AM335x BeagleBone Black
DRAM:  512 MiB
Core:  160 devices, 18 uclasses, devicetree: separate
WDT:   Started wdt@44e35000 with servicing every 1000ms (60s timeout)
NAND:  0 MiB
MMC:   OMAP SD/MMC: 0, OMAP SD/MMC: 1
Loading Environment from MMC... OK
Net:   eth2: ethernet@4a100000using musb-hdrc, OUT ep1out IN ep1in STATUS ep2in
MAC de:ad:be:ef:00:01
HOST MAC de:ad:be:ef:00:00
RNDIS ready
, eth3: usb_ether
Saving Environment to MMC... Writing to MMC(0)... failed
Failed (1)
Hit any key to stop autoboot:  0
omap_hsmmc_send_cmd: timedout waiting on cmd inhibit to clear
** No partition table - mmc 0 **
Couldn't find partition mmc 0:3
Can't set block device
** No partition table - mmc 0 **
Couldn't find partition mmc 0:3
Can't set block device
zimage: Bad magic!

This is when it is trying to access MMC 0, the sd-card.
Why does the artifact behave differently to the sd-card image?
Do I have to adjust the u-boot for the artifact when using the sd-card?
Ultimately the code will be in the eMMC, so should I transfer the *.sdimg to the eMMC and try again?
There is no recovery, the boot just stops and the sd-card image is corrupted so will never boot again. Not ideal - so can you point me in the right direction please?

I’ve just noted the SD-Card initial boot fails too. But on a subsequent power cycle it does boot.
I thought I’d not pressed the S2 button correctly and so created the failure, but no, it always fails the first time and works then on.
The SD-Card failure is:

U-Boot SPL 2024.01 (Jan 08 2024 - 15:37:48 +0000)
Trying to boot from MMC1


U-Boot 2024.01 (Jan 08 2024 - 15:37:48 +0000)

CPU  : AM335X-GP rev 2.1
Model: TI AM335x BeagleBone Black
DRAM:  512 MiB
Core:  160 devices, 18 uclasses, devicetree: separate
WDT:   Started wdt@44e35000 with servicing every 1000ms (60s timeout)
NAND:  0 MiB
MMC:   OMAP SD/MMC: 0, OMAP SD/MMC: 1
Loading Environment from MMC... *** Warning - bad CRC, using default environment

<ethaddr> not set. Validating first E-fuse MAC
Net:   eth2: ethernet@4a100000using musb-hdrc, OUT ep1out IN ep1in STATUS ep2in
MAC de:ad:be:ef:00:01
HOST MAC de:ad:be:ef:00:00
RNDIS ready
, eth3: usb_ether
Hit any key to stop autoboot:  0
Saving Environment to MMC... Writing to MMC(0)... failed
Failed (1)
omap_hsmmc_send_cmd: timedout waiting on cmd inhibit to clear
** No partition table - mmc 0 **
Couldn't find partition mmc 0:2
Can't set block device
** No partition table - mmc 0 **
Couldn't find partition mmc 0:2
Can't set block device
zimage: Bad magic!

which is very similar to the Mender artifact boot except for “Loading Environment from MMC… *** Warning - bad CRC, using default environment”

I’ve just noticed that in the uboot envs:

bootcmd=run mender_setup; 
setenv bootargs root=${mender_kernel_root} ${bootargs}; 
if test "${fdt_addr_r}" != ""; then load ${mender_uboot_root} ${fdt_addr_r} /boot/${mender_dtb_name}; fi; 
load ${mender_uboot_root} ${kernel_addr_r} /boot/${mender_kernel_name}; 
${mender_boot_kernel_type} ${kernel_addr_r} - ${fdt_addr_r}; 
run mender_try_to_recover

but mender_uboot_root is not defined.
mender_uboot_boot is however. Is this a typo?
Some Mender patches set the …_root var:

 "setenv mender_uboot_root " MENDER_UBOOT_STORAGE_INTERFACE " " __stringify(MENDER_UBOOT_STORAGE_DEVICE) ":${mender…

[EDIT]
ok, it should be set by mender_setup dynamically.

mender_setup=
if test "${mender_saveenv_canary}" != "1"; 
  then setenv mender_saveenv_canary 1; saveenv; fi; 
if test "${mender_pre_setup_commands}" != ""; 
  then run mender_pre_setup_commands; fi; 
if test "${mender_systemd_machine_id}" != ""; 
  then setenv bootargs "systemd.machine_id=${mender_systemd_machine_id} ${bootargs}"; fi; 
setenv mender_kernel_root /dev/mmcblk0p${mender_boot_part}; 
if test ${mender_boot_part} = 2; 
  then setenv mender_boot_part_name /dev/mmcblk0p2; 
else setenv mender_boot_part_name /dev/mmcblk0p3; fi; 
setenv mender_kernel_root_name ${mender_boot_part_name}; 
setenv mender_uboot_root mmc 0:${mender_boot_part_hex}; 
setenv mender_uboot_root_name ${mender_boot_part_name}; 
setenv expand_bootargs "setenv bootargs \\"${bootargs}\\""; 
run expand_bootargs; setenv expand_bootargs; 
if test "${mender_post_setup_commands}" != ""; 
  then run mender_post_setup_commands; fi

setenv mender_uboot_root mmc 0:${mender_boot_part_hex};

So any clues as to why the first boot from SD-Card fails with:

Saving Environment to MMC... Writing to MMC(0)... failed
Failed (1)
omap_hsmmc_send_cmd: timedout waiting on cmd inhibit to clear
** No partition table - mmc 0 **
Couldn't find partition mmc 0:2
Can't set block device
** No partition table - mmc 0 **
Couldn't find partition mmc 0:2
Can't set block device
zimage: Bad magic!

Hi @BenH,

I just replicated a beaglebone-uboot setup here and it works as expected as far as I can tell.
The one thing you might look into is the bootloader on the integrated eMMC interfering with whatever is on the uSD card. As far as I know for recent BBBs this should not be a problem, but older ones always try to read something from the eMMC, and only resort to the uSD if nothing is found there, or it is faulty.

On my board, I’ve (brutally) disabled that by overwriting the eMMC boot process through something like

dd if=/dev/zero of=/dev/mmcblk0 bs=1M count=16

which zeroes the first 16M of the eMMC out. After such, the boot process from uSD is stable, regardless of the S2 switch.

If you want to replicate this, please triple check the devices.

Greetz,
Josef

Hi @TheYoctoJester .
Thanks for taking the time to test this.
I’ve tried going back to the GRUB build on Scarthgap and this fails too.

I’m just testing your suggestion of the

dd if=/dev/zero of=/dev/mmcblk0 bs=1M count=16

I need to use /dev/mmcblk1 when booted from SDCard.
I need to do some more testing to check if it works.
My device was not registering on the Mender UI yesterday so need to sort that out.
Thanks

Hi @BenH,

What I mean is that you should zero out the boot sector of the integrated eMMC, not the one of the SD card. Just to be clear.

Greetz,
Josef

Yes, that’s clear.
What I didn’t realise is that the /dev/mmcblk* changes depending on what you boot from, so if you boot from SDCard, /dev/mmcblk0 is SD Card /dev/mmcblk1 is eMMC so I initially wiped the SDCard.

Hi @TheYoctoJester
It’s working now, from SDCard and from artifact deployment.
The deleting the eMMC was one factor, the other was the size of the SDCard.
I was using a 32GB card, but now I’m using an old 4GB one and it works fine.
Perhaps there is a stage where the 32GB is repartitioned and this stalls the boot ??

Anyway, thanks for your help on this.

BR,
Ben