Variscite DART-6UL - upgrade zeus to scarthgap

I’m attempting to update a Zeus build to Yocto Scarthgap. The Zeus build was created by following the instructions in the first post in this thread.
A few things required tweaks to support Scarthgap - the u-boot patches needed re-applying, and manually tweaking the conf files.
Now it’s building fine, however the boot process is having trouble.
This is the 2nd attempt I’ve had at this - the first was targeting Kirkstone, however it ran into this exact same boot issue:

Here is the boot log:

U-Boot SPL 2018.03+g9d5b55aeed8+p0 (Oct 18 2024 - 19:06:56 +0000)

Part number: VSM-VS6UL-101
Assembly: AS2104164477
Date of production: 2021 May 04
SOM configuration: eMMC WiFi
Trying to boot from MMC1
MMC Boot Device: mmc0 (SD)


U-Boot 2018.03+g9d5b55aeed8+p0 (Oct 18 2024 - 19:06:56 +0000)

CPU:   Freescale i.MX6ULL rev1.1 900 MHz (running at 396 MHz)
CPU:   Commercial temperature grade (0C to 95C) at 37C
Reset cause: POR
Board: Variscite VAR-SOM-6UL
I2C:   ready
DRAM:  512 MiB
MMC:   FSL_SDHC: 0, FSL_SDHC: 1
Loading Environment from MMC... OK
Display: VAR-WVGA-LCD (800x480)
Video: 800x480x24
** Unrecognized filesystem type **
Error: could not access storage.
In:    serial
Out:   serial
Err:   serial
eMMC:  7.3 GiB
Codec: wm8731
Net:   got MAC0 address from fuse: f8:dc:7a:5b:a4:1e
FEC0 [PRIME], usb_ether
Error: usb_ether address not set.

Normal Boot
Hit any key to stop autoboot:  0
switch to partitions #0, OK
mmc0 is current device
Scanning mmc 0:1...
32167 bytes read in 20 ms (1.5 MiB/s)
Found EFI removable media binary efi/boot/bootarm.efi
Scanning disks on usb...
Disk usb0 not ready
Disk usb1 not ready
Disk usb2 not ready
Disk usb3 not ready
Scanning disks on mmc...
MMC Device 2 not found
MMC Device 3 not found
Found 10 disks
466944 bytes read in 39 ms (11.4 MiB/s)
## Starting EFI application at 82000000 ...


EFI stub: Booting Linux Kernel...
EFI stub: Entering in SVC mode with MMU enabled
EFI stub: ERROR: install_configuration_table() failed
error: start_image() returned 0x80000009.

Dropping to grub prompt for unknown reason. Should never get here.

One thing to note - If I boot into the kernel directly from u-boot, it boots fine & everything seems correct. AFAICT it seems that the problem only arises when uefi/grub are in the loop.
These u-boot commands let me boot into the kernel directly:

setenv bootargs "console=ttymxc0,115200n8 rootwait root=/dev/mmcblk0p2" 
load mmc 0:2 ${loadaddr} boot/zImage 
load mmc 0:2 ${fdt_addr_r} boot/imx6ull-var-som-emmc-sd-card.dtb
bootz ${loadaddr} - ${fdt_addr_r}

Also, perhaps unsurprisingly, if I overwrite the zImage with one from the Zeus build (kernel 5.4.3-imx6ul+g9447bf00ed07), it boots fine.
And the opposite is true too - if I use an SD card with the Zeus build, and overwrite the zImage with one from Scarthgap, I get the exact same install_configuration_table & start_image errors.
To me, that indicates that the bootloaders are OK, and that the issue is the kernel itself - or at least, the handoff done via u-boot/grub.

Any help or guidance on how to proceed would be greatly appreciated. I’m guessing its a kernel configuration issue, however I think the config is correct. Here are the efi options in the kernel config, if it’s relevant:

root@imx6ul-var-dart-ts:~# zcat /proc/config.gz   | grep -i efi
CONFIG_EFI_STUB=y
CONFIG_EFI=y
CONFIG_EFI_PARTITION=y
# EFI (Extensible Firmware Interface) Support
CONFIG_EFI_ESRT=y
CONFIG_EFI_PARAMS_FROM_FDT=y
CONFIG_EFI_RUNTIME_WRAPPERS=y
CONFIG_EFI_GENERIC_STUB=y
CONFIG_EFI_ARMSTUB_DTB_LOADER=y
# CONFIG_EFI_BOOTLOADER_CONTROL is not set
# CONFIG_EFI_CAPSULE_LOADER is not set
# CONFIG_EFI_TEST is not set
# CONFIG_EFI_DISABLE_PCI_DMA is not set
# CONFIG_EFI_DISABLE_RUNTIME is not set
# CONFIG_EFI_COCO_SECRET is not set
# end of EFI (Extensible Firmware Interface) Support
# Software defined radio USB devices
# CONFIG_FB_EFI is not set
# CONFIG_RTC_DRV_EFI is not set
CONFIG_EFIVAR_FS=m

Hi @tloten,

Thinking a bit about your description, I think your analysis is right - unless we are both overlooking something weird. As a first shot, can you diff the kernel configurations? It should be possible to extract both from the build directory.

Greetz,
Josef

Sure, both configs are in this gist (will need to scroll down to find the 2nd file):

Hi @tloten,

Maybe a long shot, but can you check if this problem also affects you? Upgrading Mender client 2.6 to 4.0 - #18 by TheYoctoJester

Greetz,
Josef

Thanks - good idea, but didn’t seem to have an effect unfortunately. I stripped out a few areas like display/audio/bluetooth, and got the zImage to 6.9MB (down from 9.1MB). But still getting the same error :frowning:
For reference, the Zeus kernel was 8.5MB.

It’ll be a few weeks before I can get back onto this, but my next steps will be to try a few intermediary Yocto versions & their kernels to at least narrow it down. But it’s fairly tedious to do each time! I’ll also try my luck with Variscite support, but not sure how far I’ll get given it’s outside their typical boot process.

Hi @tloten,

Sorry to hear not having luck yet. I’ll also try to give the configs some more inspection, will let you know if I find something.:crossed_fingers:

Greetz,
Josef