Ubuntu image failing to boot after mender-convert + potential solution?

Hello, mender devs!

I’ve been using mender on several NUC8BEK SBCs without an issue. Recently, however, I cloned the same golden image that I’ve been using for all of the previous NUC8BEKs and noticed that after boot, it went directly to the GRUB 2.04 shell. I tried mender-convert on several nuc11s and my laptop and the same thing happened.

I noticed that a split second before dropping the shell, it flashed the following “error: no such device: {uuid-of-device-that-doesn’t-exist}”

I then tried to manually get it to boot by runing “set prefix=/efi/boot” followed by “configfile /efi/boot/grub.cfg”

It then, returned an error about being unable to find the ‘hashsum’ command twice, followed by “environment 2 still corrupt after attempted restore. Halting”

It was then that I realized that the wrong grub.efi must be getting called. I noticed that after mender-convert, there is still an /ubuntu entry under /efi that is supposed to not be called anymore, because of the /BOOT entry that contains the mender version of grubx64.efi.

I then removed all references to hashsum in the grub.cfg and was able to get it to boot by running the set prefix and configfile commands.

After doing some more digging, I realized that the grub efi inside of /ubuntu was being called instead of the efi inside /boot.

After yet more digging, I realized the reason was that the bios was looking for the BOOTX64.CSV that’s contained in /ubuntu. So, I added my own BOOTX64.CSV inside of /boot that pointed to mender’s BOOTX64.EFI, and suddenly it booted without any issues.

Is this a potential solution, or is this just a fallback band-aid over some other issue?

This sounds reasonable to me. Each hardware and OS combination can have its own quirks requiring changes like these. The best way to handle this would be to implement a custom hook in your mender-convert config files.

Drew

Thanks, Drew. It seems like this is really common across a lot of x64 efi installs (at least two different nuc models and a laptop and desktop that I tested on so far). I’m happy to add it as a hook, but it might not be a bad idea to also add it to the generic x86 config file in the official mender-convert repo, given that it won’t affect bios firmware that isn’t looking for a Bootx64.csv.

Yeah, if the code is generic then we will certainly be happy to pull it back into the main repo. I guess the question is how do we determine on which machine/OS combinations it would be needed? I think if you prototype it as a hardcoded hook in your case then we can review it and hopefully together we can figure out when it should be applied.

Drew

I’ve run into the exact same issue, took me days to track it down; thanks for your post.

Do you have any configs floating around which can help me automate this? I’m not the best at programming!