Building image with mender-systemd-boot fails at patching systemd-boot

Hi everyone,

i am switching our existing yocto image from grub to systemd-boot, mainly to take advantage of meta-intel’s uefi-comboapp and uefi-sign classes.

In doing so, the build fails whenever i enable the mender-systemd-boot feature, and i can’t for the life of me figure out why:

Error message

ERROR: systemd-boot-250.5-r0 do_patch: Applying patch ‘systemd-boot-slotconfig.patch’ on target directory ‘/opt/yocto/build/tmp/work/core2-64-poky-linux/systemd-boot/250.5-r0/git’
CmdError(‘quilt --quiltrc /opt/yocto/build/tmp/work/core2-64-poky-linux/systemd-boot/250.5-r0/recipe-sysroot-native/etc/quiltrc push’, 0, "stdout: Applying patch systemd-boot-slotconfig.patch
can’t find file to patch at input line 23 Perhaps you used the wrong -p or --strip option? The text leading up to this was:

|From d9398201627b0dee2ca78c78113862e324a4f8b1 Mon Sep 17 00:00:00 2001 |From: Liam White McShane liam.white@timesys.com |Date: Fri, 23 Jul 2021 13:53:17 -0400 |Subject: [PATCH] systemd-boot: add A/B RFS support
|
|commit 9859c61858d6a55dc648bafef46c554a93d87285
|
| Add slot config
|
|—
| src/boot/efi/boot.c | 46 ++++++++++
| src/boot/efi/meson.build | 2 +
| src/boot/efi/slot.c | 175 +++++++++++++++++++++++++++++++++++++++
| src/boot/efi/slot.h | 25 ++++++
| 4 files changed, 248 insertions(+)
| create mode 100644 src/boot/efi/slot.c
| create mode 100644 src/boot/efi/slot.h
|
|diff --git a/src/boot/efi/boot.c b/src/boot/efi/boot.c
|index b4f3b9605a…2dc8a3bbb1 100644
|— a/src/boot/efi/boot.c

+++ b/src/boot/efi/boot.c
No file to patch. Skipping patch.
4 out of 4 hunks ignored
can’t find file to patch at input line 101
Perhaps you used the wrong -p or --strip option?
The text leading up to this was:

|diff --git a/src/boot/efi/meson.build b/src/boot/efi/meson.build
|index afdf739d9b…200cf38fc0 100644
|— a/src/boot/efi/meson.build

+++ b/src/boot/efi/meson.build
No file to patch. Skipping patch.
2 out of 2 hunks ignored
patching file src/boot/efi/slot.c
patching file src/boot/efi/slot.h
Patch systemd-boot-slotconfig.patch does not apply (enforce with -f)

stderr: ")
ERROR: Logfile of failure stored in: /opt/yocto/build/tmp/work/core2-64-poky-linux/systemd-boot/250.5-r0/temp/log.do_patch.2270288
ERROR: Task (/opt/yocto/meta/recipes-core/systemd/systemd-boot_250.5.bb:do_patch) failed with exit code ‘1’

Excluding the patch file via SRC_URI:remove = "file://systemd-boot-slotconfig.patch" results in a meson build failure and some licence checksum issues:

Different error output

ERROR: mc:pa5:systemd-boot-250.5-r0 do_populate_lic: QA Issue: systemd-boot: LIC_FILES_CHKSUM points to an invalid file: /opt/yocto/build/tmp/work/corei7-64-poky-linux/systemd-boot/250.5-r0/git/LICENSE.GPL2 [license-checksum]
ERROR: mc:pa5:systemd-boot-250.5-r0 do_populate_lic: QA Issue: systemd-boot: LIC_FILES_CHKSUM points to an invalid file: /opt/yocto/build/tmp/work/corei7-64-poky-linux/systemd-boot/250.5-r0/git/LICENSE.LGPL2.1 [license-checksum]
ERROR: mc:pa5:systemd-boot-250.5-r0 do_populate_lic: Fatal QA errors were found, failing task.
ERROR: Logfile of failure stored in: /opt/yocto/build/tmp/work/corei7-64-poky-linux/systemd-boot/250.5-r0/temp/log.do_populate_lic.2188912 ERROR: Task (mc:pa5:/opt/yocto/meta/recipes-core/systemd/systemd-boot_250.5.bb:do_populate_lic) failed with exit code ‘1’
ERROR: mc:pa5:systemd-boot-250.5-r0 do_configure: meson failed ERROR: mc:pa5:systemd-boot-250.5-r0 do_configure: ExecutionError(‘/opt/yocto/build/tmp/work/corei7-64-poky-linux/systemd-boot/250.5-r0/temp/run.do_configure.2188883’, 1, None, None) ERROR: Logfile of failure stored in: /opt/yocto/build/tmp/work/corei7-64-poky-linux/systemd-boot/250.5-r0/temp/log.do_configure.2188883
Log data follows:
| DEBUG: Executing python function extend_recipe_sysroot | NOTE: Direct dependencies are [‘mc:pa5:/opt/yocto/meta/recipes-bsp/gnu-efi/gnu-efi_3.0.14.bb:do_populate_sysroot’, ‘mc:pa5:/opt/yocto/meta/recipes-core/glibc/glibc_2.35.bb:do_populate_sysroot’, ‘mc:pa5:/opt/yocto/meta/recipes-core/util-linux/util-linux_2.37.4.bb:do_populate_sysroot’, ‘mc:pa5:/opt/yocto/meta/recipes-devtools/gcc/gcc-cross_11.4.bb:do_populate_sysroot’, ‘mc:pa5:/opt/yocto/meta/recipes-devtools/gcc/gcc-runtime_11.4.bb:do_populate_sysroot’, ‘mc:pa5:/opt/yocto/meta/recipes-devtools/qemu/qemu-native_6.2.0.bb:do_populate_sysroot’, ‘mc:pa5:/opt/yocto/meta/recipes-devtools/quilt/quilt-native_0.67.bb:do_populate_sysroot’, ‘mc:pa5:/opt/yocto/meta/recipes-support/libcap/libcap_2.66.bb:do_populate_sysroot’, ‘mc:pa5:virtual:native:/opt/yocto/meta/recipes-core/gettext/gettext_0.21.bb:do_populate_sysroot’, ‘mc:pa5:virtual:native:/opt/yocto/meta/recipes-devtools/intltool/intltool_0.51.0.bb:do_populate_sysroot’, ‘mc:pa5:virtual:native:/opt/yocto/meta/recipes-devtools/meson/meson_0.61.3.bb:do_populate_sysroot’, ‘mc:pa5:virtual:native:/opt/yocto/meta/recipes-devtools/ninja/ninja_1.10.2.bb:do_populate_sysroot’, ‘mc:pa5:virtual:native:/opt/yocto/meta/recipes-devtools/patch/patch_2.7.6.bb:do_populate_sysroot’, ‘mc:pa5:virtual:native:/opt/yocto/meta/recipes-devtools/pkgconfig/pkgconfig_git.bb:do_populate_sysroot’, ‘mc:pa5:virtual:native:/opt/yocto/meta/recipes-devtools/pseudo/pseudo_git.bb:do_populate_sysroot’, ‘mc:pa5:virtual:native:/opt/yocto/meta/recipes-devtools/python/python3-jinja2_3.1.1.bb:do_populate_sysroot’, ‘mc:pa5:virtual:native:/opt/yocto/meta/recipes-devtools/python/python3_3.10.13.bb:do_populate_sysroot’, ‘mc:pa5:virtual:native:/opt/yocto/meta/recipes-extended/gperf/gperf_3.1.bb:do_populate_sysroot’]
| NOTE: Installed into sysroot:
| NOTE: Skipping as already exists in sysroot: [‘gnu-efi’, ‘glibc’, ‘util-linux’, ‘gcc-cross-x86_64’, ‘gcc-runtime’, ‘qemu-native’, ‘quilt-native’, ‘libcap’, ‘gettext-native’, ‘intltool-native’, ‘meson-native’, ‘ninja-native’, ‘patch-native’, ‘pkgconfig-native’, ‘pseudo-native’, ‘python3-jinja2-native’, ‘python3-native’, ‘gperf-native’, ‘libcap-ng’, ‘util-linux-libuuid’, ‘libxcrypt’, ‘bash-completion’, ‘zlib’, ‘opkg-utils’, ‘ncurses’, ‘zlib-native’, ‘gdbm-native’, ‘libffi-native’, ‘xz-native’, ‘openssl-native’, ‘util-linux-libuuid-native’, ‘bzip2-native’, ‘libtool-native’, ‘readline-native’, ‘libtirpc-native’, ‘ncurses-native’, ‘libnsl2-native’, ‘sqlite3-native’, ‘zstd-native’, ‘mpfr-native’, ‘flex-native’, ‘linux-libc-headers’, ‘texinfo-dummy-native’, ‘gmp-native’, ‘binutils-cross-x86_64’, ‘libmpc-native’, ‘gnu-config-native’, ‘re2c-native’, ‘libxml-parser-perl-native’, ‘gettext-minimal-native’, ‘python3-installer-native’, ‘python3-wheel-native’, ‘python3-markupsafe-native’, ‘python3-setuptools-native’, ‘libgcc’, ‘attr-native’, ‘glib-2.0-native’, ‘perl-native’, ‘m4-native’, ‘expat-native’, ‘unzip-native’, ‘python3-flit-core-native’, ‘util-linux-native’, ‘libpcre-native’, ‘make-native’, ‘cmake-native’, ‘systemd-systemctl-native’, ‘libcap-ng-native’, ‘libpcre2-native’, ‘curl-native’]
| DEBUG: Python function extend_recipe_sysroot finished
| DEBUG: Executing shell function do_configure
| NOTE: Executing meson -Defi=true -Dgnu-efi=true -Defi-includedir=/opt/yocto/build/tmp/work/corei7-64-poky-linux/systemd-boot/250.5-r0/recipe-sysroot/usr/include/efi -Defi-libdir=/opt/yocto/build/tmp/work/corei7-64-poky-linux/systemd-boot/250.5-r0/recipe-sysroot/usr/lib -Defi-ld=bfd -Dman=false --cross-file /opt/yocto/build/tmp/work/corei7-64-poky-linux/systemd-boot/250.5-r0/meson-systemd-boot.cross …
|
| ERROR: Neither directory contains a build file meson.build.
| ERROR: meson failed
| WARNING: exit code 1 from a shell command.
ERROR: Task (mc:pa5:/opt/yocto/meta/recipes-core/systemd/systemd-boot_250.5.bb:do_configure) failed with exit code ‘1’

bblayers.conf:

# POKY_BBLAYERS_CONF_VERSION is increased each time build/conf/bblayers.conf
# changes incompatibly
POKY_BBLAYERS_CONF_VERSION = “2”

BBPATH = “${TOPDIR}”
BBFILES ?= “”

BBLAYERS ?= "
/opt/yocto/meta
/opt/yocto/meta-poky
/opt/yocto/meta-intel
/opt/yocto/meta-mender/meta-mender-core
/opt/yocto/meta-openembedded/meta-oe"

And a minimal example local.conf (excluding lines unchanged from the default file):

MACHINE_FEATURES:remove = “pcbios”
MACHINE_FEATURES:append = " efi"
MACHINE_FEATURES_NATIVE:append = " efi"
EFI_PROVIDER = “systemd-boot”

DISTRO_FEATURES:append = " keyboard nfs smbfs systemd usbhost opengl"

INIT_MANAGER = “systemd”

INHERIT += “mender-full”
INHERIT += “mender-image-systemd-boot”
ARTIFACTIMG_FSTYPE = “ext4”
MENDER_SERVER_URL = “https://hosted.mender.io
MENDER_TENANT_TOKEN = “*****”
MENDER_UPDATE_POLL_INTERVAL_SECONDS = “1800”
MENDER_INVENTORY_POLL_INTERVAL_SECONDS = “28800”
MENDER_CONNECT_USER = “root”
MENDER_FEATURES_ENABLE:append = " mender-systemd-boot"
MENDER_ARTIFACT_NAME = “sometest”
COPY_LIC_MANIFEST = “1”
COPY_LIC_DIRS = “1”
LICENSE_CREATE_PACKAGE = “1”
INITRD_IMAGE = “” # i disable the INITRD image here for simplicity, adding a initramfs did not change the behaviour

The code i provide here is a minimal reproducible example, the actual target image is of course a lot more involved, including a custom distro and targeting a Congatec board.

I would appreciate any pointers on this, i am aware the systemd-boot integration is not officially maintained, but also fairly recent and explicitly updated for kirkstone, so i see no reason for it to have broken in this manner.
I rather suspect either a tiny mistake, or major misconception in my configuration to be the issue

Also, i don’t know why the first couple of lines of the first error message are being displayed in bold, sorry about that :confused:

Hi @adrian,

Had a look at this and I can replicate the situation. However the systemd-boot patch has not been updated for kirkstone specifically - looking at its history just mentions honister. Given the experimental marker, it just didn’t get exercised in the meantime it seems.
Taking a look at the involved sources, it seems that some parts have changed noticeably, so the patch needs manual forward porting.

Greetz,
Josef

Hi @TheYoctoJester ,

Oh, yeah, looking at the code i see the same thing. I thought it was updated as it was specifically mentioned in this announcement for Meta-mender’s kirkstone release.

Do you think the people originally involved in building and porting this will be interested in updating it for kirkstone? I will certainly take a crack at doing it myself, but most of what i know about bootloaders and the intricacies of yocto in general, i have learned in the last few weeks, so i would most likely at least some assistance by someone knowledgeable in the topic.

Given the upcoming release of yocto 5.0 Scarthgap, if now one does this port now, mender-systemd-boot would be two LTS releases behind, if it gets included in meta-menders Scarthgap release at all.

Greetings and thanks for your time
Martin

Hi @Adrian,

Unfortunately I don’t have neither a matching setup nor experience with systemd-boot, otherwise I would offer helping myself. I can try to ping the original author though, maybe he can give a few pointers. Agree that getting it polished for scarthgap would be nice!

Greetz,
Josef

Hi @TheYoctoJester ,

that would be great, thanks. I’ll start looking into this port then, if I get anywhere,
I’ll submit a PR early to get input.
Might have to do it on my own time (my employer is fine with me contributing, but this rework needs to get finished sometime soon)

Greetings
Martin

1 Like

Alright. the issue turned out to be two-fold:

meta-mender-core/recipes-core/systemd/systemd-boot_%.bbappend was adding the patch file to SRC_URi incorrectly, overwriting the fetch url for systemd-boot’s code.

Secondly, the patch file itself had become incompatible with systemd. Version 249.7 (included in honnister) was working, but 250.5 (included by kirkstone) had seen some major rewrites to systemd-boot. I’m just about ready to submit a PR, but I have some questions regarding procedure I hope you can help me with @TheYoctoJester :

  • I built the patch on meta-mender’s kirkstone branch for now, as that is what I need for our project. I may run a test build on master, but I don’t think we know which systemd version scarthgape will include, so this patch will probably just have to be revisited when mender ports over. For now, should I submit the PR against kirkstone, or rebase it onto master and let someone else cherrypick that into kirkstone?
  • I adjusted the patch file manually, and while i did update the commit hashes for the a files, since I did not regenerate the patch, I don’t have any new hashes for the b side, and left those untouched. Is that acceptable?

Hi @adrian,

Submitting against kirkstone is perfectly fine, and the patch hashes are definitely not a blocker. If things are spotted then we can still regenerate the patches if needed. Given the complexity I guess that the PR will take a bit to review and test anyways, just make sure early that your signed-off-by is added properly.

One thing that was brought up last night, is that systemd-boot should have A/B support of its own by now. So are you using that one?

Greets,
Josef

Hi @TheYoctoJester ,

thanks, will do.
No, I didn’t know about that. I just updated the existing A/B support patch to work with the newer systemd-version. I’ll take a look at that, but I’m going on holiday next week, so depending on how involved that is, it might have to wait until mid-febuary.

Alternatively, I could submit what i have for now, and I (or someone else with more knowledge of how systemd-boot actually works) could switch over to the native a/b support in a separate PR.

Greetings,
Martin

Hi @adrian,

I personally would try to be resource efficient and open - means, get a draft PR out early for people to see, without spending a lot of energy upfront.

Greets,
Josef

@adrian did you manage to get this to work and/or is there a PR available ? I’m trying to get systemd-boot integrated into our build but running into the same issue regarding the patch being obsolete for the current systemd version(s)

Hi @skinny

i got as far as a buildable version that did boot successfully, but when i performed a mender upgrade, it failed to switch between the A and B partitions (it would think it did, but would not switch over properly). We had some other projects that took priority, and i have since moved away from using systemd-boot and switched back to grub, but i could look into cleaning up what i have and push it to the PR, that might save you some work.

Greetings,
Martin

I’m in the same boat and on the crossroads of deciding whether to pursuit systemd-boot or stick with grub for now. Project pressure indeed is a major factor and might keep with with grub for now as there is no “proven” solution yet. However I’d really like to take a look at your work on a new version of the patch and maybe find some time to fiddle with it.

Regards,
Mark

Hey Mark,

i packed the changes i made into a temporary commit and pushed them to my fork here. I did the work before Scarthgap came out, so it’s based on kirkstone and the patch file (meta-mender-core/recipes-core/systemd/files/systemd-boot-slotconfig.patch) is updated for the systemd version included in that (250.5 i think).

There is no PR open on the mender side right now, i’ll leave it up to you whether we do that from my fork, or you can fork from upstream and grab my changes.

I would suggest completing the update for kirkstone and getting that merged (so any kirkstone users can also benefit from it), and then moving to scarthgap from there

Greetings
Martin

1 Like