Adding MENDER_SWAP_PART_SIZE_MB="512" build failure

MarekBelisko · October 7, 2021, 9:45am

Hello, on dunfell release I’m trying to add swap partition si simply extend local.conf with:
MENDER_SWAP_PART_SIZE_MB="512"

But with this sdimg fails with error:

subprocess.CalledProcessError: Command 'cd /builds/open-source/test-system/build/tmp/work/orange_pi_zero-poky-linux-gnueabi/core-image-minimal/1.0-r0/bootfs.image_sdimg; find . -type d -print | tar --xattrs --xattrs-include='*' -cf - -S -C /builds/open-source/test-system/build/tmp/work/orange_pi_zero-poky-linux-gnueabi/core-image-minimal/1.0-r0/bootfs.image_sdimg -p --no-recursion --files-from - | tar --xattrs --xattrs-include='*' -xhf - -C /builds/open-source/test-system/build/tmp/work/orange_pi_zero-poky-linux-gnueabi/core-image-minimal/1.0-r0/deploy-core-image-minimal-image-complete/core-image-minimal-orange-pi-zero-20211005113718-sdimg/tmp.wic.1kpqjyfn/rootfs3' returned non-zero exit status 2.

Any ideas? Thanks.

kacf · October 12, 2021, 9:23am

This only gives the return code. Do you have more of the log available?

MarekBelisko · October 12, 2021, 11:37am

Yes:

2021-10-05 13:59:32 - INFO     - | DEBUG: Python function extend_recipe_sysroot finished
2021-10-05 13:59:32 - INFO     - | DEBUG: Executing shell function do_image_sdimg
2021-10-05 13:59:32 - INFO     - | + mkdir -p /builds/open-source/test-system/build/tmp/work/orange_pi_zero-poky-linux-gnueabi/core-image-minimal/1.0-r0
2021-10-05 13:59:32 - INFO     - | + true
2021-10-05 13:59:32 - INFO     - | + install -m 0644 /builds/open-source/test-system/build/tmp/deploy/images/orange-pi-zero/uboot.env /builds/open-source/test-system/build/tmp/work/orange_pi_zero-poky-linux-gnueabi/core-image-minimal/1.0-r0/
2021-10-05 13:59:32 - INFO     - | + basename /dev/mmcblk0
2021-10-05 13:59:32 - INFO     - | + ondisk_dev=mmcblk0
2021-10-05 13:59:32 - INFO     - | + wks=/builds/open-source/test-system/build/tmp/work/orange_pi_zero-poky-linux-gnueabi/core-image-minimal/1.0-r0/mender-sdimg.wks
2021-10-05 13:59:32 - INFO     - | + rm -f /builds/open-source/test-system/build/tmp/work/orange_pi_zero-poky-linux-gnueabi/core-image-minimal/1.0-r0/mender-sdimg.wks
2021-10-05 13:59:32 - INFO     - | + [ -n u-boot-sunxi-with-spl.bin ]
2021-10-05 13:59:32 - INFO     - | + install -m 0644 /builds/open-source/test-system/build/tmp/deploy/images/orange-pi-zero/u-boot-sunxi-with-spl.bin /builds/open-source/test-system/build/tmp/work/orange_pi_zero-poky-linux-gnueabi/core-image-minimal/1.0-r0/
2021-10-05 13:59:32 - INFO     - | + expr 16 % 2
2021-10-05 13:59:32 - INFO     - | + [ 0 -ne 0 ]
2021-10-05 13:59:32 - INFO     - | + bootloader_sector=16
2021-10-05 13:59:32 - INFO     - | + bootloader_file=/builds/open-source/test-system/build/tmp/work/orange_pi_zero-poky-linux-gnueabi/core-image-minimal/1.0-r0/u-boot-sunxi-with-spl.bin
2021-10-05 13:59:32 - INFO     - | + expr 16 * 512
2021-10-05 13:59:32 - INFO     - | + expr 8192 / 1024
2021-10-05 13:59:32 - INFO     - | + bootloader_align_kb=8
2021-10-05 13:59:32 - INFO     - | + stat -c %s /builds/open-source/test-system/build/tmp/work/orange_pi_zero-poky-linux-gnueabi/core-image-minimal/1.0-r0/u-boot-sunxi-with-spl.bin
2021-10-05 13:59:32 - INFO     - | + bootloader_size=454440
2021-10-05 13:59:32 - INFO     - | + expr 8 * 1024 + 454440
2021-10-05 13:59:32 - INFO     - | + bootloader_end=462632
2021-10-05 13:59:32 - INFO     - | + [ 462632 -gt 8388608 ]
2021-10-05 13:59:32 - INFO     - | + cat
2021-10-05 13:59:32 - INFO     - | + true
2021-10-05 13:59:32 - INFO     - | + [ -n 8388608 ]
2021-10-05 13:59:32 - INFO     - | + expr 8388608 / 1024
2021-10-05 13:59:32 - INFO     - | + boot_env_align_kb=8192
2021-10-05 13:59:32 - INFO     - | + cat
2021-10-05 13:59:32 - INFO     - | + expr 8388608 % 1024
2021-10-05 13:59:32 - INFO     - | + true
2021-10-05 13:59:32 - INFO     - | + [ 0 -ne 0 ]
2021-10-05 13:59:32 - INFO     - | + expr 8388608 / 1024
2021-10-05 13:59:32 - INFO     - | + alignment_kb=8192
2021-10-05 13:59:32 - INFO     - | + [ msdos = gpt ]
2021-10-05 13:59:32 - INFO     - | + part_type_params=
2021-10-05 13:59:32 - INFO     - | + echo boot.scr
2021-10-05 13:59:32 - INFO     - | + sed -r s/(^\s*)|(\s*$)//g
2021-10-05 13:59:32 - INFO     - | + IMAGE_BOOT_FILES_STRIPPED=boot.scr
2021-10-05 13:59:32 - INFO     - | + [ 16 -ne 0 ]
2021-10-05 13:59:32 - INFO     - | + mender_merge_bootfs_and_image_boot_files
2021-10-05 13:59:32 - INFO     - | + W=/builds/open-source/test-system/build/tmp/work/orange_pi_zero-poky-linux-gnueabi/core-image-minimal/1.0-r0/bootfs.image_sdimg
2021-10-05 13:59:32 - INFO     - | + rm -rf /builds/open-source/test-system/build/tmp/work/orange_pi_zero-poky-linux-gnueabi/core-image-minimal/1.0-r0/bootfs.image_sdimg
2021-10-05 13:59:32 - INFO     - | + cp -al /builds/open-source/test-system/build/tmp/work/orange_pi_zero-poky-linux-gnueabi/core-image-minimal/1.0-r0/rootfs//uboot /builds/open-source/test-system/build/tmp/work/orange_pi_zero-poky-linux-gnueabi/core-image-minimal/1.0-r0/bootfs.image_sdimg
2021-10-05 13:59:32 - INFO     - | + image_boot_files=boot.scr
2021-10-05 13:59:32 - INFO     - | + echo boot.scr
2021-10-05 13:59:32 - INFO     - | + grep -q ;
2021-10-05 13:59:32 - INFO     - | + dest=./
2021-10-05 13:59:32 - INFO     - | + echo ./
2021-10-05 13:59:32 - INFO     - | + grep -q /$
2021-10-05 13:59:32 - INFO     - | + dest_is_dir=1
2021-10-05 13:59:32 - INFO     - | + mkdir -p /builds/open-source/test-system/build/tmp/work/orange_pi_zero-poky-linux-gnueabi/core-image-minimal/1.0-r0/bootfs.image_sdimg/./
2021-10-05 13:59:32 - INFO     - | + [ 1 -eq 1 ]
2021-10-05 13:59:32 - INFO     - | + basename /builds/open-source/test-system/build/tmp/deploy/images/orange-pi-zero/boot.scr
2021-10-05 13:59:32 - INFO     - | + destfile=/builds/open-source/test-system/build/tmp/work/orange_pi_zero-poky-linux-gnueabi/core-image-minimal/1.0-r0/bootfs.image_sdimg/./boot.scr
2021-10-05 13:59:32 - INFO     - | + [ -e /builds/open-source/test-system/build/tmp/work/orange_pi_zero-poky-linux-gnueabi/core-image-minimal/1.0-r0/bootfs.image_sdimg/./boot.scr ]
2021-10-05 13:59:32 - INFO     - | + cp -l /builds/open-source/test-system/build/tmp/deploy/images/orange-pi-zero/boot.scr /builds/open-source/test-system/build/tmp/work/orange_pi_zero-poky-linux-gnueabi/core-image-minimal/1.0-r0/bootfs.image_sdimg/./boot.scr
2021-10-05 13:59:32 - INFO     - | + cat
2021-10-05 13:59:32 - INFO     - | + cat
2021-10-05 13:59:32 - INFO     - | + [ 512 -ne 0 ]
2021-10-05 13:59:32 - INFO     - | + cat
2021-10-05 13:59:32 - INFO     - | + cat
2021-10-05 13:59:32 - INFO     - | + cat
2021-10-05 13:59:32 - INFO     - | + cat
2021-10-05 13:59:32 - INFO     - | + echo ### Contents of wks file ###
2021-10-05 13:59:32 - INFO     - | ### Contents of wks file ###
2021-10-05 13:59:32 - INFO     - | + cat /builds/open-source/test-system/build/tmp/work/orange_pi_zero-poky-linux-gnueabi/core-image-minimal/1.0-r0/mender-sdimg.wks
2021-10-05 13:59:32 - INFO     - | # embed bootloader
2021-10-05 13:59:32 - INFO     - | part --source rawcopy --sourceparams="file=/builds/open-source/test-system/build/tmp/work/orange_pi_zero-poky-linux-gnueabi/core-image-minimal/1.0-r0/u-boot-sunxi-with-spl.bin" --ondisk "mmcblk0" --align 8 --no-table
2021-10-05 13:59:32 - INFO     - | part --source rawcopy --sourceparams="file=/builds/open-source/test-system/build/tmp/work/orange_pi_zero-poky-linux-gnueabi/core-image-minimal/1.0-r0/uboot.env" --ondisk "mmcblk0" --align 8192 --no-table
2021-10-05 13:59:32 - INFO     - | part --source rootfs --rootfs-dir /builds/open-source/test-system/build/tmp/work/orange_pi_zero-poky-linux-gnueabi/core-image-minimal/1.0-r0/bootfs.image_sdimg --ondisk "mmcblk0" --fstype=vfat --label boot --align 8192 --fixed-size 16 --active
2021-10-05 13:59:32 - INFO     - | part --source rawcopy --sourceparams="file=/builds/open-source/test-system/build/tmp/work/orange_pi_zero-poky-linux-gnueabi/core-image-minimal/1.0-r0/deploy-core-image-minimal-image-complete/core-image-minimal-orange-pi-zero.ext4" --ondisk "mmcblk0" --align 8192 --fixed-size 2957312k
2021-10-05 13:59:32 - INFO     - | part --source rawcopy --sourceparams="file=/builds/open-source/test-system/build/tmp/work/orange_pi_zero-poky-linux-gnueabi/core-image-minimal/1.0-r0/deploy-core-image-minimal-image-complete/core-image-minimal-orange-pi-zero.ext4" --ondisk "mmcblk0" --align 8192 --fixed-size 2957312k
2021-10-05 13:59:32 - INFO     - | part swap --ondisk "mmcblk0" --fstype=swap --label swap --align 8192 --size 512
2021-10-05 13:59:32 - INFO     - | part --source rawcopy --sourceparams="file=/builds/open-source/test-system/build/tmp/work/orange_pi_zero-poky-linux-gnueabi/core-image-minimal/1.0-r0/deploy-core-image-minimal-image-complete/core-image-minimal-orange-pi-zero.dataimg" --ondisk "mmcblk0" --align 8192 --fixed-size 1024
2021-10-05 13:59:32 - INFO     - |
2021-10-05 13:59:32 - INFO     - | bootloader --ptable msdos
2021-10-05 13:59:32 - INFO     - | + echo ### End of contents of wks file ###
2021-10-05 13:59:32 - INFO     - | ### End of contents of wks file ###
2021-10-05 13:59:32 - INFO     - | + outimgname=/builds/open-source/test-system/build/tmp/work/orange_pi_zero-poky-linux-gnueabi/core-image-minimal/1.0-r0/deploy-core-image-minimal-image-complete/core-image-minimal-orange-pi-zero-20211005113718.sdimg
2021-10-05 13:59:32 - INFO     - | + wicout=/builds/open-source/test-system/build/tmp/work/orange_pi_zero-poky-linux-gnueabi/core-image-minimal/1.0-r0/deploy-core-image-minimal-image-complete/core-image-minimal-orange-pi-zero-20211005113718-sdimg
2021-10-05 13:59:32 - INFO     - | + BUILDDIR=/builds/open-source/test-system/build wic create /builds/open-source/test-system/build/tmp/work/orange_pi_zero-poky-linux-gnueabi/core-image-minimal/1.0-r0/mender-sdimg.wks --vars /builds/open-source/test-system/build/tmp/sysroots/orange-pi-zero/imgdata/ -e core-image-minimal -o /builds/open-source/test-system/build/tmp/work/orange_pi_zero-poky-linux-gnueabi/core-image-minimal/1.0-r0/deploy-core-image-minimal-image-complete/core-image-minimal-orange-pi-zero-20211005113718-sdimg/
2021-10-05 13:59:32 - INFO     - | INFO: Creating image(s)...
2021-10-05 13:59:32 - INFO     - |
2021-10-05 13:59:32 - INFO     - | Traceback (most recent call last):
2021-10-05 13:59:32 - INFO     - |   File "/builds/open-source/test-system/poky/scripts/wic", line 542, in <module>
2021-10-05 13:59:32 - INFO     - |     sys.exit(main(sys.argv[1:]))
2021-10-05 13:59:32 - INFO     - |   File "/builds/open-source/test-system/poky/scripts/wic", line 537, in main
2021-10-05 13:59:32 - INFO     - |     return hlp.invoke_subcommand(args, parser, hlp.wic_help_usage, subcommands)
2021-10-05 13:59:32 - INFO     - |   File "/builds/open-source/test-system/poky/scripts/lib/wic/help.py", line 83, in invoke_subcommand
2021-10-05 13:59:32 - INFO     - |     subcmd[0](args, usage)
2021-10-05 13:59:32 - INFO     - |   File "/builds/open-source/test-system/poky/scripts/wic", line 219, in wic_create_subcommand
2021-10-05 13:59:32 - INFO     - |     engine.wic_create(wks_file, rootfs_dir, bootimg_dir, kernel_dir,
2021-10-05 13:59:32 - INFO     - |   File "/builds/open-source/test-system/poky/scripts/lib/wic/engine.py", line 190, in wic_create
2021-10-05 13:59:32 - INFO     - |     plugin.do_create()
2021-10-05 13:59:32 - INFO     - |   File "/builds/open-source/test-system/poky/scripts/lib/wic/plugins/imager/direct.py", line 96, in do_create
2021-10-05 13:59:32 - INFO     - |     self.create()
2021-10-05 13:59:32 - INFO     - |   File "/builds/open-source/test-system/poky/scripts/lib/wic/plugins/imager/direct.py", line 180, in create
2021-10-05 13:59:32 - INFO     - |     self._image.prepare(self)
2021-10-05 13:59:32 - INFO     - |   File "/builds/open-source/test-system/poky/scripts/lib/wic/plugins/imager/direct.py", line 354, in prepare
2021-10-05 13:59:32 - INFO     - |     part.prepare(imager, imager.workdir, imager.oe_builddir,
2021-10-05 13:59:32 - INFO     - |   File "/builds/open-source/test-system/poky/scripts/lib/wic/partition.py", line 182, in prepare
2021-10-05 13:59:32 - INFO     - |     plugin.do_prepare_partition(self, srcparams_dict, creator,
2021-10-05 13:59:32 - INFO     - |   File "/builds/open-source/test-system/poky/scripts/lib/wic/plugins/source/rootfs.py", line 112, in do_prepare_partition
2021-10-05 13:59:32 - INFO     - |     copyhardlinktree(orig_dir, new_rootfs)
2021-10-05 13:59:32 - INFO     - |   File "/builds/open-source/test-system/poky/meta/lib/oe/path.py", line 121, in copyhardlinktree
2021-10-05 13:59:32 - INFO     - |     subprocess.check_output(cmd, shell=True, stderr=subprocess.STDOUT)
2021-10-05 13:59:32 - INFO     - |   File "/builds/open-source/test-system/build/tmp/work/orange_pi_zero-poky-linux-gnueabi/core-image-minimal/1.0-r0/recipe-sysroot-native/usr/lib/python3.8/subprocess.py", line 415, in check_output
2021-10-05 13:59:32 - INFO     - |     return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
2021-10-05 13:59:32 - INFO     - |   File "/builds/open-source/test-system/build/tmp/work/orange_pi_zero-poky-linux-gnueabi/core-image-minimal/1.0-r0/recipe-sysroot-native/usr/lib/python3.8/subprocess.py", line 516, in run
2021-10-05 13:59:32 - INFO     - |     raise CalledProcessError(retcode, process.args,
2021-10-05 13:59:32 - INFO     - | subprocess.CalledProcessError: Command 'cd /builds/open-source/test-system/build/tmp/work/orange_pi_zero-poky-linux-gnueabi/core-image-minimal/1.0-r0/bootfs.image_sdimg; find . -type d -print | tar --xattrs --xattrs-include='*' -cf - -S -C /builds/open-source/test-system/build/tmp/work/orange_pi_zero-poky-linux-gnueabi/core-image-minimal/1.0-r0/bootfs.image_sdimg -p --no-recursion --files-from - | tar --xattrs --xattrs-include='*' -xhf - -C /builds/open-source/test-system/build/tmp/work/orange_pi_zero-poky-linux-gnueabi/core-image-minimal/1.0-r0/deploy-core-image-minimal-image-complete/core-image-minimal-orange-pi-zero-20211005113718-sdimg/tmp.wic.1kpqjyfn/rootfs3' returned non-zero exit status 2.
2021-10-05 13:59:32 - INFO     - | + bb_exit_handler
2021-10-05 13:59:32 - INFO     - | + ret=1
2021-10-05 13:59:32 - INFO     - | + echo WARNING: exit code 1 from a shell command.
2021-10-05 13:59:32 - INFO     - | WARNING: exit code 1 from a shell command.
2021-10-05 13:59:32 - INFO     - | + exit 1
2021-10-05 13:59:32 - INFO     - | ERROR: Execution of '/builds/open-source/test-system/build/tmp/work/orange_pi_zero-poky-linux-gnueabi/core-image-minimal/1.0-r0/temp/run.do_image_sdimg.3162571' failed with exit code 1
2021-10-05 13:59:32 - INFO     - NOTE: recipe core-image-minimal-1.0-r0: task do_image_sdimg: Failed```

kacf · October 15, 2021, 7:46am

Looks like it’s hiding the output due to using check_output. Perhaps you can change it to check_call to see what it is.

Note that this could cause it to misbehave later if it needs the output from check_output, but at least you should be able to see whatever message it’s printing.

MarekBelisko · October 27, 2021, 6:19am

@kacf it doesn’t help either ;). But when I run command which is printed above manually it fails with:

Cannot open: No such file or directory
tar: Error is not recoverable: exiting now

and then return code was 2 as stated in script. Seems I need to dig more to wic creation and debug it somehow

I remember enable swap was working fine before so it could be dunfell regression? Thanks

MarekBelisko · October 27, 2021, 11:13am

When run wic create manually image is created :), this is really strange. Also strange is that it’s during creating of : bootfs.image_sdimg ?

MarekBelisko · November 22, 2021, 3:40pm

@kacf after 2 days of poky debugging I found solution/workaround. Seems there is something funky in wic which want to copy fstab to rootfs but we use rootfs type for bootloader partition and this then confuse it and fails with above issue. Without using swap updating fstab is not an option (checked variables) and everything went smooth.

Fix which works for me is below:

index 7e1c1c0..d37ecee 100644
--- a/scripts/lib/wic/plugins/imager/direct.py
+++ b/scripts/lib/wic/plugins/imager/direct.py
@@ -115,7 +115,7 @@ class DirectPlugin(ImagerPlugin):
         updated = False
         for part in self.parts:
             if not part.realnum or not part.mountpoint \
-               or part.mountpoint == "/":
+               or part.mountpoint == "/" or part.mountpoint == "swap":
                 continue
 
             if part.use_uuid:

I’m not wic expert but with this change everything back to normal. I’m curious nobody is using swap on dunfell relase branch? Thanks.

kacf · November 23, 2021, 9:24am

Nice investigation, @MarekBelisko! That is indeed surprising. AFAIK, we are not using swap in any of our testing pipelines, which explains why we have not seen it.

I trust you will be submitting this to upstream poky?

MarekBelisko · November 23, 2021, 9:52am

Yes I did but dunno if it will take any feedback yocto@lists.yoctoproject.org | wic crash when building image with multiple partitions

kacf · November 24, 2021, 7:16am

You might also have more luck with openembedded-core@lists.openembedded.org, since that’s where wic originally comes from, AFAIK.

Topic		Replies	Views
Issue on Dunfell (/etc/fstab invalid entry) - fail to boot General Discussions yocto , x86-grub	3	1148	December 16, 2020
Bumping project from thud to dunfell issue with MENDER_STORAGE_TOTAL_SIZE_MB General Discussions	6	354	March 9, 2021
Running on x86 General Discussions yocto	10	959	October 6, 2021
ERROR: Actual rootfs size (21223 kB) is larger than allowed size 16384 kB General Discussions	3	1627	April 28, 2022
Dunfell hddimg support General Discussions yocto	16	1778	March 24, 2021

Adding MENDER_SWAP_PART_SIZE_MB="512" build failure

Related Topics