I have just noticed using mender sumo branch in yocto, that I am seeing the contents of the data partition in the mender rootfs artifact. However the sdimg looks correct with nothing in the /data folder in the A/B rootfs partitions and the content in the data partition.
Is there something in particular I should be doing to make the content of data partition not appear in the rootfs mender artifact?. I thought with the addition of the IMAGE_ROOTFS_EXCLUDE_PATH code this shouldn’t happen?
This thread might provide some additional insights on this:
I think you are experiencing the same problems, though you might not have an large files installed to /data so are you getting build errors.
This has been fixed and backported to thud only, do not think that this was back ported to sumo.
no build errors, at all, just see device_type in mender folder, which i guess the mender recipe is installing, and fw_env.config in the uboot folder which i guess is also coming from the mender recipe/uboot patches.
I will take a look at your suggestion, many thanks. I also need to bring the various sumo layers up to date, so maybe that will help
the problem seems to be that the /data partiton files are making their way into the ext4 image which is used by mender-artifact
your link does look very much like the problem, I’ll take a look through it tommorrow and see if i can patch it into my sumo branch.
Ah-hah, found the problem. @mirzak you link was very helpful.
Adding back in ext4 to IMAGE_FSTYPES fixed the problem, many thanks
Great to hear. Thank you reporting back. I will mark your report as “solution”
just noticed that whilst adding ext4 into IMAGE_FSTYPES fixes the ext4, the tar and tar.xz still have the problem even though these are contained in IMAGE_FSTYPES.
currently re-building a clean build, incase something dirty in the build is causing the tar and tar.xz to not been cleaned of data directory. I can see in the do_image_tar logs that the functions prepare_excluded_directories and cleanup_excluded_directories are being executed
clean build didn’t fix it. I can see the mender tmp folders for rootfs tar in WORKDIR which doesn’t have the data directory, however the final tar contains the data directory.
Hi @dellgreen I can confirm. I had a build locally and see the same behavior.
@mirzak any idea how to apply the excludes to the tarballs?
ok cool, thought I was going crazy for a minute.
when i dump the command being used for IMAGE_CMD_tar it points to build/tmp/debug/work/ty_k_turn_2_arm_varsommx6-poky-linux-gnueabi/turn-core-image-minimal/1.0-r0/rootfs and not the mender temporary rootfs folder, so this may explain whe we can see the prepare_excluded_directories executing, but the tar command is using the original rootfs folder. variable expansion issue? race/dependency/ task evaluation order issue?
more log.task_order
do_prepare_recipe_sysroot (24269): log.do_prepare_recipe_sysroot.24269
do_populate_lic (10629): log.do_populate_lic.10629
do_rootfs (10280): log.do_rootfs.10280
do_image_qa (27025): log.do_image_qa.27025
do_image (27030): log.do_image.27030
do_image_iw_imageupdate_ext4 (27053): log.do_image_iw_imageupdate_ext4.27053
do_image_ext4 (27054): log.do_image_ext4.27054
do_rootfs_wicenv (27055): log.do_rootfs_wicenv.27055
do_image_tar (27060): log.do_image_tar.27060
do_image_sdimg (27081): log.do_image_sdimg.27081
do_image_mender (27260): log.do_image_mender.27260
do_image_complete (28013): log.do_image_complete.28013
To fix this “fully”, there needs to be some backports,
To try to explain the problem again, if you have,
IMAGE_FSTYPES = "tar.bz2"
The pre/post hooks will be installed on do_image_tar.bz2 task, but there is no such task and you want the hooks on do_image_tar as the bz2 is just a “conversion” image.
So to get around this problem you would set:
IMAGE_FSTYPES = "tar tar.bz2"
This way you would make sure that the pre/post hooks are applied to do_image_tar and /data would be cleaned up properly. This applies to all “conversion” images an the linked fix should take care of this.
The pre/post hooks are applied to do_image_tar for me and are executing, however the resulting tar file has the original rootfs content not the temporary mender created rootfs with data removed
I have IMAGE_FSTYPES += “ext4 tar”
ext4 works as expected, tar doesnt
Could it be that IMAGE_CMD_tar is a variable and not a function so the change we make to IMAGE_ROOTFS is ignored since the variable was already expanded before our pre-function executed?
I dont know anything about ‘prefuncs’, but could this be executing after the tar command is evaluating ${
IMAGE_ROOTFS}
as in image_types.bbclass it does this:
IMAGE_CMD_tar = "${IMAGE_CMD_TAR} --numeric-owner -cf ${IMGDEPLOYDIR}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.tar -C ${IMAGE_ROOTFS} . || [ $? -eq 1 ]"Yeah, that feels like what is happening. I’m playing with a change to the prepare_excluded and cleanup_excluded functions so that changing the value of IMAGE_ROOTFS is not necessary. Basically it will back up the original in prepare and restore it in cleanup.
ok great, sounds good, many thanks