# Data folder present in mender artifact rootfs update

**URL:** <https://hub.mender.io/t/data-folder-present-in-mender-artifact-rootfs-update/1329>\
**Category:** General Discussions\
**Created:** [December 5, 2019, 9:07pm UTC](https://hub.mender.io/t/data-folder-present-in-mender-artifact-rootfs-update/1329 "2019-12-05T21:07:03Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![dellgreen](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/dellgreen/32/85_2.png) [@dellgreen](https://hub.mender.io/u/dellgreen)\
**Post date:** [December 5, 2019, 9:07pm UTC](https://hub.mender.io/t/data-folder-present-in-mender-artifact-rootfs-update/1329/1 "2019-12-05T21:07:03Z")

</div>

I have just noticed using mender sumo branch in yocto, that I am seeing the contents of the data partition in the mender rootfs artifact. However the sdimg looks correct with nothing in the /data folder in the A/B rootfs partitions and the content in the data partition.

Is there something in particular I should be doing to make the content of data partition not appear in the rootfs mender artifact?. I thought with the addition of the IMAGE\_ROOTFS\_EXCLUDE\_PATH code this shouldn’t happen?

---

<div class="post-metadata">

**Author:** ![mirzak](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/mirzak/32/2056_2.png) [@mirzak](https://hub.mender.io/u/mirzak)\
**Post date:** [December 5, 2019, 9:14pm UTC](https://hub.mender.io/t/data-folder-present-in-mender-artifact-rootfs-update/1329/2 "2019-12-05T21:14:39Z")

</div>

This thread might provide some additional insights on this:

> [@Large file in /data generate do\_image\_mender error](https://hub.mender.io/t/large-file-in-data-generate-do-image-mender-error/979):
>
> Hi, I’m using yocto (thud) using the meta-mender (thud commit e4d0ca3928523ba2a57d91b75fe758f08ad6d0f5) I’m trying to integrate a large file(~8GB) in the /data partition but I have a size error at the do\_image\_mender step. See below for the error. ERROR: installer-image-1.0-r0 do\_image\_mender: Size of rootfs is greater than the calculated partition space (10356370432 \> 3489660928). This image won't fit on a device with the current storage configuration. Try reducing IMAGE\_OVERHEAD\_FACTOR …

I think you are experiencing the same problems, though you might not have an large files installed to /data so are you getting build errors.

This has been fixed and backported to `thud` only, do not think that this was back ported to sumo.

---

<div class="post-metadata">

**Author:** ![dellgreen](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/dellgreen/32/85_2.png) [@dellgreen](https://hub.mender.io/u/dellgreen)\
**Post date:** [December 5, 2019, 9:28pm UTC](https://hub.mender.io/t/data-folder-present-in-mender-artifact-rootfs-update/1329/3 "2019-12-05T21:28:38Z")

</div>

no build errors, at all, just see device\_type in mender folder, which i guess the mender recipe is installing, and fw\_env.config in the uboot folder which i guess is also coming from the mender recipe/uboot patches.

I will take a look at your suggestion, many thanks. I also need to bring the various sumo layers up to date, so maybe that will help

---

<div class="post-metadata">

**Author:** ![dellgreen](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/dellgreen/32/85_2.png) [@dellgreen](https://hub.mender.io/u/dellgreen)\
**Post date:** [December 5, 2019, 9:29pm UTC](https://hub.mender.io/t/data-folder-present-in-mender-artifact-rootfs-update/1329/4 "2019-12-05T21:29:56Z")

</div>

the problem seems to be that the /data partiton files are making their way into the ext4 image which is used by mender-artifact

---

<div class="post-metadata">

**Author:** ![dellgreen](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/dellgreen/32/85_2.png) [@dellgreen](https://hub.mender.io/u/dellgreen)\
**Post date:** [December 5, 2019, 9:35pm UTC](https://hub.mender.io/t/data-folder-present-in-mender-artifact-rootfs-update/1329/5 "2019-12-05T21:35:08Z")

</div>

your link does look very much like the problem, I’ll take a look through it tommorrow and see if i can patch it into my sumo branch.

---

<div class="post-metadata">

**Author:** ![dellgreen](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/dellgreen/32/85_2.png) [@dellgreen](https://hub.mender.io/u/dellgreen)\
**Post date:** [December 6, 2019, 2:12pm UTC](https://hub.mender.io/t/data-folder-present-in-mender-artifact-rootfs-update/1329/6 "2019-12-06T14:12:56Z")

</div>

Ah-hah, found the problem. @mirzak you link was very helpful.  
Adding back in ext4 to IMAGE\_FSTYPES fixed the problem, many thanks

---

<div class="post-metadata">

**Author:** ![mirzak](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/mirzak/32/2056_2.png) [@mirzak](https://hub.mender.io/u/mirzak)\
**Post date:** [December 6, 2019, 2:15pm UTC](https://hub.mender.io/t/data-folder-present-in-mender-artifact-rootfs-update/1329/7 "2019-12-06T14:15:42Z")

</div>

Great to hear. Thank you reporting back. I will mark your report as “solution”

---

<div class="post-metadata">

**Author:** ![dellgreen](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/dellgreen/32/85_2.png) [@dellgreen](https://hub.mender.io/u/dellgreen)\
**Post date:** [December 9, 2019, 1:15pm UTC](https://hub.mender.io/t/data-folder-present-in-mender-artifact-rootfs-update/1329/8 "2019-12-09T13:15:27Z")

</div>

just noticed that whilst adding ext4 into IMAGE\_FSTYPES fixes the ext4, the tar and tar.xz still have the problem even though these are contained in IMAGE\_FSTYPES.

---

<div class="post-metadata">

**Author:** ![dellgreen](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/dellgreen/32/85_2.png) [@dellgreen](https://hub.mender.io/u/dellgreen)\
**Post date:** [December 9, 2019, 1:52pm UTC](https://hub.mender.io/t/data-folder-present-in-mender-artifact-rootfs-update/1329/9 "2019-12-09T13:52:19Z")

</div>

currently re-building a clean build, incase something dirty in the build is causing the tar and tar.xz to not been cleaned of data directory. I can see in the do\_image\_tar logs that the functions prepare\_excluded\_directories and cleanup\_excluded\_directories are being executed

---

<div class="post-metadata">

**Author:** ![dellgreen](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/dellgreen/32/85_2.png) [@dellgreen](https://hub.mender.io/u/dellgreen)\
**Post date:** [December 9, 2019, 3:39pm UTC](https://hub.mender.io/t/data-folder-present-in-mender-artifact-rootfs-update/1329/10 "2019-12-09T15:39:15Z")

</div>

clean build didn’t fix it. I can see the mender tmp folders for rootfs tar in WORKDIR which doesn’t have the data directory, however the final tar contains the data directory.

---

<div class="post-metadata">

**Author:** ![drewmoseley](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/drewmoseley/32/47_2.png) [@drewmoseley](https://hub.mender.io/u/drewmoseley)\
**Post date:** [December 9, 2019, 3:41pm UTC](https://hub.mender.io/t/data-folder-present-in-mender-artifact-rootfs-update/1329/11 "2019-12-09T15:41:28Z")

</div>

Hi @dellgreen I can confirm. I had a build locally and see the same behavior.

@mirzak any idea how to apply the excludes to the tarballs?

---

<div class="post-metadata">

**Author:** ![dellgreen](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/dellgreen/32/85_2.png) [@dellgreen](https://hub.mender.io/u/dellgreen)\
**Post date:** [December 9, 2019, 3:58pm UTC](https://hub.mender.io/t/data-folder-present-in-mender-artifact-rootfs-update/1329/12 "2019-12-09T15:58:32Z")

</div>

ok cool, thought I was going crazy for a minute.

when i dump the command being used for IMAGE\_CMD\_tar it points to build/tmp/debug/work/ty\_k\_turn\_2\_arm\_varsommx6-poky-linux-gnueabi/turn-core-image-minimal/1.0-r0/rootfs and not the mender temporary rootfs folder, so this may explain whe we can see the prepare\_excluded\_directories executing, but the tar command is using the original rootfs folder. variable expansion issue? race/dependency/ task evaluation order issue?

---

<div class="post-metadata">

**Author:** ![dellgreen](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/dellgreen/32/85_2.png) [@dellgreen](https://hub.mender.io/u/dellgreen)\
**Post date:** [December 9, 2019, 4:01pm UTC](https://hub.mender.io/t/data-folder-present-in-mender-artifact-rootfs-update/1329/13 "2019-12-09T16:01:50Z")

</div>

more log.task\_order  
do\_prepare\_recipe\_sysroot (24269): log.do\_prepare\_recipe\_sysroot.24269  
do\_populate\_lic (10629): log.do\_populate\_lic.10629  
do\_rootfs (10280): log.do\_rootfs.10280  
do\_image\_qa (27025): log.do\_image\_qa.27025  
do\_image (27030): log.do\_image.27030  
do\_image\_iw\_imageupdate\_ext4 (27053): log.do\_image\_iw\_imageupdate\_ext4.27053  
do\_image\_ext4 (27054): log.do\_image\_ext4.27054  
do\_rootfs\_wicenv (27055): log.do\_rootfs\_wicenv.27055  
do\_image\_tar (27060): log.do\_image\_tar.27060  
do\_image\_sdimg (27081): log.do\_image\_sdimg.27081  
do\_image\_mender (27260): log.do\_image\_mender.27260  
do\_image\_complete (28013): log.do\_image\_complete.28013

---

<div class="post-metadata">

**Author:** ![mirzak](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/mirzak/32/2056_2.png) [@mirzak](https://hub.mender.io/u/mirzak)\
**Post date:** [December 9, 2019, 4:04pm UTC](https://hub.mender.io/t/data-folder-present-in-mender-artifact-rootfs-update/1329/14 "2019-12-09T16:04:16Z")

</div>

To fix this “fully”, there needs to be some backports,

> <https://github.com/mendersoftware/meta-mender/pull/824>

To try to explain the problem again, if you have,

```auto
IMAGE_FSTYPES = "tar.bz2"

```

The pre/post hooks will be installed on `do_image_tar.bz2` task, but there is no such task and you want the hooks on `do_image_tar` as the `bz2` is just a “conversion” image.

So to get around this problem you would set:

```auto
IMAGE_FSTYPES = "tar tar.bz2"

```

This way you would make sure that the pre/post hooks are applied to `do_image_tar` and `/data` would be cleaned up properly. This applies to all “conversion” images an the linked fix should take care of this.

---

<div class="post-metadata">

**Author:** ![dellgreen](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/dellgreen/32/85_2.png) [@dellgreen](https://hub.mender.io/u/dellgreen)\
**Post date:** [December 9, 2019, 4:13pm UTC](https://hub.mender.io/t/data-folder-present-in-mender-artifact-rootfs-update/1329/15 "2019-12-09T16:13:58Z")

</div>

The pre/post hooks are applied to `do_image_tar` for me and are executing, however the resulting tar file has the original rootfs content not the temporary mender created rootfs with data removed

I have IMAGE\_FSTYPES += “ext4 tar”

---

<div class="post-metadata">

**Author:** ![dellgreen](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/dellgreen/32/85_2.png) [@dellgreen](https://hub.mender.io/u/dellgreen)\
**Post date:** [December 9, 2019, 4:14pm UTC](https://hub.mender.io/t/data-folder-present-in-mender-artifact-rootfs-update/1329/16 "2019-12-09T16:14:15Z")

</div>

ext4 works as expected, tar doesnt

---

<div class="post-metadata">

**Author:** ![drewmoseley](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/drewmoseley/32/47_2.png) [@drewmoseley](https://hub.mender.io/u/drewmoseley)\
**Post date:** [December 9, 2019, 4:17pm UTC](https://hub.mender.io/t/data-folder-present-in-mender-artifact-rootfs-update/1329/17 "2019-12-09T16:17:00Z")

</div>

Could it be that IMAGE\_CMD\_tar is a variable and not a function so the change we make to IMAGE\_ROOTFS is ignored since the variable was already expanded before our pre-function executed?

---

<div class="post-metadata">

**Author:** ![dellgreen](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/dellgreen/32/85_2.png) [@dellgreen](https://hub.mender.io/u/dellgreen)\
**Post date:** [December 9, 2019, 4:18pm UTC](https://hub.mender.io/t/data-folder-present-in-mender-artifact-rootfs-update/1329/18 "2019-12-09T16:18:12Z")

</div>

I dont know anything about ‘prefuncs’, but could this be executing after the tar command is evaluating ${  
IMAGE\_ROOTFS}

as in image\_types.bbclass it does this:

```
IMAGE_CMD_tar = "${IMAGE_CMD_TAR} --numeric-owner -cf ${IMGDEPLOYDIR}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.tar -C ${IMAGE_ROOTFS} . || [$? -eq 1]"
```

---

<div class="post-metadata">

**Author:** ![drewmoseley](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/drewmoseley/32/47_2.png) [@drewmoseley](https://hub.mender.io/u/drewmoseley)\
**Post date:** [December 9, 2019, 4:26pm UTC](https://hub.mender.io/t/data-folder-present-in-mender-artifact-rootfs-update/1329/19 "2019-12-09T16:26:45Z")

</div>

Yeah, that feels like what is happening. I’m playing with a change to the prepare\_excluded and cleanup\_excluded functions so that changing the value of IMAGE\_ROOTFS is not necessary. Basically it will back up the original in prepare and restore it in cleanup.

---

<div class="post-metadata">

**Author:** ![dellgreen](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/dellgreen/32/85_2.png) [@dellgreen](https://hub.mender.io/u/dellgreen)\
**Post date:** [December 9, 2019, 4:30pm UTC](https://hub.mender.io/t/data-folder-present-in-mender-artifact-rootfs-update/1329/20 "2019-12-09T16:30:11Z")

</div>

ok great, sounds good, many thanks

[Next page](https://hub.mender.io/t/data-folder-present-in-mender-artifact-rootfs-update/1329.md?page=2)