Large file in /data generate do_image_mender error

Hi,

I’m using yocto (thud) using the meta-mender (thud commit e4d0ca3928523ba2a57d91b75fe758f08ad6d0f5)

I’m trying to integrate a large file(~8GB) in the /data partition but I have a size error at the do_image_mender step. See below for the error.

ERROR: installer-image-1.0-r0 do_image_mender: Size of rootfs is greater than the calculated     partition space (10356370432 > 3489660928). This image won't fit on a device with the current storage configuration. Try reducing IMAGE_OVERHEAD_FACTOR if it is higher than 1.0, or raise MENDER_STORAGE_TOTAL_SIZE_MB if the device in fact has more storage.ERROR: installer-image-1.0-r0 do_image_mender: Function failed: do_image_mender (log file is located at /workdir/build/tmp/work/mmwe-effenco-linux-gnueabi/installer-image/1.0-r0/temp/log.do_image_mender.130)
ERROR: Logfile of failure stored in: /workdir/build/tmp/work/mmwe-effenco-linux-gnueabi/installer-image/1.0-r0/temp/log.do_image_mender.130ERROR: Task (/workdir/build/../sources/meta-effenco/images/installer-image.bb:do_image_mender) failed with exit code '1'

It seems that the rootfs size still contain the /data folder.

On line 25 of https://github.com/mendersoftware/meta-mender/blob/e4d0ca3928523ba2a57d91b75fe758f08ad6d0f5/meta-mender-core/classes/mender-artifactimg.bbclass we have:

rootfs_size=$(stat -Lc %s ${IMGDEPLOYDIR}/${IMAGE_LINK_NAME}.${ARTIFACTIMG_FSTYPE})

It looks like it’s pointing to the rootfs that wasn’t strip from /data? Could it be a configuration issue?

I also set the following configuration in my distro conf file

# Mender configuration
MENDER_ARTIFACT_NAME = "${DISTRO_NAME}-${DISTRO_VERSION}"
ARTIFACTIMG_FSTYPE = "ext4"

# This will make sure that our SPL + u-boot.img is embedded in the sdimg at
# a 1kb offset.
MENDER_IMAGE_BOOTLOADER_FILE = "u-boot-spl.img"
MENDER_IMAGE_BOOTLOADER_BOOTSECTOR_OFFSET = "2"

# Remove the unneeded VFAT boot partition
IMAGE_BOOT_FILES = ""
MENDER_BOOT_PART_SIZE_MB = "0"

# Standard partition alignment from upstream wks file
MENDER_PARTITION_ALIGNMENT = "4194304"

MENDER_STORAGE_DEVICE = "/dev/mmcblk1"
MENDER_UBOOT_STORAGE_DEVICE = "0"

# Memory card with 16GiB of storage.
MENDER_STORAGE_TOTAL_SIZE_MB = "14305"
MENDER_DATA_PART_SIZE_MB = "7629"

MENDER_FEATURES_ENABLE_append = " mender-image mender-install mender-image-sd mender-uboot "
MENDER_FEATURES_DISABLE_append = " mender-grub mender-image-uefi"

# Fix size due to sdimg added to rootfs
IMAGE_ROOTFS_MAXSIZE = "12000000"

Thanks,

Hm, I believe that this might be a bug but unclear how to resolve it though.

We do remove the /data content when creating sdimg, but this class generates a separate ext4 file. Though looking at the master branch this no longer seems to be the case due to this change,

@kacf might some additional insights on this.

I just tried to cherry pick the previous commit and still ends up with the size error. See log below.

Would it be a good strategy to take the rootfs folder that was stripped from /data and create a {IMGDEPLOYDIR}/<NEW_ROOTFS_WITHOUT_DATA>.{ARTIFACTIMG_FSTYPE} and use that rootfs to check the size?

ERROR: installer-image-1.0-r0 do_image_mender: Size of rootfs is greater than the calculated partition space (10357413888 > 3489660928). This image won't fit on a device with the current storage configuration. Try reducing IMAGE_OVERHEAD_FACTOR if it is higher than 1.0, or raise MENDER_STORAGE_TOTAL_SIZE_MB if the device in fact has more storage.ERROR: installer-image-1.0-r0 do_image_mender: Function failed: do_image_mender (log file is located at /workdir/build/tmp/work/linux-gnueabi/installer-image/1.0-r0/temp/log.do_image_mender.205)
ERROR: Logfile of failure stored in: /workdir/build/tmp/work/linux-gnueabi/installer-image/1.0-r0/temp/log.do_image_mender.205 
NOTE: recipe installer-image-1.0-r0: task do_image_mender: Failed 
ERROR: Task (/workdir/build/../sources/meta-/images/installer-image.bb:do_image_mender) failed with exit code '1' 
ERROR: installer-image-1.0-r0 do_image_sdimg: Function failed: do_image_sdimg (log file is located at /workdir/build/tmp/work/linux-gnueabi/installer-image/1.0-r0/temp/log.do_image_sdimg.206)
ERROR: Logfile of failure stored in: /workdir/build/tmp/work/linux-gnueabi/installer-image/1.0-r0/temp/log.do_image_sdimg.206 
Log data follows: 
| DEBUG: Executing python function prepare_excluded_directories
| DEBUG: 'IMAGE_ROOTFS_EXCLUDE_PATH' is set but 'respect_exclude_path' variable flag is 0 for this image type, so ignoring it
| DEBUG: Python function prepare_excluded_directories finished
| DEBUG: Executing python function set_image_size
| DEBUG: 10012262.000000 = 7701740 * 1.300000
| DEBUG: 10114662.000000 = max(10012262.000000, 3305472)[10012262.000000] + 1
| DEBUG: 10114662.000000 = int(10114662.000000)
| DEBUG: 10114662 = aligned(10114662)
| DEBUG: returning 10114662
| DEBUG: Python function set_image_size finished
| DEBUG: Executing python function extend_recipe_sysroot
| NOTE: Direct dependencies are ['/workdir/build/../sources/poky/meta/recipes-core/glibc/glibc_2.28.bb:do_populate_sysroot', 'virtual:native:/workdir/build/../sources/poky/meta/recipes-extended/pigz/pigz_2.4.bb:do_populate_sysroot', '/workdir/build/../sources/poky/meta/recipes-kernel/kmod/depmodwrapper-cross_1.0.bb:do_populate_sysroot', '/workdir/build/../sources/poky/meta/recipes-devtools/python/python-native_2.7.15.bb:do_populate_sysroot', 'virtual:native:/workdir/build/../sources/poky/meta/recipes-devtools/opkg/opkg_0.3.6.bb:do_populate_sysroot', '/workdir/build/../sources/poky/meta/recipes-devtools/mklibs/mklibs-native_0.1.43.bb:do_populate_sysroot', 'virtual:native:/workdir/build/../sources/poky/meta/recipes-devtools/mtools/mtools_4.0.18.bb:do_populate_sysroot', 'virtual:native:/workdir/build/../sources/poky/meta/recipes-extended/parted/parted_3.2.bb:do_populate_sysroot', 'virtual:native:/workdir/build/../sources/poky/meta/recipes-devtools/dnf/dnf_2.7.5.bb:do_populate_sysroot', '/workdir/build/../sources/poky/meta/recipes-devtools/gcc/gcc-cross_8.2.bb:do_populate_sysroot', 'virtual:native:/workdir/build/../sources/poky/meta/recipes-core/update-rc.d/update-rc.d_0.8.bb:do_populate_sysroot', 'virtual:native:/workdir/build/../sources/poky/meta/recipes-devtools/e2fsprogs/e2fsprogs_1.44.3.bb:do_populate_sysroot', 'virtual:native:/workdir/build/../sources/poky/meta/recipes-devtools/dosfstools/dosfstools_4.1.bb:do_populate_sysroot', 'virtual:native:/workdir/build/../sources/poky/meta/recipes-devtools/swig/swig_3.0.12.bb:do_populate_sysroot', 'virtual:native:/workdir/build/../sources/poky/meta/recipes-devtools/opkg-utils/opkg-utils_0.3.6.bb:do_populate_sysroot', 'virtual:native:/workdir/build/../sources/poky/meta/recipes-devtools/fdisk/gptfdisk_1.0.4.bb:do_populate_sysroot', 'virtual:native:/workdir/build/../sources/poky/meta/recipes-devtools/createrepo-c/createrepo-c_git.bb:do_populate_sysroot', '/workdir/build/../sources/poky/meta/recipes-core/meta/wic-tools.bb:do_populate_sysroot', '/workdir/build/../sources/poky/meta/recipes-devtools/qemu/qemuwrapper-cross_1.0.bb:do_populate_sysroot', '/workdir/build/../sources/poky/meta/recipes-devtools/quilt/quilt-native_0.65.bb:do_populate_sysroot', 'virtual:native:/workdir/build/../sources/poky/meta/recipes-devtools/prelink/prelink_git.bb:do_populate_sysroot', 'virtual:native:/workdir/build/../sources/poky/meta/recipes-extended/bc/bc_1.07.1.bb:do_populate_sysroot', 'virtual:native:/workdir/build/../sources/poky/meta/recipes-connectivity/openssl/openssl_1.1.1a.bb:do_populate_sysroot', '/workdir/build/../sources/poky/meta/recipes-devtools/gcc/gcc-runtime_8.2.bb:do_populate_sysroot', 'virtual:native:/workdir/build/../sources/poky/meta/recipes-devtools/makedevs/makedevs_1.0.1.bb:do_populate_sysroot', 
'/workdir/build/../sources/poky/meta/recipes-core/glibc/cross-localedef-native_2.28.bb:do_populate_sysroot', 'virtual:native:/workdir/build/../sources/poky/meta/recipes-core/coreutils/coreutils_8.30.bb:do_populate_sysroot', '/workdir/build/../sources/poky/meta/recipes-core/glibc/ldconfig-native_2.12.1.bb:do_populate_sysroot', 'virtual:native:/workdir/build/../sources/poky/meta/recipes-devtools/rpm/rpm_4.14.2.bb:do_populate_sysroot', 'virtual:native:/workdir/build/../sources/poky/meta/recipes-devtools/pseudo/pseudo_git.bb:do_populate_sysroot', 'virtual:native:/workdir/build/../sources/poky/meta/recipes-support/bmap-tools/bmap-tools_3.4.bb:do_populate_sysroot']
| NOTE: Installed into sysroot: []
| NOTE: Skipping as already exists in sysroot: ['glibc', 'pigz-native', 'depmodwrapper-cross', 'python-native', 'opkg-native', 'mklibs-native', 'mtools-native', 'parted-native', 'dnf-native', 'gcc-cross-arm', 'update-rc.d-native', 'e2fsprogs-native', 'dosfstools-native', 'swig-native', 'opkg-utils-native', 'gptfdisk-native', 'createrepo-c-native', 'wic-tools', 'qemuwrapper-cross', 'quilt-native', 'prelink-native', 'bc-native', 'openssl-native', 'gcc-runtime', 'makedevs-native', 'cross-localedef-native', 'coreutils-native', 'ldconfig-native', 'rpm-native', 'pseudo-native', 'bmap-tools-native', 'linux-libc-headers', 'zlib-native', 'systemd-systemctl-native', 'automake-native', 'gnu-config-native', 'libtool-native', 'sqlite3-native', 'db-native', 'expat-native', 'autoconf-native', 'bzip2-native', 'xz-native', 'gdbm-native', 'pkgconfig-native', 'readline-native', 'libsolv-native', 'libarchive-native', 'qemu-native', 'texinfo-dummy-native', 'python3-iniparse-native', 'ninja-native', 'libdnf-native', 'librepo-native', 'cmake-native', 
'libcomps-native', 'gettext-minimal-native', 'python3-native', 'shadow-native', 'libpcre-native', 'libgcc', 'attr-native', 'nss-native', 'dbus-native', 'popt-native', 'elfutils-native', 'file-native', 'python3-setuptools-native', 'kmod-native', 'util-linux-native', 'ncurses-native', 'debianutils-native', 'binutils-cross-arm', 'flex-native', 'mpfr-native', 'libmpc-native', 'gmp-native', 'glib-2.0-native', 'libxml2-native', 'curl-native', 'binutils-native', 'm4-native', 'lzo-native', 'alsa-lib-native', 'pixman-native', 'dtc-native', 'python3-six-native', 're2c-native', 'libcheck-native', 'gtk-doc-native', 'gobject-introspection-native', 'gpgme-native', 'nspr-native', 'unzip-native', 'gettext-native', 'libffi-native', 'libpng-native', 'util-macros-native', 'libgpg-error-native', 'libassuan-native']
| DEBUG: Python function extend_recipe_sysroot finished
| DEBUG: Executing shell function do_image_sdimg
| + mkdir -p /workdir/build/tmp/work/linux-gnueabi/installer-image/1.0-r0
| + true
| + install -m 0644 /workdir/build/tmp/deploy/images/mmwe/uboot.env /workdir/build/tmp/work/linux-gnueabi/installer-image/1.0-r0/
| + basename /dev/mmcblk1
| + ondisk_dev=mmcblk1
| + wks=/workdir/build/tmp/work/linux-gnueabi/installer-image/1.0-r0/mender-sdimg.wks
| + rm -f /workdir/build/tmp/work/linux-gnueabi/installer-image/1.0-r0/mender-sdimg.wks
| + [ -n u-boot-spl.img ]
| + install -m 0644 /workdir/build/tmp/deploy/images/mmwe/u-boot-spl.img /workdir/build/tmp/work/linux-gnueabi/installer-image/1.0-r0/
| + expr 2 % 2
| + [ 0 -ne 0 ]
| + bootloader_sector=2
| + bootloader_file=/workdir/build/tmp/work/linux-gnueabi/installer-image/1.0-r0/u-boot-spl.img
| + expr 2 * 512
| + expr 1024 / 1024
| + bootloader_align_kb=1
| + stat -c %s /workdir/build/tmp/work/linux-gnueabi/installer-image/1.0-r0/u-boot-spl.img
| + bootloader_size=443584
| + expr 1 * 1024 + 443584
| + bootloader_end=444608
| + [ 444608 -gt 4194304 ]
| + cat
| + true
| + [ -n 4194304 ]
| + expr 4194304 / 1024
| + boot_env_align_kb=4096
| + cat
| + expr 4194304 % 1024
| + true
| + [ 0 -ne 0 ]
| + expr 4194304 / 1024
| + alignment_kb=4096
| + echo
| + sed -r s/(^\s*)|(\s*$)//g
| + IMAGE_BOOT_FILES_STRIPPED=
| + [ 0 -ne 0 ]
| + [ -n  ]
| + cat
| + [  -ne 0 ]
| /workdir/build/tmp/work/linux-gnueabi/installer-image/1.0-r0/temp/run.do_image_sdimg.206: 205: [: Illegal number:
| + cat
| + echo ### Contents of wks file ###
| ### Contents of wks file ###
| + cat /workdir/build/tmp/work/linux-gnueabi/installer-image/1.0-r0/mender-sdimg.wks
| # embed bootloader
| part --source rawcopy --sourceparams="file=/workdir/build/tmp/work/linux-gnueabi/installer-image/1.0-r0/u-boot-spl.img" --ondisk "mmcblk1" --align 1 --no-table| part --source rawcopy --sourceparams="file=/workdir/build/tmp/work/linux-gnueabi/installer-image/1.0-r0/uboot.env" --ondisk "mmcblk1" --align 4096 --no-table  
| part --source rawcopy --sourceparams="file=/workdir/build/tmp/work/linux-gnueabi/installer-image/1.0-r0/deploy-installer-image-image-complete/installer-image-mmwe.ext4" --ondisk "mmcblk1" --align 4096 --fixed-size 3407872k
| part --source rawcopy --sourceparams="file=/workdir/build/tmp/work/linux-gnueabi/installer-image/1.0-r0/deploy-installer-image-image-complete/installer-image-mmwe.ext4" --ondisk "mmcblk1" --align 4096 --fixed-size 3407872k
| part --source rawcopy --sourceparams="file=/workdir/build/tmp/work/linux-gnueabi/installer-image/1.0-r0/deploy-installer-image-image-complete/installer-image-mmwe.dataimg" --ondisk "mmcblk1" --align 4096 --fixed-size 7629 --mkfs-extraopts=''
| bootloader --ptable msdos
| + echo ### End of contents of wks file ###
| ### End of contents of wks file ###
| + outimgname=/workdir/build/tmp/work/linux-gnueabi/installer-image/1.0-r0/deploy-installer-image-image-complete/installer-image-mmwe--20190904120005.sdimg     
| + wicout=/workdir/build/tmp/work/linux-gnueabi/installer-image/1.0-r0/deploy-installer-image-image-complete/installer-image-mmwe--20190904120005-sdimg
| + BUILDDIR=/workdir/build wic create /workdir/build/tmp/work/linux-gnueabi/installer-image/1.0-r0/mender-sdimg.wks --vars /workdir/build/tmp/sysroots/mmwe/imgdata/ -e installer-image -o /workdir/build/tmp/work/linux-gnueabi/installer-image/1.0-r0/deploy-installer-image-image-complete/installer-image-mmwe--20190904120005-sdimg/ --no-fstab-update
| INFO: Creating image(s)...
| 
| ERROR: File system image of partition None is larger (10114662 kB) than its allowed size 3407872 kB
| 
| + bb_exit_handler
| + ret=1
| + echo WARNING: exit code 1 from a shell command.
| WARNING: exit code 1 from a shell command.
| + exit 1
| ERROR: Function failed: do_image_sdimg (log file is located at /workdir/build/tmp/work/linux-gnueabi/installer-image/1.0-r0/temp/log.do_image_sdimg.206)       
NOTE: recipe installer-image-1.0-r0: task do_image_sdimg: Failed
ERROR: Task (/workdir/build/../sources/meta-/images/installer-image.bb:do_image_sdimg) failed with exit code '1'
NOTE: Tasks Summary: Attempted 3748 tasks of which 3742 didn't need to be rerun and 2 failed.

Sorry, I can’t see anything obvious popping out. I propose that you start by looking at prepare_excluded_directories, maybe add some bb.warn("MSG") debug messages to make sure it’s getting into the expected places.

I was able to produce the same error that you have @dkhouya, by installing a 1GB file to /data but only allocating ~500MB for rootfs.

(I have some extra debug prints in below log for clarity and I only ran bitbake core-image-base -c do_image_tar to produce below)

WARNING: IMAGE_FSTYPES: ext4 sdimg.gz mender tar.bz2  wic wic.bmap mender mender.bmap sdimg sdimg.bmap
WARNING: task: do_image_wic
WARNING: task: do_image_sdimg
WARNING: task: do_image_wic.bmap
WARNING: task: do_image_mender.bmap
WARNING: task: do_image_sdimg.bmap
WARNING: task: do_image_sdimg.gz
WARNING: task: do_image_tar.bz2
WARNING: task: do_image_ext4
WARNING: task: do_image_mender
ERROR: core-image-base-1.0-r0 do_image_tar: The rootfs size 1138240(K) overrides IMAGE_ROOTFS_MAXSIZE: 507904(K)
ERROR: core-image-base-1.0-r0 do_image_tar: Function failed: set_image_size
ERROR: Logfile of failure stored in: /home/admin/mender-beaglebone/build/tmp/work/beaglebone_yocto-poky-linux-gnueabi/core-image-base/1.0-r0/temp/log.do_image_tar.3022
ERROR: Task (/home/admin/mender-beaglebone/build/../sources/poky/meta/recipes-core/images/core-image-base.bb:do_image_tar) failed with exit code '1'

Note that I have tar.bz2 in my IMAGE_FSTYPES, and that do_image_tar is failing, because the logic,

will not add prefuncs/postfuncs on do_image_tar and instead adds it to do_image_tar.bz2. I suppose this applies to any image type that has a image/compression suffix.

If I add tar explicitly to IMAGE_FSTYPES then the error goes away.

@dkhouya, can you share the content of your IMAGE_FSTYPES, e.g by running:

bitbake <image> -e | grep ^IMAGE_FSTYPES

Just ran it and had

IMAGE_FSTYPES="    mender mender.bmap sdimg sdimg.bmap  "
IMAGE_FSTYPES_DEBUGFS="tar.gz"

I’ll test it when adding tar to my IMAGE_FSTYPES

I have filed a ticket for this case,

https://tracker.mender.io/browse/MEN-2749

Thanks I’ll follow the issue!

If I find something I’ll post it.

@mirzak will this issue will be ported to thud branch when it will be merged in master?

Thanks,

Indeed it will,

Just tested the thud branch without any issue.

Thanks for the support!

Awesome, thanks for reporting back.