Large file in /data generate do_image_mender error

Hi,

I’m using yocto (thud) using the meta-mender (thud commit e4d0ca3928523ba2a57d91b75fe758f08ad6d0f5)

I’m trying to integrate a large file(~8GB) in the /data partition but I have a size error at the do_image_mender step. See below for the error.

ERROR: installer-image-1.0-r0 do_image_mender: Size of rootfs is greater than the calculated     partition space (10356370432 > 3489660928). This image won't fit on a device with the current storage configuration. Try reducing IMAGE_OVERHEAD_FACTOR if it is higher than 1.0, or raise MENDER_STORAGE_TOTAL_SIZE_MB if the device in fact has more storage.ERROR: installer-image-1.0-r0 do_image_mender: Function failed: do_image_mender (log file is located at /workdir/build/tmp/work/mmwe-effenco-linux-gnueabi/installer-image/1.0-r0/temp/log.do_image_mender.130)
ERROR: Logfile of failure stored in: /workdir/build/tmp/work/mmwe-effenco-linux-gnueabi/installer-image/1.0-r0/temp/log.do_image_mender.130ERROR: Task (/workdir/build/../sources/meta-effenco/images/installer-image.bb:do_image_mender) failed with exit code '1'

It seems that the rootfs size still contain the /data folder.

On line 25 of https://github.com/mendersoftware/meta-mender/blob/e4d0ca3928523ba2a57d91b75fe758f08ad6d0f5/meta-mender-core/classes/mender-artifactimg.bbclass we have:

rootfs_size=$(stat -Lc %s ${IMGDEPLOYDIR}/${IMAGE_LINK_NAME}.${ARTIFACTIMG_FSTYPE})

It looks like it’s pointing to the rootfs that wasn’t strip from /data? Could it be a configuration issue?

I also set the following configuration in my distro conf file

# Mender configuration
MENDER_ARTIFACT_NAME = "${DISTRO_NAME}-${DISTRO_VERSION}"
ARTIFACTIMG_FSTYPE = "ext4"

# This will make sure that our SPL + u-boot.img is embedded in the sdimg at
# a 1kb offset.
MENDER_IMAGE_BOOTLOADER_FILE = "u-boot-spl.img"
MENDER_IMAGE_BOOTLOADER_BOOTSECTOR_OFFSET = "2"

# Remove the unneeded VFAT boot partition
IMAGE_BOOT_FILES = ""
MENDER_BOOT_PART_SIZE_MB = "0"

# Standard partition alignment from upstream wks file
MENDER_PARTITION_ALIGNMENT = "4194304"

MENDER_STORAGE_DEVICE = "/dev/mmcblk1"
MENDER_UBOOT_STORAGE_DEVICE = "0"

# Memory card with 16GiB of storage.
MENDER_STORAGE_TOTAL_SIZE_MB = "14305"
MENDER_DATA_PART_SIZE_MB = "7629"

MENDER_FEATURES_ENABLE_append = " mender-image mender-install mender-image-sd mender-uboot "
MENDER_FEATURES_DISABLE_append = " mender-grub mender-image-uefi"

# Fix size due to sdimg added to rootfs
IMAGE_ROOTFS_MAXSIZE = "12000000"

Thanks,

Hm, I believe that this might be a bug but unclear how to resolve it though.

We do remove the /data content when creating sdimg, but this class generates a separate ext4 file. Though looking at the master branch this no longer seems to be the case due to this change,

@kacf might some additional insights on this.

I just tried to cherry pick the previous commit and still ends up with the size error. See log below.

Would it be a good strategy to take the rootfs folder that was stripped from /data and create a {IMGDEPLOYDIR}/<NEW_ROOTFS_WITHOUT_DATA>.{ARTIFACTIMG_FSTYPE} and use that rootfs to check the size?

ERROR: installer-image-1.0-r0 do_image_mender: Size of rootfs is greater than the calculated partition space (10357413888 > 3489660928). This image won't fit on a device with the current storage configuration. Try reducing IMAGE_OVERHEAD_FACTOR if it is higher than 1.0, or raise MENDER_STORAGE_TOTAL_SIZE_MB if the device in fact has more storage.ERROR: installer-image-1.0-r0 do_image_mender: Function failed: do_image_mender (log file is located at /workdir/build/tmp/work/linux-gnueabi/installer-image/1.0-r0/temp/log.do_image_mender.205)
ERROR: Logfile of failure stored in: /workdir/build/tmp/work/linux-gnueabi/installer-image/1.0-r0/temp/log.do_image_mender.205 
NOTE: recipe installer-image-1.0-r0: task do_image_mender: Failed 
ERROR: Task (/workdir/build/../sources/meta-/images/installer-image.bb:do_image_mender) failed with exit code '1' 
ERROR: installer-image-1.0-r0 do_image_sdimg: Function failed: do_image_sdimg (log file is located at /workdir/build/tmp/work/linux-gnueabi/installer-image/1.0-r0/temp/log.do_image_sdimg.206)
ERROR: Logfile of failure stored in: /workdir/build/tmp/work/linux-gnueabi/installer-image/1.0-r0/temp/log.do_image_sdimg.206 
Log data follows: 
| DEBUG: Executing python function prepare_excluded_directories
| DEBUG: 'IMAGE_ROOTFS_EXCLUDE_PATH' is set but 'respect_exclude_path' variable flag is 0 for this image type, so ignoring it
| DEBUG: Python function prepare_excluded_directories finished
| DEBUG: Executing python function set_image_size
| DEBUG: 10012262.000000 = 7701740 * 1.300000
| DEBUG: 10114662.000000 = max(10012262.000000, 3305472)[10012262.000000] + 1
| DEBUG: 10114662.000000 = int(10114662.000000)
| DEBUG: 10114662 = aligned(10114662)
| DEBUG: returning 10114662
| DEBUG: Python function set_image_size finished
| DEBUG: Executing python function extend_recipe_sysroot
| NOTE: Direct dependencies are ['/workdir/build/../sources/poky/meta/recipes-core/glibc/glibc_2.28.bb:do_populate_sysroot', 'virtual:native:/workdir/build/../sources/poky/meta/recipes-extended/pigz/pigz_2.4.bb:do_populate_sysroot', '/workdir/build/../sources/poky/meta/recipes-kernel/kmod/depmodwrapper-cross_1.0.bb:do_populate_sysroot', '/workdir/build/../sources/poky/meta/recipes-devtools/python/python-native_2.7.15.bb:do_populate_sysroot', 'virtual:native:/workdir/build/../sources/poky/meta/recipes-devtools/opkg/opkg_0.3.6.bb:do_populate_sysroot', '/workdir/build/../sources/poky/meta/recipes-devtools/mklibs/mklibs-native_0.1.43.bb:do_populate_sysroot', 'virtual:native:/workdir/build/../sources/poky/meta/recipes-devtools/mtools/mtools_4.0.18.bb:do_populate_sysroot', 'virtual:native:/workdir/build/../sources/poky/meta/recipes-extended/parted/parted_3.2.bb:do_populate_sysroot', 'virtual:native:/workdir/build/../sources/poky/meta/recipes-devtools/dnf/dnf_2.7.5.bb:do_populate_sysroot', '/workdir/build/../sources/poky/meta/recipes-devtools/gcc/gcc-cross_8.2.bb:do_populate_sysroot', 'virtual:native:/workdir/build/../sources/poky/meta/recipes-core/update-rc.d/update-rc.d_0.8.bb:do_populate_sysroot', 'virtual:native:/workdir/build/../sources/poky/meta/recipes-devtools/e2fsprogs/e2fsprogs_1.44.3.bb:do_populate_sysroot', 'virtual:native:/workdir/build/../sources/poky/meta/recipes-devtools/dosfstools/dosfstools_4.1.bb:do_populate_sysroot', 'virtual:native:/workdir/build/../sources/poky/meta/recipes-devtools/swig/swig_3.0.12.bb:do_populate_sysroot', 'virtual:native:/workdir/build/../sources/poky/meta/recipes-devtools/opkg-utils/opkg-utils_0.3.6.bb:do_populate_sysroot', 'virtual:native:/workdir/build/../sources/poky/meta/recipes-devtools/fdisk/gptfdisk_1.0.4.bb:do_populate_sysroot', 'virtual:native:/workdir/build/../sources/poky/meta/recipes-devtools/createrepo-c/createrepo-c_git.bb:do_populate_sysroot', '/workdir/build/../sources/poky/meta/recipes-core/meta/wic-tools.bb:do_populate_sysroot', '/workdir/build/../sources/poky/meta/recipes-devtools/qemu/qemuwrapper-cross_1.0.bb:do_populate_sysroot', '/workdir/build/../sources/poky/meta/recipes-devtools/quilt/quilt-native_0.65.bb:do_populate_sysroot', 'virtual:native:/workdir/build/../sources/poky/meta/recipes-devtools/prelink/prelink_git.bb:do_populate_sysroot', 'virtual:native:/workdir/build/../sources/poky/meta/recipes-extended/bc/bc_1.07.1.bb:do_populate_sysroot', 'virtual:native:/workdir/build/../sources/poky/meta/recipes-connectivity/openssl/openssl_1.1.1a.bb:do_populate_sysroot', '/workdir/build/../sources/poky/meta/recipes-devtools/gcc/gcc-runtime_8.2.bb:do_populate_sysroot', 'virtual:native:/workdir/build/../sources/poky/meta/recipes-devtools/makedevs/makedevs_1.0.1.bb:do_populate_sysroot', 
'/workdir/build/../sources/poky/meta/recipes-core/glibc/cross-localedef-native_2.28.bb:do_populate_sysroot', 'virtual:native:/workdir/build/../sources/poky/meta/recipes-core/coreutils/coreutils_8.30.bb:do_populate_sysroot', '/workdir/build/../sources/poky/meta/recipes-core/glibc/ldconfig-native_2.12.1.bb:do_populate_sysroot', 'virtual:native:/workdir/build/../sources/poky/meta/recipes-devtools/rpm/rpm_4.14.2.bb:do_populate_sysroot', 'virtual:native:/workdir/build/../sources/poky/meta/recipes-devtools/pseudo/pseudo_git.bb:do_populate_sysroot', 'virtual:native:/workdir/build/../sources/poky/meta/recipes-support/bmap-tools/bmap-tools_3.4.bb:do_populate_sysroot']
| NOTE: Installed into sysroot: []
| NOTE: Skipping as already exists in sysroot: ['glibc', 'pigz-native', 'depmodwrapper-cross', 'python-native', 'opkg-native', 'mklibs-native', 'mtools-native', 'parted-native', 'dnf-native', 'gcc-cross-arm', 'update-rc.d-native', 'e2fsprogs-native', 'dosfstools-native', 'swig-native', 'opkg-utils-native', 'gptfdisk-native', 'createrepo-c-native', 'wic-tools', 'qemuwrapper-cross', 'quilt-native', 'prelink-native', 'bc-native', 'openssl-native', 'gcc-runtime', 'makedevs-native', 'cross-localedef-native', 'coreutils-native', 'ldconfig-native', 'rpm-native', 'pseudo-native', 'bmap-tools-native', 'linux-libc-headers', 'zlib-native', 'systemd-systemctl-native', 'automake-native', 'gnu-config-native', 'libtool-native', 'sqlite3-native', 'db-native', 'expat-native', 'autoconf-native', 'bzip2-native', 'xz-native', 'gdbm-native', 'pkgconfig-native', 'readline-native', 'libsolv-native', 'libarchive-native', 'qemu-native', 'texinfo-dummy-native', 'python3-iniparse-native', 'ninja-native', 'libdnf-native', 'librepo-native', 'cmake-native', 
'libcomps-native', 'gettext-minimal-native', 'python3-native', 'shadow-native', 'libpcre-native', 'libgcc', 'attr-native', 'nss-native', 'dbus-native', 'popt-native', 'elfutils-native', 'file-native', 'python3-setuptools-native', 'kmod-native', 'util-linux-native', 'ncurses-native', 'debianutils-native', 'binutils-cross-arm', 'flex-native', 'mpfr-native', 'libmpc-native', 'gmp-native', 'glib-2.0-native', 'libxml2-native', 'curl-native', 'binutils-native', 'm4-native', 'lzo-native', 'alsa-lib-native', 'pixman-native', 'dtc-native', 'python3-six-native', 're2c-native', 'libcheck-native', 'gtk-doc-native', 'gobject-introspection-native', 'gpgme-native', 'nspr-native', 'unzip-native', 'gettext-native', 'libffi-native', 'libpng-native', 'util-macros-native', 'libgpg-error-native', 'libassuan-native']
| DEBUG: Python function extend_recipe_sysroot finished
| DEBUG: Executing shell function do_image_sdimg
| + mkdir -p /workdir/build/tmp/work/linux-gnueabi/installer-image/1.0-r0
| + true
| + install -m 0644 /workdir/build/tmp/deploy/images/mmwe/uboot.env /workdir/build/tmp/work/linux-gnueabi/installer-image/1.0-r0/
| + basename /dev/mmcblk1
| + ondisk_dev=mmcblk1
| + wks=/workdir/build/tmp/work/linux-gnueabi/installer-image/1.0-r0/mender-sdimg.wks
| + rm -f /workdir/build/tmp/work/linux-gnueabi/installer-image/1.0-r0/mender-sdimg.wks
| + [ -n u-boot-spl.img ]
| + install -m 0644 /workdir/build/tmp/deploy/images/mmwe/u-boot-spl.img /workdir/build/tmp/work/linux-gnueabi/installer-image/1.0-r0/
| + expr 2 % 2
| + [ 0 -ne 0 ]
| + bootloader_sector=2
| + bootloader_file=/workdir/build/tmp/work/linux-gnueabi/installer-image/1.0-r0/u-boot-spl.img
| + expr 2 * 512
| + expr 1024 / 1024
| + bootloader_align_kb=1
| + stat -c %s /workdir/build/tmp/work/linux-gnueabi/installer-image/1.0-r0/u-boot-spl.img
| + bootloader_size=443584
| + expr 1 * 1024 + 443584
| + bootloader_end=444608
| + [ 444608 -gt 4194304 ]
| + cat
| + true
| + [ -n 4194304 ]
| + expr 4194304 / 1024
| + boot_env_align_kb=4096
| + cat
| + expr 4194304 % 1024
| + true
| + [ 0 -ne 0 ]
| + expr 4194304 / 1024
| + alignment_kb=4096
| + echo
| + sed -r s/(^\s*)|(\s*$)//g
| + IMAGE_BOOT_FILES_STRIPPED=
| + [ 0 -ne 0 ]
| + [ -n  ]
| + cat
| + [  -ne 0 ]
| /workdir/build/tmp/work/linux-gnueabi/installer-image/1.0-r0/temp/run.do_image_sdimg.206: 205: [: Illegal number:
| + cat
| + echo ### Contents of wks file ###
| ### Contents of wks file ###
| + cat /workdir/build/tmp/work/linux-gnueabi/installer-image/1.0-r0/mender-sdimg.wks
| # embed bootloader
| part --source rawcopy --sourceparams="file=/workdir/build/tmp/work/linux-gnueabi/installer-image/1.0-r0/u-boot-spl.img" --ondisk "mmcblk1" --align 1 --no-table| part --source rawcopy --sourceparams="file=/workdir/build/tmp/work/linux-gnueabi/installer-image/1.0-r0/uboot.env" --ondisk "mmcblk1" --align 4096 --no-table  
| part --source rawcopy --sourceparams="file=/workdir/build/tmp/work/linux-gnueabi/installer-image/1.0-r0/deploy-installer-image-image-complete/installer-image-mmwe.ext4" --ondisk "mmcblk1" --align 4096 --fixed-size 3407872k
| part --source rawcopy --sourceparams="file=/workdir/build/tmp/work/linux-gnueabi/installer-image/1.0-r0/deploy-installer-image-image-complete/installer-image-mmwe.ext4" --ondisk "mmcblk1" --align 4096 --fixed-size 3407872k
| part --source rawcopy --sourceparams="file=/workdir/build/tmp/work/linux-gnueabi/installer-image/1.0-r0/deploy-installer-image-image-complete/installer-image-mmwe.dataimg" --ondisk "mmcblk1" --align 4096 --fixed-size 7629 --mkfs-extraopts=''
| bootloader --ptable msdos
| + echo ### End of contents of wks file ###
| ### End of contents of wks file ###
| + outimgname=/workdir/build/tmp/work/linux-gnueabi/installer-image/1.0-r0/deploy-installer-image-image-complete/installer-image-mmwe--20190904120005.sdimg     
| + wicout=/workdir/build/tmp/work/linux-gnueabi/installer-image/1.0-r0/deploy-installer-image-image-complete/installer-image-mmwe--20190904120005-sdimg
| + BUILDDIR=/workdir/build wic create /workdir/build/tmp/work/linux-gnueabi/installer-image/1.0-r0/mender-sdimg.wks --vars /workdir/build/tmp/sysroots/mmwe/imgdata/ -e installer-image -o /workdir/build/tmp/work/linux-gnueabi/installer-image/1.0-r0/deploy-installer-image-image-complete/installer-image-mmwe--20190904120005-sdimg/ --no-fstab-update
| INFO: Creating image(s)...
| 
| ERROR: File system image of partition None is larger (10114662 kB) than its allowed size 3407872 kB
| 
| + bb_exit_handler
| + ret=1
| + echo WARNING: exit code 1 from a shell command.
| WARNING: exit code 1 from a shell command.
| + exit 1
| ERROR: Function failed: do_image_sdimg (log file is located at /workdir/build/tmp/work/linux-gnueabi/installer-image/1.0-r0/temp/log.do_image_sdimg.206)       
NOTE: recipe installer-image-1.0-r0: task do_image_sdimg: Failed
ERROR: Task (/workdir/build/../sources/meta-/images/installer-image.bb:do_image_sdimg) failed with exit code '1'
NOTE: Tasks Summary: Attempted 3748 tasks of which 3742 didn't need to be rerun and 2 failed.

Sorry, I can’t see anything obvious popping out. I propose that you start by looking at prepare_excluded_directories, maybe add some bb.warn("MSG") debug messages to make sure it’s getting into the expected places.

I was able to produce the same error that you have @dkhouya, by installing a 1GB file to /data but only allocating ~500MB for rootfs.

(I have some extra debug prints in below log for clarity and I only ran bitbake core-image-base -c do_image_tar to produce below)

WARNING: IMAGE_FSTYPES: ext4 sdimg.gz mender tar.bz2  wic wic.bmap mender mender.bmap sdimg sdimg.bmap
WARNING: task: do_image_wic
WARNING: task: do_image_sdimg
WARNING: task: do_image_wic.bmap
WARNING: task: do_image_mender.bmap
WARNING: task: do_image_sdimg.bmap
WARNING: task: do_image_sdimg.gz
WARNING: task: do_image_tar.bz2
WARNING: task: do_image_ext4
WARNING: task: do_image_mender
ERROR: core-image-base-1.0-r0 do_image_tar: The rootfs size 1138240(K) overrides IMAGE_ROOTFS_MAXSIZE: 507904(K)
ERROR: core-image-base-1.0-r0 do_image_tar: Function failed: set_image_size
ERROR: Logfile of failure stored in: /home/admin/mender-beaglebone/build/tmp/work/beaglebone_yocto-poky-linux-gnueabi/core-image-base/1.0-r0/temp/log.do_image_tar.3022
ERROR: Task (/home/admin/mender-beaglebone/build/../sources/poky/meta/recipes-core/images/core-image-base.bb:do_image_tar) failed with exit code '1'

Note that I have tar.bz2 in my IMAGE_FSTYPES, and that do_image_tar is failing, because the logic,

will not add prefuncs/postfuncs on do_image_tar and instead adds it to do_image_tar.bz2. I suppose this applies to any image type that has a image/compression suffix.

If I add tar explicitly to IMAGE_FSTYPES then the error goes away.

@dkhouya, can you share the content of your IMAGE_FSTYPES, e.g by running:

bitbake <image> -e | grep ^IMAGE_FSTYPES

Just ran it and had

IMAGE_FSTYPES="    mender mender.bmap sdimg sdimg.bmap  "
IMAGE_FSTYPES_DEBUGFS="tar.gz"

I’ll test it when adding tar to my IMAGE_FSTYPES

I have filed a ticket for this case,

https://tracker.mender.io/browse/MEN-2749

Thanks I’ll follow the issue!

If I find something I’ll post it.

@mirzak will this issue will be ported to thud branch when it will be merged in master?

Thanks,

Indeed it will,

1 Like

Just tested the thud branch without any issue.

Thanks for the support!

Awesome, thanks for reporting back.