I’m trying to convert a fresh Ubuntu 20 server installed with the following autoinstaller
#cloud-config
autoinstall:
identity:
hostname: ubuntu-server
password: "$6$exDY1mhS4KUYCE/2$zmn9ToZwTKLhCw.b4/b.ZRTIZM30JZ4QrOQ2aOXJ8yk96xpcCof0kxKwuX1kqLG/ygbJ1f8wxED22bTL4F46P0"
username: ubuntu
kernel:
package: linux-generic
locale: en_US.UTF-8
storage:
config:
- ptable: gpt
path: /dev/vda
wipe: superblock-recursive
preserve: false
name: ''
grub_device: true
type: disk
id: disk-vda
- device: disk-vda
size: 1048576
flag: bios_grub
number: 1
preserve: false
grub_device: false
type: partition
id: partition-0
- device: disk-vda
size: 17176723456
wipe: superblock
flag: ''
number: 2
preserve: false
grub_device: false
type: partition
id: partition-1
- fstype: ext4
volume: partition-1
preserve: false
type: format
id: format-0
- path: /
device: format-0
type: mount
id: mount-0
updates: security
version: 1
Using KVM I can install the OS into an disk image. Without further modification of the OS I use ran it though mender-convert with following configurations
# Set the device file corresponding to the root filesystem partitions.
MENDER_STORAGE_DEVICE="/dev/sda"
MENDER_STORAGE_DEVICE_BASE="/dev/sda" ### TEST
# They can also bet defined manually
#MENDER_BOOT_PART="/dev/sda1"
#MENDER_ROOTFS_PART_A="/dev/sda2"
#MENDER_ROOTFS_PART_B="/dev/sda3"
#MENDER_DATA_PART="/dev/sda4"
#MENDER_BOOT_PART_MOUNT_LOCATION="/boot/efi"
# The size of the storage medium of the device. 119.2GB (set to 118GB)
#MENDER_STORAGE_TOTAL_SIZE_MB="120832"
MENDER_STORAGE_TOTAL_SIZE_MB="65536"
# Calculate rootfs size
#rootfs_part_sectors=$(((${MENDER_STORAGE_TOTAL_SIZE_MB(126701535232)} - ${MENDER_DATA_PART_SIZE_MB(75161927680)} - ${boot_part_sectors(536870912)} - ${overhead_sectors(65536)}) / 2))
# The size of the boot partition.
MENDER_BOOT_PART_SIZE_MB="512"
#The size of each of the two rootfs filesystems, in KiB.
#If this is 0, mender-convert will use the size of the filesystem content as a basis.
#If the value is -1, mender-convert will use the maximum size that will fit inside the created partition.
#The size is further modified by IMAGE_ROOTFS_EXTRA_SPACE and IMAGE_OVERHEAD_FACTOR.
# NOTE! By not setting this the finished image will think it only have ex 11GB space, eventhough the partition is 24GB
# (11GB filesystem 24GB partition size)
IMAGE_ROOTFS_SIZE="-1"
# The size of the root partitions
# Mender client version
#
# This is used to fetch the correct binaries
#
# Valid values are "latest" (default), "master" or a specific version
MENDER_CLIENT_VERSION="latest"
# Install Mender Connect addon
#
MENDER_ADDON_CONNECT_INSTALL="n"
# Install Mender Configure addon
#
MENDER_ADDON_CONFIGURE_INSTALL="n"
# The size of the Mender data partition. 30720
#MENDER_DATA_PART_SIZE_MB="71680" # 70 GB --> root parts (128-70)/2 = 24GB root
MENDER_DATA_PART_SIZE_MB="16384"
# https://www.unitconverters.net/data-storage/gb-to-mb.htm
# Enable/Disable automatically growing the filesystem to fill the physical storage device.
# This feature is utilizing x−systemd.growfs and will only work on systemd 242
# and newer and nothing will be done if an older systemd version is used.
MENDER_DATA_PART_GROWFS="y"
#MENDER_DATA_PART_GROWFS="y"
# Our devices are using 245
# However Their system is somewhat wierd, since if we want to control the size of root partition.
# We can do that by setting total size and data size.
#MENDER_PARTITIONING_OVERHEAD_KB="32768"
# This is useful when you have large disk images,
# compressing them makes it easier to transfer them between a build server
# and a local machine, and saves space.
# Values: gzip(default)/lzma/none
MENDER_COMPRESS_DISK_IMAGE="gzip"
MENDER_ARTIFACT_COMPRESSION="gzip"
# The partition alignment expressed in bytes.
MENDER_PARTITION_ALIGNMENT="8388608"
# Explicitly set the name of the generated update Artifact.
# Required for the conversion to succeed.
# However, should be specified on the command line, and not in the configuration.
MENDER_ARTIFACT_NAME="$(date +'%Y.%m.%d')"
#MENDER_FEATURES_ENABLE_append = " <FEATURE>"
#MENDER_FEATURES_DISABLE_append = " mender-bios"
#DISTRO_FEATURES="mender-grub mender-convert "
MENDER_FEATURES_ENABLE_append="mender-grub mender-image-uefi"
MENDER_FEATURES_DISABLE_append="mender-uboot mender-image-sd mender-bios"
MENDER_FEATURES_ENABLE="mender-grub mender-image-uefi"
MENDER_FEATURES_DISABLE="mender-uboot mender-image-sd"
# Specific Linux kernel boot arguments
#
# Typically you would read the content of /proc/cmdline on a "golden image"
# add the appropriate arguments here.
#
# This will override the defaults set by by grub-mender-grubenv, if not an
# empty string
MENDER_GRUB_KERNEL_BOOT_ARGS="console=tty0,115200n8 console=ttyO0,115200n8 console=ttyAMA0,115200n8"
#MENDER_GRUB_KERNEL_IMAGETYPE=""
DEPLOY_DIR_IMAGE="/mender-convert/deploy"
MENDER_MACHINE="robotti_mender"
# Set the device type specified by the Artifact. (robotti 150D 150LR... etc.. This way it will be more easy to send specific updates)
# If left empty it will default to the value of '/etc/hostname'.
MENDER_DEVICE_TYPE="x86_64"
# Copy the boot gap, which is the area from sector 1 until first parition,
# this is done because some images might contain the bootloader embedded here
# and we need to copy the bootloader to the output image.
#
# In most cases you would like this enabled.
# Nothing to copy
MENDER_COPY_BOOT_GAP="n"
function platform_modify() {
#
# Make sure /lib64 exists since the Mender binary requires it.
# Some systems put everything under /lib (ie Yocto) and a simple
# symlink is enough to find everything Mender needs.
#
if [ ! -e work/rootfs/lib64 ]; then
run_and_log_cmd "ln -s /lib work/rootfs/lib64"
fi
}
function write_mender_version() {
cat <<- EOF > work/rootfs/data/mender/artifact_version.txt
ARTIFACT_VERSION_PART_A=$MENDER_ARTIFACT_NAME
ARTIFACT_VERSION_PART_B=$MENDER_ARTIFACT_NAME
EOF
}
PLATFORM_MODIFY_HOOKS+=(write_mender_version)
function check_system_permissions_and_groups_1() {
if ! [ "stat -c '%U' work/rootfs/" == "root" ];
then
run_and_log_cmd "sudo chown root:root work/rootfs/"
log_info "owner and group is now:"
run_and_log_cmd "stat -c '%U' work/rootfs/"
else
log_info "owner and group matches for /"
run_and_log_cmd "stat -c '%U' work/rootfs/"
fi
if ! [ "stat -c '%U' work/rootfs/etc" == "root" ];
then
run_and_log_cmd "sudo chown root:root work/rootfs/etc"
log_info "owner and group is now:"
run_and_log_cmd "stat -c '%U' work/rootfs/etc"
else
log_info "owner and group matches for /etc"
run_and_log_cmd "stat -c '%U' work/rootfs/etc"
fi
if ! [ "stat -c '%U' work/rootfs/etc/mender" == "root" ];
then
run_and_log_cmd "sudo chown root:root work/rootfs/etc/mender"
log_info "owner and group is now:"
run_and_log_cmd "stat -c '%U' work/rootfs/etc/mender"
else
log_info "owner and group matches /etc/mender"
run_and_log_cmd "stat -c '%U' work/rootfs/etc/mender"
fi
}
PLATFORM_MODIFY_HOOKS+=(check_system_permissions_and_groups_1)
function check_system_permissions_and_groups_2() {
if ! [ "stat -c '%U' work/rootfs/" == "root" ];
then
run_and_log_cmd "sudo chown root:root work/rootfs/"
log_info "owner and group is now:"
run_and_log_cmd "stat -c '%U' work/rootfs/"
else
log_info "owner and group matches for /"
run_and_log_cmd "stat -c '%U' work/rootfs/"
fi
if ! [ "stat -c '%U' work/rootfs/etc" == "root" ];
then
run_and_log_cmd "sudo chown root:root work/rootfs/etc"
log_info "owner and group is now:"
run_and_log_cmd "stat -c '%U' work/rootfs/etc"
else
log_info "owner and group matches for /etc"
run_and_log_cmd "stat -c '%U' work/rootfs/etc"
fi
if ! [ "stat -c '%U' work/rootfs/etc/mender" == "root" ];
then
run_and_log_cmd "sudo chown root:root work/rootfs/etc/mender"
log_info "owner and group is now:"
run_and_log_cmd "stat -c '%U' work/rootfs/etc/mender"
else
log_info "owner and group matches /etc/mender"
run_and_log_cmd "stat -c '%U' work/rootfs/etc/mender"
fi
}
OVERLAY_MODIFY_HOOKS+=(check_system_permissions_and_groups_2)
function check_system_permissions_and_groups_3() {
if ! [ "stat -c '%U' work/rootfs/" == "root" ];
then
run_and_log_cmd "sudo chown root:root work/rootfs/"
log_info "owner and group is now:"
run_and_log_cmd "stat -c '%U' work/rootfs/"
else
log_info "owner and group matches for /"
run_and_log_cmd "stat -c '%U' work/rootfs/"
fi
if ! [ "stat -c '%U' work/rootfs/etc" == "root" ];
then
run_and_log_cmd "sudo chown root:root work/rootfs/etc"
log_info "owner and group is now:"
run_and_log_cmd "stat -c '%U' work/rootfs/etc"
else
log_info "owner and group matches for /etc"
run_and_log_cmd "stat -c '%U' work/rootfs/etc"
fi
if ! [ "stat -c '%U' work/rootfs/etc/mender" == "root" ];
then
run_and_log_cmd "sudo chown root:root work/rootfs/etc/mender"
log_info "owner and group is now:"
run_and_log_cmd "stat -c '%U' work/rootfs/etc/mender"
else
log_info "owner and group matches /etc/mender"
run_and_log_cmd "stat -c '%U' work/rootfs/etc/mender"
fi
}
#function check_partition_sizes() {
#if [ "df --output=size work/roofs/dev/sda2 | awk 'NR==2 {print}'" < "sudo fdisk -h work/roofs/dev/sda2" ];
# then
# log_info "Current size:"
# run_and_log_cmd "df -h --out=size /dev/sda2"
#run_and_log_cmd "sudo e2fsck -f /dev/sda2"
#run_and_log_cmd "sudo resize2fs /dev/sda2"
# log_info "New size:"
# run_and_log_cmd "df -h --out=size /dev/sda2"
# else
# log_info "size match for sda2"
# run_and_log_cmd "df -h --out=size /dev/sda2"
#fi
#if [ "df --output=size /dev/sda3 | awk 'NR==2 {print}'" < "sudo fdisk -h /dev/sda3" ];
# then
# log_info "Current size:"
# run_and_log_cmd "df -h --out=size /dev/sda3"
# #run_and_log_cmd "sudo e2fsck -f /dev/sda3"
# #run_and_log_cmd "sudo resize2fs /dev/sda3"
# log_info "New size:"
# run_and_log_cmd "df -h --out=size /dev/sda3"
#
# else
# log_info "size match for sda3"
# run_and_log_cmd "df -h --out=size /dev/sda3"
#fi
#}
#PLATFORM_MODIFY_HOOKS+=(check_partition_sizes)
Then mender-convert is executed like the following
MENDER_ARTIFACT_NAME=release-2 ./mender-convert
--disk-image $INPUT_DISK_IMAGE
--config $CONFIG_1
--config $CONFIG_2
--overlay $CONFIG_OVERLAY
--overlay $DEVICE_OVERLAY
However mender-convert fails with the following
Running mender-convert-extract: [...]
[...]
2023-06-02 09:29:54 [INFO] [mender-convert-extract] Validating disk image
2023-06-02 09:29:54 [INFO] [mender-convert-extract] Disk parsed successfully
2023-06-02 09:29:54 [INFO] [mender-convert-extract] NUMBER OF PARTS: 2 TYPE: gpt
2023-06-02 09:29:54 [INFO] [mender-convert-extract] PART 1: SIZE: 1M TYPE: 21686148-6449-6e6f-744e-656564454649
2023-06-02 09:29:54 [INFO] [mender-convert-extract] PART 1: extracting to work/part-1.fs
2023-06-02 09:29:54 [INFO] [mender-convert-extract] PART 2: SIZE: 16G TYPE: 0fc63daf-8483-4772-8e79-3d69d8477de4
2023-06-02 09:29:54 [INFO] [mender-convert-extract] PART 2: extracting to work/part-2.fs
mender-convert-extract has finished. Cleaning up...
Running mender-convert-modify: [...]
mount: .../mender-convert/work/boot: wrong fs type, bad option, bad superblock on /dev/loop33, missing codepage or helper program, or other error.
mender-convert-modify has finished. Cleaning up...
umount: work/boot: not mounted.
2023-06-02 09:30:16 [ERROR] [mender-convert] mender-convert failed
2023-06-02 09:30:16 [DEBUG] [mender-convert-extract] When running: (modules/disk.sh:63): run_and_log_cmd():
dd if=[...] of=work/part-2.fs skip=4096 bs=512 count=33548288 conv=sparse status=none
Tried to run the dd command manually and there’s no error. So I’m clueless of what could’ve went wrong, any advice is appreciated.