Help with docker-mender-convert for Ubuntu 20.04 image

General Discussions
1

I’m trying to use mender-convert on an Ubuntu 20.04 image. I’m following the instructions in the generic_x86-64_hdd_config file and I’m getting the following output/error.

MENDER_ARTIFACT_NAME=release-1 ./docker-mender-convert --disk-image input/golden-image-1.img --config configs/generic_x86-64_hdd_config --overlay rootfs_overlay_demo/
Running mender-convert --disk-image input/golden-image-1.img --config configs/generic_x86-64_hdd_config --overlay rootfs_overlay_demo/
Running mender-convert-extract: --config ./work/override_compression_config --disk-image input/golden-image-1.img --config configs/generic_x86-64_hdd_config --overlay rootfs_overlay_demo/
2022-01-10 07:53:44 [INFO] [mender-convert-extract] Using configuration file: configs/mender_convert_config
2022-01-10 07:53:44 [INFO] [mender-convert-extract] Using configuration file: ./work/override_compression_config
2022-01-10 07:53:44 [INFO] [mender-convert-extract] Using configuration file: configs/generic_x86-64_hdd_config
2022-01-10 07:53:44 [INFO] [mender-convert-extract] Validating disk image
2022-01-10 07:53:44 [INFO] [mender-convert-extract] Disk parsed successfully
2022-01-10 07:53:44 [INFO] [mender-convert-extract] NUMBER OF PARTS: 3 TYPE: gpt
2022-01-10 07:53:44 [INFO] [mender-convert-extract] PART 1: SIZE: 1M TYPE: 21686148-6449-6e6f-744e-656564454649
2022-01-10 07:53:44 [INFO] [mender-convert-extract] PART 1: extracting to work/part-1.fs
2022-01-10 07:53:44 [INFO] [mender-convert-extract] PART 2: SIZE: 512M TYPE: 0fc63daf-8483-4772-8e79-3d69d8477de4
2022-01-10 07:53:44 [INFO] [mender-convert-extract] PART 2: extracting to work/part-2.fs
2022-01-10 07:53:52 [INFO] [mender-convert-extract] PART 3: SIZE: 8G TYPE: 0fc63daf-8483-4772-8e79-3d69d8477de4
2022-01-10 07:53:52 [INFO] [mender-convert-extract] PART 3: extracting to work/part-3.fs
mender-convert-extract has finished. Cleaning up...
Running mender-convert-modify: --config ./work/override_compression_config --disk-image input/golden-image-1.img --config configs/generic_x86-64_hdd_config --overlay rootfs_overlay_demo/
2022-01-10 07:55:48 [INFO] [mender-convert-modify] Using configuration file: configs/mender_convert_config
2022-01-10 07:55:48 [INFO] [mender-convert-modify] Using configuration file: ./work/override_compression_config
2022-01-10 07:55:48 [INFO] [mender-convert-modify] Using configuration file: configs/generic_x86-64_hdd_config
mount: /mender-convert/work/boot: wrong fs type, bad option, bad superblock on /dev/loop8, missing codepage or helper program, or other error.
mender-convert-modify has finished. Cleaning up...
umount: work/boot: not mounted.
2022-01-10 07:55:49 [ERROR] [mender-convert] mender-convert failed
2022-01-10 07:55:48 [DEBUG] [mender-convert-extract] When running: (modules/disk.sh:64): run_and_log_cmd():  

	dd if=input/golden-image-1.img of=work/part-3.fs skip=1052672 bs=512 count=16777216 conv=sparse status=none
2022-01-10 07:55:48 [INFO] [mender-convert-modify] Using configuration file: configs/mender_convert_config
2022-01-10 07:55:48 [INFO] [mender-convert-modify] Using configuration file: ./work/override_compression_config
2022-01-10 07:55:48 [INFO] [mender-convert-modify] Using configuration file: configs/generic_x86-64_hdd_config
2022-01-10 07:55:49 [ERROR] [mender-convert] mender-convert failed
2022-01-10 07:55:49 [ERROR] [mender-convert] mender-convert exit code: 32

I manually partitioned the SSD with 512M boot partition and 8G root partition. I took an image of the first 9G of the SSD with

dd if=/dev/sda of=/mnt/external/mender-convert/input/golden-image-1.img bs=1M count=9000 status=progress

I then moved that image to my linux VM (arm64 on M1 Mac) and ran the sgdisk -e input/golden-image-1.img command to size the image correctly.

I then checked the image with fdisk to confirm it looked valid with.

$ fdisk -l input/golden-image-1.img
Disk golden-image-1.img: 8.81 GiB, 9437184000 bytes, 18432000 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 2E5ED8F1-6E16-49C5-A24F-F0A4C94DD404

Device                Start      End  Sectors  Size Type
golden-image-1.img1    2048     4095     2048    1M BIOS boot
golden-image-1.img2    4096  1052671  1048576  512M Linux filesystem
golden-image-1.img3 1052672 17829887 16777216    8G Linux filesystem

The first partition was automatically created by the Ubuntu installer as the “BIOS grub spacer”. Might this be causing the issue with mender-convert?

2

Yes, I would expect so. The tool expects an ext4 or FAT fs in the first partition.

3

I think mender-convert can only handle one UEFI (boot) partition and one root partition. The first partition you have there looks a bit weird, only 1M size…? It says BIOS boot, which indicates this might be a legacy BIOS bootloader, which mender-convert doesn’t support. What kind of board is this?

4

The board is a pretty standard x86 board; it’s not an embedded board. I’m booting from a SATA SSD with GPT partitioning.

The Ubuntu 20 installer creates a 1M partition, which is reserved for boot information. I’m not an expert on this, but I think previously, the initial part of the disk was reserved via convention; with the first partition containing data starting some distance (e.g. 1M) away from the start of the disk. The more recent convention seems to be to reserve this space using an explicit partition.

I did manage to get this to work, using the steps below:

  1. Install Ubuntu 20 with 512M /boot partition and 8G / partition.
  2. Use sfdisk to rewrite the partition table so that partition 1 is removed, and partitions 2 and 3 and renumbered to 1 and 2. I did this by sfdisk -d > sda.sfdisk then editing the sda.sfdisk file, removing the line containing partition 1 and renumbering the remaining partitions. Then I used sfdisk < sda.sfdisk to write the partitions back. This left the partition locations unchanged (i.e. space was still reserved for boot information at the start of the disk) but renumbered.
  3. Reinstall the grub bootloader after the partitions were renumbered. Note that you have to use grub-install /dev/sda --force on modern grub-install versions to install grub without the 1MB EFI partition.
  4. Take the image and run through mender-convert.
  5. dd the mender image to the disk
  6. Grow the last partition with growpart /dev/sda 4 to expand the /data partition. The data parition is marked with growfs in fstab, but the partition needs to be expanded first for this to work.
  7. Fix the partition table with sgdisk -e /dev/sda
  8. Re-install grub on the destination disk
  9. Boot the machine. The file system for /data expands on boot as the data partition was expanded manually.