Unable to convert images using Mender-Convert

As per your suggestion I did it with “if=/dev/sdb” and after that checked with command “>> fdisk -l golden-image-1.img”, generated image
Output:

Disk golden-image-1.img: 14.6 GiB, 15631122432 bytes, 30529536 sectors
               unit sectors of 1 * 512 = 512bytes
               Sector Size (logical/physical): 512 bytes / 512 bytes
               I/O Size (minimum/optimal): 512 bytes / 512 bytes
              Disklabel type: dos
              Disk identifier: 0x1ce243a8

              Device                                 Boot          Start             End        Sectors     Size   Id  Type
              golden-image-1.img1       *                2048   22468607  22466560  10.7G  83 Linux         
              golden-image-1.img2                22470654   30527487    8056834    3.9G    5  Extended
              golden-image-1.img5                22470656   30527487    8056832    3.9G   82 Linux Swap / Solaris

I think these are partition table should exist in the image, assuming it is correct now. So, I have proceeded with mender-convert 2.3 version with this command

MENDER_ARTIFACT_NAME=release-1 ./docker-mender-convert --disk-image input/golden-image-1.img --config configs/raspberrypi3_config --overlay rootfs_overlay_demo

Output: Running mender-convert: --disk-image input/golden-image-1.img --config configs/raspberrypi3_config --overlay rootfs_overlay_demo
               Running mender-convert-extract: --disk-image input/golden-image-1.img --config configs/raspberrypi3_config --overlay rootfs_overlay_demo
               [Date and TimeStamp] [INFO] [mender-convert-extract] Using configuration file: configs/mender_convert_config
               [Date and TimeStamp] [INFO] [mender-convert-extract] Using configuration file: configs/ raspberrypi3_config
               [Date and TimeStamp] [INFO] [mender-convert-extract] Validating Disk Image
               [Date and TimeStamp] [INFO] [mender-convert-extract] Disk Parsed Successfully
               [Date and TimeStamp] [INFO] [mender-convert-extract] NUMBER OF PARTS: 3 TYPE: dos
               [Date and TimeStamp] [INFO] [mender-convert-extract] PART1: SIZE: 10.7G TYPE: 0x83
               [Date and TimeStamp] [INFO] [mender-convert-extract] PART1: extracting to work/part-1.fs
               [Date and TimeStamp] [INFO] [mender-convert-extract] PART2: SIZE: 3.9G TYPE: 0x5
               [Date and TimeStamp] [INFO] [mender-convert-extract] PART2: extracting to work/part-2.fs
               [Date and TimeStamp] [INFO] [mender-convert-extract] PART3: SIZE: TYPE: 
               [Date and TimeStamp] [INFO] [mender-convert-extract] PART3: extracting to work/part-3.fs
               Modules/disk.sh: line 61: $4: unbound variable

This error is seen on the terminal window. Then I checked particular line of script file “Modules/disk.sh”, here I assume “$4” is for writing of output files i.e. deploy folder as per the instructions, which isn’t happening, So I tried modifying the line in the script file i.e. run_and_log_cmd “dd if=$1 of=/media/user_name/external_HDD/folder_name/mender-convert skip=$2 bs=512 count=$3 conv=sparse status=none” Since it is looking for a path to write output files.

After this modification I ran the mender-convert command again and checked, received this error

Output: Running mender-convert: --disk-image input/golden-image-1.img --config configs/raspberrypi3_config --overlay rootfs_overlay_demo
               Running mender-convert-extract: --disk-image input/golden-image-1.img --config configs/raspberrypi3_config --overlay rootfs_overlay_demo
               [Date and TimeStamp] [INFO] [mender-convert-extract] Using configuration file: configs/mender_convert_config
               [Date and TimeStamp] [INFO] [mender-convert-extract] Using configuration file: configs/ raspberrypi3_config
               [Date and TimeStamp] [INFO] [mender-convert-extract] Validating Disk Image
               [Date and TimeStamp] [INFO] [mender-convert-extract] Disk Parsed Successfully
               [Date and TimeStamp] [INFO] [mender-convert-extract] NUMBER OF PARTS: 3 TYPE: dos
               [Date and TimeStamp] [INFO] [mender-convert-extract] PART1: SIZE: 10.7G TYPE: 0x83
              Dd: failed to open ‘/user_name/external_HDD/folder_name/mender-convert’: No such file or directory

Just to ensure, given all write permissions to this folder as well.

Please give your inputs on this, whether I have missed any step before executing the mender-convert command.

Edit: @mirzak: formatting

It seems to have problems with identifying the swap partition,

[Date and TimeStamp] [INFO] [mender-convert-extract] PART3: SIZE: TYPE: 
[Date and TimeStamp] [INFO] [mender-convert-extract] PART3: extracting to work/part-3.fs

Which then leads to the error. But I see other problems with your input disk image,

  1. mender-convert will assume that the first partition is the boot partition (I do not think this is the case in your image)
  2. mender-convert will assume that the second partition is the rootfs (I do not think this is the case in your image)
  3. mender-convert is not able to handle swap partitions

So without making modifications to mender-convert I do not think that your image will work.

A typical image that mender-convert is able to handle looks like this,

$ fdisk -l Ubuntu-Bionic-x86-64.img 
Disk Ubuntu-Bionic-x86-64.img: 2,12 GiB, 2270532608 bytes, 4434634 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 3B4BAD4E-D5CB-5F40-B03D-7BBF5F306C3E

Device                     Start     End Sectors  Size Type
Ubuntu-Bionic-x86-64.img1   2048  526335  524288  256M EFI System
Ubuntu-Bionic-x86-64.img2 526336 4434600 3908265  1,9G Linux root (x86-64)

As you mentioned about modifications needed in mender-convert, Requesting you to provide/guide me on those changes which will help support in generating 4 partition image.

Do you have any other suggestion for debian 9.8 OS.

Hi @Sash you are using the raspberrypi3_config file which is expecting the extra FAT partition that is needed for that platform. You will need a different config file at a mimimum here. I suggest you start with the Beaglebone config files as they use Debian. But they may also use the FAT partition so you will need to adjust that.

Drewmoseley/Mirzak,

Even i tried creating a FAT partition for the above, as various other ways, i couldn’t succeed with Mender-Convert.

Here am trying a fresh Debian 9.8/10.0 image which is in iso format, having partition table in it as well.

So my question here is did you people try mender-convert on any Debian images?

If yes and successful for you, please share me the steps for it? Since i have followed the steps in doc.mender.io 2.3 version.

Thanks in Advance

these files listed here are the ones tested by the community.

Is it possible to start out with Raspbian maybe? It is based on Debian :slight_smile:

Hi @Sash I have tested with the Beaglebone Black and the images provided by the BBB community. We have not tested mender-convert on X86/Debian platforms. It’s very possible to make it work but based on your reporting it requires some effort to make it happen.

Drew

@mirzak @drewmoseley

Please have a look!

Here am done with some additional trails performed to generate golden-image-1.img

Trail 1:
Steps Followed for generating golden-image-1.img

  1. Downloaded Fresh Debian 10.0 buster image.

  2. Flashed OS onto the USB using Balena etcher application.

  3. Then installed the Debian 10.0 OS on the Hardware using the USB stick.

  4. Check the output of the installed OS by fdisk -l /dev/mmcblk1 i.e.

    root@root:/home/user# fdisk -l /dev/mmcblk1
    Disk /dev/mmcblk1: 27.5 GiB, 29527900160 bytes, 57671680 sectors
    Units: sectors of 1 * 512 = 512 bytes
    Sector size (logical/physical): 512 bytes / 512 bytes
    I/O size (minimum/optimal): 512 bytes / 512 bytes
    Disklabel type: gpt
    Disk identifier: 150F6BD7-4C64-4455-8C57-7E38CB0C8AD1

    Device Start End Sectors Size Type
    /dev/mmcblk1p1 2048 1050623 1048576 512M EFI System
    /dev/mmcblk1p2 1050624 40968703 48558080 23.2G Linux filesystem
    /dev/mmcblk1p3 40968704 57669631 8060928 3.9G Linux swap

  5. Installed other packages in the hardware.

  6. Connected a clean USB stick to the hardware, to create a golden-image-1.img from the hardware, then proceeded with “dd” i.e.

    root@root:/home/user# dd if=/dev/mmcblk1 of=/media/user/USB-stick-Name/golden-image-1.img bs=1M count=10000 conv=fdatasync status=progress
    10485760000 bytes (10 GB, 9.8 GiB) copied, 912s, 11.5 MB/s
    10000+0 records in
    10000+0 records out
    10485760000 bytes (10 GB, 9.8 GiB) copied, 912.113s, 11.5 MB/s

  7. Verifying the generated golden-image-1.img from the USB stick i.e. fdisk
    Note: Here I copied the generated golden-image-1.img to the hardware/device where mender-convert exists (LINUX Machine).

    root@root:/home/user# fdisk -l golden-image-1.img
    GPT PMBR size mismatch (57671679 != 20479999) will be corrected by write.
    Disk golden-image-1.img: 9.8Gi, 10485760000 bytes, 20480000 sectors
    Units: sectors of 1 * 512 = 512 bytes
    Sector size (logical/physical): 512 bytes / 512 bytes
    I/O size (minimum/optimal): 512 bytes / 512 bytes
    Disklabel type: dos
    Disk identifier: 0x00000000

    Device Boot Start End Sectors Size Type
    golden-image-1.img 1 20479999 20479999 9.8G GPT

  8. I assume here the generated image isn’t the right input to mender-convert. Since Partition Table isn’t correct as well DiskLabel Type.

  9. Next I tried to fix the free space using “sgdisk” command.

    root@root:/home/user# sgdisk -e golden-image-1.img

    Output: It didn’t fix the partitions, the response was with warnings and aborted.

Trail 2:

Downloaded Debian 10.0 buster image & Flash OS onto the USB using Balena etcher application.
Copied the contents of USB to the workstation using dd command and try to generate Mender artifacts using generic x86 config file which is present in the master branch
We have cherry-picked one commit(mender-conversion-tools) to mender convert repo as mentioned in other thread https://github.com/nandra/mender-conversion-tools/commit/0f500f215a9c944c85df731b2e499c6e111a5cc8 and still unsuccessful in generating mender artifacts

Please find fdisk -l output

abc@HP-Notebook:~/try5/mender-convert$ fdisk -l input/golden-image-1.img
Disk input/golden-image-1.img: 14.5 GiB, 15597568000 bytes, 30464000 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: BC8FB979-A015-41E8-9823-8257C56D0767

Device Start End Sectors Size Type
input/golden-image-1.img1 2048 1050623 1048576 512M EFI System
input/golden-image-1.img2 1050624 13858815 12808192 6.1G Linux filesystem
input/golden-image-1.img3 13858816 30461951 16603136 7.9G Linux swap

abc@HP-Notebook:~/try5/mender-convert$ sudo MENDER_ARTIFACT_NAME=release-1 ./docker-mender-convert --disk-image input/golden-image-1.img --config configs/generic_x86-64_hdd_config --overlay rootfs_overlay_demo/
Running mender-convert --disk-image input/golden-image-1.img --config configs/generic_x86-64_hdd_config --overlay rootfs_overlay_demo/
Running mender-convert-extract: --disk-image input/golden-image-1.img --config configs/generic_x86-64_hdd_config --overlay rootfs_overlay_demo/
2020-05-26 04:16:51 [INFO] [mender-convert-extract] Using configuration file: configs/mender_convert_config
2020-05-26 04:16:51 [INFO] [mender-convert-extract] Using configuration file: configs/generic_x86-64_hdd_config
2020-05-26 04:16:52 [INFO] [mender-convert-extract] Validating disk image
2020-05-26 04:16:52 [INFO] [mender-convert-extract] Disk parsed successfully
2020-05-26 04:16:52 [INFO] [mender-convert-extract] NUMBER OF PARTS: 3 TYPE: gpt
2020-05-26 04:16:52 [INFO] [mender-convert-extract] PART 1: SIZE: 512M TYPE: c12a7328-f81f-11d2-ba4b-00a0c93ec93b
2020-05-26 04:16:52 [INFO] [mender-convert-extract] PART 1: extracting to work/part-1.fs
2020-05-26 04:17:00 [INFO] [mender-convert-extract] PART 2: SIZE: 6.1G TYPE: 0fc63daf-8483-4772-8e79-3d69d8477de4
2020-05-26 04:17:00 [INFO] [mender-convert-extract] PART 2: extracting to work/part-2.fs
2020-05-26 04:20:11 [INFO] [mender-convert-extract] PART 3: SIZE: 7.9G TYPE: 0657fd6d-a4ab-43c4-84e5-0933c84b4f4f
2020-05-26 04:20:11 [INFO] [mender-convert-extract] PART 3: extracting to work/part-3.fs
mender-convert-extract has finished. Cleaning up…
Running mender-convert-modify: --disk-image input/golden-image-1.img --config configs/generic_x86-64_hdd_config --overlay rootfs_overlay_demo/
2020-05-26 04:23:26 [INFO] [mender-convert-modify] Using configuration file: configs/mender_convert_config
2020-05-26 04:23:26 [INFO] [mender-convert-modify] Using configuration file: configs/generic_x86-64_hdd_config
mount: /mender-convert/work/boot: wrong fs type, bad option, bad superblock on /dev/loop17, missing codepage or helper program, or other error.
mender-convert-modify has finished. Cleaning up…
umount: work/boot: not mounted.
2020-05-26 04:23:28 [ERROR] [mender-convert] mender-convert failed
2020-05-26 04:23:26 [DEBUG] [mender-convert-extract] When running: (modules/disk.sh:61): run_and_log_cmd():

dd if=input/golden-image-1.img of=work/part-3.fs skip=13858816 bs=512 count=16603136 conv=sparse status=none
2020-05-26 04:23:28 [ERROR] [mender-convert] mender-convert exit code: 32

Some additional queries/doubts which I have:

  1. Does mender-convert only support UEFI based boot system images? (That means, It doesn’t support Legacy Boot Systems.)
  2. If I wish to proceed with Yocto-Project does it support Legacy Boot Systems and Grub? (Here I have gone through the Board Integration Table Still need Confirmation on this)
  3. What did am missing in Trail-1 while generating golden-image-1.img?
  4. What does [ERROR] Exit code 32 mean from the Trail-2?
  5. Should Mender-convert input golden-image-1.img have only 2 Partitions with (One EFI, Other rootfs)?
  6. Wondering here how to generate/procedure for golden-image-1.img? Basically, asking what did I miss?

Note: Performed various other trails to generate the golden image. But aren’t successful.

Regards,
Sashank

From my tests golden image is created in way that OS is installed by manual partitioning (not using automatic partitioner which can create also LVM partitions) but manually create EFI + rootfs (for testing purposes rootfs can be 5G or so). Then use dd as above and before starting mender-convert try sgdisk -e image.img. Then try run mender convert and flash device back. After flashing run sgdisk -e /dev/xxx to correct GPT and test.

For details please try to follow readme in config file

1 Like

Thanks, MarekBelisko

I could successfully generate a “Debian Buster” image using the MKOSI command, and proceeded to the next step with a mender-convert. Here is the snapshot of the log on the converted image. I believe here the image conversion was successful.

Here comes the issue while performing the write operation of the generated “debian-trail-x86_64-mender.img.gz” on the hardware i.e. Intel Atom E3900 processor with the following command

  1. Current OS running in the device was Debian 10.0 Buster with this partition table.
    root@root:/home/user# fdisk -l /dev/mmcblk1
    Disk /dev/mmcblk1: 27.5 GiB, 29527900160 bytes, 57671680 sectors
    Units: sectors of 1 * 512 = 512 bytes
    Sector size (logical/physical): 512 bytes / 512 bytes
    I/O size (minimum/optimal): 512 bytes / 512 bytes
    Disklabel type: gpt
    Disk identifier: 150F6BD7-4C64-4455-8C57-7E38CB0C8AD1
    Device Start End Sectors Size Type
    /dev/mmcblk1p1 2048 1050623 1048576 512M EFI System
    /dev/mmcblk1p2 1050624 40968703 48558080 23.2G Linux filesystem
    /dev/mmcblk1p3 40968704 57669631 8060928 3.9G Linux swap

  2. Performed zcat command to do write operation on the device
    root@root:/home/user# zcat /media/user-name/usb-name/debian-trail-x86_64-mender.img.gz | dd of=/dev/mmcblk1 bs=1M status=progress
    XXXXXXXXXXXXXX bytes (17 GB, 16.9 GiB) copied, 912s, 11.5 MB/s
    YYYYYY+0 records in
    YYYYYY +0 records out
    XXXXXXXXXXXXXX bytes (17 GB, 16.9 GiB) copied, 912.113s, 11.5 MB/s
    Operation successfully wrote from the shell window.

  3. Next step, I checked few commands in the same shell window for examples lsblk, df -hT, cat /etc/os-release, etc
    All these gave me a segmentation fault. I am not sure what basically happened.

  4. Gave a reboot to the hardware then ended up in this issue where grub 2.04 looks initramfs, initrd files to load. Entering into Dracut emergency shell window. Here even I could see the logs mention Dracut-init queue failing.

  5. Made some additional modification in the configuration file generic_x86-64_hdd_config and mender_convert_config. Since the HW device name uses mmcblk1 you can refer to the previous partition table.
    MENDER_STORAGE_DEVICE_BASE=/dev/mmcblk1
    MENDER_BOOT_PART_NUMBER=”p1”
    MENDER_ROOTFS_PART_A_NUMBER=”p2”
    MENDER_ROOTFS_PART_B_NUMBER=”p3”
    MENDER_DATA_PART_NUMBER=”p4”

Here is a different result

  1. While browsing across the community I have come across this link Mender
    For supporting some additional HW boards, I even tried including these files, then generated the mender-converted images, Obtained the same previous results.

Could you please help me, while flashing the generated images what additional I should take care of? Why am I ending up these issues?
Regards,
Sashank

Hi,

thanks for update.

Please add parameter to dd command conv=fsync (or call sync after flash) and then please call sgdisk -e /dev/mmcblk0 to fix GPT table after flashing.

When you have grub emergency shell you can enter to grub command line hitting ‘c’ button. The please try to inspect if files are there (grub have commands like ls …) Also be sure you remove installation media because when still plugged it can confuse grub. Please report back your finding. Thanks.

Thanks @MarekBelisko

As per your instructions, I did try including conv=fsync in the process of flashing the images. Then did sgdisk -e /dev/mmcblk1 to fix GPT Table. After this step, it did mention the table will be fixed after reboot.

So I did a reboot, without any external media or USB stick. Still eventually ended up in GNU Grub Version 2.04 window. The screen mentions about *Dummy Entry for Debug. The image is been attached in the previous comment, point 5.

Here I tried both the options to edit which ‘e’ to edit or ‘c’ for the command line. Both these interfaces ran into an infinite loop. Not allowing the user to enter anything:frowning_face:

Thanks
Sashank

Version 2.04 should be one from mender-convert. grub entry comes from mender grub.cfg:

menuentry 'Dummy Entry for Debug.' 'Wait' {
   echo Menu Entry for debug/command prompt access
}

so looks like it start mender-convert grub but for unknown reason it won’t boot image. I think getting grub shell is must cos then we can debug bit more why image cannot boot. What do you mean by infinite loop when try to get grub prompt? Thanks.

Hey @MarekBelisko

Yes, agree to you Grub 2.04 comes from mender-convert. Here are configuration details of mkosi created image.

[Distribution]
Distribution=debian
Release=buster

[Output]
Format=gpt_ext4
Bootable=yes

[Packages]
Packages= vim, nano, linux-image-amd64, libpam-sytemd

[Validation]
Password=password

Once the image.raw is been generated this is been converted using dd command which gives “.img”, then given input mender-convert.

Followed by writing images using zcat command and reboot the hardware machine. The machine booted up but, Ended up Grub 2.04 window looking for files to load (kernel, initramfs, initrd). The previous image refers to that.

This one which I am mentioning about infinite loop running, once you press ‘c’ command to debug the grub window screen, here is where I ran into this issue.

Thanks

as an side, you shouldn’t need the intermediate dd command from .raw to .img, just rename the .raw to .img so mender-convert can process it.

@Sash I can offer looking into problem locally just would need to mkosi generated image. Can you please share it somewhere so I can fetch it? I have ubuntu 16.04 and mkosi doesn’t work for me :wink: Thanks.

2 Likes

Thank You, Marek :slight_smile:

Here is the complete cmd, which helped me to generate the image using mkosi

mkosi -d debian -r buster -b -t gpt_ext4 --checksum --password password -f --package=vim --package=nano --package=linux-image-amd64 --package=libpam-systemd -o image.raw

Here is the generated image, Google Drive Link Please check whether you can access it?

The image is untouched when given to mender-convert as input. Some modifications which I made in config files, here is the details

MENDER_STORAGE_DEVICE_BASE="/dev/mmcblk1"

MENDER_BOOT_PART="/dev/mmcblk1p1"
MENDER_ROOTFS_PART_A="/dev/mmcblk1p2"
MENDER_ROOTFS_PART_B="/dev/mmcblk1p3"

Let me know, what did I miss if image is appropriate, what changes I should perform so that it doesn’t fail from GRUB 2.04 window.

Regards,
Sashank

Yes I fetched it thanks. I’ll try to install it first and then convert and play with it. I cannot promise today or tomorrow but during weekend should be feasible. :wink:

1 Like

@Sash sorry had busy weekend and cannot test it but today at least try to flash original image to my device to verify it boots (at it was booting :wink: ). I converted image using mender-convert and tomorrow I’ll try to verify what can be cause of issue and get back to you. Thank you for patience :+1:

@Sash I flashed mender-convert image to my x86 device and it boots perfectly fine (+ adding patch on top of actual tree). My config is following:

MENDER_STORAGE_DEVICE_BASE=/dev/mmcblk0p
MENDER_DEVICE_TYPE="x86_64"

MENDER_STORAGE_TOTAL_SIZE_MB=3000

# Nothing to copy
MENDER_COPY_BOOT_GAP="n"

so maybe try to use it and it will work also on your setup. Thanks.

2 Likes