Getting error during artefacts creation from Ubuntu 18.04 x64 image using mender-convert-next

General Discussions
1

Hello,

With ubuntu test image. i have “.sdimg” and “.mender” files But I am getting error below with Ubuntu 18.04.3 LTS (Bionic beaver) Desktop image after execute mender-convert using mender-convert-next repo.

I have followed the steps as below.

  1. Download Ubuntu 18.04.3 LTS (Bionic beaver) Desktop iso image.
    Ubuntu 18.04.6 LTS (Bionic Beaver)

    $ fdisk -l -u ubuntu-18.04.3-desktop-amd64.iso
    Disk ubuntu-18.04.3-desktop-amd64.iso: 2 GiB, 2082816000 bytes, 4068000 sectors
    Units: sectors of 1 * 512 = 512 bytes
    Sector size (logical/physical): 512 bytes / 512 bytes
    I/O size (minimum/optimal): 512 bytes / 512 bytes
    Disklabel type: dos
    Disk identifier: 0x5092863d

    Device Boot Start End Sectors Size Id Type
    ubuntu-18.04.3-desktop-amd64.iso1 * 0 4067999 4068000 2G 0 Empty
    ubuntu-18.04.3-desktop-amd64.iso2 3989132 3994059 4928 2.4M ef EFI (FAT-12/16/32)

  2. Could you please assist me, what is the right way to convert iso file into .img file for mender?

    I have tried iso to img conversion as below.

    a. Convert using dd command as below

     sudo dd if=ubuntu-18.04.3-desktop-amd64.iso of=ubuntu-18.04.3-desktop-amd64_withdd.img bs=1M status=progress conv=fdatasync

    $ fdisk -l -u ubuntu-18.04.3-desktop-amd64_withdd.img
    Disk ubuntu-18.04.3-desktop-amd64_withdd.img: 2 GiB, 2082816000 bytes, 4068000 sectors
    Units: sectors of 1 * 512 = 512 bytes
    Sector size (logical/physical): 512 bytes / 512 bytes
    I/O size (minimum/optimal): 512 bytes / 512 bytes
    Disklabel type: dos
    Disk identifier: 0x5092863d

    Device Boot Start End Sectors Size Id Type
    ubuntu-18.04.3-desktop-amd64_withdd.img1 * 0 4067999 4068000 2G 0 Empty
    ubuntu-18.04.3-desktop-amd64_withdd.img2 3989132 3994059 4928 2.4M ef EFI (FAT-12/16/32)

    Here .img image booted successfully on X86 m/c.

    b. Just rename XXX.iso into XXX.img

    $ fdisk -l -u ubuntu-18.04.3-desktop-amd64_rename.img
    Disk ubuntu-18.04.3-desktop-amd64_rename.img: 1.8 GiB, 1875902464 bytes, 3663872 sectors
    Units: sectors of 1 * 512 = 512 bytes
    Sector size (logical/physical): 512 bytes / 512 bytes
    I/O size (minimum/optimal): 512 bytes / 512 bytes
    Disklabel type: dos
    Disk identifier: 0x5092863d

    Device Boot Start End Sectors Size Id Type
    ubuntu-18.04.3-desktop-amd64_rename.img1 * 0 4067999 4068000 2G 0 Empty
    ubuntu-18.04.3-desktop-amd64_rename.img2 3989132 3994059 4928 2.4M ef EFI (FAT-12/16/32)

    Here also .img image booted successfully on X86 m/c.

    c. Burn iso image into sdcard (.iso booted successfully on X86 m/c.) then copy sdcard image into .img using dd command and then boot again on X86 m/c successfully.

Note: Here all three above ways .img successfully booted on X86 m/c.

  1. Download mender-convert source code
    git clone -b next GitHub - mendersoftware/mender-convert: An easy way to integrate the Mender client into an existing image

  2. copy .img under input folder
    cd mender-convert
    mkdir -p input
    #Copy the .img file under input folder

  3. Execute below commands

    ./scripts/bootstrap-rootfs-overlay-demo-server.sh --output-dir ${PWD}/rootfs_overlay_demo --server-ip 10.77.164.6
    sudo su
    ./docker-build

    MENDER_ARTIFACT_NAME=release-1 ./docker-mender-convert --disk-image input/ubuntu-18.04.3-desktop-amd64.img --overlay rootfs_overlay_demo --config configs/qemux86-64_config

Terminal Output 7 Error:

2019-12-02 05:56:53 [INFO] [mender-convert-extract] Using configuration file: configs/mender_convert_config
2019-12-02 05:56:53 [INFO] [mender-convert-extract] Using configuration file: configs/qemux86-64_config
2019-12-02 05:56:53 [INFO] [mender-convert-extract] Validating disk image
2019-12-02 05:56:53 [INFO] [mender-convert-extract] Disk parsed succesfully
2019-12-02 05:56:53 [INFO] [mender-convert-extract] NUMBER OF PARTS: 2 TYPE: dos
2019-12-02 05:56:53 [INFO] [mender-convert-extract] PART 1: SIZE: 2G TYPE: 0x0
2019-12-02 05:56:53 [INFO] [mender-convert-extract] PART 1: extracting to work/part-1.fs
2019-12-02 05:57:48 [INFO] [mender-convert-extract] PART 2: SIZE: 2.4M TYPE: 0xef
2019-12-02 05:57:48 [INFO] [mender-convert-extract] PART 2: extracting to work/part-2.fs
dd: invalid number: ‘-1’

Anyone assist me to resolve error during create artifacts from Ubuntu 18.04 x64 desktop image using mender-convert-next repo.

can anyone please replicate same above and guide me if i missed something.

2

Hi @Rohita83,

I don’t think mender-convert will function on the ISO files directly. That could certainly be added but it would not be terribly useful since that is the Ubuntu installer image and on an “installed” Ubuntu. I think the best approach is to boot this ISO in a machine (a VM is fine), do the installation, and then use “dd” to make an image of the installed partition.

Drew

3

@drewmoseley, Thanks for the quick informative response.

if possible, Could you please share the steps or docs to make an iso image (mender compatible) from ubuntu18.04 desktop iso through VM.

4

I didn’t actually validate these instructions but this wiki looks pretty helpful for doing the actual installation of Ubuntu into Virtualbox. Basically, just create a new VirtualBox machine from the “Machine” menu, make sure to add a disk (VDI mode); then edit the machine settings and add the Ubuntu ISO to the IDE Storage controller as a CDROM device. Then boot the machine and select the “Install Ubuntu” option. Once that completes, power off the virtual machine.

Now you need to convert the VDI image to raw format. Note that this will create a file as large as you specified the disk to be, so make sure you have sufficient disk space. On my Ubuntu Laptop, the disk image is stored in “/home/dmoseley/Documents/VirtualBox/Ubuntu 18/Ubuntu 18.vdi”. It may be different in your case:

$ VBoxManage clonehd ~/Documents/VirtualBox/Ubuntu\ 18/Ubuntu\ 18.vdi /work/dmoseley/scratch/foo.img --format raw

This foo.img file should be usable with mender-convert however please note that we have not added support for Ubuntu/X86 in mender-convert yet so you will likely have further development to do to get this to work.

HTH,
Drew

5

I have tried the same as above and created a mender image (sdimg.gz 5.55 GB) successfully.
But failed during flashing.

After that, I have checked SD Card in GParted. It shows 2 partitions of 100 GB and another 3 partitions.

Is there any specific configuration to create .vdi (VirtualBox image) or with mender?

6

@drewmoseley, Thanks so much for suggesting the right steps. The steps given is working fine for qemux86_64 emulator.
Is it possible to convert for x86_64 generic hardware machine?.

7

Supporting a generic X86 machine is possible but will likely require some customization in mender-convert. We have not tested that machine type yet and would love a pull request implementing it.

8

Hi @sandiprakhasiya, welcome to Mender Hub.
I’m not quite sure what you are asking here. Mender convert will output a multiple partition image with the SDIMG suffix. You may need to run either qemu-img or VBoxManage to convert that to something directly usable by VirtualBox but I’ve not tried that.

Drew

9

Yes I will love to implement for ubuntu generic x86_64. Can you please guide from where i can start? Ubuntu working for qemux86_64 nice one and Thanks for the steps.

10

The docs are in the process of being updated. Take a look here for details: https://github.com/mendersoftware/mender-docs/blob/62d9b1cba2130f0162f96902a504149be1d3507a/04.Artifacts/02.Debian-family/02.image-configuration/docs.md

Drew

11

Thanks for the documents. I am able to boot on generic-x86_64 machine with Micro SD card as a boot-able media. Trying to run on computer hard disk HDD but the facing issue while booting.

Thanks and Regards,
Sachin

12

What issue are you seeing? You need to ensure that the MENDER_STORAGE_DEVICE variable is changed to whatever is appropriate for the hard disk.

13

Micro SD card set MENDER_STORAGE_DEVICE=/dev/sdb and that it perfectly working. For hard disk HDD set MENDER_STORAGE_DEVICE=/dev/sda but that it not working. Just blank after power on. Set MENDER_GRUB_STORAGE_DEVICE=hd0 for both sd card and HDD.

14

Whether those device nodes are correct is going to depend on your hardware and how specifically the devices are enumerated by the BIOS.

Also, some BIOSes renumber things when external devices are connected. I’ve seen some where they boot find when the HDD is the only device connected but then when you put a USB key in and boot, the HDD is pushed to HD1. I’m working on a fix for that but in the meantime if you just make sure that the HDD is the only block device in the system, that may help.

Drew

15

Hello Drew,

Only with HDD device also i tried but not booting. Here is more information. SD image (.sdimg)after mender-convert programmed with dd command in to HDD from sd card and that’s not booting but when i do same procedure for .img image before mender-convert it is booting from HDD. if i connect micro sd card and inside that in grub.cfg file if i give the boot and roots path of HDD storage then it is booting from HDD.

Thanks and Regards,
Sachin

16

It sounds like there is some changes needed in the mender-convert setup for the GRUB installation. We have not currently validated X86 platforms with mender-convert so that’s not terribly surprising. We’d happily accept a PR implementing this if you are so inclined.
Drew

17

Hello Drew,

Tried for the Another x86-64 machine that’s working perfectly and One of the x86-64 machine it was not working even after trying repeatedly. Both machines with HDD memory only. Most probably as you said before maybe because of the BIOS settings. Another machine where it is working fine is in legacy boot mode. PR? pull request, I am not aware?

Thanks and Regards,
Sachin

18

Yes, PR meant Pull Request. Sorry for being unclear.

Indeed, if it works on one machine and not the other then it is likely something odd in the device numbering as handled by the BIOS. You can try using the mender-partuuid setup but I’m not sure how easy that is to do in mender-convert. If you grep through the meta-mender sources for partuuid, that may give you an idea.

Drew