CL200 series x86_64 mender convertion

[mikhailtchernychev]

Description

CL200 series is industrial x86_64 computer.

The manual is available here

CL200_Series_User_Guide.pdf

This post follows the one for NUC computer:

INTEL NUC x86-64 Ubuntu 18.04

Board description The Intel® NUC is a powerful 4x4-inch mini PC with entertainment, gaming, and productivity features, including a customizable board that is ready to accept the memory, storage, and operating systems that you want. From fully configured, ready-to-use Mini PCs to kits and boards for do-it-yourselfers, find the Intel® NUC that is right for you. [56-102-184_R01.jpg] URL: https://www.intel.com/content/www/us/en/products/boards-kits/nuc.html Wiki: https://en.wikipedia.org/wiki/N…

And uses mender-convert 2.1.0

The only difference is in the conversion process is that

MENDER_STORAGE_DEVICE_BASE=/dev/mmcblk1p

in the cl200_x86-64_hdd_config file. Other then that, it is identical to generic_x86-64_hdd_config

The conversion went smoothly.

Problem: resulting HD does not boot

GRUB is dropping into prompt (grub>)

It appeared that two created A and B partitions have the same UUID:
(showed on the disk image here)

/dev/loop0p2: UUID="121ddda3-31e6-449e-a880-fbc4c06b1f14" TYPE="ext4" PARTLABEL="primary" PARTUUID="bbad2e8c-5453-4549-85f6-2df5dbd2929a"
/dev/loop0p3: UUID="121ddda3-31e6-449e-a880-fbc4c06b1f14" TYPE="ext4" PARTLABEL="primary" PARTUUID="9eeb7225-d801-4762-9f96-da30c06c5634"
/dev/loop0p4: UUID="ac67c903-d527-4a03-a89f-18030f69b8b9" TYPE="ext4" PARTLABEL="primary" PARTUUID="f098d7ce-6ee4-4d15-9b10-1f4c54496ba6"
/dev/loop0: PTUUID="6c17766b-aaf0-40e8-ab95-e496c6ab59ed" PTTYPE="gpt"
/dev/loop0p1: UUID="B9E1-2DA7" TYPE="vfat" PARTLABEL="ESP" PARTUUID="00abec87-ab23-48fa-978d-f2736e60b114"

From other hand /EFI/ubuntu/grub.cfg looks like:

search.fs_uuid 828235fc-0cfd-4d29-baaf-f0254e8d5a1d root 
set prefix=($root)'/boot/grub'
configfile $prefix/grub.cfg

Further search showed that UUID 828235fc-0cfd-4d29-baaf-f0254e8d5a1d is embedded into
different configuration files on the disk image.

Possible fix (???)

Use tune2fs to assign UUID from grub.cfg to both partitions, like below:
(this is done using USB booted live linux distro, grml https://grml.org)

tune2fs -U 828235fc-0cfd-4d29-baaf-f0254e8d5a1d /dev/mmcblk1p2
tune2fs -U 828235fc-0cfd-4d29-baaf-f0254e8d5a1d /dev/mmcblk1p3

After this operation box is booting just fine, everything appear to work, I was able
to take snapshot, upload with stand-alone client, reboot, observe partition change,
and commit.

I used NUC procedure on another UEFI computer with root name /dev/sda and
it worked without need to change UUIDs; however partitions and grub.cfg UUIDs were
different.

I hope this could be useful.

Mikhail

[mirzak]

@mikhailtchernychev

From other hand /EFI/ubuntu/grub.cfg looks like

After you have converted an image with mender-convert, it should not be using /EFI/ubuntu/grub.cfg as a new grub.cfg is generated that is compatible with Mender.

Can you try removing /EFI/ubuntu/grub.cfg ? As you should also have /EFI/BOOT/grub.cfg which is the correct one for it to use.

[mikhailtchernychev]

There are two files /EFI/BOOT/grub.cfg (long one, appeared to be copied from the original ubuntu with some changes) and short one /EFI/ubuntu/grub.cfg which seem to me mender-specific

[mikhailtchernychev]

Sorry for deleting the posts. I wanted to delete in one place, it deleted everywhere.

I was under impression that these both grub.cfg are used (chained) but if only one is supposed to be
in use, you are right.

[MarekBelisko]

@mikhailtchernychev just wild guess, can you please apply this patch and try to convert again? Thanks.

[mikhailtchernychev]

I will but probably next week - I don’t have this thing right now. Thank you for looking!

[MarekBelisko]

No problem ;). BTW thanks for sharing board integration

[mikhailtchernychev]

@mirzak

/EFI/ubuntu/grub.cfg

Removing /EFI/ubuntu/grub.cfg does not look the right thing to do; on my other system which runs
fine without need to adjust UUIDs renaming this file brighs system to grub prompt.

[MarekBelisko]

@mikhailtchernychev can you please try mentioned patch if it resolve your issue? Thanks

[mikhailtchernychev]

I will during the week. Box is not in my possession yet

[mikhailtchernychev]

Hi,

I used exactly this commit

0f500f215a9c944c85df731b2e499c6e111a5cc8

Got past grub with messages “Welcome to GRUB” and “lock0” two times.

Did not go beyond this:

It reads: mdadm: No devices listed in conf file were found

USB keyboard also stopped working, so I could not do anything suggested.

Double-checked original image before conversion, it boots just fine.

Here is config file:

    #This configuration can be used to run any distribution on a any x86-64 machine.
    # As example is used CentOS distribution but can be applied to others also.
    #
    # Install CentOS using classic partition scheme (not use LVM) with minimal
    # 2 partitons /boot/efi and /root. For root please choose only necessary size
    # e.g. 10G (or desired size).
    #
    # This has been tested on images generated with the following command:
    #
    # Boot live USB and copy the data from internal storage (hdd, eMMC) e.g. for eMMC:
    #
    # 	dd if=/dev/mmcblk0 of=/media/usb/centos.img bs=1M count=10000 status=progress
    #
    # Before running mender-convert we need to fix GPT partition table (as we didn't copy
    # whole hdd). Run following command to fix GPT partition table on copied image:
    #
    #	sgdisk -e /media/usb/centos.img
    #
    # Converted with the following command:
    #
    #	MENDER_ARTIFACT_NAME=release-1 ./docker-mender-convert \
    #		--disk-image input/centos.img \
    #		--config configs/centos_x86-64_hdd_config \
    #		--overlay rootfs_overlay_demo/
    #
    # and image must be then copied back after conversion to internal storage (when booted again from live USB) e.g. for hdd:
    #
    #	zcat /media/usb/centos-x86_64-mender.img.gz | sudo dd of=/dev/sda bs=1M status=progress
    #
    # after copy run again (to fix GPT partition table):
    #	sgdisk -e /dev/sda
    #

    MENDER_STORAGE_DEVICE_BASE=/dev/mmcblk1p
    MENDER_DEVICE_TYPE="x86_64"

    MENDER_STORAGE_TOTAL_SIZE_MB=16000

    # Nothing to copy
    MENDER_COPY_BOOT_GAP="n"

    function platform_modify() {
        #
        # Make sure /lib64 exists since the Mender binary requires it.
        # Some systems put everything under /lib (ie Yocto) and a simple
        # symlink is enough to find everything Mender needs.
        #
        if [ ! -e work/rootfs/lib64 ]; then
            run_and_log_cmd "ln -s /lib work/rootfs/lib64"
        fi
    }

Apparently, partition /dev/mmcblk0p2 did not become available.

What kind of troubleshooting could be done?

Thank you!
Mikhail

[mikhailtchernychev]

I tried removing mdadm by booting original system using apt remove --purge mdadm, and it was apparently removed, re-imaging everything just to arrive to the same result as above. It seem there should be mo mdadm anymore, but it is.

[MarekBelisko]

Do you use lvm partition setup? I think this is not supported only ext4 in mender convert.

[mikhailtchernychev]

It was done exactly as for NUC. It is GPT / ext4, I am not using lvm.

Here is partition table:

1 grml@grml ~ % sudo parted  /dev/mmcblk1                                                                                   :(
GNU Parted 3.3
Using /dev/mmcblk1
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) p                                                                
Model: MMC DF4032 (sd/mmc)
Disk /dev/mmcblk1: 31.3GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags: 

Number  Start   End     Size    File system  Name     Flags
 1      8389kB  545MB   537MB   fat32        ESP      boot, esp
 2      545MB   8582MB  8036MB  ext4         primary
 3      8582MB  16.6GB  8036MB  ext4         primary
 4      16.6GB  16.8GB  134MB   ext4         primary

[drewmoseley]

Just a shot in the dark but I’m not sure fat32 works here. Can anyone confirm or refute that?

Drew

[mikhailtchernychev]

It did work on the other box. And boot got past loading from reading efi partition.
Also, original image with two partitions (before mender-convert) does have fat32 and ext4
partitions and boots fine.

[MarekBelisko]

@mikhailtchernychev

No devices listed in conf file were found

Does it have kind of raid or 2 disks plugged in? Could that be a source of problem?

[mikhailtchernychev]

Just installed fresh Ubuntu 20 server, imaged, converted with using commit with patch, moved to CL200 and ended up with grub> In other words it did not work at all.

[mikhailtchernychev]

But I actually fixed the problem by this:

File:
EFI/BOOT/grub.cfg

Original:

# Start of ---------- 00_mender_grubenv_defines_grub.cfg ----------
mender_rootfsa_part=2
mender_rootfsb_part=3
mender_kernel_root_base=/dev/mmcblk1p
mender_grub_storage_device=hd0
kernel_imagetype=
initrd_imagetype=
kernel_devicetree=kernel.dtb

Note kernel_imagetype= and initrd_imagetype= are blank

Correction:

kernel_imagetype=vmlinuz
initrd_imagetype=initrd.img

Fixes the boot. Yes I know these are symlinks.

[MarekBelisko]

Hmm this is weird. Those should be populated by mender-convert. Can you post convert log? I didn’t try Ubuntu 20 only 18.04. Thanks.