First my input Debian 9.8 buster image (followed the recommended steps to copy the contents into golden-image-1.img) and given input to mender-convert.
Here am trying it for Demo Server, so ran ./scripts/test/run-tests.sh –output-dir ${PWD}/rootfs_overlay_demo/ --tenant-token “dummy”
The next step given is MENDER_ARTIFACT… Output is [FATAL] [mender-convert-extract] Sorry but the provided option is not supported: --overlays, tried removing the overlays options as well.
Along with that attaching image for more understanding! Whether it should generate some files in “rootfs_overlay_demo”, which’ll support artifacts for the convert?
In the latest release 2.3, Here my target is of amd64 and Debian OS, which configuration file I should use while converting?
Which lower version of mender supports mender-convert, if I want to try this feature of mender?
As per your suggestion I did it with “if=/dev/sdb” and after that checked with command “>> fdisk -l golden-image-1.img”, generated image
Output:
Disk golden-image-1.img: 14.6 GiB, 15631122432 bytes, 30529536 sectors
unit sectors of 1 * 512 = 512bytes
Sector Size (logical/physical): 512 bytes / 512 bytes
I/O Size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x1ce243a8
Device Boot Start End Sectors Size Id Type
golden-image-1.img1 * 2048 22468607 22466560 10.7G 83 Linux
golden-image-1.img2 22470654 30527487 8056834 3.9G 5 Extended
golden-image-1.img5 22470656 30527487 8056832 3.9G 82 Linux Swap / Solaris
I think these are partition table should exist in the image, assuming it is correct now. So, I have proceeded with mender-convert 2.3 version with this command
Output: Running mender-convert: --disk-image input/golden-image-1.img --config configs/raspberrypi3_config --overlay rootfs_overlay_demo
Running mender-convert-extract: --disk-image input/golden-image-1.img --config configs/raspberrypi3_config --overlay rootfs_overlay_demo
[Date and TimeStamp] [INFO] [mender-convert-extract] Using configuration file: configs/mender_convert_config
[Date and TimeStamp] [INFO] [mender-convert-extract] Using configuration file: configs/ raspberrypi3_config
[Date and TimeStamp] [INFO] [mender-convert-extract] Validating Disk Image
[Date and TimeStamp] [INFO] [mender-convert-extract] Disk Parsed Successfully
[Date and TimeStamp] [INFO] [mender-convert-extract] NUMBER OF PARTS: 3 TYPE: dos
[Date and TimeStamp] [INFO] [mender-convert-extract] PART1: SIZE: 10.7G TYPE: 0x83
[Date and TimeStamp] [INFO] [mender-convert-extract] PART1: extracting to work/part-1.fs
[Date and TimeStamp] [INFO] [mender-convert-extract] PART2: SIZE: 3.9G TYPE: 0x5
[Date and TimeStamp] [INFO] [mender-convert-extract] PART2: extracting to work/part-2.fs
[Date and TimeStamp] [INFO] [mender-convert-extract] PART3: SIZE: TYPE:
[Date and TimeStamp] [INFO] [mender-convert-extract] PART3: extracting to work/part-3.fs
Modules/disk.sh: line 61: $4: unbound variable
This error is seen on the terminal window. Then I checked particular line of script file “Modules/disk.sh”, here I assume “$4” is for writing of output files i.e. deploy folder as per the instructions, which isn’t happening, So I tried modifying the line in the script file i.e. run_and_log_cmd “dd if=$1 of=/media/user_name/external_HDD/folder_name/mender-convert skip=$2 bs=512 count=$3 conv=sparse status=none” Since it is looking for a path to write output files.
After this modification I ran the mender-convert command again and checked, received this error
Output: Running mender-convert: --disk-image input/golden-image-1.img --config configs/raspberrypi3_config --overlay rootfs_overlay_demo
Running mender-convert-extract: --disk-image input/golden-image-1.img --config configs/raspberrypi3_config --overlay rootfs_overlay_demo
[Date and TimeStamp] [INFO] [mender-convert-extract] Using configuration file: configs/mender_convert_config
[Date and TimeStamp] [INFO] [mender-convert-extract] Using configuration file: configs/ raspberrypi3_config
[Date and TimeStamp] [INFO] [mender-convert-extract] Validating Disk Image
[Date and TimeStamp] [INFO] [mender-convert-extract] Disk Parsed Successfully
[Date and TimeStamp] [INFO] [mender-convert-extract] NUMBER OF PARTS: 3 TYPE: dos
[Date and TimeStamp] [INFO] [mender-convert-extract] PART1: SIZE: 10.7G TYPE: 0x83
Dd: failed to open ‘/user_name/external_HDD/folder_name/mender-convert’: No such file or directory
Just to ensure, given all write permissions to this folder as well.
Please give your inputs on this, whether I have missed any step before executing the mender-convert command.
As you mentioned about modifications needed in mender-convert, Requesting you to provide/guide me on those changes which will help support in generating 4 partition image.
Do you have any other suggestion for debian 9.8 OS.
Hi @Sash you are using the raspberrypi3_config file which is expecting the extra FAT partition that is needed for that platform. You will need a different config file at a mimimum here. I suggest you start with the Beaglebone config files as they use Debian. But they may also use the FAT partition so you will need to adjust that.
Hi @Sash I have tested with the Beaglebone Black and the images provided by the BBB community. We have not tested mender-convert on X86/Debian platforms. It’s very possible to make it work but based on your reporting it requires some effort to make it happen.
Device Start End Sectors Size Type
/dev/mmcblk1p1 2048 1050623 1048576 512M EFI System
/dev/mmcblk1p2 1050624 40968703 48558080 23.2G Linux filesystem
/dev/mmcblk1p3 40968704 57669631 8060928 3.9G Linux swap
Installed other packages in the hardware.
Connected a clean USB stick to the hardware, to create a golden-image-1.img from the hardware, then proceeded with “dd” i.e.
root@root:/home/user# dd if=/dev/mmcblk1 of=/media/user/USB-stick-Name/golden-image-1.img bs=1M count=10000 conv=fdatasync status=progress
10485760000 bytes (10 GB, 9.8 GiB) copied, 912s, 11.5 MB/s
10000+0 records in
10000+0 records out
10485760000 bytes (10 GB, 9.8 GiB) copied, 912.113s, 11.5 MB/s
Verifying the generated golden-image-1.img from the USB stick i.e. fdisk
Note: Here I copied the generated golden-image-1.img to the hardware/device where mender-convert exists (LINUX Machine).
root@root:/home/user# fdisk -l golden-image-1.img GPT PMBR size mismatch (57671679 != 20479999) will be corrected by write.
Disk golden-image-1.img: 9.8Gi, 10485760000 bytes, 20480000 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x00000000
Device Boot Start End Sectors Size Type
golden-image-1.img 1 20479999 20479999 9.8G GPT
I assume here the generated image isn’t the right input to mender-convert. Since Partition Table isn’t correct as well DiskLabel Type.
Next I tried to fix the free space using “sgdisk” command.
Output: It didn’t fix the partitions, the response was with warnings and aborted.
Trail 2:
Downloaded Debian 10.0 buster image & Flash OS onto the USB using Balena etcher application.
Copied the contents of USB to the workstation using dd command and try to generate Mender artifacts using generic x86 config file which is present in the master branch
We have cherry-picked one commit(mender-conversion-tools) to mender convert repo as mentioned in other thread https://github.com/nandra/mender-conversion-tools/commit/0f500f215a9c944c85df731b2e499c6e111a5cc8 and still unsuccessful in generating mender artifacts
Does mender-convert only support UEFI based boot system images? (That means, It doesn’t support Legacy Boot Systems.)
If I wish to proceed with Yocto-Project does it support Legacy Boot Systems and Grub? (Here I have gone through the Board Integration Table Still need Confirmation on this)
What did am missing in Trail-1 while generating golden-image-1.img?
What does [ERROR] Exit code 32 mean from the Trail-2?
Should Mender-convert input golden-image-1.img have only 2 Partitions with (One EFI, Other rootfs)?
Wondering here how to generate/procedure for golden-image-1.img? Basically, asking what did I miss?
Note: Performed various other trails to generate the golden image. But aren’t successful.
From my tests golden image is created in way that OS is installed by manual partitioning (not using automatic partitioner which can create also LVM partitions) but manually create EFI + rootfs (for testing purposes rootfs can be 5G or so). Then use dd as above and before starting mender-convert try sgdisk -e image.img. Then try run mender convert and flash device back. After flashing run sgdisk -e /dev/xxx to correct GPT and test.
For details please try to follow readme in config file
I could successfully generate a “Debian Buster” image using the MKOSI command, and proceeded to the next step with a mender-convert. Here is the snapshot of the log on the converted image. I believe here the image conversion was successful.
Here comes the issue while performing the write operation of the generated “debian-trail-x86_64-mender.img.gz” on the hardware i.e. Intel Atom E3900 processor with the following command
Current OS running in the device was Debian 10.0 Buster with this partition table.
root@root:/home/user# fdisk -l /dev/mmcblk1
Disk /dev/mmcblk1: 27.5 GiB, 29527900160 bytes, 57671680 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 150F6BD7-4C64-4455-8C57-7E38CB0C8AD1
Device Start End Sectors Size Type
/dev/mmcblk1p1 2048 1050623 1048576 512M EFI System
/dev/mmcblk1p2 1050624 40968703 48558080 23.2G Linux filesystem
/dev/mmcblk1p3 40968704 57669631 8060928 3.9G Linux swap
Performed zcat command to do write operation on the device
root@root:/home/user# zcat /media/user-name/usb-name/debian-trail-x86_64-mender.img.gz | dd of=/dev/mmcblk1 bs=1M status=progress
XXXXXXXXXXXXXX bytes (17 GB, 16.9 GiB) copied, 912s, 11.5 MB/s
YYYYYY+0 records in
YYYYYY +0 records out
XXXXXXXXXXXXXX bytes (17 GB, 16.9 GiB) copied, 912.113s, 11.5 MB/s
Operation successfully wrote from the shell window.
Next step, I checked few commands in the same shell window for examples lsblk, df -hT, cat /etc/os-release, etc
All these gave me a segmentation fault. I am not sure what basically happened.
Gave a reboot to the hardware then ended up in this issue where grub 2.04 looks initramfs, initrd files to load. Entering into Dracut emergency shell window. Here even I could see the logs mention Dracut-init queue failing.
Made some additional modification in the configuration file generic_x86-64_hdd_config and mender_convert_config. Since the HW device name uses mmcblk1 you can refer to the previous partition table.
MENDER_STORAGE_DEVICE_BASE=/dev/mmcblk1
MENDER_BOOT_PART_NUMBER=”p1”
MENDER_ROOTFS_PART_A_NUMBER=”p2”
MENDER_ROOTFS_PART_B_NUMBER=”p3”
MENDER_DATA_PART_NUMBER=”p4”
While browsing across the community I have come across this link Mender
For supporting some additional HW boards, I even tried including these files, then generated the mender-converted images, Obtained the same previous results.
Could you please help me, while flashing the generated images what additional I should take care of? Why am I ending up these issues?
Regards,
Sashank
Please add parameter to dd command conv=fsync (or call sync after flash) and then please call sgdisk -e /dev/mmcblk0 to fix GPT table after flashing.
When you have grub emergency shell you can enter to grub command line hitting ‘c’ button. The please try to inspect if files are there (grub have commands like ls …) Also be sure you remove installation media because when still plugged it can confuse grub. Please report back your finding. Thanks.
As per your instructions, I did try including conv=fsync in the process of flashing the images. Then did sgdisk -e /dev/mmcblk1 to fix GPT Table. After this step, it did mention the table will be fixed after reboot.
So I did a reboot, without any external media or USB stick. Still eventually ended up in GNU Grub Version 2.04 window. The screen mentions about *Dummy Entry for Debug. The image is been attached in the previous comment, point 5.
Here I tried both the options to edit which ‘e’ to edit or ‘c’ for the command line. Both these interfaces ran into an infinite loop. Not allowing the user to enter anything:frowning_face:
Version 2.04 should be one from mender-convert. grub entry comes from mender grub.cfg:
menuentry 'Dummy Entry for Debug.' 'Wait' {
echo Menu Entry for debug/command prompt access
}
so looks like it start mender-convert grub but for unknown reason it won’t boot image. I think getting grub shell is must cos then we can debug bit more why image cannot boot. What do you mean by infinite loop when try to get grub prompt? Thanks.
Once the image.raw is been generated this is been converted using dd command which gives “.img”, then given input mender-convert.
Followed by writing images using zcat command and reboot the hardware machine. The machine booted up but, Ended up Grub 2.04 window looking for files to load (kernel, initramfs, initrd). The previous image refers to that.
This one which I am mentioning about infinite loop running, once you press ‘c’ command to debug the grub window screen, here is where I ran into this issue.
@Sash I can offer looking into problem locally just would need to mkosi generated image. Can you please share it somewhere so I can fetch it? I have ubuntu 16.04 and mkosi doesn’t work for me Thanks.
Thanks.