New iteration of the mender-convert tool

General Discussions
1

Hi,

As you might be aware of we have a tool we call mender-convert to enable support for Mender on binary distributions.

We are happy that this has worked out, enabling people to securely update their devices on non-Yocto devices. Even though we are happy with the functionality of the current tool, there are a couple drawbacks in the current design of the tool, which we want to improve. We looked hard at the current code and decided, to achieve the desired design we would need to do a re-write.

We obviously learned a lot while developing the first iteration and we are trying to apply that knowledge to the current development effort.

The goals of the re-write was to achieve the following (lacking today):

  1. It should be easy to extend the tool to support other boards or distributions
  2. We should not compile code in the tool (rely on binaries built elsewhere)
    • compiling code increases complexity, due to requirement of toolchains etc.
  3. The tool shall not be designed around specific hardware/platform types
    • This is the case today with the usage of --device-type flag
  4. The tool should be to convert images without knowing anything about the hardware/platform
    • relates to above 3.
  5. Configuration interface should be simplified
    • command line flags -> configuration files
  6. Platform specific code shall be provided trough “hooks”, and are not part of the “core” mender-convert code
  7. It shall be easy to extend functionality
    • support for rootfs overlay to inject user applications/configurations
    • ability to override how the Mender Artifact is generated (to be able to sign and include state-scripts)
  8. Code structure should be modular
    • Eases Maintenance and possibility of making isolated changes

With that being said, the source code of the new iteration that tries to implement above mentioned goals currently lives here,

https://github.com/mendersoftware/mender-convert/tree/next

Considered this a early beta release, and would love to get feedback on the design and if you are able to test things out and report any issues.

There is still a lot of documentation to be written, and you will probably need to inspect the source code to get a full understanding of how certain things work but that is how it is :smiley:.

The following platforms have been run-time tested:

  • Raspbian (Raspberry Pi 3) which is covered in the README
  • Ubuntu Bionic (x86-64) - Instructions on how to test this can be found here
  • Armbian (Pine64 ROCKPro64, SD & eMMC)
  • Debian on BeagleBone Black (SD & eMMC)

There is no defined timeline when we would make the switch to make this tool the official one, and we will maintain both until we decide on this. Though new development will be focused on the updated version, especially adding support for new boards or distributions.

Thanks!

Edit: @mirzak: Updated link to the next branch of mender-convert repository, which is the current development branch for this new version of the tool

5 Likes
2

Hi,

Thanks for the new iteration, I use it to alter a ubuntu image on x86 64.

Which branch do you suggest to use? the v2 branch or the master-replaced-v1-with-v2?

I tested the v2 branch, it can mender -install <new.mender> and can boot into it, but on boot, it says:

error: no such device: (hd0, msdos1)/EFI/BOOT/EFI/BOOT/grub.cfg.
lock: OK
lock: OK

Also, the esp partition is a bit messy (EFI nested in /boot/efi), is this intended?

# hand-typed
$ ls /boot
System.map-4.15.0 xxxx
config-4.15.0-20-generic
loader
a5f4ae(a long string, seems like a checksum)
abi-4.15.0-20-generic
efi
initrd.img-xxx
retpoline-4.15xxx
vmlinuz-4.15xxx
/boot/efi/EFI/BOOT
3

Actually I would recommend to try this branch,

https://github.com/mendersoftware/mender-convert/tree/next

Forgot to update this thread with the progress

4

What is the changes needed to support BIOS GRUB?

5

@mirzak @lluiscampos @kacf any comments here? I know we’ve tested it with Yocto but in mender-convert I don’t now.

6

The best source of inspiration on how this can achieved is probably this commit in meta-mender,

Where we added BIOS support, and the primary tasks would be:

  1. GRUB needs to built “differently” to produce a boot.img
  2. The boot.img must be embedded in the sdimg after the MBR

Also this tutorial provides some insights how it all works,

We can probably produce the correct GRUB binaries using Yocto, this is what we already are doing for the EFI binaries which are uploaded to an S3 bucket and mender-convert fetches them.

7

Hi mirzak,
I would like some clarification so I will not go to dead end direction.
I’m not sure what should I do so the output image file will boot.
Do you suggest a way to convert BIOS GRUB image into new Uboot image, UEFI GRUB or BIOS GRUB? I prefer getting BIOS GRUB at the end.
What is the relation of meta-mender and mender-convert “next” branch? I guess I should change mender-convert “next” branch in the same way meta-mender was changed?
Thank You,
Israel

8

I think this would be would be possible.

What is the relation of meta-mender and mender-convert “next” branch?

meta-mender is meta data for the Yocto Project and there is no strict relation between meta-mender and mender-convert.

But meta-mender does contain logic that could be ported to mender-convert, e.g the x86 BIOS support I linked.

I guess I should change mender-convert “next” branch in the same way meta-mender was changed?

Yes, to some extent. Everything is not applicable and some code is for Yocto project only.

9

Hi,
I understand that the ability of the Mender Client service to come up is the indication that a full OS update is OK.
Does Mender full OS update enable supplying user-exit script that will check that the upgrade is OK?
Thank You,
Israel

10

There are ways of extending the sanity check.

Please have a look at,

https://docs.mender.io/2.2/artifacts/state-scripts

and specifically,

https://docs.mender.io/2.2/artifacts/state-scripts#custom-sanity-checks-after-the-update-is-installed

1 Like