New online

I forgot to mention that before converting I’ve cloning the repo

    oga@oga-UX550VD:~/devel/mender_tools$ git clone -b 2.3.0 https://github.com/mendersoftware/mender-convert.git
    Cloning into 'mender-convert'...
    remote: Enumerating objects: 140, done.
    remote: Counting objects: 100% (140/140), done.
    remote: Compressing objects: 100% (79/79), done.
    remote: Total 2591 (delta 68), reused 125 (delta 61), pack-reused 2451
    Receiving objects: 100% (2591/2591), 1.13 MiB | 462.00 KiB/s, done.
    Resolving deltas: 100% (1562/1562), done.
    Note: switching to '7e924d6b2fb7479de96066bebe8b8ba57d6356a3'.

    You are in 'detached HEAD' state. You can look around, make experimental
    changes and commit them, and you can discard any commits you make in this
    state without impacting any branches by switching back to a branch.

    If you want to create a new branch to retain commits you create, you may
    do so (now or later) by using -c with the switch command. Example:

      git switch -c <new-branch-name>

    Or undo this operation with:

      git switch -

    Turn off this advice by setting config variable advice.detachedHead to false

after that I built the docker container using convenience script

oga@oga-UX550VD:~/devel/mender_tools/mender-convert$ ./docker-build 
[+] Building 360.4s (16/16) FINISHED                                                                     
 => [internal] load build definition from Dockerfile                                                0.0s
 => => transferring dockerfile: 2.32kB                                                              0.0s
 => [internal] load .dockerignore                                                                   0.0s
 => => transferring context: 74B                                                                    0.0s
 => [internal] load metadata for docker.io/library/ubuntu:20.04                                     1.6s
 => [internal] load build context                                                                   0.0s
 => => transferring context: 906B                                                                   0.0s
 => [build 1/4] FROM docker.io/library/ubuntu:20.04@sha256:3c9c713e0979e9bd6061ed52ac1e9e1f246c94  69.7s
 => => resolve docker.io/library/ubuntu:20.04@sha256:3c9c713e0979e9bd6061ed52ac1e9e1f246c9495aa063  0.0s
 => => sha256:3c9c713e0979e9bd6061ed52ac1e9e1f246c9495aa063619d9d695fb8039aa1f 1.20kB / 1.20kB      0.0s
 => => sha256:5403064f94b617f7975a19ba4d1a1299fd584397f6ee4393d0e16744ed11aab1 943B / 943B          0.0s
 => => sha256:26b77e58432b01665d7e876248c9056fa58bf4a7ab82576a024f5cf3dac146d6 3.32kB / 3.32kB      0.0s
 => => sha256:a70d879fa5984474288d52009479054b8bb2993de2a1859f43b5480600cecb24 28.57MB / 28.57MB   68.7s
 => => sha256:c4394a92d1f8760cf7d17fee0bcee732c94c5b858dd8d19c7ff02beecf3b4e83 848B / 848B          1.7s
 => => sha256:10e6159c56c084c858f5de2416454ac0a49ddda47b764e4379c5d5a147c9bf5f 187B / 187B          0.2s
 => => extracting sha256:a70d879fa5984474288d52009479054b8bb2993de2a1859f43b5480600cecb24           0.6s
 => => extracting sha256:c4394a92d1f8760cf7d17fee0bcee732c94c5b858dd8d19c7ff02beecf3b4e83           0.0s
 => => extracting sha256:10e6159c56c084c858f5de2416454ac0a49ddda47b764e4379c5d5a147c9bf5f           0.0s
 => [stage-1 2/9] RUN apt-get update && env DEBIAN_FRONTEND=noninteractive apt-get install -y     223.0s
 => [build 2/4] RUN apt-get update && env DEBIAN_FRONTEND=noninteractive apt-get install -y       265.8s
 => [build 3/4] RUN git clone https://github.com/jnovy/pxz.git /root/pxz                            0.9s
 => [build 4/4] RUN cd /root/pxz && make                                                            0.7s 
 => [stage-1 3/9] COPY --from=build /root/pxz/pxz /usr/bin/pxz                                      0.0s 
 => [stage-1 4/9] RUN echo "Defaults        secure_path="/usr/local/sbin:/usr/local/bin:/usr/sbin:  0.4s 
 => [stage-1 5/9] RUN chmod 0440 /etc/sudoers.d/secure_path_override                                0.5s 
 => [stage-1 6/9] RUN sed -i -e 's/,metadata_csum//' /etc/mke2fs.conf                               0.4s 
 => [stage-1 7/9] RUN wget -q -O /usr/bin/mender-artifact https://downloads.mender.io/mender-arti  18.7s 
 => [stage-1 8/9] COPY docker-entrypoint.sh /usr/local/bin/                                         0.0s 
 => exporting to image                                                                              1.6s 
 => => exporting layers                                                                             1.6s 
 => => writing image sha256:001dba185dc01aa4ed21fbebb6902d80a0cfc1da6b682e70ba291fdc8e4ca966        0.0s 
 => => naming to docker.io/library/mender-convert     

sorry for letting token visible but it’s not a big issue as it’s just a test account the compagny one will be different
and thank’s for editing and anonymizing my post

I don’t see anything obviously wrong there. What command did you use to write the SDCard?

Do you have a serial console? That is usually more helpful than the HDMI output and if you don’t have one I suggest you do for this and future troubleshooting efforts. This link is a great intro to using them and you can buy the cables readily on Amazon or other services. If you can share the serial console logs that would be the best next step.

Drew

than’ks for your analyse
for flashing image into sd I used 3 different methods (all same result)
balena / rpi-imager
and dd

sudo cat ./bareminimal-raspberrypi4-mender.img | sudo dd bs=4M of=/dev/sdb status=progress

by the past I got some issue flashing too large image files from linux on sd but I tried on window with diferent sd cards and card reader still same issue … :frowning:

unfortunately I have enable the serial but disable the console ( because I will use if for something else)
I will re-flash a standard PI OS image and try again through serial console

ok so I’ve :

  • tried a brand new PIOS lite image
  • got check that serial console wxorking properly
  • converted it to mender as before
  • re flashing it in a new sd
    and result is same ( not a big surprise as the boot sequence looks not starting correctly )
    just small video to showcase the issue
    mender converted image issue - YouTube

let me explain my objective and why i need a converted image and not a live client added as in getting started demo example
I want that end of product line technician flash an image with minimal image that already integrate mender client but :
I cant use an already registered pi image with an already existing id in mender server (like the command line in getting started demo is going to do) otherwise all products will get same id on server
and
I cant use a standard pi OS image and ask to the product technician to get logged onto mender server and do an ssh onto pi and copy past to a shell etc
that’s why I want the pi os already integrate the client but with a new ID and n ot registered yet to mender server (I will probably doing scripting to send the UID as preregistered onto mender server)

it acts as if it don’t know in boot-loader part on what partition (A or B) to boot on and stays in boot-loader for ever

I definitely understand the need for the converted image. It’s definitely the way to go for anything beyond just playing around with the system.

When you say you disabled the console, how did you do that? We discovered recently that “enable_uart=1” was required for U-Boot to function due to something related to GPU clock dividers or some such. See this thread for the full context.

Drew

usually in raspi-config you can choose to enable console over serial or just enable serial acting a standard tty that what I plane to use (or disable every thing) but anyway just for testing I enabled console over serial a redo the test ( the one shown in video)
when setting uart_enable=yes in /boot/config.txt you just enable the uart to enable console you need to add console=serial0,115200 console=tty1 in cdmline.txt
but that’s not the issue right here because

  • I took a standard pi OS witch enable serial console by default
  • I tested that serial console is working before converting the image
  • took the image out of sd
  • converting
  • and shoot it on an sd again
    and I still get black screen

now I’m trying a new approach : setup a standard pi image that will call the download.mender.io script (in a service for example) at first boot with right tenant token and remove itself at the end (or event better do the update that will remove it from root fs)

Can you share your two images with me?

sure !! (thanks again for your strong help)
here you can find the non converted image :

and here this is the converted one :

OK, I took a look at your images and I cannot explain the behavior. The image files look ok to me from the partition layout:

$ sudo fdisk -l blanklite_console.img; sudo fdisk -l blanklite_console-raspberrypi4-mender.img
Disk blanklite_console.img: 14.86 GiB, 15931539456 bytes, 31116288 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x249f3474

Device                 Boot  Start      End  Sectors  Size Id Type
blanklite_console.img1        8192   532479   524288  256M  c W95 FAT32 (LBA)
blanklite_console.img2      532480 31116287 30583808 14.6G 83 Linux
Disk blanklite_console-raspberrypi4-mender.img: 8 GiB, 8589934592 bytes, 16777216 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x0b5af379

Device                                     Boot    Start      End Sectors  Size Id Type
blanklite_console-raspberrypi4-mender.img1 *       24576   548863  524288  256M  c W95 FAT32 (LBA)
blanklite_console-raspberrypi4-mender.img2        548864  8519679 7970816  3.8G 83 Linux
blanklite_console-raspberrypi4-mender.img3       8519680 16490495 7970816  3.8G 83 Linux
blanklite_console-raspberrypi4-mender.img4      16490496 16752639  262144  128M 83 Linux

But when I write the blanklite_console-raspberrypi4-mender.img file to a 64GB SDCard, I get no errors but it does not appear to have a partition table. I remove the sdcard and reinsert it and the dmesg log shows the /dev/sdb device but no partitions. Very odd.

I’m wondering if the large disparity in partition sizes is somehow at fault. Your input image contains a partition of 14.6G but your output partitions are 3.8G. It looks like it should fit since the used sides of your input image is 1.2G. Maybe we have a partition alignment issue.

Can you try increasing MENDER_STORAGE_TOTAL_SIZE_MB to 32GB and 64GB and see if the behavior is different?

@lluiscampos @kacf any ideas on this?

Drew

thanks again for your help
my input img is 14Gb because I extracted it from a 16 Gb SD but I got same result with a non extended root fs freshly downloaded from PIOS website (what I did at the beginning)
where should I change MENDER_STORAGE_TOTAL_SIZE_MB ? on the host env before calling convert script that runs docker container ?
On my side after flashing a mender converted image I well find the 4 partitions (boot / root-fs A / B and data ) on the host system

I have 2 other questions :

as I have an issue with converted image and as I really want we use your product on our next items I tried to find another option to get a production ready mender preinstalled image

  • so I took a golden image with no mender inside
  • booted it in a raspberry
  • installed by hand mender client (following the doc Client installation | Mender documentation)
  • disable the mender-client.service
  • removed from hosted server the device record trough curl API
  • and I have even prepare some script for the device register alone at first boot to be pre-authorised before enabling the mender-client.service because I don’t want to have to create a different image for each new device even with a script it’s too bulky to setup on production line
  • (I have also ad /proc/cpuinfo serial to the mender-device-identity and it works great)

and it looks working perfectly except I don’t have the 2 root-fs swapping partitions for system updates (I was thinking client will do that if it’s not already done at boot but i was wrong)
so I took another look at the get.mender.io install script that do the partitions transforms and I don’t see where in the install procedure it do the re-partitioning stage
I seen that it install also mender-connect package but I don’t seen any info about that in the documentation

so my questions are :

  • where in the scripts are done the partitions resizing when using the getting started installation procedure ? ( or can I set the partition scheme manually ?)
  • what is mender-connect used for ?
  • when creating a system snapshot artifact does it require the device to have the 2 root-fs a/b ready for the deploy ? because in the mender client install documentation nowhere I seen something about partition setup

finaly I tried to setup all my system over the buster distrib you provide here https://d4o6e0uccgv40.cloudfront.net/2020-05-27-raspios-buster-lite-armhf/arm/2020-05-27-raspios-buster-lite-armhf-raspberrypi4-mender-2.5.0.img.xz
this one is booting correctly unfortunatly enabling the camera looks impossible because missing update and wend doing an apt update I have no support for lots of things starting by keyboard ! :frowning:

but it works on ssh so it’s ok for me
last things I need to acheive to get my img almost complete is to know if I can get read only partitions even on DATA because we dont want the sd card beeing corrupted by too long swap operation and/or power failure
and is it possible to encrypt partitions to avoid getting anyone looking keys
and sorry for my missunderstanding about mender install I understand now that partitioning is not done byt client itself

The variables are set in configuration files. Specifically configs/images/raspberrypi_raspbian_config. You can modify it there or add a local configuration file to override it.

We never modify the partitions in a live system. It’s not possible to do it robustly. We do have this link which is a method to do that but it can result in a bricked system.

mender-connect is a remote shell feature.

I honestly don’t know what will happen if you use the snapshot feature on a system without the dual partition setup. It may well create the artifact but you won’t be able to deploy it.

I’ll take another look at your converted image and see if I can find anything.

Drew

thank’s again for all the time you spend on my poor work …
I will try to move forward with the pre partitioned img you provide and go for a manual install or client (thanks for the enlightenment on mender-connect)

Hi @profff I had a simiilar behavior when using the 2.3.0 tag but it seems to be resolved in the master branch. Can you try the latest HEAD of master to see if your issue is addressed there?

Drew

OK !!! it works with last HEAD Master branch
I didnt took time to make a diff between conf files to spot the bug
I need to move forward now !!
thank’s again
ps : may be could be a good idea to change this in documentation :
https://docs.mender.io/system-updates-debian-family/convert-a-mender-debian-image

@lluiscampos @kacf is there something we need to document or backport here? It seems that Rpi4 does not work on any of the official releases.

Drew

This has been fixed and backported already, it just hasn’t been released yet. The fix will be in mender-convert 2.3.1 and 2.4.0. This is the relevant bug: MEN-4567. As mentioned in the ticket, a workaround for now is to use the 2020-05 image for Raspberry Pi.