# RaspiOS versions vs mender-convert versions **URL:** \ **Category:** General Discussions\ **Tags:** mender-convert, raspberry-pi-4\ **Created:** [March 19, 2021, 5:12pm UTC](https://hub.mender.io/t/raspios-versions-vs-mender-convert-versions/3398 "2021-03-19T17:12:36Z")\ **Posts on this page:** 16\ **Page:** 1
**Author:** ![wombat](https://avatars.discourse-cdn.com/v4/letter/w/e480ec/32.png) [@wombat](https://hub.mender.io/u/wombat)\ **Post date:** [March 19, 2021, 5:12pm UTC](https://hub.mender.io/t/raspios-versions-vs-mender-convert-versions/3398/1 "2021-03-19T17:12:36Z")
Apologies for the short post - I can follow up with debug info in due course (as necessary). I recently started trying out mender ([hosted.mender.io](http://hosted.mender.io)) and initially had success converting bootable images (and deployable mender rootfs artifacts) using: - mender-convert at version [2.3.0](https://github.com/mendersoftware/mender-convert/releases/tag/2.3.0) - RaspiOS Lite 2020-05-27 I have not been able to use any more recent versions of RaspiOS Lite, because the resulting ‘.img’ does not boot. I’ve been watching [MEN-3941](https://tracker.mender.io/browse/MEN-3941) and [MEN-3945](https://tracker.mender.io/browse/MEN-3945) with interest. The most recent version I’ve tried to use (with no success) is: - mender-convert on branch 2.3.x at commit [dab3afdf](https://github.com/mendersoftware/mender-convert/commit/dab3afdfef01e03ca12b8678fbeda836ea276558) - RaspiOS Lite 2021-01-11 I have two initial questions: - Q. Does this apparent failure in the integration tests for current HEAD of 2.3.x significant? - Q. How should I use mender-convert to convert a RaspiOS Lite 2021-01-11 system image? (assuming MEN-3945 is fixed) Thanks! ---
**Author:** ![drewmoseley](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/drewmoseley/32/47_2.png) [@drewmoseley](https://hub.mender.io/u/drewmoseley)\ **Post date:** [March 19, 2021, 9:24pm UTC](https://hub.mender.io/t/raspios-versions-vs-mender-convert-versions/3398/2 "2021-03-19T21:24:36Z")
Hi @wombat, I was just able to run a converted image using the commit and RaspiOS Lite that you referenced. It took me 2 tries because it was a bit too big for my SDCard and I didn’t notice the dd failure but otherwise I can’t see anything wrong. I’m using a Raspberry Pi 3A device but I don’t _think_ that should make a difference. Can you provide a serial console log of your device with this configuration while it is booting? Drew ---
**Author:** ![wombat](https://avatars.discourse-cdn.com/v4/letter/w/e480ec/32.png) [@wombat](https://hub.mender.io/u/wombat)\ **Post date:** [March 20, 2021, 8:23am UTC](https://hub.mender.io/t/raspios-versions-vs-mender-convert-versions/3398/3 "2021-03-20T08:23:40Z")
Thanks for the quick reply Drew! Apologies - forgot to mention that I’m running on **RPi4**. Also, is this [integration test failure at commit dab3afdf](https://gitlab.com/Northern.tech/Mender/mender-convert/-/pipelines/267995521) significant? As far as I understand, I’ll need to get it booted normally then use `rpi-eeprom-config` to [enable BOOT\_UART](https://www.raspberrypi.org/documentation/hardware/raspberrypi/bcm2711_bootloader_config.md) to get early console; I have an old bus-pirate here (with its 3.3V UART mode) ready to connect…I’ll post when I have some logs. Many thanks again! ---
**Author:** ![drewmoseley](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/drewmoseley/32/47_2.png) [@drewmoseley](https://hub.mender.io/u/drewmoseley)\ **Post date:** [March 20, 2021, 2:31pm UTC](https://hub.mender.io/t/raspios-versions-vs-mender-convert-versions/3398/4 "2021-03-20T14:31:36Z")
I doubt if that test failure is significant but @oleorhagen should be able to confirm. You should not need to jump through too many hoops to get a serial log. I just use one of [these adapters](https://learn.adafruit.com/adafruits-raspberry-pi-lesson-5-using-a-console-cable) and add ‘enable\_uart=1’ to the /boot/config.txt file. ---
**Author:** ![oleorhagen](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/oleorhagen/32/297_2.png) [@oleorhagen](https://hub.mender.io/u/oleorhagen)\ **Post date:** [March 22, 2021, 11:19am UTC](https://hub.mender.io/t/raspios-versions-vs-mender-convert-versions/3398/5 "2021-03-22T11:19:51Z")
Nope, the test failure is not significant. I have manually tested `2021-01-11` on RPi4 and it worked just fine. Why is `MEN-3945` critical to you @wombat ? ---
**Author:** ![wombat](https://avatars.discourse-cdn.com/v4/letter/w/e480ec/32.png) [@wombat](https://hub.mender.io/u/wombat)\ **Post date:** [March 22, 2021, 1:22pm UTC](https://hub.mender.io/t/raspios-versions-vs-mender-convert-versions/3398/6 "2021-03-22T13:22:55Z")
I’m getting a very strange behaviour: - When the UART is not enabled (like default `config.txt), then the system does NOT boot. *Specifically, my `config.txt`has`enable\_uart=1` commented out.\* - When the UART is enabled (`enable_uart=1` in `config.txt` file), then the system boots OK. Before running `docker-mender-convert` again, I pruned all my docker image layers/resources and then reran ‘docker-build’ (to build the mender-convert container image) at the [current HEAD of branch `2.3.x`](https://github.com/mendersoftware/mender-convert/commit/dab3afdfef01e03ca12b8678fbeda836ea276558). I ran ‘docker-mender-convert’ this against [RaspiOS Lite `2021-01-11`](https://downloads.raspberrypi.org/raspios_lite_armhf/images/raspios_lite_armhf-2021-01-12/). With the UART enabled, I was able to boot the system. I then used `rpi-eeprom-config` to set `BOOT_UART=1` in the EEPROM configuration, so that I could try and capture some console output to try and see what was happening (see below): I’ve been using my old bus-pirate (with its 3.3V UART) to attach to the primary UART (GPIO 14/15) - captured logs are below. **Default non-booting (`enable_uart=1` commented out):** ``` PM_RSTS: 0x00001000 RPi: BOOTLOADER release VERSION:c305221a DATE: Sep 3 2020 TIME: 13:11:46 BOOTMODE: 0x00000006 part: 0 BUILD_TIMESTAMP=1599135103 0xdf179f44 0x00d03114 uSD voltage 3.3V Initialising SDRAM 'Micron' 32Gb x2 total-size: 64 Gbit 3200 VLI: HUB2: 0xfff00000 0x24e6 MCU: 0xfff20000 0x15218 VL805 0xfff00000 0xfff20000 XHCI-STOP xHC ver: 256 HCS: 05000420 fc000031 00e70004 HCC: 002841eb xHC ports 5 slots 32 intrs 4 Boot mode: SD (01) order 0 SD HOST: 250000000 CTL0: 0x00000000 BUS: 100000 Hz actual: 100000 HZ div: 2500 (1250) status: 0x1fff0000 delay: 1080 SD HOST: 250000000 CTL0: 0x00000f00 BUS: 100000 Hz actual: 100000 HZ div: 2500 (1250) status: 0x1fff0000 delay: 1080 CID: 00275048534431364730014148f60122 CSD: 400e00325b590000737f7f800a400000 SD: bus-width: 4 spec: 2 SCR: 0x02358002 0x01000000 SD HOST: 250000000 CTL0: 0x00000f04 BUS: 50000000 Hz actual: 41666666 HZ div: 6 (3) status: 0x1fff0000 delay: 2 MBR: 0x00006000, 524288 type: 0x0c MBR: 0x00086000,14327808 type: 0x83 MBR: 0x00e30000,14327808 type: 0x83 MBR: 0x01bda000, 1048576 type: 0x83 lba: 24576 oem: 'mkfs.fat' volume: ' boot ' rsc 32 fat-sectors 4033 c-count 516190 c-size 1 r-dir 2 r-sec 0 PM_RSTS: 0x00001000 Partition: 0 lba: 24576 oem: 'mkfs.fat' volume: ' boot ' rsc 32 fat-sectors 4033 c-count 516190 c-size 1 r-dir 2 r-sec 0 Read config.txt bytes 1800 hnd 0x00005ef5 hash '6910d15434027841' recover4.elf not found (6) recovery.elf not found (6) Read start4.elf bytes 2215776 hnd 0x0000d407 hash '906e34e8700956d7' Read fixup4.dat bytes 5429 hnd 0x000003f8 hash '1d0725a9904ebc93' 0x00d03114 0x00000000 0x0000003f MEM GPU: 76 ARM: 948 TOTAL: 1024 Starting start4.elf @ 0xfec00200 partition 0 ``` **Booting OK (`enable_uart=1` enabled):** ``` PM_RSTS: 0x00001000 RPi: BOOTLOADER release VERSION:c305221a DATE: Sep 3 2020 TIME: 13:11:46 BOOTMODE: 0x00000006 part: 0 BUILD_TIMESTAMP=1599135103 0xdf179f44 0x00d03114 uSD voltage 3.3V Initialising SDRAM 'Micron' 32Gb x2 total-size: 64 Gbit 3200 VLI: HUB2: 0xfff00000 0x24e6 MCU: 0xfff20000 0x15218 VL805 0xfff00000 0xfff20000 XHCI-STOP xHC ver: 256 HCS: 05000420 fc000031 00e70004 HCC: 002841eb xHC ports 5 slots 32 intrs 4 Boot mode: SD (01) order 0 SD HOST: 250000000 CTL0: 0x00000000 BUS: 100000 Hz actual: 100000 HZ div: 2500 (1250) status: 0x1fff0000 delay: 1080 SD HOST: 250000000 CTL0: 0x00000f00 BUS: 100000 Hz actual: 100000 HZ div: 2500 (1250) status: 0x1fff0000 delay: 1080 CID: 00275048534431364730014148f60122 CSD: 400e00325b590000737f7f800a400000 SD: bus-width: 4 spec: 2 SCR: 0x02358002 0x01000000 SD HOST: 250000000 CTL0: 0x00000f04 BUS: 50000000 Hz actual: 41666666 HZ div: 6 (3) status: 0x1fff0000 delay: 2 MBR: 0x00006000, 524288 type: 0x0c MBR: 0x00086000,14327808 type: 0x83 MBR: 0x00e30000,14327808 type: 0x83 MBR: 0x01bda000, 1048576 type: 0x83 lba: 24576 oem: 'mkfs.fat' volume: ' boot ' rsc 32 fat-sectors 4033 c-count 516190 c-size 1 r-dir 2 r-sec 0 PM_RSTS: 0x00001000 Partition: 0 lba: 24576 oem: 'mkfs.fat' volume: ' boot ' rsc 32 fat-sectors 4033 c-count 516190 c-size 1 r-dir 2 r-sec 0 Read config.txt bytes 1799 hnd 0x00005eed hash '68ca07b419dcf6ab' recover4.elf not found (6) recovery.elf not found (6) Read start4.elf bytes 2215776 hnd 0x0000d407 hash '906e34e8700956d7' Read fixup4.dat bytes 5429 hnd 0x000003f8 hash '1d0725a9904ebc93' 0x00d03114 0x00000000 0x0000003f MEM GPU: 76 ARM: 948 TOTAL: 1024 Starting start4.elf @ 0xfec00200 partition 0 U-Boot 2020.01-g83cf4883ec (Mar 09 2021 - 08:12:31 +0000) DRAM: 3.9 GiB RPI 4 Model B (0xd03114) MMC: mmcnr@7e300000: 1, emmc2@7e340000: 0 Loading Environment from MMC... OK In: serial Out: serial Err: serial Net: Net Initialization Skipped No ethernet found. Hit any key to stop autoboot: 0 switch to partitions #0, OK mmc0 is current device Scanning mmc 0:1... Found U-Boot script /boot.scr 568 bytes read in 49 ms (10.7 KiB/s) ## Executing script at 02400000 switch to partitions #0, OK mmc0 is current device ... ``` Also, I tried setting `config.txt` options `start_file=start4db.elf` and `fixup_file=fixup4db.dat`, but this doesn’t yield anything different in the console output. If I understand the [RPi4 UART configuration](https://www.raspberrypi.org/documentation/configuration/uart.md) correctly: - There are 6 UARTs (5 x PL011, 1 x ‘mini UART’), and the “primary UART” is connected to GPIO 14/15. - The “primary UART” is of type ‘mini UART’, which is disabled by default. - The ‘mini UART’ type requires a fixed VPU core clock frequency, and setting `enable_uart=1` sets the GPU core clock to 250 MHz. _Could this (the GPU variable clock) be a factor?_ A colleague of mine also sees this behaviour (with the same mender-converted image) on his RPi4 - i.e. that setting `enable_uart=1` allows the system to boot OK. So for now we can continue developing, but it would be good to know what’s causing this behaviour. Thanks again for all your help! ---
**Author:** ![wombat](https://avatars.discourse-cdn.com/v4/letter/w/e480ec/32.png) [@wombat](https://hub.mender.io/u/wombat)\ **Post date:** [March 22, 2021, 2:55pm UTC](https://hub.mender.io/t/raspios-versions-vs-mender-convert-versions/3398/7 "2021-03-22T14:55:28Z")
Hi! All I am currently interested in, is making a bootable mender-converted system based on RaspiOS Lite (preferable a recent release). _Background - we are Linux users/developers but are not embedded Linux developers (yet) :-). In other words, the convenience of Debian and mender-convert is appreciated/valued as quick way for us to get up and running with OTA update prototyping; in the future we may look to Yocto or other routes to building a custom Linux for our application._ Initially, I got started very quickly with mender-convert (at tag 2.3.0) and RaspiOS Lite 2020-05-27 and everything was working as expected “out of the box”. However, I noticed that if I tried later versions of RaspiOS, the resulting img (from mender-convert) would not boot. Also, I had seen [MEN-3941](https://tracker.mender.io/browse/MEN-3941) and then later [MEN-4395](https://tracker.mender.io/browse/MEN-4395), so I was hoping/expecting to be able to use RaspiOS Lite 2020-01-11 to continue our evaluation of [hosted.mender.io](http://hosted.mender.io) (v2.6) and mender-convert. **So, any guidance you can offer would be most appreciated; right now it looks like we either have to stick with the older 2020-05-27 RaspiOS Lite, or (assuming there are no other issues) just set `enable_uart=1` in our config.txt in order to make our mender-converted 2021-01-11 RaspiOS Lite bootable.** Apologies if I’ve misunderstood anything or missed some documentation; this is all part of our exploration of mender. Thanks again to you and the mender team for a great set of tools! Kind regards Tom (wombat) ---
**Author:** ![drewmoseley](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/drewmoseley/32/47_2.png) [@drewmoseley](https://hub.mender.io/u/drewmoseley)\ **Post date:** [March 22, 2021, 4:47pm UTC](https://hub.mender.io/t/raspios-versions-vs-mender-convert-versions/3398/8 "2021-03-22T16:47:18Z")
Hi Tom, It sounds like leaving the UART enabled is a reasonable workaround for you here. I suppose it is possible that the GPU clock could be part of the problem but it’s hard to say for certain. Can you confirm that you do _not_ see this behavior in the standard RaspiOS versions? The one big change that I think could affect this is the use of U-Boot with Mender. When you get a board in the non-booting state, do you have a display of some kind connected? Is there any output? Drew ---
**Author:** ![oleorhagen](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/oleorhagen/32/297_2.png) [@oleorhagen](https://hub.mender.io/u/oleorhagen)\ **Post date:** [March 23, 2021, 10:30am UTC](https://hub.mender.io/t/raspios-versions-vs-mender-convert-versions/3398/9 "2021-03-23T10:30:37Z")
Good instincts @drewmoseley I found [this](https://lists.denx.de/pipermail/u-boot/2016-March/249623.html) on the mailing list. So indeed it seems the GPU clock is divided for the UART, and this is causing some problems. Indeed, I see that [this blogpost](https://andrei.gherzan.ro/linux/uboot-on-rpi/) mentions: ```auto The enable_uart setting is required because U-Boot assumes the VideoCore firmware is configured to use the mini UART (rather than PL011) for the serial console. Without this, U-Boot will not boot at all. ``` ---
**Author:** ![wombat](https://avatars.discourse-cdn.com/v4/letter/w/e480ec/32.png) [@wombat](https://hub.mender.io/u/wombat)\ **Post date:** [March 23, 2021, 2:38pm UTC](https://hub.mender.io/t/raspios-versions-vs-mender-convert-versions/3398/10 "2021-03-23T14:38:17Z")
Thanks for confirming (finding that uboot post)! I happened across this behaviour by chance, enabling the mini UART in order to get more dianostics and seeing that it actually allowed uboot to proceed. 🙂 Regarding Drew’s question, yes I have an HDMI display attached. When the system is not booting (i.e. `enable_uart=1` is not set present and console output stops at `Starting start4.elf...`), the HDMI output is black. The EEPROM HDMI debug screen has disappeared at this point in the boot process. Tom ---
**Author:** ![drewmoseley](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/drewmoseley/32/47_2.png) [@drewmoseley](https://hub.mender.io/u/drewmoseley)\ **Post date:** [March 23, 2021, 9:30pm UTC](https://hub.mender.io/t/raspios-versions-vs-mender-convert-versions/3398/11 "2021-03-23T21:30:22Z")
Hi Tom, Given the comment from @oleorhagen and the posts he references, I guess the behavior you describe with HDMI is unsurprising. It sounds like `enable_uart=1` should be considered mandatory here and not just as a workaround. Drew ---
**Author:** ![wombat](https://avatars.discourse-cdn.com/v4/letter/w/e480ec/32.png) [@wombat](https://hub.mender.io/u/wombat)\ **Post date:** [March 24, 2021, 8:47pm UTC](https://hub.mender.io/t/raspios-versions-vs-mender-convert-versions/3398/12 "2021-03-24T20:47:48Z")
Agreed. 🙂 Given the `enable_uart` requirement of uboot in RPi4 has been [commented in the uboot source for the last year](https://source.denx.de/u-boot/u-boot/-/commit/c6bcf05fcd61fce1092b7fca945982a65d0c723b), I’m a little curious to know how mender-convert 2.3.x was tested without noticing this issue before. _Was it perhaps that the test systems had this option enabled already so it went unnoticed?_ ---
**Author:** ![drewmoseley](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/drewmoseley/32/47_2.png) [@drewmoseley](https://hub.mender.io/u/drewmoseley)\ **Post date:** [March 25, 2021, 1:12am UTC](https://hub.mender.io/t/raspios-versions-vs-mender-convert-versions/3398/13 "2021-03-25T01:12:39Z")
Yeah, my guess is all testing has been done on devices with UART enabled. I know that’s the first thing I do on any of these devices. @oleorhagen would it make sense to modify mender-convert to ensure that the UART is enabled? Drew ---
**Author:** ![oleorhagen](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/oleorhagen/32/297_2.png) [@oleorhagen](https://hub.mender.io/u/oleorhagen)\ **Post date:** [March 25, 2021, 7:47am UTC](https://hub.mender.io/t/raspios-versions-vs-mender-convert-versions/3398/14 "2021-03-25T07:47:53Z")
Yes, it is tested with UART enabled by default (just gives more output from which you can debug). It never occured to me that this might come back and bite us like this, hah. We are having an internal discussion about how to approach this right now, I will get back to you within the day with a response on what we will decide to do here 🙂 ---
**Author:** ![lluiscampos](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/lluiscampos/32/112_2.png) [@lluiscampos](https://hub.mender.io/u/lluiscampos)\ **Post date:** [March 25, 2021, 2:42pm UTC](https://hub.mender.io/t/raspios-versions-vs-mender-convert-versions/3398/15 "2021-03-25T14:42:41Z")
We decided to enable uart for all boards. See pull request [here](https://github.com/mendersoftware/mender-convert/pull/328). ---
**Author:** ![wombat](https://avatars.discourse-cdn.com/v4/letter/w/e480ec/32.png) [@wombat](https://hub.mender.io/u/wombat)\ **Post date:** [March 31, 2021, 10:03am UTC](https://hub.mender.io/t/raspios-versions-vs-mender-convert-versions/3398/16 "2021-03-31T10:03:56Z")
Thanks again to everyone on the Mender team for the quick follow-up and responses to this topic! _ **It’s much appreciated!** _