Raspberry Pi Stuck in U-Boot

General Discussions
1

Rather new to Mender, so I basically followed this post: Raspberry Pi 3 Model B/B+ Raspbian. I substituted the official raspbian image with a custom one built via pi-gen (Only modifications around boot is enabling i2c). I wrote the .sdimg to the sd card using etcher. The image does work without using mender-convert.

I used the following to convert:

./docker-mender-convert from-raw-disk-image                       \
            --raw-disk-image $RAW_DISK_IMAGE                      \
            --mender-disk-image $MENDER_DISK_IMAGE                \
            --device-type raspberrypi3                          \
            --artifact-name $ARTIFACT_NAME                        \
            --bootloader-toolchain arm-buildroot-linux-gnueabihf  \
            --server-url "https://mender.example.com"

I noticed the pi wasn’t booting, so I connected a serial and saw the following:

U-Boot 2018.07-g981cc831e3 (May 13 2019 - 01:06:40 +0000)

DRAM:  948 MiB
RPI 3 Model B (0xa22082)
MMC:   mmc@7e202000: 0, sdhci@7e300000: 1
Loading Environment from FAT... *** Warning - bad CRC, using default environment

Failed (-5)
Loading Environment from MMC... OK
In:    serial
Out:   serial
Err:   serial
Net:   No ethernet found.
Hit any key to stop autoboot:  0
switch to partitions #0, OK
mmc0 is current device
Scanning mmc 0:1...
Found U-Boot script /boot.scr
300 bytes read in 0 ms
## Executing script at 02400000
libfdt fdt_check_header(): FDT_ERR_BADMAGIC
switch to partitions #0, OK
mmc0 is current device
4944992 bytes read in 209 ms (22.6 MiB/s)
Kernel image @ 0x080000 [ 0x000000 - 0x4b7460 ]
ERROR: Did not find a cmdline Flattened Device Tree
Could not find a valid device tree
SCRIPT FAILED: continuing...
starting USB...

I’m not quite sure where to start troubleshooting, any help would be appreciated.

2

Hi @Silvenga,

First of all welcome to Mender Hub :slight_smile:.

Did you try the official Raspbian image?

But looks very similar to the problem reported here,

https://tracker.mender.io/browse/MEN-2436

Could you try removing init=/usr/lib/raspi-config/init_resize.sh from cmdline.txt before attempting the first boot. I suspect it is related to this, though we have not found out why.

3

I appreciate the guidance, you put me on the right path! I removed that directive and my custom image boots (and reboots correctly). I’m wondering if there was some error on first boot now… I’ll reflash again and monitor first boot for anything.

Thanks! Anything I can provide to help track down that bug?

4

I’m wondering if there was some error on first boot now… I’ll reflash again and monitor first boot for anything.

What that line does is that it will on first boot try to expand the “data” partition to occupy the remaining space available on the disk and by removing this line you have skipped this step.

There is probably something going wrong in the “expand” procedure that corrupts the SD.

Thanks! Anything I can provide to help track down that bug?

You could try to manually run the scripts, which is also mentioned in [MEN-2436] - Mender and CFEngine (by Northern.tech) Jira and try to isolate which one of the breaks it.

5

I ran the script manually (well parts of it):

$ROOT_PART_NAME: mmcblk0p4
$ROOT_DEV_NAME: mmcblk0
$ROOT_DEV: /dev/mmcblk0
$ROOT_PART_NUM: 4
$BOOT_PART_DEV: /dev/mmcblk0p1
$BOOT_PART_NAME: mmcblk0p1
$BOOT_DEV_NAME: mmcblk0
$BOOT_PART_NUM: 1
$ROOT_DEV_SIZE: 62518272
$TARGET_END: 62518271
$PARTITION_TABLE: BYT; /dev/mmcblk0:62518272:d/mmc:512:512:mdo:SD 00000:; 1:24576:114687:90112:fat32::boot, lba; 2:114688:8118271:8003584:ext4::; 3:8118272:16121855:8003584:ext4::; 4:16121856:16383999:262144:ext4::;
$LAST_PART_NUM: 4
$ROOT_PART_LINE: 4:16121856:16383999:262144:ext4::;
$ROOT_PART_START: 16121856
$ROOT_PART_END: 16383999

# Effective command:
parted -m /dev/mmcblk0 u s resizepart 4 62518271

And the pi still works. The /usr/sbin/resizefs.sh script just aborts.

I re-imaged, and I got the following on the boot right after the resize reboot (without the init work around). The error message differs from the third reboot.

[    7.476983] sysrq: SysRq : Resetting
[    7.480635] CPU0: stopping
[    7.483637] CPU1: stopping
[    7.486595] CPU3: stopping


U-Boot 2018.07-g981cc831e3 (May 13 2019 - 01:06:40 +0000)

DRAM:  948 MiB
RPI 3 Model B (0xa22082)
MMC:   mmc@7e202000: 0, sdhci@7e300000: 1
Loading Environment from FAT... *** Warning - bad CRC, using default environment

Failed (-5)
Loading Environment from MMC... OK
In:    serial
Out:   serial
Err:   serial
Net:   No ethernet found.
Hit any key to stop autoboot:  0
switch to partitions #0, OK
mmc0 is current device
Scanning mmc 0:1...
Found U-Boot script /boot.scr
300 bytes read in 1 ms (293 KiB/s)
## Executing script at 02400000
libfdt fdt_path_offset() returned FDT_ERR_BADSTRUCTURE
switch to partitions #0, OK
mmc0 is current device
4944992 bytes read in 208 ms (22.7 MiB/s)
Kernel image @ 0x080000 [ 0x000000 - 0x4b7460 ]
## Flattened Device Tree blob at 2eff9800
   Booting using the fdt blob at 0x2eff9800
   reserving fdt memory region: addr=0 size=1000
   reserving fdt memory region: addr=2eff9800 size=7800
   Using Device Tree in place at 2eff9800, end 2f003fff
WARNING: could not set serial-number FDT_ERR_BADSTRUCTURE.
ERROR: root node setup failed
 - must RESET the board to recover.

FDT creation failed! hanging...### ERROR ### Please RESET the board ###
6

There has been some progress on this, thanks to @MarekBelisko, more info here,

7

@Silvenga
if you want to try my fix pls apply this patch on top of mender-convert:

diff --git a/rpi-convert-stage-5.sh b/rpi-convert-stage-5.sh
index 799e652..1ef88c0 100755
--- a/rpi-convert-stage-5.sh
+++ b/rpi-convert-stage-5.sh
@@ -33,8 +33,8 @@ declare -a mender_partitions_regular=("boot" "primary" "secondary" "data")
 build_uboot_files() {
   local CROSS_COMPILE=${1}-
   local ARCH=arm
-  local branch="mender-rpi-2018.07"
-  local commit="981cc831e3"
+  local branch="mbe-test"
+  local commit="44dc10bb7225c3c578e75b77a03d81cbff56225c"
   local uboot_repo_vc_dir=$uboot_dir/.git
   local defconfig="rpi_3_32b_defconfig"
 
@@ -50,7 +50,7 @@ build_uboot_files() {
   log "\tBuilding U-Boot related files."
 
   if [ ! -d $uboot_repo_vc_dir ]; then
-    git clone https://github.com/mendersoftware/uboot-mender.git -b $branch >> "$build_log" 2>&1
+    git clone https://github.com/nandra/uboot-mender.git -b $branch >> "$build_log" 2>&1
   fi
 
   cd $uboot_dir
8

And it’s alive! I can confirm those changes to uboot-mender do resolve the problem. Thanks for the help @mirzak @MarekBelisko!

9

@Silvenga there have been update pushed fix this issue. Please update your repo to origin/1.1.x

10

I am doing the same thing as OP after doing a dry run of mender a few months ago.

Can we update Raspberry Pi 3 Model B/B+ Raspbian or mention this somewhere on that guide also?

I have been working all night on this stuff, and I got to the end of it all, and it seems I am having the same experience. I do not know for sure yet, but I am working w/ 1.1

I am starting over…

I guess I just

git clone --no-checkout
git checkout 1.1.x

Thanks for working on all of this, it is pretty late here, so I apologize for my dry lifeless post.

11

Thanks for bringing this up, it makes sense to update Raspberry Pi 3 Model B/B+ Raspbian to utilize 1.1.x branch which does contain the U-boot fix.

12

I am going to have 1.1.x tested over here in a sec too…

1 Like
13

I just got done removing

init=/usr/lib/raspi-config/init_resize.sh

From a written image just like in this guide, and it did solve my issue. I am grateful this fix is here.

I am about to write a 1.1.x convert, and I will report on that. I can also report on artifact application, because that is the last step in my testing.

The report on artifact testing will most likely be later today.

14

1.1.x is working great!

1 Like
15

Thanks for reporting.

16

Switch over the that branch, and Yay! Everything reboots successfully and the resize of the data partition occurs. Thanks for your help.

Unrelated, I’m not sure if resizefs.sh is executing correctly.

# fdisk -l /dev/mmcblk0p4
Disk /dev/mmcblk0p4: 22.1 GiB
# df -h /dev/mmcblk0p4
/dev/mmcblk0p4  120M  1.6M  110M   2% /data

However, the resize unit definitely executed, but it exited with Data partition already resized, aborting on first boot and disabled itself:

# systemctl status resizefs.service
â—Ź resizefs.service - Expand data partition file system
   Loaded: loaded (/lib/systemd/system/resizefs.service; disabled; vendor preset: enabled)
   Active: inactive (dead)
17

Thanks for reporting, I created a ticket for this,

https://tracker.mender.io/browse/MEN-2564

18

It seems to work for me, and I have not examined it on other devices but:

/dev/sda1        58G   18G   41G  30% /media/usb0
/dev/mmcblk0p1   44M   18M   26M  40% /uboot
/dev/mmcblk0p4  6.3G  3.2M  6.0G   1% /data

What was your mender-convert command…it could be that doing this:
--storage-total-size-mb 24000 \
Helped?