Updating the boot partition on Raspberry using Debian (non Yocto)

dwjbosman · February 11, 2021, 1:06pm

I am using mender 2.5 and Debian on Raspberry 3B

I need to update the boot partition. I am following these steps:

Configure the golden device
Use mender-convert to create an artifact and .img
Mount the boot partition of the generated .img and copy the /uboot and /boot files into an overlay folder /boot_update/uboot, /boot_update/boot
Regenerate the artifact including the /boot_update folder
An ArtifactInstall_Leave script uses rsync to copy the /boot_update folders to /uboot and /boot
Mender performs a reboot

It doesn’t seem to work correctly.
*The kernel is updated
*For some time (several minutes) the mender server doesn’t see the device
*In the logs I don’t see the artifactinstall_leave script called
*The device boots up with a black screen and resets after sometime. As if a rollback is being performed? But in the end the device runs the updated firmware…

Should this approach updating /uboot and /boot work?

Will Mender specific configuration settings on /uboot (the active A/B partition?) be kept, or are these overwritten?

oleorhagen · February 11, 2021, 3:01pm

Hi @dwjbosman

Any chance you could post the logs to begin with?

Also, have a look at this, even though it is not for mender-convert, I think it is informal.

In general, /boot is mounted as a separate partition. Have a look in fstab, to make sure if it is boot, or /uboot (I think it is /uboot/ in this case).

So, in short, everything written to the boot partition is kept, as it is a separate partition.

The rest of your questions I think I need a log to look further into

dwjbosman · February 12, 2021, 12:08am

Hi @oleorhagen,

I made the mistake thinking that the newly downloaded rootfs was mounted at step 5. So the script was actually rsync’ing the old/currently installed boot files…

So now the steps become:

Configure the golden device
Use mender-convert to create an artifact and .img
Mount the boot partition of the generated .img and copy the /uboot and /boot files into an overlay folder /boot_update/uboot, /boot_update/boot
Regenerate the artifact including the /boot_update folder
An ArtifactReboot_Leave script uses rsync to copy the /boot_update folders from the newly installed rootfs to /uboot and /boot
5.1. The ArtifactReboot_Leave script will need to reboot the device again so that the new kernel boots.

Is it allowed to do another reboot in 5.1. Or will mender then assume that the install failed (and try to rollback)?

oleorhagen · February 12, 2021, 7:47am

Hi

For 5.1, you cannot reboot. This will mark the udpate as failed, and mender will fall back.
However, you can do a reboot in ArtifactCommit_Leave. This will do a reboot, and not roll back to the old update, as the new image is already committed.

Please give it a try, and tell me how it goes

dwjbosman · February 12, 2021, 12:13pm

Used the ArtifactCommit_Leave script. It includes a ‘reboot’ command.

This currently results in an error:

retry [ArtifactCommit_Enter] → update-commit [ArtifactCommit_Enter]
2021-02-12 11:30:53 +0000 UTC info: Committing update
2021-02-12 11:30:53 +0000 UTC info: Probing the Bootloader environment for which separator to use
2021-02-12 11:30:53 +0000 UTC info: Writing map[mender_uboot_separator:1] to the U-Boot environment, using separator:
2021-02-12 11:30:54 +0000 UTC info: Writing map[mender_uboot_separator:] to the U-Boot environment, using separator:
2021-02-12 11:30:54 +0000 UTC info: Writing map[upgrade_available:0] to the U-Boot environment, using separator:
2021-02-12 11:30:54 +0000 UTC info: State transition: update-commit [ArtifactCommit_Enter] → update-after-first-commit [none]
2021-02-12 11:30:54 +0000 UTC info: State transition: update-after-first-commit [none] → update-after-commit [ArtifactCommit_Leave]
2021-02-12 11:30:54 +0000 UTC info: State transition: update-after-commit [ArtifactCommit_Leave] → cleanup [none]
2019-02-14 10:12:23 +0000 UTC info: Running Mender client version: 2.4.1
2019-02-14 10:12:23 +0000 UTC error: Mender shut down in state: update-after-commit
2019-02-14 10:12:23 +0000 UTC info: State transition: init [none] → update-after-commit [ArtifactCommit_Leave]
2019-02-14 10:12:23 +0000 UTC info: State transition: update-after-commit [ArtifactCommit_Leave] → cleanup [none]
2019-02-14 10:12:24 +0000 UTC info: Running Mender client version: 2.4.1
2019-02-14 10:12:24 +0000 UTC error: Mender shut down in state: update-after-commit
2019-02-14 10:12:24 +0000 UTC info: State transition: init [none] → update-after-commit [ArtifactCommit_Leave]
2019-02-14 10:12:24 +0000 UTC info: State transition: update-after-commit [ArtifactCommit_Leave] → cleanup [none]
2019-02-14 10:12:25 +0000 UTC info: Running Mender client version: 2.4.1
2019-02-14 10:12:25 +0000 UTC error: Mender shut down in state: update-after-commit
2019-02-14 10:12:25 +0000 UTC info: State transition: init [none] → update-after-commit [ArtifactCommit_Leave]
2019-02-14 10:12:25 +0000 UTC info: State transition: update-after-commit [ArtifactCommit_Leave] → cleanup [none]
2019-02-14 10:12:24 +0000 UTC info: Running Mender client version: 2.4.1
2019-02-14 10:12:24 +0000 UTC error: Mender shut down in state: update-after-commit
2019-02-14 10:12:24 +0000 UTC info: State transition: init [none] → update-after-commit [ArtifactCommit_Leave]
2019-02-14 10:12:24 +0000 UTC info: State transition: update-after-commit [ArtifactCommit_Leave] → cleanup [none]
2019-02-14 10:12:24 +0000 UTC info: Running Mender client version: 2.4.1
2019-02-14 10:12:24 +0000 UTC error: Mender shut down in state: update-after-commit
2019-02-14 10:12:24 +0000 UTC info: State transition: init [none] → update-after-commit [ArtifactCommit_Leave]
2019-02-14 10:12:24 +0000 UTC info: State transition: update-after-commit [ArtifactCommit_Leave] → cleanup [none]
2019-02-14 10:14:12 +0000 UTC info: Running Mender client version: 2.4.1
2019-02-14 10:14:12 +0000 UTC error: Mender shut down in state: update-after-commit
2019-02-14 10:14:12 +0000 UTC info: State transition: init [none] → update-after-commit [ArtifactCommit_Leave]
2019-02-14 10:14:12 +0000 UTC info: State transition: update-after-commit [ArtifactCommit_Leave] → cleanup [none]
2019-02-14 10:12:23 +0000 UTC info: Running Mender client version: 2.4.1
2019-02-14 10:12:23 +0000 UTC error: Mender shut down in state: update-after-commit
2019-02-14 10:12:23 +0000 UTC info: State transition: init [none] → update-after-commit [ArtifactCommit_Leave]
2019-02-14 10:12:23 +0000 UTC info: State transition: update-after-commit [ArtifactCommit_Leave] → cleanup [none]
2019-02-14 10:12:23 +0000 UTC info: Running Mender client version: 2.4.1
2019-02-14 10:12:23 +0000 UTC error: Mender shut down in state: update-after-commit
2019-02-14 10:12:23 +0000 UTC info: State transition: init [none] → update-after-commit [ArtifactCommit_Leave]
2019-02-14 10:12:23 +0000 UTC info: State transition: update-after-commit [ArtifactCommit_Leave] → cleanup [none]
2019-02-14 10:12:24 +0000 UTC info: Running Mender client version: 2.4.1
2019-02-14 10:12:24 +0000 UTC error: Mender shut down in state: update-after-commit
2019-02-14 10:12:24 +0000 UTC info: State transition: init [none] → update-after-commit [ArtifactCommit_Leave]
2019-02-14 10:12:24 +0000 UTC info: State transition: update-after-commit [ArtifactCommit_Leave] → cleanup [none]
2019-02-14 10:12:24 +0000 UTC info: Running Mender client version: 2.4.1
2019-02-14 10:12:24 +0000 UTC error: Mender shut down in state: update-after-commit
2019-02-14 10:12:24 +0000 UTC info: State transition: init [none] → update-after-commit [ArtifactCommit_Leave]
2019-02-14 10:12:24 +0000 UTC info: State transition: update-after-commit [ArtifactCommit_Leave] → cleanup [none]
2019-02-14 10:12:24 +0000 UTC info: Running Mender client version: 2.4.1
2019-02-14 10:12:24 +0000 UTC error: Mender shut down in state: update-after-commit
2019-02-14 10:12:24 +0000 UTC info: State transition: init [none] → update-after-commit [ArtifactCommit_Leave]
2019-02-14 10:12:24 +0000 UTC info: State transition: update-after-commit [ArtifactCommit_Leave] → cleanup [none]
2019-02-14 10:12:23 +0000 UTC info: Running Mender client version: 2.4.1
2019-02-14 10:12:23 +0000 UTC error: Mender shut down in state: update-after-commit
2019-02-14 10:12:23 +0000 UTC info: State transition: init [none] → update-after-commit [ArtifactCommit_Leave]
2019-02-14 10:12:23 +0000 UTC info: State transition: update-after-commit [ArtifactCommit_Leave] → cleanup [none]
2019-02-14 10:12:23 +0000 UTC info: Running Mender client version: 2.4.1
2019-02-14 10:12:23 +0000 UTC error: Mender shut down in state: update-after-commit
2019-02-14 10:12:23 +0000 UTC info: State transition: init [none] → update-after-commit [ArtifactCommit_Leave]
2019-02-14 10:12:23 +0000 UTC info: State transition: update-after-commit [ArtifactCommit_Leave] → cleanup [none]
2019-02-14 10:12:23 +0000 UTC info: Running Mender client version: 2.4.1
2019-02-14 10:12:23 +0000 UTC error: Mender shut down in state: update-after-commit
2019-02-14 10:12:23 +0000 UTC info: State transition: init [none] → update-after-commit [ArtifactCommit_Leave]
2019-02-14 10:12:23 +0000 UTC info: State transition: update-after-commit [ArtifactCommit_Leave] → cleanup [none]
2019-02-14 10:12:23 +0000 UTC info: Running Mender client version: 2.4.1
2019-02-14 10:12:23 +0000 UTC error: Mender shut down in state: update-after-commit
2019-02-14 10:12:23 +0000 UTC info: State transition: init [none] → update-after-commit [ArtifactCommit_Leave]
2019-02-14 10:12:23 +0000 UTC info: State transition: update-after-commit [ArtifactCommit_Leave] → cleanup [none]
2019-02-14 10:12:23 +0000 UTC info: Running Mender client version: 2.4.1
2019-02-14 10:12:23 +0000 UTC error: Mender shut down in state: update-after-commit
2019-02-14 10:12:23 +0000 UTC info: State transition: init [none] → update-after-commit [ArtifactCommit_Leave]
2019-02-14 10:12:23 +0000 UTC info: State transition: update-after-commit [ArtifactCommit_Leave] → cleanup [none]
2019-02-14 10:12:22 +0000 UTC info: Running Mender client version: 2.4.1
2019-02-14 10:12:22 +0000 UTC error: Mender shut down in state: update-after-commit
2019-02-14 10:12:22 +0000 UTC info: State transition: init [none] → update-after-commit [ArtifactCommit_Leave]
2019-02-14 10:12:22 +0000 UTC info: State transition: update-after-commit [ArtifactCommit_Leave] → cleanup [none]
2019-02-14 10:12:23 +0000 UTC info: Running Mender client version: 2.4.1
2019-02-14 10:12:23 +0000 UTC error: Mender shut down in state: update-after-commit
2019-02-14 10:12:23 +0000 UTC info: State transition: init [none] → update-after-commit [ArtifactCommit_Leave]
2019-02-14 10:12:23 +0000 UTC info: State transition: update-after-commit [ArtifactCommit_Leave] → cleanup [none]
2019-02-14 10:12:24 +0000 UTC info: Running Mender client version: 2.4.1
2019-02-14 10:12:24 +0000 UTC error: Mender shut down in state: update-after-commit
2019-02-14 10:12:24 +0000 UTC info: State transition: init [none] → update-after-commit [ArtifactCommit_Leave]
2019-02-14 10:12:24 +0000 UTC info: State transition: update-after-commit [ArtifactCommit_Leave] → cleanup [none]
2019-02-14 10:12:24 +0000 UTC info: Running Mender client version: 2.4.1
2019-02-14 10:12:24 +0000 UTC error: Mender shut down in state: update-after-commit
2019-02-14 10:12:24 +0000 UTC info: State transition: init [none] → update-after-commit [ArtifactCommit_Leave]
2019-02-14 10:12:24 +0000 UTC info: State transition: update-after-commit [ArtifactCommit_Leave] → cleanup [none]
2019-02-14 10:12:23 +0000 UTC info: Running Mender client version: 2.4.1
2019-02-14 10:12:23 +0000 UTC error: Mender shut down in state: update-after-commit
2019-02-14 10:12:23 +0000 UTC info: State transition: init [none] → update-after-commit [ArtifactCommit_Leave]
2019-02-14 10:12:23 +0000 UTC info: State transition: update-after-commit [ArtifactCommit_Leave] → cleanup [none]
2019-02-14 10:12:23 +0000 UTC info: Running Mender client version: 2.4.1
2019-02-14 10:12:23 +0000 UTC error: Mender shut down in state: update-after-commit
2019-02-14 10:12:23 +0000 UTC info: State transition: init [none] → update-after-commit [ArtifactCommit_Leave]
2019-02-14 10:12:23 +0000 UTC info: State transition: update-after-commit [ArtifactCommit_Leave] → cleanup [none]
2019-02-14 10:12:24 +0000 UTC info: Running Mender client version: 2.4.1
2019-02-14 10:12:24 +0000 UTC error: Mender shut down in state: update-after-commit
2019-02-14 10:12:24 +0000 UTC info: State transition: init [none] → update-after-commit [ArtifactCommit_Leave]
2019-02-14 10:12:24 +0000 UTC info: State transition: update-after-commit [ArtifactCommit_Leave] → cleanup [none]
2019-02-14 10:12:24 +0000 UTC info: Running Mender client version: 2.4.1
2019-02-14 10:12:24 +0000 UTC error: Mender shut down in state: update-after-commit
2019-02-14 10:12:24 +0000 UTC info: State transition: init [none] → update-after-commit [ArtifactCommit_Leave]
2019-02-14 10:12:24 +0000 UTC info: State transition: update-after-commit [ArtifactCommit_Leave] → cleanup [none]
2019-02-14 10:12:23 +0000 UTC info: Running Mender client version: 2.4.1
2019-02-14 10:12:23 +0000 UTC error: Mender shut down in state: update-after-commit
2019-02-14 10:12:23 +0000 UTC info: State transition: init [none] → update-after-commit [ArtifactCommit_Leave]
2019-02-14 10:12:23 +0000 UTC info: State transition: update-after-commit [ArtifactCommit_Leave] → cleanup [none]
2019-02-14 10:12:23 +0000 UTC info: Running Mender client version: 2.4.1
2019-02-14 10:12:23 +0000 UTC error: Mender shut down in state: update-after-commit
2019-02-14 10:12:23 +0000 UTC info: State transition: init [none] → update-after-commit [ArtifactCommit_Leave]
2019-02-14 10:12:23 +0000 UTC info: State transition: update-after-commit [ArtifactCommit_Leave] → cleanup [none]
2019-02-14 10:12:29 +0000 UTC info: Running Mender client version: 2.4.1
2019-02-14 10:12:29 +0000 UTC error: Mender shut down in state: update-after-commit
2019-02-14 10:12:29 +0000 UTC info: State transition: init [none] → update-after-commit [ArtifactCommit_Leave]
2019-02-14 10:12:29 +0000 UTC info: State transition: update-after-commit [ArtifactCommit_Leave] → cleanup [none]
2019-02-14 10:12:23 +0000 UTC info: Running Mender client version: 2.4.1
2019-02-14 10:12:23 +0000 UTC error: State transition loop detected in state update-after-commit: Forcefully aborting update. The system is likely to be in an inconsistent state after this.

I will turn on debugging and retest

oleorhagen · February 12, 2021, 11:46pm

Oh, this is because of this. I’m sorry, I did not think it was also handled in commit-leave.

This means that you’re script needs to have some sort of state (like a file?), so that you will only reboot the one time

dwjbosman · February 12, 2021, 11:46pm

I used the ArtifactCommit_Leave script to do the reboot. In the script I just deleted the script, so that no multiple reboot would take place

sh update_kernel.sh
rm -f /data/mender/scripts/ArtifactCommit_Leave_02
reboot