Hi everyone,
I encountered an issue where the system update failed at the reboot stage.
The artifact I used for the system update is based on Rocky Linux 8.
Although the menderized Rocky Linux 8 image can be launched successfully, the artifact cannot be used for a system update due to a failure at the reboot stage.
This is puzzling because the conversion steps are the same as those used for the menderized CentOS 7 and RHEL 8 images, both of which can be updated successfully.
Main error message:
2023-11-29 23:49:36 +0000 UTC error: transient error: Reboot to the new update failed. Expected "upgrade_available" flag to be true but it was false. Either the switch to the new partition was unsuccessful, or the bootloader rolled back
Full system update log
2023-11-29 23:38:44 +0000 UTC info: Running Mender client version: 3.5.1
2023-11-29 23:38:44 +0000 UTC info: State transition: update-fetch [Download_Enter] -> update-store [Download_Enter]
2023-11-29 23:38:44 +0000 UTC info: No public key was provided for authenticating the artifact
2023-11-29 23:38:44 +0000 UTC info: Opening device "/dev/sda3" for writing
2023-11-29 23:38:44 +0000 UTC info: Native sector size of block device /dev/sda3 is 512 bytes. Mender will write in chunks of 1048576 bytes
2023-11-29 23:40:46 +0000 UTC info: All bytes were successfully written to the new partition
2023-11-29 23:40:46 +0000 UTC info: The optimized block-device writer wrote a total of 20481 frames, where 3330 frames did need to be rewritten (i.e., skipped)
2023-11-29 23:40:47 +0000 UTC info: Wrote 21474836480/21474836480 bytes to the inactive partition
2023-11-29 23:40:47 +0000 UTC info: State transition: update-store [Download_Enter] -> update-after-store [Download_Leave]
2023-11-29 23:40:47 +0000 UTC info: State transition: update-after-store [Download_Leave] -> mender-update-control-refresh-maps [none]
2023-11-29 23:40:47 +0000 UTC info: Executing script: Download_Leave_00_retainconfig
2023-11-29 23:40:47 +0000 UTC info: Output (stdout) from command "/etc/mender/scripts/Download_Leave_00_retainconfig": test
2023-11-29 23:40:47 +0000 UTC info: State transition: mender-update-control-refresh-maps [none] -> mender-update-control [none]
2023-11-29 23:40:47 +0000 UTC info: State transition: mender-update-control [none] -> update-install [ArtifactInstall]
2023-11-29 23:40:47 +0000 UTC info: Enabling partition with new image installed to be a boot candidate: 3
2023-11-29 23:40:47 +0000 UTC info: State transition: update-install [ArtifactInstall] -> mender-update-control-refresh-maps [none]
2023-11-29 23:40:47 +0000 UTC info: State transition: mender-update-control-refresh-maps [none] -> mender-update-control [none]
2023-11-29 23:40:47 +0000 UTC info: State transition: mender-update-control [none] -> reboot [ArtifactReboot_Enter]
2023-11-29 23:40:47 +0000 UTC info: State transition: reboot [ArtifactReboot_Enter] -> mender-update-control [none]
2023-11-29 23:40:47 +0000 UTC info: Update Control: Pausing before entering reboot state
2023-11-29 23:40:47 +0000 UTC info: State transition: mender-update-control [none] -> mender-update-control-pause [none]
2023-11-29 23:40:47 +0000 UTC info: Next update refresh from the server in: 29.978172692s
2023-11-29 23:40:47 +0000 UTC info: Forced wake-up from sleep
2023-11-29 23:40:47 +0000 UTC info: State transition: mender-update-control-pause [none] -> mender-update-control [none]
2023-11-29 23:40:47 +0000 UTC info: Update Control: Pausing before entering reboot state
2023-11-29 23:40:47 +0000 UTC info: State transition: mender-update-control [none] -> mender-update-control-pause [none]
2023-11-29 23:40:47 +0000 UTC info: Next update refresh from the server in: 29.966081343s
2023-11-29 23:41:17 +0000 UTC info: State transition: mender-update-control-pause [none] -> mender-update-control-refresh-maps [none]
2023-11-29 23:41:17 +0000 UTC info: State transition: mender-update-control-refresh-maps [none] -> mender-update-control [none]
2023-11-29 23:41:17 +0000 UTC info: Update Control: Pausing before entering reboot state
2023-11-29 23:41:17 +0000 UTC info: State transition: mender-update-control [none] -> mender-update-control-pause [none]
2023-11-29 23:41:17 +0000 UTC info: Next update refresh from the server in: 29.992164449s
2023-11-29 23:41:17 +0000 UTC info: Forced wake-up from sleep
2023-11-29 23:41:17 +0000 UTC info: State transition: mender-update-control-pause [none] -> mender-update-control [none]
2023-11-29 23:41:17 +0000 UTC info: Update Control: Pausing before entering reboot state
2023-11-29 23:41:17 +0000 UTC info: State transition: mender-update-control [none] -> mender-update-control-pause [none]
2023-11-29 23:41:17 +0000 UTC info: Next update refresh from the server in: 29.98224103s
2023-11-29 23:41:47 +0000 UTC info: State transition: mender-update-control-pause [none] -> mender-update-control-refresh-maps [none]
2023-11-29 23:41:47 +0000 UTC info: State transition: mender-update-control-refresh-maps [none] -> mender-update-control [none]
2023-11-29 23:41:47 +0000 UTC info: Update Control: Pausing before entering reboot state
2023-11-29 23:41:47 +0000 UTC info: State transition: mender-update-control [none] -> mender-update-control-pause [none]
2023-11-29 23:41:47 +0000 UTC info: Next update refresh from the server in: 29.99113852s
2023-11-29 23:41:47 +0000 UTC info: Forced wake-up from sleep
2023-11-29 23:41:47 +0000 UTC info: State transition: mender-update-control-pause [none] -> mender-update-control [none]
2023-11-29 23:41:47 +0000 UTC info: Update Control: Pausing before entering reboot state
2023-11-29 23:41:47 +0000 UTC info: State transition: mender-update-control [none] -> mender-update-control-pause [none]
2023-11-29 23:41:47 +0000 UTC info: Next update refresh from the server in: 29.981054641s
2023-11-29 23:42:17 +0000 UTC info: State transition: mender-update-control-pause [none] -> mender-update-control-refresh-maps [none]
2023-11-29 23:42:17 +0000 UTC info: State transition: mender-update-control-refresh-maps [none] -> mender-update-control [none]
2023-11-29 23:42:17 +0000 UTC info: Update Control: Pausing before entering reboot state
2023-11-29 23:42:17 +0000 UTC info: State transition: mender-update-control [none] -> mender-update-control-pause [none]
2023-11-29 23:42:17 +0000 UTC info: Next update refresh from the server in: 29.989602881s
2023-11-29 23:42:17 +0000 UTC info: Forced wake-up from sleep
2023-11-29 23:42:17 +0000 UTC info: State transition: mender-update-control-pause [none] -> mender-update-control [none]
2023-11-29 23:42:17 +0000 UTC info: Update Control: Pausing before entering reboot state
2023-11-29 23:42:17 +0000 UTC info: State transition: mender-update-control [none] -> mender-update-control-pause [none]
2023-11-29 23:42:17 +0000 UTC info: Next update refresh from the server in: 29.974077962s
2023-11-29 23:42:47 +0000 UTC info: State transition: mender-update-control-pause [none] -> mender-update-control-refresh-maps [none]
2023-11-29 23:42:47 +0000 UTC info: State transition: mender-update-control-refresh-maps [none] -> mender-update-control [none]
2023-11-29 23:42:47 +0000 UTC info: Update Control: Pausing before entering reboot state
2023-11-29 23:42:47 +0000 UTC info: State transition: mender-update-control [none] -> mender-update-control-pause [none]
2023-11-29 23:42:47 +0000 UTC info: Next update refresh from the server in: 29.99204547s
2023-11-29 23:42:47 +0000 UTC info: Forced wake-up from sleep
2023-11-29 23:42:47 +0000 UTC info: State transition: mender-update-control-pause [none] -> mender-update-control [none]
2023-11-29 23:42:47 +0000 UTC info: Update Control: Pausing before entering reboot state
2023-11-29 23:42:47 +0000 UTC info: State transition: mender-update-control [none] -> mender-update-control-pause [none]
2023-11-29 23:42:47 +0000 UTC info: Next update refresh from the server in: 29.977211342s
2023-11-29 23:43:17 +0000 UTC info: State transition: mender-update-control-pause [none] -> mender-update-control-refresh-maps [none]
2023-11-29 23:43:17 +0000 UTC info: State transition: mender-update-control-refresh-maps [none] -> mender-update-control [none]
2023-11-29 23:43:17 +0000 UTC info: Update Control: Pausing before entering reboot state
2023-11-29 23:43:17 +0000 UTC info: State transition: mender-update-control [none] -> mender-update-control-pause [none]
2023-11-29 23:43:17 +0000 UTC info: Next update refresh from the server in: 29.990708472s
2023-11-29 23:43:17 +0000 UTC info: Forced wake-up from sleep
2023-11-29 23:43:17 +0000 UTC info: State transition: mender-update-control-pause [none] -> mender-update-control [none]
2023-11-29 23:43:17 +0000 UTC info: Update Control: Pausing before entering reboot state
2023-11-29 23:43:17 +0000 UTC info: State transition: mender-update-control [none] -> mender-update-control-pause [none]
2023-11-29 23:43:17 +0000 UTC info: Next update refresh from the server in: 29.966029924s
2023-11-29 23:43:47 +0000 UTC info: State transition: mender-update-control-pause [none] -> mender-update-control-refresh-maps [none]
2023-11-29 23:43:47 +0000 UTC info: State transition: mender-update-control-refresh-maps [none] -> mender-update-control [none]
2023-11-29 23:43:47 +0000 UTC info: Update Control: Pausing before entering reboot state
2023-11-29 23:43:47 +0000 UTC info: State transition: mender-update-control [none] -> mender-update-control-pause [none]
2023-11-29 23:43:47 +0000 UTC info: Next update refresh from the server in: 29.968659172s
2023-11-29 23:43:47 +0000 UTC info: Forced wake-up from sleep
2023-11-29 23:43:47 +0000 UTC info: State transition: mender-update-control-pause [none] -> mender-update-control [none]
2023-11-29 23:43:47 +0000 UTC info: Update Control: Pausing before entering reboot state
2023-11-29 23:43:47 +0000 UTC info: State transition: mender-update-control [none] -> mender-update-control-pause [none]
2023-11-29 23:43:47 +0000 UTC info: Next update refresh from the server in: 29.953219104s
2023-11-29 23:44:17 +0000 UTC info: State transition: mender-update-control-pause [none] -> mender-update-control-refresh-maps [none]
2023-11-29 23:44:17 +0000 UTC info: State transition: mender-update-control-refresh-maps [none] -> mender-update-control [none]
2023-11-29 23:44:17 +0000 UTC info: Update Control: Pausing before entering reboot state
2023-11-29 23:44:17 +0000 UTC info: State transition: mender-update-control [none] -> mender-update-control-pause [none]
2023-11-29 23:44:17 +0000 UTC info: Next update refresh from the server in: 29.988241241s
2023-11-29 23:44:17 +0000 UTC info: Forced wake-up from sleep
2023-11-29 23:44:17 +0000 UTC info: State transition: mender-update-control-pause [none] -> mender-update-control [none]
2023-11-29 23:44:17 +0000 UTC info: Update Control: Pausing before entering reboot state
2023-11-29 23:44:17 +0000 UTC info: State transition: mender-update-control [none] -> mender-update-control-pause [none]
2023-11-29 23:44:17 +0000 UTC info: Next update refresh from the server in: 29.974621463s
2023-11-29 23:44:47 +0000 UTC info: State transition: mender-update-control-pause [none] -> mender-update-control-refresh-maps [none]
2023-11-29 23:44:47 +0000 UTC info: State transition: mender-update-control-refresh-maps [none] -> mender-update-control [none]
2023-11-29 23:44:47 +0000 UTC info: Update Control: Pausing before entering reboot state
2023-11-29 23:44:47 +0000 UTC info: State transition: mender-update-control [none] -> mender-update-control-pause [none]
2023-11-29 23:44:47 +0000 UTC info: Next update refresh from the server in: 29.988879601s
2023-11-29 23:44:47 +0000 UTC info: Forced wake-up from sleep
2023-11-29 23:44:47 +0000 UTC info: State transition: mender-update-control-pause [none] -> mender-update-control [none]
2023-11-29 23:44:47 +0000 UTC info: Update Control: Pausing before entering reboot state
2023-11-29 23:44:47 +0000 UTC info: State transition: mender-update-control [none] -> mender-update-control-pause [none]
2023-11-29 23:44:47 +0000 UTC info: Next update refresh from the server in: 29.965446092s
2023-11-29 23:45:17 +0000 UTC info: State transition: mender-update-control-pause [none] -> mender-update-control-refresh-maps [none]
2023-11-29 23:45:17 +0000 UTC info: State transition: mender-update-control-refresh-maps [none] -> mender-update-control [none]
2023-11-29 23:45:17 +0000 UTC info: Update Control: Pausing before entering reboot state
2023-11-29 23:45:17 +0000 UTC info: State transition: mender-update-control [none] -> mender-update-control-pause [none]
2023-11-29 23:45:17 +0000 UTC info: Next update refresh from the server in: 29.989439822s
2023-11-29 23:45:17 +0000 UTC info: Forced wake-up from sleep
2023-11-29 23:45:17 +0000 UTC info: State transition: mender-update-control-pause [none] -> mender-update-control [none]
2023-11-29 23:45:17 +0000 UTC info: Update Control: Pausing before entering reboot state
2023-11-29 23:45:17 +0000 UTC info: State transition: mender-update-control [none] -> mender-update-control-pause [none]
2023-11-29 23:45:17 +0000 UTC info: Next update refresh from the server in: 29.978640183s
2023-11-29 23:45:47 +0000 UTC info: State transition: mender-update-control-pause [none] -> mender-update-control-refresh-maps [none]
2023-11-29 23:45:47 +0000 UTC info: State transition: mender-update-control-refresh-maps [none] -> mender-update-control [none]
2023-11-29 23:45:47 +0000 UTC info: Update Control: Pausing before entering reboot state
2023-11-29 23:45:47 +0000 UTC info: State transition: mender-update-control [none] -> mender-update-control-pause [none]
2023-11-29 23:45:47 +0000 UTC info: Next update refresh from the server in: 29.984661711s
2023-11-29 23:45:47 +0000 UTC info: Forced wake-up from sleep
2023-11-29 23:45:47 +0000 UTC info: State transition: mender-update-control-pause [none] -> mender-update-control [none]
2023-11-29 23:45:47 +0000 UTC info: Update Control: Pausing before entering reboot state
2023-11-29 23:45:47 +0000 UTC info: State transition: mender-update-control [none] -> mender-update-control-pause [none]
2023-11-29 23:45:47 +0000 UTC info: Next update refresh from the server in: 29.974328632s
2023-11-29 23:46:17 +0000 UTC info: State transition: mender-update-control-pause [none] -> mender-update-control-refresh-maps [none]
2023-11-29 23:46:17 +0000 UTC info: State transition: mender-update-control-refresh-maps [none] -> mender-update-control [none]
2023-11-29 23:46:17 +0000 UTC info: Update Control: Pausing before entering reboot state
2023-11-29 23:46:17 +0000 UTC info: State transition: mender-update-control [none] -> mender-update-control-pause [none]
2023-11-29 23:46:17 +0000 UTC info: Next update refresh from the server in: 29.99090791s
2023-11-29 23:46:17 +0000 UTC info: Forced wake-up from sleep
2023-11-29 23:46:17 +0000 UTC info: State transition: mender-update-control-pause [none] -> mender-update-control [none]
2023-11-29 23:46:17 +0000 UTC info: Update Control: Pausing before entering reboot state
2023-11-29 23:46:17 +0000 UTC info: State transition: mender-update-control [none] -> mender-update-control-pause [none]
2023-11-29 23:46:17 +0000 UTC info: Next update refresh from the server in: 29.976397072s
2023-11-29 23:46:47 +0000 UTC info: State transition: mender-update-control-pause [none] -> mender-update-control-refresh-maps [none]
2023-11-29 23:46:47 +0000 UTC info: State transition: mender-update-control-refresh-maps [none] -> mender-update-control [none]
2023-11-29 23:46:47 +0000 UTC info: State transition: mender-update-control [none] -> reboot [ArtifactReboot_Enter]
2023-11-29 23:46:47 +0000 UTC info: Rebooting device(s)
2023-11-29 23:46:47 +0000 UTC info: Mender rebooting from active partition: /dev/sda2
2023-11-29 23:46:48 +0000 UTC info: Daemon terminated with SIGTERM
2023-11-29 23:49:36 +0000 UTC info: Running Mender client version: 3.5.1
2023-11-29 23:49:36 +0000 UTC info: State transition: init [none] -> after-reboot [ArtifactReboot_Leave]
2023-11-29 23:49:36 +0000 UTC error: transient error: Reboot to the new update failed. Expected "upgrade_available" flag to be true but it was false. Either the switch to the new partition was unsuccessful, or the bootloader rolled back
2023-11-29 23:49:36 +0000 UTC info: State transition: after-reboot [ArtifactReboot_Leave] -> rollback [ArtifactRollback]
2023-11-29 23:49:36 +0000 UTC info: Performing rollback
2023-11-29 23:49:36 +0000 UTC info: No update available, so no rollback needed.
2023-11-29 23:49:36 +0000 UTC info: State transition: rollback [ArtifactRollback] -> rollback-reboot [ArtifactRollbackReboot_Enter]
2023-11-29 23:49:36 +0000 UTC info: Rebooting device(s) after rollback
2023-11-29 23:49:36 +0000 UTC info: Mender rebooting from inactive partition: /dev/sda2
2023-11-29 23:49:36 +0000 UTC info: Daemon terminated with SIGTERM
2023-11-29 23:50:03 +0000 UTC info: Running Mender client version: 3.5.1
2023-11-29 23:50:03 +0000 UTC info: Mender shut down in state: rollback-reboot
2023-11-29 23:50:03 +0000 UTC info: State transition: init [none] -> verify-rollback-reboot [ArtifactRollbackReboot_Leave]
2023-11-29 23:50:03 +0000 UTC info: State transition: verify-rollback-reboot [ArtifactRollbackReboot_Leave] -> after-rollback-reboot [ArtifactRollbackReboot_Leave]
2023-11-29 23:50:03 +0000 UTC info: State transition: after-rollback-reboot [ArtifactRollbackReboot_Leave] -> update-error [ArtifactFailure]
2023-11-29 23:50:03 +0000 UTC info: State transition: update-error [ArtifactFailure] -> cleanup [Error]
2023-11-29 23:50:03 +0000 UTC info: State transition: cleanup [Error] -> update-status-report [none]
2023-11-29 23:50:03 +0000 UTC info: Device unauthorized; attempting reauthorization
2023-11-29 23:50:03 +0000 UTC info: successfully received new authorization data from server https://docker.mender.io
2023-11-29 23:50:03 +0000 UTC info: Local proxy started
2023-11-29 23:50:03 +0000 UTC info: Reauthorization successful