Start-update-notifier fails

Hi, I am trying to run an update 2 state scripts. I thought they were the problem but I think something else happened.

Board: up squared
Version: sumo
Mender: 1.7

My 2 state scripts are:
ArtifactInstall_Leave_01_store-parameters
ArtifactReboot_Leave_01_reenable-services

The failing one is:
ArtifactReboot_Leave_02_start-update-notifier-0.1.3

I can’t find where it comes from. I also just found out my storage certificate was invalid, could this be connected?

Here is the full log

2019-06-24 11:29:19 +0000 UTC debug: handle update fetch state
2019-06-24 11:29:19 +0000 UTC debug: status reported, response 204 No Content
2019-06-24 11:29:20 +0000 UTC debug: Received fetch update response &{200 OK 200 HTTP/1.1 1 1 map[Vary:[Origin] X-Xss-Protection:[1; mode=block] Strict-Transport-Security:[max-age=63072000; includeSubdomains; preload] Server:[openresty/1.13.6.2] Date:[Mon, 24 Jun 2019 11:29:20 GMT] Content-Type:[application/vnd.mender-artifact] Connection:[keep-alive] Content-Security-Policy:[block-all-mixed-content] Content-Length:[152658944] Accept-Ranges:[bytes] X-Frame-Options:[DENY] Etag:["f51dd55c5155251047d34f5416c55ca5"] Last-Modified:[Mon, 24 Jun 2019 10:53:18 GMT] X-Amz-Request-Id:[15AB1EFB5BB0984C] X-Content-Type-Options:[nosniff]] 0xc4203d20c0 152658944 [] false false map[] 0xc4203a2300 0xc420392370}+
2019-06-24 11:29:20 +0000 UTC info: State transition: update-fetch [Download] -> update-store [Download]
2019-06-24 11:29:20 +0000 UTC debug: handle update install state
2019-06-24 11:29:21 +0000 UTC debug: status reported, response 204 No Content
2019-06-24 11:29:21 +0000 UTC debug: Read data from device manifest file: device_type=up-board
2019-06-24 11:29:21 +0000 UTC debug: Found needed line: device_type=up-board
2019-06-24 11:29:21 +0000 UTC debug: Current manifest data: up-board
2019-06-24 11:29:21 +0000 UTC info: no public key was provided for authenticating the artifact
2019-06-24 11:29:21 +0000 UTC debug: checking if device [up-board] is on compatibile device list: [up-board]
2019-06-24 11:29:21 +0000 UTC debug: installer: processing script: ArtifactInstall_Leave_01_store-parameters
2019-06-24 11:29:21 +0000 UTC debug: installer: processing script: ArtifactReboot_Leave_01_reenable-services
2019-06-24 11:29:21 +0000 UTC debug: installer: processing script: ArtifactReboot_Leave_02_start-update-notifier-0.1.3
2019-06-24 11:29:21 +0000 UTC debug: installer: processing script: ArtifactInstall_Enter_00_store-version
2019-06-24 11:29:21 +0000 UTC debug: installing update oblac-box-os-up-board.ext4 of size 4269801472
2019-06-24 11:29:21 +0000 UTC debug: Trying to install update of size: 4269801472
2019-06-24 11:29:21 +0000 UTC debug: Active partition: /dev/mmcblk0p2
2019-06-24 11:29:21 +0000 UTC debug: Detected inactive partition /dev/mmcblk0p3, based on active partition /dev/mmcblk0p2
2019-06-24 11:29:21 +0000 UTC info: opening device /dev/mmcblk0p3 for writing
2019-06-24 11:29:21 +0000 UTC info: partition /dev/mmcblk0p3 size: 4269801472
2019-06-24 11:32:33 +0000 UTC error: Download connection broken: read tcp 172.20.3.138:43598->35.174.233.226:9000: read: connection timed out
2019-06-24 11:32:33 +0000 UTC info: Resuming download in 1m0s
2019-06-24 11:33:33 +0000 UTC info: Attempting to resume artifact download from offset 139591126
2019-06-24 11:33:34 +0000 UTC debug: Content-Range received from server: 'bytes 139591126-152658943/152658944'
2019-06-24 11:37:45 +0000 UTC info: wrote 4269801472/4269801472 bytes of update to device /dev/mmcblk0p3
2019-06-24 11:37:48 +0000 UTC debug: installer: successfully read artifact [name: 0.1.3; version: 2; compatible devices: [up-board]]
2019-06-24 11:37:48 +0000 UTC debug: status reported, response 204 No Content
2019-06-24 11:37:49 +0000 UTC info: State transition: update-store [Download] -> update-install [ArtifactInstall]
2019-06-24 11:37:49 +0000 UTC debug: statescript: timeout for executing scripts is not defined; using default of 60 seconds
2019-06-24 11:37:49 +0000 UTC debug: statescript: timeout for executing scripts is not defined; using default of 60 seconds
2019-06-24 11:37:49 +0000 UTC debug: start executing script: ArtifactInstall_Enter_00_store-version
2019-06-24 11:37:49 +0000 UTC debug: status reported, response 204 No Content
2019-06-24 11:37:49 +0000 UTC debug: status reported, response 204 No Content
2019-06-24 11:37:49 +0000 UTC debug: status reported, response 204 No Content
2019-06-24 11:37:49 +0000 UTC debug: Inactive partition: /dev/mmcblk0p3
2019-06-24 11:37:49 +0000 UTC debug: Marking inactive partition (/dev/mmcblk0p3) as the new boot candidate.
2019-06-24 11:37:49 +0000 UTC info: Enabling partition with new image installed to be a boot candidate: 3
2019-06-24 11:37:49 +0000 UTC debug: Have U-Boot variable: mender_check_saveenv_canary=
2019-06-24 11:37:49 +0000 UTC debug: List of U-Boot variables:map[mender_check_saveenv_canary:]
2019-06-24 11:37:50 +0000 UTC debug: Marking inactive partition as a boot candidate successful.
2019-06-24 11:37:50 +0000 UTC info: State transition: update-install [ArtifactInstall] -> reboot [ArtifactReboot_Enter]
2019-06-24 11:37:50 +0000 UTC debug: statescript: timeout for executing scripts is not defined; using default of 60 seconds
2019-06-24 11:37:50 +0000 UTC debug: start executing script: ArtifactInstall_Leave_01_store-parameters
2019-06-24 11:37:50 +0000 UTC debug: status reported, response 204 No Content
2019-06-24 11:37:50 +0000 UTC debug: status reported, response 204 No Content
2019-06-24 11:37:50 +0000 UTC debug: statescript: timeout for executing scripts is not defined; using default of 60 seconds
2019-06-24 11:37:50 +0000 UTC debug: handling reboot state
2019-06-24 11:37:50 +0000 UTC debug: status reported, response 204 No Content
2019-06-24 11:37:51 +0000 UTC info: rebooting device
2019-06-24 11:37:51 +0000 UTC info: Mender rebooting from active partition: /dev/mmcblk0p2
2019-06-24 11:38:15 +0000 UTC debug: handling state after reboot
2019-06-24 11:38:15 +0000 UTC info: State transition: after-reboot [ArtifactReboot_Leave] -> update-verify [ArtifactReboot_Leave]
2019-06-24 11:38:15 +0000 UTC debug: handle update verify state
2019-06-24 11:38:15 +0000 UTC debug: Have U-Boot variable: mender_check_saveenv_canary=
2019-06-24 11:38:15 +0000 UTC debug: List of U-Boot variables:map[mender_check_saveenv_canary:]
2019-06-24 11:38:15 +0000 UTC debug: Have U-Boot variable: upgrade_available=1
2019-06-24 11:38:16 +0000 UTC debug: List of U-Boot variables:map[upgrade_available:1]
2019-06-24 11:38:16 +0000 UTC info: State transition: update-verify [ArtifactReboot_Leave] -> update-commit [ArtifactCommit]
2019-06-24 11:38:16 +0000 UTC debug: statescript: timeout for executing scripts is not defined; using default of 60 seconds
2019-06-24 11:38:16 +0000 UTC debug: start executing script: ArtifactReboot_Leave_01_reenable-services
2019-06-24 11:38:17 +0000 UTC debug: status reported, response 204 No Content
2019-06-24 11:38:17 +0000 UTC debug: start executing script: ArtifactReboot_Leave_02_start-update-notifier-0.1.3
2019-06-24 11:38:17 +0000 UTC debug: status reported, response 204 No Content
2019-06-24 11:38:17 +0000 UTC debug: status reported, response 204 No Content
2019-06-24 11:38:18 +0000 UTC debug: status reported, response 204 No Content
2019-06-24 11:38:18 +0000 UTC error: error executing leave script for update-verify state: error running leave state script(s) for ArtifactReboot_Leave state: statescript: error executing 'ArtifactReboot_Leave_02_start-update-notifier-0.1.3': 1 : exit status 1
2019-06-24 11:38:18 +0000 UTC error: will transit to error state from: update-verify [ArtifactReboot_Leave]
2019-06-24 11:38:18 +0000 UTC info: State transition: update-verify [ArtifactReboot_Leave] -> rollback [ArtifactRollback]
2019-06-24 11:38:18 +0000 UTC debug: transitioning to error state
2019-06-24 11:38:18 +0000 UTC debug: statescript: timeout for executing scripts is not defined; using default of 60 seconds
2019-06-24 11:38:18 +0000 UTC debug: statescript: timeout for executing scripts is not defined; using default of 60 seconds
2019-06-24 11:38:18 +0000 UTC info: performing rollback

Thanks

The failing one is:
ArtifactReboot_Leave_02_start-update-notifier-0.1.3

This is not something that is provided by any of the meta-mender layers and you would need to dig out where it is coming from and what it contains.

Thanks @mirzak.
I had a script called start-update-notifier which was removed afterwards. I deleted the .mender in the deploy/images folder, rebuilt the same image and it worked. I can’t say why this happened. Something cached this script was executed while it was not anymore.