Hi there!
I’m working with Mender 2.7 and suddenly deployments stop working. In logs, I can see that client doesn’t arrive to Download state and the deployment keeps waiting for device. It happened without changing any configuration, I tried to reinstall the client but nothing.
I also tried to fetch the device to an other account and the deployments work fine.
Any idea why deployments aren’t working on my account?
Thanks
Hi @Mikel can you provide the output from running journalctl -u mender-client on one of the devices?
Are the devices visible in the web UI?
Do they show any recent checkins?
Drew
Yes, I can see the device in the web UI and the last check-in is a few seconds ago.
The output of journalctl -u mender-client is:
After getting this, I tried to do a deployment and nothing happened, nothing more added to journalctl.
Can you share that journalctl as text? It’s missing quite a bit of context off the right side of the image file.
What kind of artifact are you deploying?
You are using hosted Mender correct? Can you provide the ID of the deployment in question? It should be availabe in the web UI.
Drew
I tried to do configure update and dir-overlay update.
Yes, I’m using hosted Mender. The ID of the configure update: object":{“id”:“9a493466-6af3-410b-af4b-c558520056bb”,“type”:“device”},
journalctl as text:
May 04 08:22:06 dlk00011 mender[11670]: time=“2021-05-04T08:22:06Z” level=info msg="State transition: check-wait [Idle] → update-check [Sync]
May 04 08:22:06 dlk00011 mender[11670]: time=“2021-05-04T08:22:06Z” level=info msg=“Forced wake-up from sleep”
May 04 08:03:19 dlk00011 mender[11670]: time=“2021-05-04T08:03:19Z” level=info msg="State transition: update-check [Sync] → check-wait [Idle]
May 04 08:03:19 dlk00011 mender[11670]: time=“2021-05-04T08:03:19Z” level=info msg="State transition: check-wait [Idle] → update-check [Sync]
May 04 07:33:20 dlk00011 mender[11670]: time=“2021-05-04T07:33:20Z” level=info msg="State transition: update-check [Sync] → check-wait [Idle]
May 04 07:33:19 dlk00011 mender[11670]: time=“2021-05-04T07:33:19Z” level=info msg="State transition: check-wait [Idle] → update-check [Sync]
May 04 07:03:48 dlk00011 mender[11670]: time=“2021-05-04T07:03:48Z” level=info msg="State transition: inventory-update [Sync] → check-wait [I
May 04 07:03:46 dlk00011 mender[11670]: time=“2021-05-04T07:03:46Z” level=warning msg="Inventory tool /usr/share/mender/inventory/mender-inven
May 04 07:03:46 dlk00011 mender[11670]: time=“2021-05-04T07:03:46Z” level=warning msg="Inventory tool /usr/share/mender/inventory/mender-inven
May 04 07:03:45 dlk00011 mender[11670]: time=“2021-05-04T07:03:45Z” level=info msg="State transition: check-wait [Idle] → inventory-update [S
May 04 07:03:45 dlk00011 mender[11670]: time=“2021-05-04T07:03:45Z” level=info msg=“State transition: idle [Idle] → check-wait [Idle]”
May 04 07:03:45 dlk00011 mender[11670]: time=“2021-05-04T07:03:45Z” level=info msg="State transition: update-status-report [none] → idle [Idl
May 04 07:03:45 dlk00011 mender[11670]: time=“2021-05-04T07:03:45Z” level=info msg="State transition: cleanup [none] → update-status-report [
May 04 07:03:45 dlk00011 mender[11670]: time=“2021-05-04T07:03:45Z” level=info msg="State transition: update-after-commit [ArtifactCommit_Leav
May 04 07:03:45 dlk00011 mender[11670]: time=“2021-05-04T07:03:45Z” level=info msg="State transition: update-after-first-commit [none] → upda
May 04 07:03:45 dlk00011 mender[11670]: time=“2021-05-04T07:03:45Z” level=info msg="State transition: update-commit [ArtifactCommit_Enter] →
May 04 07:03:45 dlk00011 mender[11670]: time=“2021-05-04T07:03:45Z” level=info msg="State transition: update-install [ArtifactInstall] → upda
May 04 07:03:44 dlk00011 sudo[12291]: pam_unix(sudo:session): session closed for user root
May 04 07:03:44 dlk00011 sudo[12291]: pam_unix(sudo:session): session opened for user root by (uid=0)
May 04 07:03:44 dlk00011 sudo[12291]: root : TTY=unknown ; PWD=/var/lib/mender/modules/v3/payloads/0000/tree ; USER=root ; COMMAND=/bin/sy
May 04 07:03:42 dlk00011 mender[11670]: time=“2021-05-04T07:03:42Z” level=info msg="Update module output: Executing /usr/lib/mender-configure/
May 04 07:03:42 dlk00011 sudo[11873]: pam_unix(sudo:session): session closed for user root
May 04 07:03:24 dlk00011 sudo[11873]: pam_unix(sudo:session): session opened for user root by (uid=0)
May 04 07:03:24 dlk00011 sudo[11873]: root : TTY=unknown ; PWD=/var/lib/mender/modules/v3/payloads/0000/tree ; USER=root ; COMMAND=/bin/sy
May 04 07:03:24 dlk00011 mender[11670]: time=“2021-05-04T07:03:24Z” level=info msg="Update module output: Executing /usr/lib/mender-configure/
May 04 07:03:24 dlk00011 sudo[11864]: pam_unix(sudo:session): session closed for user root
May 04 07:03:24 dlk00011 sudo[11864]: pam_unix(sudo:session): session opened for user root by (uid=0)
May 04 07:03:24 dlk00011 sudo[11864]: root : TTY=unknown ; PWD=/var/lib/mender/modules/v3/payloads/0000/tree ; USER=root ; COMMAND=/usr/bi
May 04 07:03:24 dlk00011 mender[11670]: time=“2021-05-04T07:03:24Z” level=info msg="Update module output: Executing /usr/lib/mender-configure/
May 04 07:03:23 dlk00011 sudo[11843]: pam_unix(sudo:session): session closed for user root
May 04 07:03:23 dlk00011 sudo[11843]: pam_unix(sudo:session): session opened for user root by (uid=0)
May 04 07:03:23 dlk00011 sudo[11843]: root : TTY=unknown ; PWD=/var/lib/mender/modules/v3/payloads/0000/tree ; USER=root ; COMMAND=/bin/sy
May 04 07:03:21 dlk00011 mender[11670]: time=“2021-05-04T07:03:21Z” level=info msg="Update module output: Executing /usr/lib/mender-configure/
May 04 07:03:20 dlk00011 mender[11670]: time=“2021-05-04T07:03:20Z” level=info msg="State transition: update-after-store [Download_Leave] → u
May 04 07:03:20 dlk00011 mender[11670]: time=“2021-05-04T07:03:20Z” level=info msg="State transition: update-store [Download_Enter] → update-
May 04 07:03:20 dlk00011 mender[11670]: time=“2021-05-04T07:03:20Z” level=info msg="No public key was provided for authenticating the artifact
May 04 07:03:19 dlk00011 mender[11670]: time=“2021-05-04T07:03:19Z” level=info msg="State transition: update-fetch [Download_Enter] → update-
May 04 07:03:19 dlk00011 mender[11670]: time=“2021-05-04T07:03:19Z” level=info msg=“Running Mender client version: 2.6.0”
May 04 07:03:19 dlk00011 mender[11670]: time=“2021-05-04T07:03:19Z” level=info msg="State transition: update-check [Sync] → update-fetch [Dow
May 04 07:03:19 dlk00011 mender[11670]: time=“2021-05-04T07:03:19Z” level=info msg="Correct request for getting image from: https://hosted.men
May 04 07:03:19 dlk00011 mender[11670]: time=“2021-05-04T07:03:19Z” level=info msg="State transition: check-wait [Idle] → update-check [Sync]
May 04 07:03:19 dlk00011 mender[11670]: time=“2021-05-04T07:03:19Z” level=info msg="State transition: inventory-update [Sync] → check-wait [I
May 04 07:03:18 dlk00011 mender[11670]: time=“2021-05-04T07:03:18Z” level=warning msg="Inventory tool /usr/share/mender/inventory/mender-inven
May 04 07:03:18 dlk00011 mender[11670]: time=“2021-05-04T07:03:18Z” level=warning msg="Inventory tool /usr/share/mender/inventory/mender-inven
May 04 07:03:17 dlk00011 mender[11670]: time=“2021-05-04T07:03:17Z” level=info msg="State transition: check-wait [Idle] → inventory-update [S
May 04 07:03:17 dlk00011 mender[11670]: time=“2021-05-04T07:03:17Z” level=info msg=“State transition: idle [Idle] → check-wait [Idle]”
May 04 07:03:17 dlk00011 mender[11670]: time=“2021-05-04T07:03:17Z” level=info msg="State transition: update-status-report [none] → idle [Idl
May 04 07:03:16 dlk00011 mender[11670]: time=“2021-05-04T07:03:16Z” level=info msg="State transition: cleanup [Error] → update-status-report
May 04 07:03:16 dlk00011 mender[11670]: time=“2021-05-04T07:03:16Z” level=info msg=“State transition: init [none] → cleanup [Error]”
May 04 07:03:16 dlk00011 mender[11670]: time=“2021-05-04T07:03:16Z” level=error msg=“Mender shut down in state: update-store”
May 04 07:03:16 dlk00011 mender[11670]: time=“2021-05-04T07:03:16Z” level=error msg=“Mender shut down in state: update-store”
May 04 07:03:16 dlk00011 mender[11670]: time=“2021-05-04T07:03:16Z” level=info msg=“Running Mender client version: 2.6.0”
May 04 07:03:16 dlk00011 mender[11670]: time=“2021-05-04T07:03:16Z” level=info msg=“Handling loaded state: update-store”
May 04 07:03:16 dlk00011 mender[11670]: time=“2021-05-04T07:03:16Z” level=info msg=“State transition: init [none] → init [none]”
May 04 07:03:16 dlk00011 mender[11670]: time=“2021-05-04T07:03:16Z” level=info msg=“No dual rootfs configuration present”
May 04 07:03:16 dlk00011 mender[11670]: time=“2021-05-04T07:03:16Z” level=info msg=“Loaded configuration file: /etc/mender/mender.conf”
May 04 07:03:16 dlk00011 systemd[1]: Started Mender OTA update service.
Mikel
@kacf any ideas here?
@tranchitella @peter can you help check on the status of the deployment in case there is anything in the server logs?
Drew
@Mikel can you please confirm you are on Hosted Mender? If yes, can you please share the name of your organization and the ID of the device so we can look up the logs?
