Mender rootfs artifact deploy not starting

General Discussions
1

Hi Guys,
Using Jetson Nano device, yocto dunfell base branch, I am trying to setup the server side for mender.
Have been able to make it work, server version 2.7.0, device is shown on the server and inventory looks good.

However, when putting a release and deploying to device, it is pending forever, waiting for device.
From the device side I see the following in the logs:

Jun 3 08:22:42 jetson-nano-emmc-cot user.info mender[729693]: time=“2021-06-03T08:22:42+03:00” level=info msg=“State transition: check-wait [Idle] → update-check [Sync]” func=github.com/mendersoftware/mender/app.transitionState file="/home/tirank/GIT/nvidia_1.0.5_new/tegra-l4t/bu
Jun 3 08:22:42 jetson-nano-emmc-cot user.warn mender[729693]: time=“2021-06-03T08:22:42+03:00” level=warning msg=“Client received invalid response status code: 503” func=github.com/mendersoftware/mender/client.processUpdateResponse file="/home/tirank/GIT/nvidia_1.0.5_new/tegra-l4t
Jun 3 08:22:42 jetson-nano-emmc-cot user.err mender[729693]: time=“2021-06-03T08:22:42+03:00” level=error msg=“Error receiving scheduled update data: (request_id: ): Invalid response received from server server error message: failed to parse server response: json: cannot unmarsha
Jun 3 08:22:42 jetson-nano-emmc-cot user.err mender[729693]: time=“2021-06-03T08:22:42+03:00” level=error msg=“Update check failed: transient error: (request_id: ): Invalid response received from server server error message: failed to parse server response: json: cannot unmarshal
Jun 3 08:22:42 jetson-nano-emmc-cot user.info mender[729693]: time=“2021-06-03T08:22:42+03:00” level=info msg=“State transition: update-check [Sync] → error [Error]” func=github.com/mendersoftware/mender/app.transitionState file=”/home/tirank/GIT/nvidia_1.0.5_new/tegra-l4t/build-
Jun 3 08:22:42 jetson-nano-emmc-cot user.info mender[729693]: time=“2021-06-03T08:22:42+03:00” level=info msg=“Handling error state, current error: transient error: (request_id: ): Invalid response received from server server error message: failed to parse server response: json: c
Jun 3 08:22:42 jetson-nano-emmc-cot user.info mender[729693]: time=“2021-06-03T08:22:42+03:00” level=info msg=“State transition: error [Error] → idle [Idle]” func=github.com/mendersoftware/mender/app.transitionState file=”/home/tirank/GIT/nvidia_1.0.5_new/tegra-l4t/build-nano-emm
Jun 3 08:22:42 jetson-nano-emmc-cot user.info mender[729693]: time=“2021-06-03T08:22:42+03:00” level=info msg=“State transition: idle [Idle] → check-wait [Idle]” func=github.com/mendersoftware/mender/app.transitionState file=”/home/tirank/GIT/nvidia_1.0.5_new/tegra-l4t/build-nano

On the device side, mender client version is 2.6.0 (couldnt find a client version 2.7.0 and dont know the client to server compatibility matrix).

Anyone seen this error?

Thanks

2

For the compatibility have a look here.

For the error: I have not seen it previously, any change you could run the client in debug mode, and paste the full logs (without it being truncated)?

:slight_smile:

3

Sure, I will need to rebuild for this, will do.

In the meantime, I looked at the server logs and saw this (mender deploy docker log):
time=“2021-06-03T11:05:10Z” level=error msg="error reaching artifact storage service: SerializationError: failed to decode REST XML response\n\tstatus code: 200, request id: \ncaused by: XML syntax error on line 8: element closed by " file=response_helpers.go func=rest_utils.restErrWithLogMsg line=110 request_id=03887c01-8b74-46fd-a0f2-4d28c5d9d1ab

4

Interesting, how have you setup your Artifact storage?

5

Regular production installation, on the same host name, post was set to 9000 in the compose yaml.

6

Hi @oleorhagen
Here is the client debug log:

Jun 06 20:56:31 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:56:31Z” level=debug msg=“device_type=jetson-nano-emmcsd-juganu-cot” func=github.com/mendersoftware/mender/de>
Jun 06 20:56:31 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:56:31Z” level=debug msg=“Current manifest data: jetson-nano-emmcsd-juganu-cot” func=github.com/mendersoftwar>
Jun 06 20:56:31 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:56:31Z” level=debug msg=“received the GET_AUTH_TOKEN action” func="github.com/mendersoftware/mender/app.(*me>
Jun 06 20:56:31 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:56:31Z” level=debug msg=“Connecting to server ota.nvidia.local.juganu.com” func="github.com/mendersoftware/m>
Jun 06 20:56:31 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:56:31Z” level=debug msg=“device provides not accepted by the server. Response code: 503” func="github.com/me>
Jun 06 20:56:31 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:56:31Z” level=debug msg=“Connecting to server ota.nvidia.local.juganu.com” func="github.com/mendersoftware/m>
Jun 06 20:56:31 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:56:31Z” level=debug msg=“Received response:503 Service Unavailable” func=github.com/mendersoftware/mender/cl>
Jun 06 20:56:31 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:56:31Z” level=warning msg=“Client received invalid response status code: 503” func=github.com/mendersoftware>
Jun 06 20:56:31 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:56:31Z” level=error msg="Error receiving scheduled update data: (request_id: ): Invalid response received fr>
Jun 06 20:56:31 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:56:31Z” level=error msg="Update check failed: transient error: (request_id: ): Invalid response received fro>
Jun 06 20:56:31 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:56:31Z” level=info msg=“State transition: update-check [Sync] → error [Error]” func=github.com/mendersoftwa>
Jun 06 20:56:31 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:56:31Z” level=debug msg=“Transitioning to error state” func=github.com/mendersoftware/mender/app.transitionS>
Jun 06 20:56:31 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:56:31Z” level=debug msg="statescript: The timeout for executing scripts is not defined; using default of 1h0>
Jun 06 20:56:31 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:56:31Z” level=debug msg="statescript: The timeout for executing scripts is not defined; using default of 1h0>
Jun 06 20:56:31 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:56:31Z” level=info msg="Handling error state, current error: transient error: (request_id: ): Invalid respon>
Jun 06 20:56:31 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:56:31Z” level=info msg=“State transition: error [Error] → idle [Idle]” func=github.com/mendersoftware/mende>
Jun 06 20:56:31 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:56:31Z” level=debug msg="statescript: The timeout for executing scripts is not defined; using default of 1h0>
Jun 06 20:56:31 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:56:31Z” level=debug msg="statescript: The timeout for executing scripts is not defined; using default of 1h0>
Jun 06 20:56:31 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:56:31Z” level=debug msg=“received the GET_AUTH_TOKEN action” func="github.com/mendersoftware/mender/app.(*me>
Jun 06 20:56:31 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:56:31Z” level=info msg=“State transition: idle [Idle] → check-wait [Idle]” func=github.com/mendersoftware/m>
Jun 06 20:56:31 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:56:31Z” level=debug msg=“Handle check wait state” func="github.com/mendersoftware/mender/app.(*checkWaitStat>
Jun 06 20:56:31 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:56:31Z” level=debug msg="Check wait state; next checks: (update: 2021-06-06 20:57:01.353348861 +0000 UTC m=+>
Jun 06 20:56:31 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:56:31Z” level=debug msg="Next check: 2021-06-06 20:57:00.294817806 +0000 UTC m=+390.405576111:inventory-upda>
Jun 06 20:56:31 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:56:31Z” level=debug msg=“Waiting 28.840409732s for the next state” func=“github.com/mendersoftware/mender/ap>
Jun 06 20:57:00 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:00Z” level=debug msg=“Wait complete” func=“github.com/mendersoftware/mender/app.(*waitState).Wait” file=”>
Jun 06 20:57:00 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:00Z” level=info msg=“State transition: check-wait [Idle] → inventory-update [Sync]” func=github.com/mend>
Jun 06 20:57:00 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:00Z” level=debug msg="statescript: The timeout for executing scripts is not defined; using default of 1h0>
Jun 06 20:57:00 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:00Z” level=debug msg="statescript: The timeout for executing scripts is not defined; using default of 1h0>
Jun 06 20:57:00 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:00Z” level=debug msg=“Returning artifact name from /etc/mender/artifact_info file.” func="github.com/mend>
Jun 06 20:57:00 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:00Z” level=debug msg=“Reading data from the device manifest file: /etc/mender/artifact_info” func=github.>
Jun 06 20:57:00 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:00Z” level=debug msg=“artifact_name=juganu-image-prod_3.1-d63ee59-tirank” func=github.com/mendersoftware/>
Jun 06 20:57:00 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:00Z” level=debug msg=“Current manifest data: juganu-image-prod_3.1-d63ee59-tirank” func=github.com/mender>
Jun 06 20:57:00 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:00Z” level=debug msg=“Reading data from the device manifest file: /var/lib/mender/device_type” func=githu>
Jun 06 20:57:00 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:00Z” level=debug msg=“device_type=jetson-nano-emmcsd-juganu-cot” func=github.com/mendersoftware/mender/de>
Jun 06 20:57:00 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:00Z” level=debug msg=“Current manifest data: jetson-nano-emmcsd-juganu-cot” func=github.com/mendersoftwar>
Jun 06 20:57:00 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:00Z” level=debug msg=“received the GET_AUTH_TOKEN action” func="github.com/mendersoftware/mender/app.(*me>
Jun 06 20:57:00 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:00Z” level=debug msg=“Connecting to server ota.nvidia.local.juganu.com” func="github.com/mendersoftware/m>
Jun 06 20:57:00 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:00Z” level=debug msg="Inventory update sent, response &{200 OK 200 HTTP/1.1 1 1 map[Content-Length:[0] Co>
Jun 06 20:57:00 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:00Z” level=debug msg=“Inventory refresh complete” func="github.com/mendersoftware/mender/app.(*inventoryU>
Jun 06 20:57:00 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:00Z” level=info msg=“State transition: inventory-update [Sync] → check-wait [Idle]” func=github.com/mend>
Jun 06 20:57:00 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:00Z” level=debug msg="statescript: The timeout for executing scripts is not defined; using default of 1h0>
Jun 06 20:57:00 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:00Z” level=debug msg="statescript: The timeout for executing scripts is not defined; using default of 1h0>
Jun 06 20:57:00 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:00Z” level=debug msg=“Handle check wait state” func="github.com/mendersoftware/mender/app.(*checkWaitStat>
Jun 06 20:57:00 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:00Z” level=debug msg="Check wait state; next checks: (update: 2021-06-06 20:57:01.353348861 +0000 UTC m=+>
Jun 06 20:57:00 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:00Z” level=debug msg="Next check: 2021-06-06 20:57:01.353348861 +0000 UTC m=+391.464107374:update-check, >
Jun 06 20:57:00 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:00Z” level=debug msg=“Waiting 385.897902ms for the next state” func=“github.com/mendersoftware/mender/app>
Jun 06 20:57:01 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:01Z” level=debug msg=“Wait complete” func=“github.com/mendersoftware/mender/app.(*waitState).Wait” file=”>
Jun 06 20:57:01 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:01Z” level=info msg=“State transition: check-wait [Idle] → update-check [Sync]” func=github.com/menderso>
Jun 06 20:57:01 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:01Z” level=debug msg="statescript: The timeout for executing scripts is not defined; using default of 1h0>
Jun 06 20:57:01 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:01Z” level=debug msg="statescript: The timeout for executing scripts is not defined; using default of 1h0>
Jun 06 20:57:01 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:01Z” level=debug msg=“Handle update check state” func="github.com/mendersoftware/mender/app.(*updateCheck>
Jun 06 20:57:01 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:01Z” level=debug msg=“Returning artifact name from /etc/mender/artifact_info file.” func="github.com/mend>
Jun 06 20:57:01 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:01Z” level=debug msg=“Reading data from the device manifest file: /etc/mender/artifact_info” func=github.>
Jun 06 20:57:01 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:01Z” level=debug msg=“artifact_name=juganu-image-prod_3.1-d63ee59-tirank” func=github.com/mendersoftware/>
Jun 06 20:57:01 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:01Z” level=debug msg=“Current manifest data: juganu-image-prod_3.1-d63ee59-tirank” func=github.com/mender>
Jun 06 20:57:01 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:01Z” level=debug msg=“Reading data from the device manifest file: /var/lib/mender/device_type” func=githu>
Jun 06 20:57:01 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:01Z” level=debug msg=“device_type=jetson-nano-emmcsd-juganu-cot” func=github.com/mendersoftware/mender/de>
Jun 06 20:57:01 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:01Z” level=debug msg=“Current manifest data: jetson-nano-emmcsd-juganu-cot” func=github.com/mendersoftwar>
Jun 06 20:57:01 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:01Z” level=debug msg=“received the GET_AUTH_TOKEN action” func="github.com/mendersoftware/mender/app.(*me>
Jun 06 20:57:01 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:01Z” level=debug msg=“Connecting to server ota.nvidia.local.juganu.com” func="github.com/mendersoftware/m>
Jun 06 20:57:01 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:01Z” level=debug msg=“device provides not accepted by the server. Response code: 503” func="github.com/me>
Jun 06 20:57:01 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:01Z” level=debug msg=“Connecting to server ota.nvidia.local.juganu.com” func="github.com/mendersoftware/m>
Jun 06 20:57:01 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:01Z” level=debug msg=“Received response:503 Service Unavailable” func=github.com/mendersoftware/mender/cl>
Jun 06 20:57:01 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:01Z” level=warning msg=“Client received invalid response status code: 503” func=github.com/mendersoftware>
Jun 06 20:57:01 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:01Z” level=error msg="Error receiving scheduled update data: (request_id: ): Invalid response received fr>
Jun 06 20:57:01 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:01Z” level=error msg="Update check failed: transient error: (request_id: ): Invalid response received fro>
Jun 06 20:57:01 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:01Z” level=info msg=“State transition: update-check [Sync] → error [Error]” func=github.com/mendersoftwa>
Jun 06 20:57:01 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:01Z” level=debug msg=“Transitioning to error state” func=github.com/mendersoftware/mender/app.transitionS>
Jun 06 20:57:01 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:01Z” level=debug msg="statescript: The timeout for executing scripts is not defined; using default of 1h0>
Jun 06 20:57:01 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:01Z” level=debug msg="statescript: The timeout for executing scripts is not defined; using default of 1h0>
Jun 06 20:57:01 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:01Z” level=info msg="Handling error state, current error: transient error: (request_id: ): Invalid respon>
Jun 06 20:57:01 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:01Z” level=info msg=“State transition: error [Error] → idle [Idle]” func=github.com/mendersoftware/mende>
Jun 06 20:57:01 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:01Z” level=debug msg="statescript: The timeout for executing scripts is not defined; using default of 1h0>
Jun 06 20:57:01 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:01Z” level=debug msg="statescript: The timeout for executing scripts is not defined; using default of 1h0>
Jun 06 20:57:01 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:01Z” level=debug msg=“received the GET_AUTH_TOKEN action” func="github.com/mendersoftware/mender/app.(*me>
Jun 06 20:57:01 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:01Z” level=info msg=“State transition: idle [Idle] → check-wait [Idle]” func=github.com/mendersoftware/m>
Jun 06 20:57:01 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:01Z” level=debug msg=“Handle check wait state” func="github.com/mendersoftware/mender/app.(*checkWaitStat>
Jun 06 20:57:01 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:01Z” level=debug msg="Check wait state; next checks: (update: 2021-06-06 20:57:31.353348861 +0000 UTC m=+>
Jun 06 20:57:01 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:01Z” level=debug msg="Next check: 2021-06-06 20:57:30.294817806 +0000 UTC m=+420.405576111:inventory-upda>
Jun 06 20:57:01 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:01Z” level=debug msg=“Waiting 28.853989902s for the next state” func=“github.com/mendersoftware/mender/ap>
Jun 06 20:57:30 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:30Z” level=debug msg=“Wait complete” func=“github.com/mendersoftware/mender/app.(*waitState).Wait” file=”>
Jun 06 20:57:30 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:30Z” level=info msg=“State transition: check-wait [Idle] → inventory-update [Sync]” func=github.com/mend>
Jun 06 20:57:30 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:30Z” level=debug msg="statescript: The timeout for executing scripts is not defined; using default of 1h0>
Jun 06 20:57:30 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:30Z” level=debug msg="statescript: The timeout for executing scripts is not defined; using default of 1h0>
Jun 06 20:57:30 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:30Z” level=debug msg=“Returning artifact name from /etc/mender/artifact_info file.” func="github.com/mend>
Jun 06 20:57:30 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:30Z” level=debug msg=“Reading data from the device manifest file: /etc/mender/artifact_info” func=github.>
Jun 06 20:57:30 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:30Z” level=debug msg=“artifact_name=juganu-image-prod_3.1-d63ee59-tirank” func=github.com/mendersoftware/>
Jun 06 20:57:30 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:30Z” level=debug msg=“Current manifest data: juganu-image-prod_3.1-d63ee59-tirank” func=github.com/mender>
Jun 06 20:57:30 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:30Z” level=debug msg=“Reading data from the device manifest file: /var/lib/mender/device_type” func=githu>
Jun 06 20:57:30 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:30Z” level=debug msg=“device_type=jetson-nano-emmcsd-juganu-cot” func=github.com/mendersoftware/mender/de>
Jun 06 20:57:30 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:30Z” level=debug msg=“Current manifest data: jetson-nano-emmcsd-juganu-cot” func=github.com/mendersoftwar>
Jun 06 20:57:30 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:30Z” level=debug msg=“received the GET_AUTH_TOKEN action” func="github.com/mendersoftware/mender/app.(*me>
Jun 06 20:57:30 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:30Z” level=debug msg=“Connecting to server ota.nvidia.local.juganu.com” func="github.com/mendersoftware/m>
Jun 06 20:57:30 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:30Z” level=debug msg="Inventory update sent, response &{200 OK 200 HTTP/1.1 1 1 map[Content-Length:[0] Co>
Jun 06 20:57:30 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:30Z” level=debug msg=“Inventory refresh complete” func="github.com/mendersoftware/mender/app.(*inventoryU>
Jun 06 20:57:30 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:30Z” level=info msg=“State transition: inventory-update [Sync] → check-wait [Idle]” func=github.com/mend>
Jun 06 20:57:30 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:30Z” level=debug msg="statescript: The timeout for executing scripts is not defined; using default of 1h0>
Jun 06 20:57:30 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:30Z” level=debug msg="statescript: The timeout for executing scripts is not defined; using default of 1h0>
Jun 06 20:57:30 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:30Z” level=debug msg=“Handle check wait state” func="github.com/mendersoftware/mender/app.(*checkWaitStat>
Jun 06 20:57:30 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:30Z” level=debug msg="Check wait state; next checks: (update: 2021-06-06 20:57:31.353348861 +0000 UTC m=+>
Jun 06 20:57:30 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:30Z” level=debug msg="Next check: 2021-06-06 20:57:31.353348861 +0000 UTC m=+421.464107374:update-check, >
Jun 06 20:57:30 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:30Z” level=debug msg=“Waiting 429.982749ms for the next state” func=“github.com/mendersoftware/mender/app>
Jun 06 20:57:31 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:31Z” level=debug msg=“Wait complete” func=“github.com/mendersoftware/mender/app.(*waitState).Wait” file=”>
Jun 06 20:57:31 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:31Z” level=info msg=“State transition: check-wait [Idle] → update-check [Sync]” func=github.com/menderso>
Jun 06 20:57:31 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:31Z” level=debug msg="statescript: The timeout for executing scripts is not defined; using default of 1h0>
Jun 06 20:57:31 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:31Z” level=debug msg="statescript: The timeout for executing scripts is not defined; using default of 1h0>
Jun 06 20:57:31 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:31Z” level=debug msg=“Handle update check state” func="github.com/mendersoftware/mender/app.(*updateCheck>
Jun 06 20:57:31 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:31Z” level=debug msg=“Returning artifact name from /etc/mender/artifact_info file.” func="github.com/mend>
Jun 06 20:57:31 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:31Z” level=debug msg=“Reading data from the device manifest file: /etc/mender/artifact_info” func=github.>
Jun 06 20:57:31 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:31Z” level=debug msg=“artifact_name=juganu-image-prod_3.1-d63ee59-tirank” func=github.com/mendersoftware/>
Jun 06 20:57:31 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:31Z” level=debug msg=“Current manifest data: juganu-image-prod_3.1-d63ee59-tirank” func=github.com/mender>
Jun 06 20:57:31 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:31Z” level=debug msg=“Reading data from the device manifest file: /var/lib/mender/device_type” func=githu>
Jun 06 20:57:31 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:31Z” level=debug msg=“device_type=jetson-nano-emmcsd-juganu-cot” func=github.com/mendersoftware/mender/de>
Jun 06 20:57:31 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:31Z” level=debug msg=“Current manifest data: jetson-nano-emmcsd-juganu-cot” func=github.com/mendersoftwar>
Jun 06 20:57:31 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:31Z” level=debug msg=“received the GET_AUTH_TOKEN action” func="github.com/mendersoftware/mender/app.(*me>
Jun 06 20:57:31 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:31Z” level=debug msg=“Connecting to server ota.nvidia.local.juganu.com” func="github.com/mendersoftware/m>
Jun 06 20:57:31 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:31Z” level=debug msg=“device provides not accepted by the server. Response code: 503” func="github.com/me>
Jun 06 20:57:31 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:31Z” level=debug msg=“Connecting to server ota.nvidia.local.juganu.com” func="github.com/mendersoftware/m>
Jun 06 20:57:31 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:31Z” level=debug msg=“Received response:503 Service Unavailable” func=github.com/mendersoftware/mender/cl>
Jun 06 20:57:31 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:31Z” level=warning msg=“Client received invalid response status code: 503” func=github.com/mendersoftware>
Jun 06 20:57:31 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:31Z” level=error msg="Error receiving scheduled update data: (request_id: ): Invalid response received fr>
Jun 06 20:57:31 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:31Z” level=error msg="Update check failed: transient error: (request_id: ): Invalid response received fro>
Jun 06 20:57:31 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:31Z” level=info msg=“State transition: update-check [Sync] → error [Error]” func=github.com/mendersoftwa>
Jun 06 20:57:31 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:31Z” level=debug msg=“Transitioning to error state” func=github.com/mendersoftware/mender/app.transitionS>
Jun 06 20:57:31 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:31Z” level=debug msg="statescript: The timeout for executing scripts is not defined; using default of 1h0>
Jun 06 20:57:31 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:31Z” level=debug msg="statescript: The timeout for executing scripts is not defined; using default of 1h0>
Jun 06 20:57:31 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:31Z” level=info msg="Handling error state, current error: transient error: (request_id: ): Invalid respon>
Jun 06 20:57:31 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:31Z” level=info msg=“State transition: error [Error] → idle [Idle]” func=github.com/mendersoftware/mende>
Jun 06 20:57:31 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:31Z” level=debug msg="statescript: The timeout for executing scripts is not defined; using default of 1h0>
Jun 06 20:57:31 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:31Z” level=debug msg="statescript: The timeout for executing scripts is not defined; using default of 1h0>
Jun 06 20:57:31 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:31Z” level=debug msg=“received the GET_AUTH_TOKEN action” func="github.com/mendersoftware/mender/app.(*me>
Jun 06 20:57:31 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:31Z” level=info msg=“State transition: idle [Idle] → check-wait [Idle]” func=github.com/mendersoftware/m>
Jun 06 20:57:31 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:31Z” level=debug msg=“Handle check wait state” func="github.com/mendersoftware/mender/app.(*checkWaitStat>
Jun 06 20:57:31 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:31Z” level=debug msg="Check wait state; next checks: (update: 2021-06-06 20:58:01.353348861 +0000 UTC m=+>
Jun 06 20:57:31 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:31Z” level=debug msg="Next check: 2021-06-06 20:58:00.294817806 +0000 UTC m=+450.405576111:inventory-upda>
Jun 06 20:57:31 jetson-nano-emmcsd-juganu-cot mender[104661]: time=“2021-06-06T20:57:31Z” level=debug msg=“Waiting 28.801758698s for the next state” func="github.com/mendersoftware/mender/ap>
l

7

Hi @oleorhagen
Here is the full log, I had to cut the log to match the size limitation:

Jun 06 21:00:01 jetson-nano-emmcsd-juganu-cot mender[104661]: time="2021-06-06T21:00:01Z" level=debug msg="statescript: The timeout for executing scripts is not defined; using default of 1h0m0s seconds" func=github.com/mendersoftware/mender/statescript.Launcher.getTimeout file="/home/tirank/GIT/nvidia_1.0.5_new/tegra-l4t/nano-emmcsd-prod/tmp/work/jetson_nano_emmcsd_juganu_cot-juganu-linux/mender-client/2.6.0-r0/build/src/github.com/mendersoftware/mender/statescript/executor.go:82"
Jun 06 21:00:01 jetson-nano-emmcsd-juganu-cot mender[104661]: time="2021-06-06T21:00:01Z" level=debug msg="Handle update check state" func="github.com/mendersoftware/mender/app.(*updateCheckState).Handle" file="/home/tirank/GIT/nvidia_1.0.5_new/tegra-l4t/nano-emmcsd-prod/tmp/work/jetson_nano_emmcsd_juganu_cot-juganu-linux/mender-client/2.6.0-r0/build/src/github.com/mendersoftware/mender/app/state.go:641"
Jun 06 21:00:01 jetson-nano-emmcsd-juganu-cot mender[104661]: time="2021-06-06T21:00:01Z" level=debug msg="Returning artifact name from /etc/mender/artifact_info file." func="github.com/mendersoftware/mender/device.(*DeviceManager).GetCurrentArtifactName" file="/home/tirank/GIT/nvidia_1.0.5_new/tegra-l4t/nano-emmcsd-prod/tmp/work/jetson_nano_emmcsd_juganu_cot-juganu-linux/mender-client/2.6.0-r0/build/src/github.com/mendersoftware/mender/device/device.go:134"
Jun 06 21:00:01 jetson-nano-emmcsd-juganu-cot mender[104661]: time="2021-06-06T21:00:01Z" level=debug msg="Reading data from the device manifest file: /etc/mender/artifact_info" func=github.com/mendersoftware/mender/device.GetManifestData file="/home/tirank/GIT/nvidia_1.0.5_new/tegra-l4t/nano-emmcsd-prod/tmp/work/jetson_nano_emmcsd_juganu_cot-juganu-linux/mender-client/2.6.0-r0/build/src/github.com/mendersoftware/mender/device/device.go:80"
Jun 06 21:00:01 jetson-nano-emmcsd-juganu-cot mender[104661]: time="2021-06-06T21:00:01Z" level=debug msg="artifact_name=juganu-image-prod_3.1-d63ee59-tirank" func=github.com/mendersoftware/mender/device.GetManifestData file="/home/tirank/GIT/nvidia_1.0.5_new/tegra-l4t/nano-emmcsd-prod/tmp/work/jetson_nano_emmcsd_juganu_cot-juganu-linux/mender-client/2.6.0-r0/build/src/github.com/mendersoftware/mender/device/device.go:91"
Jun 06 21:00:01 jetson-nano-emmcsd-juganu-cot mender[104661]: time="2021-06-06T21:00:01Z" level=debug msg="Current manifest data: juganu-image-prod_3.1-d63ee59-tirank" func=github.com/mendersoftware/mender/device.GetManifestData file="/home/tirank/GIT/nvidia_1.0.5_new/tegra-l4t/nano-emmcsd-prod/tmp/work/jetson_nano_emmcsd_juganu_cot-juganu-linux/mender-client/2.6.0-r0/build/src/github.com/mendersoftware/mender/device/device.go:98"
Jun 06 21:00:01 jetson-nano-emmcsd-juganu-cot mender[104661]: time="2021-06-06T21:00:01Z" level=debug msg="Reading data from the device manifest file: /var/lib/mender/device_type" func=github.com/mendersoftware/mender/device.GetManifestData file="/home/tirank/GIT/nvidia_1.0.5_new/tegra-l4t/nano-emmcsd-prod/tmp/work/jetson_nano_emmcsd_juganu_cot-juganu-linux/mender-client/2.6.0-r0/build/src/github.com/mendersoftware/mender/device/device.go:80"
Jun 06 21:00:01 jetson-nano-emmcsd-juganu-cot mender[104661]: time="2021-06-06T21:00:01Z" level=debug msg="device_type=jetson-nano-emmcsd-juganu-cot" func=github.com/mendersoftware/mender/device.GetManifestData file="/home/tirank/GIT/nvidia_1.0.5_new/tegra-l4t/nano-emmcsd-prod/tmp/work/jetson_nano_emmcsd_juganu_cot-juganu-linux/mender-client/2.6.0-r0/build/src/github.com/mendersoftware/mender/device/device.go:91"
Jun 06 21:00:01 jetson-nano-emmcsd-juganu-cot mender[104661]: time="2021-06-06T21:00:01Z" level=debug msg="Current manifest data: jetson-nano-emmcsd-juganu-cot" func=github.com/mendersoftware/mender/device.GetManifestData file="/home/tirank/GIT/nvidia_1.0.5_new/tegra-l4t/nano-emmcsd-prod/tmp/work/jetson_nano_emmcsd_juganu_cot-juganu-linux/mender-client/2.6.0-r0/build/src/github.com/mendersoftware/mender/device/device.go:98"
Jun 06 21:00:01 jetson-nano-emmcsd-juganu-cot mender[104661]: time="2021-06-06T21:00:01Z" level=debug msg="received the GET_AUTH_TOKEN action" func="github.com/mendersoftware/mender/app.(*menderAuthManagerService).run" file="/home/tirank/GIT/nvidia_1.0.5_new/tegra-l4t/nano-emmcsd-prod/tmp/work/jetson_nano_emmcsd_juganu_cot-juganu-linux/mender-client/2.6.0-r0/build/src/github.com/mendersoftware/mender/app/auth.go:352"
Jun 06 21:00:01 jetson-nano-emmcsd-juganu-cot mender[104661]: time="2021-06-06T21:00:01Z" level=debug msg="Connecting to server ota.nvidia.local.juganu.com" func="github.com/mendersoftware/mender/client.(*ApiRequest).Do" file="/home/tirank/GIT/nvidia_1.0.5_new/tegra-l4t/nano-emmcsd-prod/tmp/work/jetson_nano_emmcsd_juganu_cot-juganu-linux/mender-client/2.6.0-r0/build/src/github.com/mendersoftware/mender/client/client.go:219"
Jun 06 21:00:01 jetson-nano-emmcsd-juganu-cot mender[104661]: time="2021-06-06T21:00:01Z" level=debug msg="device provides not accepted by the server. Response code: 503" func="github.com/mendersoftware/mender/client.(*UpdateClient).getUpdateInfo" file="/home/tirank/GIT/nvidia_1.0.5_new/tegra-l4t/nano-emmcsd-prod/tmp/work/jetson_nano_emmcsd_juganu_cot-juganu-linux/mender-client/2.6.0-r0/build/src/github.com/mendersoftware/mender/client/client_update.go:103"
Jun 06 21:00:01 jetson-nano-emmcsd-juganu-cot mender[104661]: time="2021-06-06T21:00:01Z" level=debug msg="Connecting to server ota.nvidia.local.juganu.com" func="github.com/mendersoftware/mender/client.(*ApiRequest).Do" file="/home/tirank/GIT/nvidia_1.0.5_new/tegra-l4t/nano-emmcsd-prod/tmp/work/jetson_nano_emmcsd_juganu_cot-juganu-linux/mender-client/2.6.0-r0/build/src/github.com/mendersoftware/mender/client/client.go:219"
Jun 06 21:00:01 jetson-nano-emmcsd-juganu-cot mender[104661]: time="2021-06-06T21:00:01Z" level=debug msg="Received response:503 Service Unavailable" func=github.com/mendersoftware/mender/client.processUpdateResponse file="/home/tirank/GIT/nvidia_1.0.5_new/tegra-l4t/nano-emmcsd-prod/tmp/work/jetson_nano_emmcsd_juganu_cot-juganu-linux/mender-client/2.6.0-r0/build/src/github.com/mendersoftware/mender/client/client_update.go:184"
Jun 06 21:00:01 jetson-nano-emmcsd-juganu-cot mender[104661]: time="2021-06-06T21:00:01Z" level=warning msg="Client received invalid response status code: 503" func=github.com/mendersoftware/mender/client.processUpdateResponse file="/home/tirank/GIT/nvidia_1.0.5_new/tegra-l4t/nano-emmcsd-prod/tmp/work/jetson_nano_emmcsd_juganu_cot-juganu-linux/mender-client/2.6.0-r0/build/src/github.com/mendersoftware/mender/client/client_update.go:215"
Jun 06 21:00:01 jetson-nano-emmcsd-juganu-cot mender[104661]: time="2021-06-06T21:00:01Z" level=error msg="Error receiving scheduled update data: (request_id: ): Invalid response received from server server error message: failed to parse server response: json: cannot unmarshal object into Go struct field .error of type string" func="github.com/mendersoftware/mender/app.(*Mender).CheckUpdate" file="/home/tirank/GIT/nvidia_1.0.5_new/tegra-l4t/nano-emmcsd-prod/tmp/work/jetson_nano_emmcsd_juganu_cot-juganu-linux/mender-client/2.6.0-r0/build/src/github.com/mendersoftware/mender/app/mender.go:309"
Jun 06 21:00:01 jetson-nano-emmcsd-juganu-cot mender[104661]: time="2021-06-06T21:00:01Z" level=error msg="Update check failed: transient error: (request_id: ): Invalid response received from server server error message: failed to parse server response: json: cannot unmarshal object into Go struct field .error of type string" func="github.com/mendersoftware/mender/app.(*updateCheckState).Handle" file="/home/tirank/GIT/nvidia_1.0.5_new/tegra-l4t/nano-emmcsd-prod/tmp/work/jetson_nano_emmcsd_juganu_cot-juganu-linux/mender-client/2.6.0-r0/build/src/github.com/mendersoftware/mender/app/state.go:652"
Jun 06 21:00:01 jetson-nano-emmcsd-juganu-cot mender[104661]: time="2021-06-06T21:00:01Z" level=info msg="State transition: update-check [Sync] -> error [Error]" func=github.com/mendersoftware/mender/app.transitionState file="/home/tirank/GIT/nvidia_1.0.5_new/tegra-l4t/nano-emmcsd-prod/tmp/work/jetson_nano_emmcsd_juganu_cot-juganu-linux/mender-client/2.6.0-r0/build/src/github.com/mendersoftware/mender/app/mender.go:461"
Jun 06 21:00:01 jetson-nano-emmcsd-juganu-cot mender[104661]: time="2021-06-06T21:00:01Z" level=debug msg="Transitioning to error state" func=github.com/mendersoftware/mender/app.transitionState file="/home/tirank/GIT/nvidia_1.0.5_new/tegra-l4t/nano-emmcsd-prod/tmp/work/jetson_nano_emmcsd_juganu_cot-juganu-linux/mender-client/2.6.0-r0/build/src/github.com/mendersoftware/mender/app/mender.go:477"
Jun 06 21:00:01 jetson-nano-emmcsd-juganu-cot mender[104661]: time="2021-06-06T21:00:01Z" level=debug msg="statescript: The timeout for executing scripts is not defined; using default of 1h0m0s seconds" func=github.com/mendersoftware/mender/statescript.Launcher.getTimeout file="/home/tirank/GIT/nvidia_1.0.5_new/tegra-l4t/nano-emmcsd-prod/tmp/work/jetson_nano_emmcsd_juganu_cot-juganu-linux/mender-client/2.6.0-r0/build/src/github.com/mendersoftware/mender/statescript/executor.go:82"
Jun 06 21:00:01 jetson-nano-emmcsd-juganu-cot mender[104661]: time="2021-06-06T21:00:01Z" level=debug msg="statescript: The timeout for executing scripts is not defined; using default of 1h0m0s seconds" func=github.com/mendersoftware/mender/statescript.Launcher.getTimeout file="/home/tirank/GIT/nvidia_1.0.5_new/tegra-l4t/nano-emmcsd-prod/tmp/work/jetson_nano_emmcsd_juganu_cot-juganu-linux/mender-client/2.6.0-r0/build/src/github.com/mendersoftware/mender/statescript/executor.go:82"
Jun 06 21:00:01 jetson-nano-emmcsd-juganu-cot mender[104661]: time="2021-06-06T21:00:01Z" level=info msg="Handling error state, current error: transient error: (request_id: ): Invalid response received from server server error message: failed to parse server response: json: cannot unmarshal object into Go struct field .error of type string" func="github.com/mendersoftware/mender/app.(*errorState).Handle" file="/home/tirank/GIT/nvidia_1.0.5_new/tegra-l4t/nano-emmcsd-prod/tmp/work/jetson_nano_emmcsd_juganu_cot-juganu-linux/mender-client/2.6.0-r0/build/src/github.com/mendersoftware/mender/app/state.go:1208"
Jun 06 21:00:01 jetson-nano-emmcsd-juganu-cot mender[104661]: time="2021-06-06T21:00:01Z" level=info msg="State transition: error [Error] -> idle [Idle]" func=github.com/mendersoftware/mender/app.transitionState file="/home/tirank/GIT/nvidia_1.0.5_new/tegra-l4t/nano-emmcsd-prod/tmp/work/jetson_nano_emmcsd_juganu_cot-juganu-linux/mender-client/2.6.0-r0/build/src/github.com/mendersoftware/mender/app/mender.go:461"
Jun 06 21:00:01 jetson-nano-emmcsd-juganu-cot mender[104661]: time="2021-06-06T21:00:01Z" level=debug msg="statescript: The timeout for executing scripts is not defined; using default of 1h0m0s seconds" func=github.com/mendersoftware/mender/statescript.Launcher.getTimeout file="/home/tirank/GIT/nvidia_1.0.5_new/tegra-l4t/nano-emmcsd-prod/tmp/work/jetson_nano_emmcsd_juganu_cot-juganu-linux/mender-client/2.6.0-r0/build/src/github.com/mendersoftware/mender/statescript/executor.go:82"
Jun 06 21:00:01 jetson-nano-emmcsd-juganu-cot mender[104661]: time="2021-06-06T21:00:01Z" level=debug msg="statescript: The timeout for executing scripts is not defined; using default of 1h0m0s seconds" func=github.com/mendersoftware/mender/statescript.Launcher.getTimeout file="/home/tirank/GIT/nvidia_1.0.5_new/tegra-l4t/nano-emmcsd-prod/tmp/work/jetson_nano_emmcsd_juganu_cot-juganu-linux/mender-client/2.6.0-r0/build/src/github.com/mendersoftware/mender/statescript/executor.go:82"
Jun 06 21:00:01 jetson-nano-emmcsd-juganu-cot mender[104661]: time="2021-06-06T21:00:01Z" level=debug msg="received the GET_AUTH_TOKEN action" func="github.com/mendersoftware/mender/app.(*menderAuthManagerService).run" file="/home/tirank/GIT/nvidia_1.0.5_new/tegra-l4t/nano-emmcsd-prod/tmp/work/jetson_nano_emmcsd_juganu_cot-juganu-linux/mender-client/2.6.0-r0/build/src/github.com/mendersoftware/mender/app/auth.go:352"
Jun 06 21:00:01 jetson-nano-emmcsd-juganu-cot mender[104661]: time="2021-06-06T21:00:01Z" level=info msg="State transition: idle [Idle] -> check-wait [Idle]" func=github.com/mendersoftware/mender/app.transitionState file="/home/tirank/GIT/nvidia_1.0.5_new/tegra-l4t/nano-emmcsd-prod/tmp/work/jetson_nano_emmcsd_juganu_cot-juganu-linux/mender-client/2.6.0-r0/build/src/github.com/mendersoftware/mender/app/mender.go:461"
Jun 06 21:00:01 jetson-nano-emmcsd-juganu-cot mender[104661]: time="2021-06-06T21:00:01Z" level=debug msg="Handle check wait state" func="github.com/mendersoftware/mender/app.(*checkWaitState).Handle" file="/home/tirank/GIT/nvidia_1.0.5_new/tegra-l4t/nano-emmcsd-prod/tmp/work/jetson_nano_emmcsd_juganu_cot-juganu-linux/mender-client/2.6.0-r0/build/src/github.com/mendersoftware/mender/app/state.go:1102"
Jun 06 21:00:01 jetson-nano-emmcsd-juganu-cot mender[104661]: time="2021-06-06T21:00:01Z" level=debug msg="Check wait state; next checks: (update: 2021-06-06 21:00:31.353348861 +0000 UTC m=+601.464107374) (inventory: 2021-06-06 21:00:30.294817806 +0000 UTC m=+600.405576111)" func="github.com/mendersoftware/mender/app.(*checkWaitState).Handle" file="/home/tirank/GIT/nvidia_1.0.5_new/tegra-l4t/nano-emmcsd-prod/tmp/work/jetson_nano_emmcsd_juganu_cot-juganu-linux/mender-client/2.6.0-r0/build/src/github.com/mendersoftware/mender/app/state.go:1113"
Jun 06 21:00:01 jetson-nano-emmcsd-juganu-cot mender[104661]: time="2021-06-06T21:00:01Z" level=debug msg="Next check: 2021-06-06 21:00:30.294817806 +0000 UTC m=+600.405576111:inventory-update, (2021-06-06 21:00:01.484466822 +0000 UTC m=+571.595225179)" func="github.com/mendersoftware/mender/app.(*checkWaitState).Handle" file="/home/tirank/GIT/nvidia_1.0.5_new/tegra-l4t/nano-emmcsd-prod/tmp/work/jetson_nano_emmcsd_juganu_cot-juganu-linux/mender-client/2.6.0-r0/build/src/github.com/mendersoftware/mender/app/state.go:1131"
Jun 06 21:00:01 jetson-nano-emmcsd-juganu-cot mender[104661]: time="2021-06-06T21:00:01Z" level=debug msg="Waiting 28.810350932s for the next state" func="github.com/mendersoftware/mender/app.(*checkWaitState).Handle" file="/home/tirank/GIT/nvidia_1.0.5_new/tegra-l4t/nano-emmcsd-prod/tmp/work/jetson_nano_emmcsd_juganu_cot-juganu-linux/mender-client/2.6.0-r0/build/src/github.com/mendersoftware/mender/app/state.go:1159"
8

I am guessing this is not an issue on the client side, but rather with how the server is set up, considering the error code it receives when fetching the update.

@merlin perhaps you could help debugging the 503 from the server while fetching the update? And then I’ll be perusing from the sideline.

9

Hi @merlin
Following are the deployments container logs:
time=“2021-06-07T06:51:07Z” level=info msg=“204 154μs GET /api/internal/v1/deployments/alive HTTP/1.1 - Go-http-client/1.1” byteswritten=0 file=middleware.go func=“accesslog.(*AccessLogMiddleware).MiddlewareFunc.func1” line=71 method=GET path=/api/internal/v1/deployments/alive qs= request_id=d60c61f3-3f68-416b-8448-fc7aa6784edb responsetime=0.000154048 status=204 ts=“2021-06-07 06:51:07.452729674 +0000 UTC” type=http
time=“2021-06-07T06:51:07Z” level=error msg="error reaching artifact storage service: SerializationError: failed to decode REST XML response\n\tstatus code: 200, request id: \ncaused by: XML syntax error on line 8: element closed by " file=response_helpers.go func=rest_utils.restErrWithLogMsg line=110 request_id=1eb3a5a7-55ac-4fe3-9f74-7840b88e1b0a
time=“2021-06-07T06:51:07Z” level=info msg=“503 366026μs GET /api/internal/v1/deployments/health HTTP/1.1 - Go-http-client/1.1” byteswritten=286 file=middleware.go func=“accesslog.(*AccessLogMiddleware).MiddlewareFunc.func1” line=71 method=GET path=/api/internal/v1/deployments/health qs= request_id=1eb3a5a7-55ac-4fe3-9f74-7840b88e1b0a responsetime=0.366026345 status=503 ts=“2021-06-07 06:51:07.452832229 +0000 UTC” type=http
time=“2021-06-07T06:51:12Z” level=info msg=“204 164μs GET /api/internal/v1/deployments/alive HTTP/1.1 - Go-http-client/1.1” byteswritten=0 file=middleware.go func=“accesslog.(*AccessLogMiddleware).MiddlewareFunc.func1” line=71 method=GET path=/api/internal/v1/deployments/alive qs= request_id=d17d1b5a-a8cc-48bb-a88a-7f2c5445e8fa responsetime=0.0001645 status=204 ts=“2021-06-07 06:51:12.451589352 +0000 UTC” type=http
time=“2021-06-07T06:51:12Z” level=error msg="error reaching artifact storage service: SerializationError: failed to decode REST XML response\n\tstatus code: 200, request id: \ncaused by: XML syntax error on line 8: element closed by " file=response_helpers.go func=rest_utils.restErrWithLogMsg line=110 request_id=cc5e2f5e-74a8-4556-8a4d-81f4d96b62b2
time=“2021-06-07T06:51:12Z” level=info msg=“503 327650μs GET /api/internal/v1/deployments/health HTTP/1.1 - Go-http-client/1.1” byteswritten=286 file=middleware.go func=“accesslog.(*AccessLogMiddleware).MiddlewareFunc.func1” line=71 method=GET path=/api/internal/v1/deployments/health qs= request_id=cc5e2f5e-74a8-4556-8a4d-81f4d96b62b2 responsetime=0.327650855 status=503 ts=“2021-06-07 06:51:12.451752048 +0000 UTC” type=http

10

Probably the following thread is related to your issue:

Though we did not find a solution for it yet.

1 Like
11

yes seems very much related. thanks Dave I was looking for that :slight_smile:
could you provide the same information as requested here: Issue: mender-deployments SerializationError - #19 by peter

peter

12

@Tirank could you check the other thread: Issue: mender-deployments SerializationError - #29 by peter perhaps you will find it to solve your issue.

peter