Not finishing deployment

General Discussions
1

Hello,
I have a problem about deployment, and I need your help. All services are up and running. There are no services that has problem when I type ./run ps. The problem that I am having right now is that the deployment of a basic txt.file is not finishing. I am sharing te screenshots of the UI and the log of the mender-deployment service here.
Note: I am using self hosted mender version 3.4.

Approximately, every 10 seconds (10000ms), the logs saying inprogress, pending and list are repeating in the network tag of the browser.

Ekran görüntüsü 2023-09-25 084910
Ekran görüntüsü 2023-09-25 0849101920×1080 138 KB

Ekran görüntüsü 2023-09-25 085041
Ekran görüntüsü 2023-09-25 0850411920×1080 152 KB

The output of the docker logs menderproduction_mender-deployments_1 is the following:

Ekran görüntüsü 2023-09-25 090223
Ekran görüntüsü 2023-09-25 0902231920×948 138 KB

2

After a couple of tries, the deployment gave me the following failure log :

2023-09-25 08:10:29 +0000 UTC info: Running Mender client version: 3.5.1
2023-09-25 08:10:29 +0000 UTC warning: RetryPollIntervalSeconds is not defined
2023-09-25 08:10:29 +0000 UTC error: Error fetching scheduled update info: code (404)
2023-09-25 08:10:29 +0000 UTC error: Update fetch failed: error receiving scheduled update information server error message: 404 page not found
2023-09-25 08:10:29 +0000 UTC info: State transition: update-fetch [Download_Enter] → fetch-install-retry-wait [Download_Enter]
2023-09-25 08:10:29 +0000 UTC warning: UpdatePollIntervalSeconds is not defined
2023-09-25 08:40:29 +0000 UTC info: State transition: fetch-install-retry-wait [Download_Enter] → update-fetch [Download_Enter]
2023-09-25 08:40:30 +0000 UTC warning: RetryPollIntervalSeconds is not defined
2023-09-25 08:40:36 +0000 UTC error: Error fetching scheduled update info: code (404)
2023-09-25 08:40:36 +0000 UTC error: Update fetch failed: error receiving scheduled update information server error message: 404 page not found
2023-09-25 08:40:36 +0000 UTC info: State transition: update-fetch [Download_Enter] → fetch-install-retry-wait [Download_Enter]
2023-09-25 08:40:36 +0000 UTC warning: UpdatePollIntervalSeconds is not defined
2023-09-25 08:40:36 +0000 UTC info: State transition: fetch-install-retry-wait [Download_Enter] → update-error [ArtifactFailure]
2023-09-25 08:40:36 +0000 UTC info: State transition: update-error [ArtifactFailure] → cleanup [Error]
2023-09-25 08:40:36 +0000 UTC info: State transition: cleanup [Error] → update-status-report [none]

Could you please help me ? Thanks in advance.

3

ANY HELP ? I also share the related section of the prod.yml file here:

prod
prod749×978 53.4 KB

I am not sure about DEPLOYMENTS_AWS_URI and DEPLOYMENTS_AWS_EXTERNAL_URI. I set the external one same as my mender domain name, and set the DEPLOYMENTS_AWS_URI as minio:9000. I am trying to use minio not aws, so I am a little bit confused. I have only one domain by the way. Also, I am using version 3.4 with docker-compose option.

4

Hello @dekan ,
could you please verify if you could fetch the artifact storage? Preferably from the device itself, or from another host; you should issue something like this:

export AWS_ACCESS_KEY_ID=<your MINIO_ACCESS_KEY> 
export AWS_SECRET_ACCESS_KEY=<your MINIO_SECRET_KEY>
export DEPLOYMENTS_AWS_EXTERNAL_URI=https://mymenderdomain.com
aws s3 ls --endpoint-url $DEPLOYMENTS_AWS_EXTERNAL_URI s3://mender-artifact-storage/

Specifically, the DEPLOYMENTS_AWS_URI is for internal communication only, and the DEPLOYMENTS_AWS_EXTERNAL_URI is the external storage URL, accessible from the devices.

The AWS names are confusing, but kept for history reasons; it’s safe to use Minio with them.

Another test could be downloading the Artifact from the UI.

Thanks

1 Like
5

First of all, thank you so much for the reply.
I have tried to download the artifact(a basic txt file) from the UI, and it gave me 404 not found. What is the thing that I am doing wrong in this setup ?

6

Which URL is the UI providing to you?

1 Like
7

https://mymenderdomain.com/mender-artifact-storage/d4e75c24-0309-4708-96e3-8a7ab1c66af2?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=mender-deployments%2F20230926%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20230926T101905Z&X-Amz-Expires=900&X-Amz-SignedHeaders=host&response-content-disposition=attachment%3B%20filename%3D"deneme.txt.mender"&response-content-type=application%2Fvnd.mender-artifact&X-Amz-Signature=f011c5521d9558a3a9a95681f6c376c4c69def7396f1c1713c197f7ee6f68528

8

What about this test?

export AWS_ACCESS_KEY_ID=<your MINIO_ACCESS_KEY> 
export AWS_SECRET_ACCESS_KEY=<your MINIO_SECRET_KEY>
export DEPLOYMENTS_AWS_EXTERNAL_URI=https://mymenderdomain.com
aws s3 ls --endpoint-url $DEPLOYMENTS_AWS_EXTERNAL_URI s3://mender-artifact-storage/

This should display the artifact S3 URI, if any…

Of course, you have to resolve mymenderdomain.com domain

1 Like
9

I cannot execute the command starting with aws. I tried to install awscli, but I cannot do because of some errors that I cannot solve.
But, I have an idea that may causes this problem. Since I have changed the default port 443 to 44344, may the problem arises because of that? I am not including this port the DEPLOYMENTS_AWS_EXTERNAL_URI.

10

Good point, the api-gateway (port 443 by default) is also a gateway for Minio. So you can either try falling it back to 443 or by setting DEPLOYMENTS_AWS_EXTERNAL_URI=https://mymenderdomain.com:44344.
Also please set the protocol accordingly (http or https)

1 Like
11

Thank you so much, it works with no problem now.