# Mender server production installation failed with "Couldn't load deployments. Error: Bad Gateway Retrying in 26 seconds"

**URL:** <https://hub.mender.io/t/mender-server-production-installation-failed-with-couldnt-load-deployments-error-bad-gateway-retrying-in-26-seconds/5940>\
**Category:** General Discussions\
**Created:** [June 23, 2023, 10:39am UTC](https://hub.mender.io/t/mender-server-production-installation-failed-with-couldnt-load-deployments-error-bad-gateway-retrying-in-26-seconds/5940 "2023-06-23T10:39:23Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![rock63](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/rock63/32/1920_2.png) [@rock63](https://hub.mender.io/u/rock63)\
**Post date:** [June 23, 2023, 10:39am UTC](https://hub.mender.io/t/mender-server-production-installation-failed-with-couldnt-load-deployments-error-bad-gateway-retrying-in-26-seconds/5940/1 "2023-06-23T10:39:23Z")

</div>

Hi All,

I am trying to deploy the mender server production installation by following the given docs which provided by mender [Production installation | Mender documentation](https://docs.mender.io/2.4/administration/production-installation) .

below is the error which i faced:

 ![Screenshot from 2023-06-23 16-06-26](https://canada1.discourse-cdn.com/flex036/uploads/mender/original/2X/a/ab7bc397ebcc56db497643a0bfb1b8c9d2c95902.png)

below is my mender.conf file which running on board:  
**root@qemux86-64:~# cat /etc/mender/mender.conf**  
{  
“InventoryPollIntervalSeconds”: 5,  
“RetryPollIntervalSeconds”: 30,  
“ServerURL”: “[https://exaleap-mender/ui/](https://exaleap-mender/ui/)”,  
“TenantToken”: “dummy”,  
“UpdateControlMapBootExpirationTimeSeconds”: 45,  
“UpdateControlMapExpirationTimeSeconds”: 90,  
“UpdatePollIntervalSeconds”: 5

Any help will be highly appreciated.

---

<div class="post-metadata">

**Author:** ![robgio](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/robgio/32/1760_2.png) [@robgio](https://hub.mender.io/u/robgio)\
**Post date:** [June 23, 2023, 12:35pm UTC](https://hub.mender.io/t/mender-server-production-installation-failed-with-couldnt-load-deployments-error-bad-gateway-retrying-in-26-seconds/5940/2 "2023-06-23T12:35:36Z")

</div>

Hi @rock63  
do you have any specific use-case to try the 2.4 Mender version?

If not could you please restart the process with the [latest (and supported) version,](https://docs.mender.io/3.4/server-installation/installation-with-docker-compose) which is 3.4?

Also, with this version I’d recommend to use the [Helm Setup for Kubernetes](https://docs.mender.io/3.4/server-installation/production-installation-with-kubernetes).

Thanks  
Roberto

---

<div class="post-metadata">

**Author:** ![rock63](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/rock63/32/1920_2.png) [@rock63](https://hub.mender.io/u/rock63)\
**Post date:** [June 27, 2023, 4:53am UTC](https://hub.mender.io/t/mender-server-production-installation-failed-with-couldnt-load-deployments-error-bad-gateway-retrying-in-26-seconds/5940/3 "2023-06-27T04:53:19Z")

</div>

Hi @robgio

I dont have any fix use case for any version just trying to setup mender server so I took the version 3.4 but here also facing issue just look at once.

Some of the services not starting and giving error:

**mender/mender-server/production$ ./run up -d**  
Creating network “menderproduction\_mender” with the default driver  
Creating menderproduction\_minio\_1 …  
Creating menderproduction\_mender-nats\_1 …  
Creating menderproduction\_mender-mongo\_1 …  
Creating menderproduction\_mender-gui\_1 …  
Creating menderproduction\_minio\_1  
Creating menderproduction\_mender-nats\_1  
Creating menderproduction\_mender-mongo\_1  
Creating menderproduction\_mender-mongo\_1 … done  
Creating menderproduction\_mender-iot-manager\_1 …  
Creating menderproduction\_mender-workflows-worker\_1 …  
Creating menderproduction\_mender-inventory\_1 …  
Creating menderproduction\_mender-deviceconfig\_1 …  
Creating menderproduction\_mender-workflows-server\_1 …  
Creating menderproduction\_mender-create-artifact-worker\_1 …  
Creating menderproduction\_mender-useradm\_1 …  
Creating menderproduction\_mender-deviceconnect\_1 …  
Creating menderproduction\_mender-iot-manager\_1  
Creating menderproduction\_mender-workflows-worker\_1  
Creating menderproduction\_mender-inventory\_1  
Creating menderproduction\_mender-workflows-server\_1  
Creating menderproduction\_mender-deviceconfig\_1  
Creating menderproduction\_mender-useradm\_1  
Creating menderproduction\_mender-create-artifact-worker\_1  
Creating menderproduction\_mender-workflows-server\_1 … done  
Creating menderproduction\_mender-device-auth\_1 …  
Creating menderproduction\_mender-device-auth\_1 … done  
Creating menderproduction\_mender-api-gateway\_1 …  
Creating menderproduction\_mender-api-gateway\_1 … done  
Creating menderproduction\_mender-deployments\_1 …  
Creating menderproduction\_mender-deployments\_1

ERROR: for menderproduction\_mender-deployments\_1 UnixHTTPConnectionPool(host=‘localhost’, port=None): Read timed out. (read timeout=60)

ERROR: for mender-deployments UnixHTTPConnectionPool(host=‘localhost’, port=None): Read timed out. (read timeout=60)  
ERROR: An HTTP request took too long to complete. Retry with --verbose to obtain debug information.  
If you encounter this issue regularly because of slow network conditions, consider setting COMPOSE\_HTTP\_TIMEOUT to a higher value (current value: 60).

## **mender/mender-server/production$ ./run ps** Name Command State Ports

menderproduction\_mender-api-gateway\_1 /entrypoint.sh --accesslog … Up 0.0.0.0:443-\>443/tcp,:::443-\>443/tcp, 80/tcp  
menderproduction\_mender-create-artifact-worker\_1 /usr/bin/workflows --confi … Up  
menderproduction\_mender-deployments\_1 /entrypoint.sh --config /e … Up 8080/tcp  
menderproduction\_mender-device-auth\_1 /usr/bin/deviceauth --conf … Up 8080/tcp  
menderproduction\_mender-deviceconfig\_1 /usr/bin/deviceconfig --co … Exit 0  
menderproduction\_mender-deviceconnect\_1 /usr/bin/deviceconnect --c … Up 8080/tcp  
menderproduction\_mender-gui\_1 /entrypoint.sh nginx Up 80/tcp, 8080/tcp  
menderproduction\_mender-inventory\_1 /usr/bin/inventory --confi … Up 8080/tcp  
menderproduction\_mender-iot-manager\_1 /usr/bin/iot-manager serve … Up 8080/tcp  
menderproduction\_mender-mongo\_1 docker-entrypoint.sh mongod Restarting  
menderproduction\_mender-nats\_1 docker-entrypoint.sh -js Up 4222/tcp, 6222/tcp, 8222/tcp  
menderproduction\_mender-useradm\_1 /usr/bin/useradm --config … Restarting  
menderproduction\_mender-workflows-server\_1 /usr/bin/workflows --confi … Up 8080/tcp  
menderproduction\_mender-workflows-worker\_1 /usr/bin/workflows --confi … Restarting  
menderproduction\_minio\_1 /usr/bin/docker-entrypoint … Up 9000/tcp

---

<div class="post-metadata">

**Author:** ![rock63](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/rock63/32/1920_2.png) [@rock63](https://hub.mender.io/u/rock63)\
**Post date:** [June 28, 2023, 3:20am UTC](https://hub.mender.io/t/mender-server-production-installation-failed-with-couldnt-load-deployments-error-bad-gateway-retrying-in-26-seconds/5940/4 "2023-06-28T03:20:12Z")

</div>

Hi Roberto,  
After doing **$./run up -d** all the services are up without any issue but This problem is coming at the of creating user name and password, I don t known why?.

log:  
**mender/mender-server/production$ docker ps**  
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES  
a784ea416ef4 mendersoftware/deployments:mender-3.4.0 “/entrypoint.sh --co…” 41 seconds ago Up 15 seconds 8080/tcp menderproduction\_mender-deployments\_1  
efdaa23dc773 traefik:v2.8 “/entrypoint.sh --ac…” About a minute ago Up 24 seconds 80/tcp, 0.0.0.0:443-\>443/tcp, :::443-\>443/tcp menderproduction\_mender-api-gateway\_1  
f581cce00f8d mendersoftware/deviceauth:mender-3.4.0 “/usr/bin/deviceauth…” About a minute ago Up 3 seconds 8080/tcp menderproduction\_mender-device-auth\_1  
7db75f147bef mendersoftware/deviceconnect:mender-3.4.0 “/usr/bin/deviceconn…” About a minute ago Up 32 seconds 8080/tcp menderproduction\_mender-deviceconnect\_1  
17dff12591d0 mendersoftware/workflows:mender-3.4.0 “/usr/bin/workflows …” About a minute ago Up 27 seconds 8080/tcp menderproduction\_mender-workflows-server\_1  
4678073f2ac3 mendersoftware/workflows-worker:mender-3.4.0 “/usr/bin/workflows …” About a minute ago Up 22 seconds 8080/tcp menderproduction\_mender-workflows-worker\_1  
5617079a8f1a mendersoftware/create-artifact-worker:mender-3.4.0 “/usr/bin/workflows …” About a minute ago Up 22 seconds menderproduction\_mender-create-artifact-worker\_1  
793f94864289 mendersoftware/useradm:mender-3.4.0 “/usr/bin/useradm --…” About a minute ago Up 3 seconds 8080/tcp menderproduction\_mender-useradm\_1  
de5c8c75775d mendersoftware/deviceconfig:mender-3.4.0 “/usr/bin/deviceconf…” About a minute ago Up About a minute 8080/tcp menderproduction\_mender-deviceconfig\_1  
e9b6742fafc4 mendersoftware/inventory:mender-3.4.0 “/usr/bin/inventory …” About a minute ago Up About a minute 8080/tcp menderproduction\_mender-inventory\_1  
2c7d768315ea mendersoftware/iot-manager:mender-3.4.0 “/usr/bin/iot-manage…” About a minute ago Up 48 seconds 8080/tcp menderproduction\_mender-iot-manager\_1  
0aa25dd9daa1 minio/minio:RELEASE.2019-04-23T23-50-36Z “/usr/bin/docker-ent…” 2 minutes ago Up About a minute (healthy) 9000/tcp menderproduction\_minio\_1  
e977cc26eb15 mendersoftware/gui:mender-3.4.0 “/entrypoint.sh nginx” 2 minutes ago Up About a minute (healthy) 80/tcp, 8080/tcp menderproduction\_mender-gui\_1  
f9ff5007dec1 mongo:4.4 “docker-entrypoint.s…” 2 minutes ago Up About a minute 27017/tcp menderproduction\_mender-mongo\_1  
e73b8f188010 nats:2.6-alpine “docker-entrypoint.s…” 2 minutes ago Up About a minute 4222/tcp, 6222/tcp, 8222/tcp menderproduction\_mender-nats\_1

**mender/mender-server/production$ ./run exec mender-useradm /usr/bin/useradm create-user [--username=abhi@host.com](mailto:--username=abhi@host.com) --password=Walia@12345**  
**ERROR** : Container  
793f9486428963c5cf63e03de5e0c7f93946e82c1cc5fcdc4d4033338407cd82 is restarting, wait until the container is running

**mender/mender-server/production$ docker ps**  
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES  
a784ea416ef4 mendersoftware/deployments:mender-3.4.0 “/entrypoint.sh --co…” 4 minutes ago Up 3 minutes 8080/tcp menderproduction\_mender-deployments\_1  
efdaa23dc773 traefik:v2.8 “/entrypoint.sh --ac…” 4 minutes ago Up 3 minutes 80/tcp, 0.0.0.0:443-\>443/tcp, :::443-\>443/tcp menderproduction\_mender-api-gateway\_1  
**f581cce00f8d mendersoftware/deviceauth:mender-3.4.0 “/usr/bin/deviceauth…” 5 minutes ago Restarting (4) 33 seconds ago** menderproduction\_mender-device-auth\_1  
7db75f147bef mendersoftware/deviceconnect:mender-3.4.0 “/usr/bin/deviceconn…” 5 minutes ago Up 4 minutes 8080/tcp menderproduction\_mender-deviceconnect\_1  
17dff12591d0 mendersoftware/workflows:mender-3.4.0 “/usr/bin/workflows …” 5 minutes ago Up 4 minutes 8080/tcp menderproduction\_mender-workflows-server\_1  
4678073f2ac3 mendersoftware/workflows-worker:mender-3.4.0 “/usr/bin/workflows …” 5 minutes ago Up 3 minutes 8080/tcp menderproduction\_mender-workflows-worker\_1  
5617079a8f1a mendersoftware/create-artifact-worker:mender-3.4.0 “/usr/bin/workflows …” 5 minutes ago Up 3 minutes menderproduction\_mender-create-artifact-worker\_1  
**793f94864289 mendersoftware/useradm:mender-3.4.0 “/usr/bin/useradm --…” 5 minutes ago Restarting (4) 33 seconds ago** menderproduction\_mender-useradm\_1  
de5c8c75775d mendersoftware/deviceconfig:mender-3.4.0 “/usr/bin/deviceconf…” 5 minutes ago Up 5 minutes 8080/tcp menderproduction\_mender-deviceconfig\_1  
e9b6742fafc4 mendersoftware/inventory:mender-3.4.0 “/usr/bin/inventory …” 5 minutes ago Up 5 minutes 8080/tcp menderproduction\_mender-inventory\_1  
2c7d768315ea mendersoftware/iot-manager:mender-3.4.0 “/usr/bin/iot-manage…” 5 minutes ago Up 4 minutes 8080/tcp menderproduction\_mender-iot-manager\_1  
0aa25dd9daa1 minio/minio:RELEASE.2019-04-23T23-50-36Z “/usr/bin/docker-ent…” 5 minutes ago Up 5 minutes (healthy) 9000/tcp menderproduction\_minio\_1  
e977cc26eb15 mendersoftware/gui:mender-3.4.0 “/entrypoint.sh nginx” 5 minutes ago Up 5 minutes (healthy) 80/tcp, 8080/tcp menderproduction\_mender-gui\_1  
f9ff5007dec1 mongo:4.4 “docker-entrypoint.s…” 5 minutes ago Up 5 minutes 27017/tcp menderproduction\_mender-mongo\_1  
e73b8f188010 nats:2.6-alpine “docker-entrypoint.s…” 5 minutes ago Up 5 minutes 4222/tcp, 6222/tcp, 8222/tcp menderproduction\_mender-nats\_1

why some of the services are showing restarting after executing the above command to create username and password.

---

<div class="post-metadata">

**Author:** ![rock63](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/rock63/32/1920_2.png) [@rock63](https://hub.mender.io/u/rock63)\
**Post date:** [June 28, 2023, 7:29am UTC](https://hub.mender.io/t/mender-server-production-installation-failed-with-couldnt-load-deployments-error-bad-gateway-retrying-in-26-seconds/5940/5 "2023-06-28T07:29:01Z")

</div>

I fixed the issue by doing following changes in the integration repo sources.

**mender/mender-server$** sudo chmod +777 keys-generated/keys/useradm/private.key  
**mender/mender-server$** sudo chmod +777 keys-generated/keys/deviceauth/private.key

then again run the up command:  
**mender/mender-server/production$**./run up -d

create the username and password:  
**mender/mender-server/production$**./run exec mender-useradm /usr/bin/useradm create-user [--username=myuser@host.com](mailto:--username=myuser@host.com) --password=pass@12345

Now just login with your given url and enter username and password ,it is working 😊

Regards,  
rock63

---

<div class="post-metadata">

**Author:** ![rock63](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/rock63/32/1920_2.png) [@rock63](https://hub.mender.io/u/rock63)\
**Post date:** [June 28, 2023, 8:41am UTC](https://hub.mender.io/t/mender-server-production-installation-failed-with-couldnt-load-deployments-error-bad-gateway-retrying-in-26-seconds/5940/6 "2023-06-28T08:41:34Z")

</div>

Hi @robgio ,

I login mender server successfully but mender client is not able to connect with the mender server ,there is no token present in open source mender server so how to connect.can you please help me.

my server:

 ![Screenshot from 2023-06-28 14-06-07](https://canada1.discourse-cdn.com/flex036/uploads/mender/original/2X/4/44b9ff41b23369683fc8a2f0e15ad64c76d1f736.png)

---

<div class="post-metadata">

**Author:** ![robgio](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/robgio/32/1760_2.png) [@robgio](https://hub.mender.io/u/robgio)\
**Post date:** [July 1, 2023, 1:05pm UTC](https://hub.mender.io/t/mender-server-production-installation-failed-with-couldnt-load-deployments-error-bad-gateway-retrying-in-26-seconds/5940/7 "2023-07-01T13:05:26Z")

</div>

Hi @rock63 ,

> I fixed the issue by doing following changes in the integration repo sources.

Right, there’s a known issue ongoing. We’re working on that but you can use your workaround.

> there is no token present in open source mender server

That’s right, it the Organization Token is present on the multitenant setup, which is an Enterprise only feature.  
To connect a Mender device you can follow the tutorial on the UI, or the [Documentation](https://docs.mender.io/). For example if you want to install Mender on an existing Debian device you can follow this documentation: [Downloads | Mender documentation](https://docs.mender.io/3.4/downloads#mender-client)

---

<div class="post-metadata">

**Author:** ![rock63](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/rock63/32/1920_2.png) [@rock63](https://hub.mender.io/u/rock63)\
**Post date:** [July 10, 2023, 3:47am UTC](https://hub.mender.io/t/mender-server-production-installation-failed-with-couldnt-load-deployments-error-bad-gateway-retrying-in-26-seconds/5940/8 "2023-07-10T03:47:51Z")

</div>

Hi @robgio ,

The mender installation has been done by following this doc [Downloads | Mender documentation](https://docs.mender.io/3.4/downloads#mender-client) but when coming to mender client should connect with mender server which is not happening, Hosted mender server is connecting if we configure the mender client as follows:

**/etc/mender/mender.conf**  
“HttpsClient”: {},  
“Security”: {},  
“Connectivity”: {},  
“DeviceTypeFile”: “/var/lib/mender/device\_type”,  
“DBus”: {  
“Enabled”: true  
},  
“UpdateControlMapExpirationTimeSeconds”: 90,  
“UpdateControlMapBootExpirationTimeSeconds”: 45,  
“UpdatePollIntervalSeconds”: 5,  
“InventoryPollIntervalSeconds”: 5,  
“RetryPollIntervalSeconds”: 30,  
“TenantToken”: “eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJtZW5kZXIudGVuYW50IjoiNjQxNDNhYzZiNDY1ODI5ZmE3Y2IzMjlkIiwiaXNzIjoiTWVuZGVyIiwic3ViIjoiNjQxNDNhYzZiNDY1ODI5ZmE3Y2IzMjlkIn0.MaUqVy6tu6N6pB\_Ws1fUutRgOXDxH4whfvENOI9Lz4aj-76oCtwcVyawq5lBHO\_M-5D7N28hpEa5NS4SUfwQAKYtzUuoioUQF2q6qLsJvNbp7MaRzzeXg4FrJVelMTGFrak8eAuD3lOmJKmsShbG17XL90Zr-lh\_gyNKdKmK9Qj42D3DV1jOQaM6TNcHbqoIO4AeXoI7bOz2HRQw5QpELlJfvFwarkO1jNMGvJvtlr50\_UnnGSFBlbHYNCM5mmDUZ09Fj\_fcIZ-L9AwbbeSn9855PSHlv0QErQC6JVg0uQIYwrAdif4XahzThOk5hrl1EyCNpRrzBIIO\_Oe9VbXnYnt8CCPqIvgCJAN0OBR9P0z3guefUZhkAJTmo\_9qPPhZ-oPtBkpSiQrG8JPKVd5lqZTOQ5ubY7g4jGFLyCaAx0b0gtfpEFpA-dmfqNMUrr53ecyro57bCGIemwO\_OL\_SpNdiGbmIXlQDVsuDOwdIhms2guegSUpVo33IpJNEG8Mb”,  
“Servers”: [  
{  
“ServerURL”: “[https://hosted.mender.io](https://hosted.mender.io)”  
}  
]

But when I set self hosted server configurations then it is not showing any pending device on my server, here is my configurations .

**/etc/mender/mender.conf**  
“HttpsClient”: {},  
“Security”: {},  
“Connectivity”: {},  
“DeviceTypeFile”: “/var/lib/mender/device\_type”,  
“DBus”: {  
“Enabled”: true  
},  
“UpdateControlMapExpirationTimeSeconds”: 90,  
“UpdateControlMapBootExpirationTimeSeconds”: 45,  
“UpdatePollIntervalSeconds”: 5,  
“InventoryPollIntervalSeconds”: 5,  
“RetryPollIntervalSeconds”: 30,  
“ServerCertificate”: “/mender/mender-server/production/keys-generated/cert/cert.crt”,  
“Servers”: [  
{  
“ServerURL”: “[https://myserver-mender](https://myserver-mender)”  
}  
]  
Can you please check what I missed here …

---

<div class="post-metadata">

**Author:** ![robgio](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/robgio/32/1760_2.png) [@robgio](https://hub.mender.io/u/robgio)\
**Post date:** [July 12, 2023, 3:13pm UTC](https://hub.mender.io/t/mender-server-production-installation-failed-with-couldnt-load-deployments-error-bad-gateway-retrying-in-26-seconds/5940/9 "2023-07-12T15:13:18Z")

</div>

Hi @rock63,  
the conf seems good, I have to check. Anyway, the docker compose setup is going to be discontinued, I suggest to try the [Helm Chart on Kubernetes](https://docs.mender.io/3.4/server-installation/production-installation-with-kubernetes) instead.

Please don’t share your Tenant Token or anything sensible in this forum 😉

Roberto