# Production installation - Couldn't load deployments. Error: Bad Gateway Retrying in 7 seconds **URL:** \ **Category:** General Discussions\ **Created:** [July 19, 2019, 7:32am UTC](https://hub.mender.io/t/production-installation-couldnt-load-deployments-error-bad-gateway-retrying-in-7-seconds/836 "2019-07-19T07:32:15Z")\ **Posts on this page:** 16\ **Page:** 1
**Author:** ![mnachev](https://avatars.discourse-cdn.com/v4/letter/m/ce7236/32.png) [@mnachev](https://hub.mender.io/u/mnachev)\ **Post date:** [July 19, 2019, 7:32am UTC](https://hub.mender.io/t/production-installation-couldnt-load-deployments-error-bad-gateway-retrying-in-7-seconds/836/1 "2019-07-19T07:32:15Z")
Hi, Following the instructions from this page [Production installation](https://docs.mender.io/2.0/administration/production-installation), I successfully installed Mender Server for production, except the deployments part “Couldn’t load deployments. Error: Bad Gateway Retrying in 7 seconds” (this is shown on the GUI page). ./run ps ``` Name Command State Ports ``` * * * menderproduction\_mender-api-gateway\_1 /entrypoint.sh Up 0.0.0.0:443-\>443/tcp menderproduction\_mender-conductor\_1 /srv/start\_conductor.sh Up 8080/tcp, 8090/tcp menderproduction\_mender-deployments\_1 /entrypoint.sh --config /e … Restarting menderproduction\_mender-device-auth\_1 /usr/bin/deviceauth --conf … Up 8080/tcp menderproduction\_mender-elasticsearch\_1 /docker-entrypoint.sh elas … Up 9200/tcp, 9300/tcp menderproduction\_mender-gui\_1 /entrypoint.sh Up 80/tcp menderproduction\_mender-inventory\_1 /usr/bin/inventory --confi … Up 8080/tcp menderproduction\_mender-mongo\_1 docker-entrypoint.sh mongod Up 27017/tcp menderproduction\_mender-redis\_1 /redis/entrypoint.sh Up 6379/tcp menderproduction\_mender-useradm\_1 /usr/bin/useradm --config … Up 8080/tcp menderproduction\_minio\_1 /usr/bin/docker-entrypoint … Up (healthy) 9000/tcp menderproduction\_storage-proxy\_1 /usr/local/openresty/bin/o … Up 0.0.0.0:9000-\>9000/tcp Attached is prod.yml file (prod.yml.yaml). All URIs are directed to the IP address of the computer, where Production Mender Server is installed: ping [mender.bg.centrak.com](http://mender.bg.centrak.com) 64 bytes from [s3.bg.centrak.com](http://s3.bg.centrak.com) (192.168.88.50): icmp\_seq=1 ttl=64 time=0.039 ms ping [s3.bg.centrak.com](http://s3.bg.centrak.com) 64 bytes from [s3.bg.centrak.com](http://s3.bg.centrak.com) (192.168.88.50): icmp\_seq=1 ttl=64 time=0.037 ms Where is the problem? What is wrong? ---
**Author:** ![drewmoseley](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/drewmoseley/32/47_2.png) [@drewmoseley](https://hub.mender.io/u/drewmoseley)\ **Post date:** [July 19, 2019, 2:26pm UTC](https://hub.mender.io/t/production-installation-couldnt-load-deployments-error-bad-gateway-retrying-in-7-seconds/836/2 "2019-07-19T14:26:21Z")
This looks to be the culprit: > menderproduction\_mender-deployments\_1 /entrypoint.sh --config /e … Restarting My guess is that you have an older version of the docker container running or something. Is this a new installation? Can you “docker stop” and “docker rm” all existing containers and rerun? ---
**Author:** ![mnachev](https://avatars.discourse-cdn.com/v4/letter/m/ce7236/32.png) [@mnachev](https://hub.mender.io/u/mnachev)\ **Post date:** [July 22, 2019, 4:54am UTC](https://hub.mender.io/t/production-installation-couldnt-load-deployments-error-bad-gateway-retrying-in-7-seconds/836/3 "2019-07-22T04:54:41Z")
Yes, after stopping and removing all docker containers, the services started up. The installation is new, but before few days, I had made a Mender demo server installation, that should have been stopped and not working, after stopping and restarting of the computer. ---
**Author:** ![mnachev](https://avatars.discourse-cdn.com/v4/letter/m/ce7236/32.png) [@mnachev](https://hub.mender.io/u/mnachev)\ **Post date:** [July 22, 2019, 4:57am UTC](https://hub.mender.io/t/production-installation-couldnt-load-deployments-error-bad-gateway-retrying-in-7-seconds/836/4 "2019-07-22T04:57:54Z")
Now I have the same message again: “Bad Gateway Retrying in 7 Seconds” ![image](https://canada1.discourse-cdn.com/flex036/uploads/mender/original/1X/4df409447ffd01dcc9a841e2de27e232380d6077.png) ---
**Author:** ![mnachev](https://avatars.discourse-cdn.com/v4/letter/m/ce7236/32.png) [@mnachev](https://hub.mender.io/u/mnachev)\ **Post date:** [July 22, 2019, 5:01am UTC](https://hub.mender.io/t/production-installation-couldnt-load-deployments-error-bad-gateway-retrying-in-7-seconds/836/5 "2019-07-22T05:01:43Z")
I checked the processes again with “./run ps” and “menderproduction\_mender-deployments\_1 /entrypoint.sh --config /e …” is again with status “Restarting”. ---
**Author:** ![mnachev](https://avatars.discourse-cdn.com/v4/letter/m/ce7236/32.png) [@mnachev](https://hub.mender.io/u/mnachev)\ **Post date:** [July 22, 2019, 5:08am UTC](https://hub.mender.io/t/production-installation-couldnt-load-deployments-error-bad-gateway-retrying-in-7-seconds/836/6 "2019-07-22T05:08:17Z")
After stopping ("./run stop"), the following processes are active (“docker ps -a”): CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES - 2bc49fc2668e mendersoftware/api-gateway:2.0.0 “/entrypoint.sh” 18 minutes ago Exited (0) 2 minutes ago menderproduction\_mender-api-gateway\_1 - a8da6d5b14f2 mendersoftware/deviceauth:2.0.0 “/usr/bin/deviceauth…” 18 minutes ago Exited (2) 2 minutes ago menderproduction\_mender-device-auth\_1 - 497df1aa71da mendersoftware/inventory:1.6.0 “/usr/bin/inventory …” 18 minutes ago Exited (2) 2 minutes ago menderproduction\_mender-inventory\_1 - 21aeb0dcd8d4 mendersoftware/deployments:1.7.0 “/entrypoint.sh --co…” 18 minutes ago Exited (4) 2 minutes ago menderproduction\_mender-deployments\_1 - 8bfb2d96e873 mendersoftware/useradm:1.7.0 “/usr/bin/useradm --…” 18 minutes ago Exited (2) 2 minutes ago menderproduction\_mender-useradm\_1 - a862f07ec838 mendersoftware/mender-conductor:1.3.1 “/srv/start\_conducto…” 18 minutes ago Exited (137) About a minute ago menderproduction\_mender-conductor\_1 - 23ceec2272aa openresty/openresty:1.13.6.2-0-alpine “/usr/local/openrest…” 18 minutes ago Exited (0) 2 minutes ago menderproduction\_storage-proxy\_1 - 5ee3ae5987cd redis:5-alpine3.8 “/redis/entrypoint.sh” 18 minutes ago Exited (137) About a minute ago menderproduction\_mender-redis\_1 - c9d5b0fb88b8 elasticsearch:5-alpine “/docker-entrypoint.…” 18 minutes ago Exited (143) About a minute ago menderproduction\_mender-elasticsearch\_1 - ee4dd6aeb0ff mongo:3.4 “docker-entrypoint.s…” 18 minutes ago Exited (0) About a minute ago menderproduction\_mender-mongo\_1 - f144c05f2bf0 mendersoftware/gui:2.0.1 “/entrypoint.sh” 18 minutes ago Exited (137) About a minute ago menderproduction\_mender-gui\_1 - 833e2152641a minio/minio:RELEASE.2018-09-25T21-34-43Z “/usr/bin/docker-ent…” 18 minutes ago Exited (0) About a minute ago menderproduction\_minio\_1 ---
**Author:** ![mnachev](https://avatars.discourse-cdn.com/v4/letter/m/ce7236/32.png) [@mnachev](https://hub.mender.io/u/mnachev)\ **Post date:** [July 22, 2019, 5:13am UTC](https://hub.mender.io/t/production-installation-couldnt-load-deployments-error-bad-gateway-retrying-in-7-seconds/836/7 "2019-07-22T05:13:51Z")
Again stopping and removing: docker stop $(docker ps -a -q) docker rm $(docker ps -a -q) docker ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES All processes are removed. ./run up -d - Creating menderproduction\_mender-elasticsearch\_1 … done - Creating menderproduction\_mender-gui\_1 … done - Creating menderproduction\_minio\_1 … done - Creating menderproduction\_mender-redis\_1 … done - Creating menderproduction\_mender-mongo\_1 … done - Creating menderproduction\_mender-deployments\_1 … done - Creating menderproduction\_mender-inventory\_1 … done - Creating menderproduction\_mender-useradm\_1 … done - Creating menderproduction\_mender-conductor\_1 … done - Creating menderproduction\_storage-proxy\_1 … done - Creating menderproduction\_mender-device-auth\_1 … done - Creating menderproduction\_mender-api-gateway\_1 … done ./run ps ``` Name Command State Ports ``` * * * - menderproduction\_mender-api-gateway\_1 /entrypoint.sh Up 0.0.0.0:443-\>443/tcp - menderproduction\_mender-conductor\_1 /srv/start\_conductor.sh Up 8080/tcp, 8090/tcp - menderproduction\_mender-deployments\_1 /entrypoint.sh --config /e … Restarting - menderproduction\_mender-device-auth\_1 /usr/bin/deviceauth --conf … Up 8080/tcp - menderproduction\_mender-elasticsearch\_1 /docker-entrypoint.sh elas … Up 9200/tcp, 9300/tcp - menderproduction\_mender-gui\_1 /entrypoint.sh Up 80/tcp - menderproduction\_mender-inventory\_1 /usr/bin/inventory --confi … Up 8080/tcp - menderproduction\_mender-mongo\_1 docker-entrypoint.sh mongod Up 27017/tcp - menderproduction\_mender-redis\_1 /redis/entrypoint.sh Up 6379/tcp - menderproduction\_mender-useradm\_1 /usr/bin/useradm --config … Up 8080/tcp - menderproduction\_minio\_1 /usr/bin/docker-entrypoint … Up (healthy) 9000/tcp - menderproduction\_storage-proxy\_1 /usr/local/openresty/bin/o … Up 0.0.0.0:9000-\>9000/tcp ---
**Author:** ![0lmi](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/0lmi/32/133_2.png) [@0lmi](https://hub.mender.io/u/0lmi)\ **Post date:** [July 26, 2019, 8:35am UTC](https://hub.mender.io/t/production-installation-couldnt-load-deployments-error-bad-gateway-retrying-in-7-seconds/836/8 "2019-07-26T08:35:27Z")
@mnachev it’s reasonable to check logs from deployments service `docker logs menderproduction_mender-deployments_1` ---
**Author:** ![splaw1k](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/splaw1k/32/409_2.png) [@splaw1k](https://hub.mender.io/u/splaw1k)\ **Post date:** [October 21, 2019, 11:06am UTC](https://hub.mender.io/t/production-installation-couldnt-load-deployments-error-bad-gateway-retrying-in-7-seconds/836/9 "2019-10-21T11:06:56Z")
Hello, My problem seems like the same. Did you managed to solve this issue? ---
**Author:** ![mirzak](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/mirzak/32/2056_2.png) [@mirzak](https://hub.mender.io/u/mirzak)\ **Post date:** [October 21, 2019, 11:20am UTC](https://hub.mender.io/t/production-installation-couldnt-load-deployments-error-bad-gateway-retrying-in-7-seconds/836/10 "2019-10-21T11:20:55Z")
Are these requirements satisfied? > Minimum of 10 GB free disk space and 4 GB RAM available for Mender and its services. Can you share what you are seeing, e.g output of `docker ps` ---
**Author:** ![leonsgithub](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/leonsgithub/32/442_2.png) [@leonsgithub](https://hub.mender.io/u/leonsgithub)\ **Post date:** [December 2, 2019, 10:09am UTC](https://hub.mender.io/t/production-installation-couldnt-load-deployments-error-bad-gateway-retrying-in-7-seconds/836/12 "2019-12-02T10:09:06Z")
I got the same restart issues after trying the OS installation as outlined here: [https://docs.mender.io/2.2/administration/production-installation](https://docs.mender.io/2.2/administration/production-installation) root@ec2-lala:/usr/local/mender-server/production# ./run logs --tail 10 mender-deployments Attaching to menderproduction\_mender-deployments\_1 mender-deployments\_1 | RequestError: send request failed mender-deployments\_1 | caused by: Put [https://ec2-lala.eu-west-1.compute.amazonaws.com:9000/mender-artifact-storage:](https://ec2-lala.eu-west-1.compute.amazonaws.com:9000/mender-artifact-storage:) x509: certificate is not valid for any names, but wanted to match [ec2-lala.eu-west-1.compute.amazonaws.com](http://ec2-lala.eu-west-1.compute.amazonaws.com) mender-deployments\_1 | WARNING: ca-certificates.crt does not contain exactly one certificate or CRL: skipping mender-deployments\_1 | time=“2019-12-02T09:26:38Z” level=info msg=“Deployments Service, version unknown starting up” mender-deployments\_1 | time=“2019-12-02T09:26:38Z” level=info msg=“automigrate is ON, will apply migrations” file=migrations.go func=mongo.Migrate line=48 mender-deployments\_1 | time=“2019-12-02T09:26:38Z” level=info msg=“migrating deployment\_service” file=migrations.go func=mongo.MigrateSingle line=70 mender-deployments\_1 | time=“2019-12-02T09:26:38Z” level=info msg=“migration to version 1.2.1 skipped” db=deployment\_service file=migrator\_simple.go func=“migrate.(\*SimpleMigrator).Apply” line=125 mender-deployments\_1 | time=“2019-12-02T09:26:38Z” level=info msg=“DB migrated to version 1.2.1” db=deployment\_service file=migrator\_simple.go func=“migrate.(\*SimpleMigrator).Apply” line=140 mender-deployments\_1 | RequestError: send request failed mender-deployments\_1 | caused by: Put [https://ec2-lala.eu-west-1.compute.amazonaws.com:9000/mender-artifact-storage:](https://ec2-lala.eu-west-1.compute.amazonaws.com:9000/mender-artifact-storage:) x509: certificate is not valid for any names, but wanted to match [ec2-lala.eu-west-1.compute.amazonaws.com](http://ec2-lala.eu-west-1.compute.amazonaws.com) It seems i have ended in certificate hell my favorite pastime ---
**Author:** ![0lmi](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/0lmi/32/133_2.png) [@0lmi](https://hub.mender.io/u/0lmi)\ **Post date:** [February 18, 2020, 10:33pm UTC](https://hub.mender.io/t/production-installation-couldnt-load-deployments-error-bad-gateway-retrying-in-7-seconds/836/13 "2020-02-18T22:33:45Z")
Looks like the issue with storage proxy SSL certificate. ---
**Author:** ![PJK](https://avatars.discourse-cdn.com/v4/letter/p/bc79bd/32.png) [@PJK](https://hub.mender.io/u/PJK)\ **Post date:** [May 12, 2020, 2:43pm UTC](https://hub.mender.io/t/production-installation-couldnt-load-deployments-error-bad-gateway-retrying-in-7-seconds/836/14 "2020-05-12T14:43:13Z")
Has anybody managed to find a solution for this issue? Because I’m also experiencing it at the moment. ---
**Author:** ![oernst](https://avatars.discourse-cdn.com/v4/letter/o/65b543/32.png) [@oernst](https://hub.mender.io/u/oernst)\ **Post date:** [August 30, 2020, 12:09am UTC](https://hub.mender.io/t/production-installation-couldnt-load-deployments-error-bad-gateway-retrying-in-7-seconds/836/15 "2020-08-30T00:09:29Z")
+1 on the same issue list - solutions? ---
**Author:** ![tranchitella](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/tranchitella/32/606_2.png) [@tranchitella](https://hub.mender.io/u/tranchitella)\ **Post date:** [August 30, 2020, 4:20am UTC](https://hub.mender.io/t/production-installation-couldnt-load-deployments-error-bad-gateway-retrying-in-7-seconds/836/16 "2020-08-30T04:20:06Z")
@oernst can you please paste here the output of docker-compos ps and check the logs from your containers? ---
**Author:** ![funnybretzel](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/funnybretzel/32/765_2.png) [@funnybretzel](https://hub.mender.io/u/funnybretzel)\ **Post date:** [September 2, 2020, 5:43pm UTC](https://hub.mender.io/t/production-installation-couldnt-load-deployments-error-bad-gateway-retrying-in-7-seconds/836/17 "2020-09-02T17:43:15Z")
Hi, I had this issue as well but fixed it by updating the conf/prod.yml by hand. The thing is, some of the sed commands don’t seem to work quite as well and i ended up with DEPLOYMENTS\_AWS\_URI: [https://set-my-alias-here.com](https://set-my-alias-here.com) instead of DEPLOYMENTS\_AWS\_URI: [https://set-my-alias-here.com:9000](https://set-my-alias-here.com:9000). So check that by hand 🙂 Cheerio