# Self-hosted 3.0.0 fails to connect to mongo, demo runs fine
**URL:** https://hub.mender.io/t/self-hosted-3-0-0-fails-to-connect-to-mongo-demo-runs-fine/3921
**Category:** General Discussions
**Tags:** mender-server
**Created:** [July 23, 2021, 9:25pm UTC](https://hub.mender.io/t/self-hosted-3-0-0-fails-to-connect-to-mongo-demo-runs-fine/3921 "2021-07-23T21:25:57Z")
**Posts on this page:** 8
**Page:** 1
### Author:  [@Kris](https://hub.mender.io/u/Kris)
#### Post date: [July 23, 2021, 9:25pm UTC](https://hub.mender.io/t/self-hosted-3-0-0-fails-to-connect-to-mongo-demo-runs-fine/3921/1 "2021-07-23T21:25:57Z")
We’re spinning up a feature bump for a customer’s setup and completely stuck trying to get any recent version of Mender running.
We have a test 2.2.1 server that still runs, but minio complains that the version of Mender is too old.
If I clone and follow the demo instructions for 3.0.0, the demo and all docker containers run without issue.
But following the instructions for self-hosted, verbatim, ends up with a handful of containers complaining they cannot reach mongodb, with mongo itself showing no errors. When in this state, I’m not able to log in with the created user. However, it appears the user create command from commandline is failing. See below for some logs which I assume are relevant. If there is any other detail I can provide, let me know.
OS info:
```auto
Linux mender 4.15.0-151-generic #157-Ubuntu SMP Fri Jul 9 23:07:57 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Ubuntu 18.04
```
Create user command return
```auto
kris@mender:~/mender-server/production$ ./run exec mender-useradm /usr/bin/useradm create-user --username=myusername@host.com --password=mysecretpassword
kris@mender:~/mender-server/production$ echo $?
137
```
docker ps
```auto
kris@mender:~/mender-server/production$ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
12d7f5b9ce3a mendersoftware/deployments:mender-3.0.0 "/entrypoint.sh --co…" 13 minutes ago Up 2 seconds 8080/tcp menderproduction_mender-deployments_1
9544e0fca9f5 traefik:v2.4 "/entrypoint.sh --ac…" 13 minutes ago Up 13 minutes 80/tcp, 0.0.0.0:443->443/tcp, :::443->443/tcp menderproduction_mender-api-gateway_1
8972d6f36a2b mendersoftware/deviceauth:mender-3.0.0 "/usr/bin/deviceauth…" 13 minutes ago Up Less than a second 8080/tcp menderproduction_mender-device-auth_1
f63d7218db97 mendersoftware/deviceconnect:mender-3.0.0 "/usr/bin/deviceconn…" 13 minutes ago Up 1 second 8080/tcp menderproduction_mender-deviceconnect_1
58c52b1a3fa9 mendersoftware/deviceconfig:mender-3.0.0 "/usr/bin/deviceconf…" 13 minutes ago Up 19 seconds 8080/tcp menderproduction_mender-deviceconfig_1
9d4856e11cbb mendersoftware/create-artifact-worker:mender-3.0.0 "/usr/bin/workflows …" 13 minutes ago Up 6 seconds 8080/tcp menderproduction_mender-create-artifact-worker_1
7e659881fe9c mendersoftware/inventory:mender-3.0.0 "/usr/bin/inventory …" 13 minutes ago Up 19 seconds 8080/tcp menderproduction_mender-inventory_1
c76b16cbc5a2 mendersoftware/useradm:mender-3.0.0 "/usr/bin/useradm --…" 13 minutes ago Up 5 seconds 8080/tcp menderproduction_mender-useradm_1
a6afa115cb40 mendersoftware/workflows:mender-3.0.0 "/usr/bin/workflows …" 14 minutes ago Up 2 seconds 8080/tcp menderproduction_mender-workflows-server_1
6bfc5b8df0f7 mendersoftware/workflows-worker:mender-3.0.0 "/usr/bin/workflows …" 14 minutes ago Up 6 seconds menderproduction_mender-workflows-worker_1
05f8916ae244 nats:2.1.9-alpine3.12 "docker-entrypoint.s…" 14 minutes ago Up 13 minutes 4222/tcp, 6222/tcp, 8222/tcp menderproduction_mender-nats_1
2002ace8325a mendersoftware/gui:mender-3.0.0 "/entrypoint.sh nginx" 14 minutes ago Up 14 minutes (healthy) 80/tcp, 8080/tcp menderproduction_mender-gui_1
e27b5199a6da minio/minio:RELEASE.2019-04-23T23-50-36Z "/usr/bin/docker-ent…" 14 minutes ago Up 13 minutes (healthy) 9000/tcp menderproduction_minio_1
bb1e2f8d642f mongo:4.4 "docker-entrypoint.s…" 14 minutes ago Restarting (14) 33 seconds ago menderproduction_mender-mongo_1
```
mongodb log
```auto
kris@mender:~/mender-server/production$ ./run logs mender-mongo
mender-mongo_1 | {"t":{"$date":"2021-07-23T21:19:05.151+00:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"initandlisten","msg":" Frame: {frame}","attr":{"frame":{"a":"55BDA513572E","b":"55BDA4164000","o":"FD172E",
"s":"_start","s+":"2E"}}}
mender-mongo_1 | {"t":{"$date":"2021-07-23T21:20:05.704+00:00"},"s":"I", "c":"CONTROL", "id":23285, "ctx":"main","msg":"Automatically disabling TLS 1.0, to force-enable TLS 1.0 specify --sslDisabledProtocols 'none'"
}
mender-mongo_1 | {"t":{"$date":"2021-07-23T21:20:05.706+00:00"},"s":"W", "c":"ASIO", "id":22601, "ctx":"main","msg":"No TransportLayer configured during NetworkInterface startup"}
mender-mongo_1 | {"t":{"$date":"2021-07-23T21:20:05.706+00:00"},"s":"I", "c":"NETWORK", "id":4648601, "ctx":"main","msg":"Implicit TCP FastOpen unavailable. If TCP FastOpen is required, set tcpFastOpenServer, tcpFastO
penClient, and tcpFastOpenQueueSize."}
mender-mongo_1 | {"t":{"$date":"2021-07-23T21:20:05.707+00:00"},"s":"I", "c":"STORAGE", "id":4615611, "ctx":"initandlisten","msg":"MongoDB starting","attr":{"pid":1,"port":27017,"dbPath":"/data/db","architecture":"64$
bit","host":"bb1e2f8d642f"}}
mender-mongo_1 | {"t":{"$date":"2021-07-23T21:20:05.707+00:00"},"s":"I", "c":"CONTROL", "id":23403, "ctx":"initandlisten","msg":"Build Info","attr":{"buildInfo":{"version":"4.4.7","gitVersion":"abb6b9c2bf675e9e2aea$
cba05f0f8359d99e203","openSSLVersion":"OpenSSL 1.1.1f 31 Mar 2020","modules":[],"allocator":"tcmalloc","environment":{"distmod":"ubuntu2004","distarch":"x86_64","target_arch":"x86_64"}}}}
mender-mongo_1 | {"t":{"$date":"2021-07-23T21:20:05.707+00:00"},"s":"I", "c":"CONTROL", "id":51765, "ctx":"initandlisten","msg":"Operating System","attr":{"os":{"name":"Ubuntu","version":"20.04"}}}
mender-mongo_1 | {"t":{"$date":"2021-07-23T21:20:05.707+00:00"},"s":"I", "c":"CONTROL", "id":21951, "ctx":"initandlisten","msg":"Options set by command line","attr":{"options":{"net":{"bindIp":"*"}}}}
mender-mongo_1 | {"t":{"$date":"2021-07-23T21:20:05.709+00:00"},"s":"I", "c":"STORAGE", "id":22270, "ctx":"initandlisten","msg":"Storage engine to use detected by data files","attr":{"dbpath":"/data/db","storageEng$
ne":"wiredTiger"}}
mender-mongo_1 | {"t":{"$date":"2021-07-23T21:20:05.710+00:00"},"s":"I", "c":"STORAGE", "id":22297, "ctx":"initandlisten","msg":"Using the XFS filesystem is strongly recommended with the WiredTiger storage engine. S
ee http://dochub.mongodb.org/core/prodnotes-filesystem","tags":["startupWarnings"]}
mender-mongo_1 | {"t":{"$date":"2021-07-23T21:20:05.710+00:00"},"s":"I", "c":"STORAGE", "id":22315, "ctx":"initandlisten","msg":"Opening WiredTiger","attr":{"config":"create,cache_size=1460M,session_max=33000,evicti
on=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000,close_scan_interval=10,close_handle_minimum=250),statistics_log=(wai
t=0),verbose=[recovery_progress,checkpoint_progress,compact_progress],"}}
mender-mongo_1 | {"t":{"$date":"2021-07-23T21:20:07.120+00:00"},"s":"F", "c":"STORAGE", "id":4671205, "ctx":"initandlisten","msg":"This version of MongoDB is too recent to start up on the existing data files. Try Mong
oDB 4.2 or earlier."}
mender-mongo_1 | {"t":{"$date":"2021-07-23T21:20:07.122+00:00"},"s":"F", "c":"-", "id":23089, "ctx":"initandlisten","msg":"Fatal assertion","attr":{"msgid":4671205,"file":"src/mongo/db/storage/wiredtiger/wired
tiger_kv_engine.cpp","line":919}}
mender-mongo_1 | {"t":{"$date":"2021-07-23T21:20:07.123+00:00"},"s":"F", "c":"-", "id":23090, "ctx":"initandlisten","msg":"\n\n***aborting after fassert() failure\n\n"}
mender-mongo_1 | {"t":{"$date":"2021-07-23T21:20:07.123+00:00"},"s":"F", "c":"CONTROL", "id":4757800, "ctx":"initandlisten","msg":"Writing fatal message","attr":{"message":"Got signal: 6 (Aborted).\n"}}
mender-mongo_1 | {"t":{"$date":"2021-07-23T21:20:07.237+00:00"},"s":"I", "c":"CONTROL", "id":31431, "ctx":"initandlisten","msg":"BACKTRACE: {bt}","attr":{"bt":{"backtrace":[{"a":"55F58BAED69A","b":"55F588D44000","o"
:"2DA969A","s":"_ZN5mongo18stack_trace_detail12_GLOBAL__N_119printStackTraceImplERKNS1_7OptionsEPNS_14StackTraceSinkE.constprop.606","s+":"1EA"},{"a":"55F58BAEF129","b":"55F588D44000","o":"2DAB129","s":"_ZN5mongo15printStackTraceEv","s+"
:"29"},{"a":"55F58BAEC4B6","b":"55F588D44000","o":"2DA84B6","s":"_ZN5mongo12_GLOBAL__N_116abruptQuitActionEiP9siginfo_tPv","s+":"66"},{"a":"7F0959F793C0","b":"7F0959F64000","o":"153C0","s":"funlockfile","s+":"60"},{"a":"7F0959DB818B","b"
:"7F0959D72000","o":"4618B","s":"gsignal","s+":"CB"},{"a":"7F0959D97859","b":"7F0959D72000","o":"25859","s":"abort","s+":"12B"},{"a":"55F589C7D788","b":"55F588D44000","o":"F39788","s":"_ZN5mongo25fassertFailedWithLocationEiPKcj","s+":"12
B"},{"a":"55F5899513D7","b":"55F588D44000","o":"C0D3D7","s":"_ZN5mongo18WiredTigerKVEngine15_openWiredTigerERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES8_.cold.1865","s+":"1C7"},{"a":"55F589D8C642","b":"55F588D44000","o":"1048
642","s":"_ZN5mongo18WiredTigerKVEngineC2ERKNSt7 __cxx1112basic_stringIcSt11char_traitsIcESaIcEEES8_PNS_11ClockSourceES8_mmbbbb","s+":"1362"},{"a":"55F589D6A3E0","b":"55F588D44000","o":"10263E0","s":"_ZNK5mongo12_GLOBAL__ N_117WiredTigerFa
ctory6createERKNS_19StorageGlobalParamsEPKNS_21StorageEngineLockFileE","s+":"120"},{"a":"55F58A66ECE7","b":"55F588D44000","o":"192ACE7","s":"_ZN5mongo23initializeStorageEngineEPNS_14ServiceContextENS_22StorageEngineInitFlagsE","s+":"4F7"
},{"a":"55F589D162F7","b":"55F588D44000","o":"FD22F7","s":"_ZN5mongo12_GLOBAL __N_114_initAndListenEPNS_14ServiceContextEi.isra.1584","s+":"3F7"},{"a":"55F589D19F8B","b":"55F588D44000","o":"FD5F8B","s":"_ZN5mongo12_GLOBAL__ N_111mongoDbMai
nEiPPcS2_","s+":"104B"},{"a":"55F589C8CE99","b":"55F588D44000","o":"F48E99","s":"main","s+":"9"},{"a":"7F0959D990B3","b":"7F0959D72000","o":"270B3","s":"__libc_start_main","s+":"F3"},{"a":"55F589D1572E","b":"55F588D44000","o":"FD172E","s
":"_start","s+":"2E"}],"processInfo":{"mongodbVersion":"4.4.7","gitVersion":"abb6b9c2bf675e9e2aeaecba05f0f8359d99e203","compiledModules":[],"uname":{"sysname":"Linux","release":"4.15.0-151-generic","version":"#157-Ubuntu SMP Fri Jul 9 23
:07:57 UTC 2021","machine":"x86_64"},"somap":[{"b":"55F588D44000","elfType":3,"buildId":"4F8B26A04CD4B51B127724FF7CAE2E74E9FFAF32"},{"b":"7F0959F64000","path":"/lib/x86_64-linux-gnu/libpthread.so.0","elfType":3,"buildId":"E54761F7B554D0F
CC1562959665D93DFFBEBDAF0"},{"b":"7F0959D72000","path":"/lib/x86_64-linux-gnu/libc.so.6","elfType":3,"buildId":"099B9225BCB0D019D9D60884BE583EB31BB5F44E"}]}}}}
mender-mongo_1 | {"t":{"$date":"2021-07-23T21:20:07.240+00:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"initandlisten","msg":" Frame: {frame}","attr":{"frame":{"a":"55F58BAED69A","b":"55F588D44000","o":"2DA969A"
,"s":"_ZN5mongo18stack_trace_detail12_GLOBAL__N_119printStackTraceImplERKNS1_7OptionsEPNS_14StackTraceSinkE.constprop.606","s+":"1EA"}}}
mender-mongo_1 | {"t":{"$date":"2021-07-23T21:20:07.240+00:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"initandlisten","msg":" Frame: {frame}","attr":{"frame":{"a":"55F58BAEF129","b":"55F588D44000","o":"2DAB129"
,"s":"_ZN5mongo15printStackTraceEv","s+":"29"}}}
mender-mongo_1 | {"t":{"$date":"2021-07-23T21:20:07.240+00:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"initandlisten","msg":" Frame: {frame}","attr":{"frame":{"a":"55F58BAEC4B6","b":"55F588D44000","o":"2DA84B6"
,"s":"_ZN5mongo12_GLOBAL__N_116abruptQuitActionEiP9siginfo_tPv","s+":"66"}}}
mender-mongo_1 | {"t":{"$date":"2021-07-23T21:20:07.240+00:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"initandlisten","msg":" Frame: {frame}","attr":{"frame":{"a":"7F0959F793C0","b":"7F0959F64000","o":"153C0","
s":"funlockfile","s+":"60"}}}
mender-mongo_1 | {"t":{"$date":"2021-07-23T21:20:07.240+00:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"initandlisten","msg":" Frame: {frame}","attr":{"frame":{"a":"7F0959DB818B","b":"7F0959D72000","o":"4618B","
s":"gsignal","s+":"CB"}}}
mender-mongo_1 | {"t":{"$date":"2021-07-23T21:20:07.240+00:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"initandlisten","msg":" Frame: {frame}","attr":{"frame":{"a":"7F0959D97859","b":"7F0959D72000","o":"25859","
s":"abort","s+":"12B"}}}
mender-mongo_1 | {"t":{"$date":"2021-07-23T21:20:07.240+00:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"initandlisten","msg":" Frame: {frame}","attr":{"frame":{"a":"55F589C7D788","b":"55F588D44000","o":"F39788",
"s":"_ZN5mongo25fassertFailedWithLocationEiPKcj","s+":"12B"}}}
mender-mongo_1 | {"t":{"$date":"2021-07-23T21:20:07.240+00:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"initandlisten","msg":" Frame: {frame}","attr":{"frame":{"a":"55F5899513D7","b":"55F588D44000","o":"C0D3D7",
"s":"_ZN5mongo18WiredTigerKVEngine15_openWiredTigerERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES8_.cold.1865","s+":"1C7"}}}
mender-mongo_1 | {"t":{"$date":"2021-07-23T21:20:07.240+00:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"initandlisten","msg":" Frame: {frame}","attr":{"frame":{"a":"55F589D8C642","b":"55F588D44000","o":"1048642"
,"s":"_ZN5mongo18WiredTigerKVEngineC2ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES8_PNS_11ClockSourceES8_mmbbbb","s+":"1362"}}}
mender-mongo_1 | {"t":{"$date":"2021-07-23T21:20:07.240+00:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"initandlisten","msg":" Frame: {frame}","attr":{"frame":{"a":"55F589D6A3E0","b":"55F588D44000","o":"10263E0"
,"s":"_ZNK5mongo12_GLOBAL__N_117WiredTigerFactory6createERKNS_19StorageGlobalParamsEPKNS_21StorageEngineLockFileE","s+":"120"}}}
mender-mongo_1 | {"t":{"$date":"2021-07-23T21:20:07.240+00:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"initandlisten","msg":" Frame: {frame}","attr":{"frame":{"a":"55F58A66ECE7","b":"55F588D44000","o":"192ACE7"
,"s":"_ZN5mongo23initializeStorageEngineEPNS_14ServiceContextENS_22StorageEngineInitFlagsE","s+":"4F7"}}}
mender-mongo_1 | {"t":{"$date":"2021-07-23T21:20:07.240+00:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"initandlisten","msg":" Frame: {frame}","attr":{"frame":{"a":"55F589D162F7","b":"55F588D44000","o":"FD22F7",
"s":"_ZN5mongo12_GLOBAL__N_114_initAndListenEPNS_14ServiceContextEi.isra.1584","s+":"3F7"}}}
mender-mongo_1 | {"t":{"$date":"2021-07-23T21:20:07.240+00:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"initandlisten","msg":" Frame: {frame}","attr":{"frame":{"a":"55F589D19F8B","b":"55F588D44000","o":"FD5F8B",
"s":"_ZN5mongo12_GLOBAL__N_111mongoDbMainEiPPcS2_","s+":"104B"}}}
mender-mongo_1 | {"t":{"$date":"2021-07-23T21:20:07.240+00:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"initandlisten","msg":" Frame: {frame}","attr":{"frame":{"a":"55F589C8CE99","b":"55F588D44000","o":"F48E99",
"s":"main","s+":"9"}}}
mender-mongo_1 | {"t":{"$date":"2021-07-23T21:20:07.240+00:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"initandlisten","msg":" Frame: {frame}","attr":{"frame":{"a":"7F0959D990B3","b":"7F0959D72000","o":"270B3","
s":"__libc_start_main","s+":"F3"}}}
mender-mongo_1 | {"t":{"$date":"2021-07-23T21:20:07.240+00:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"initandlisten","msg":" Frame: {frame}","attr":{"frame":{"a":"55F589D1572E","b":"55F588D44000","o":"FD172E",
"s":"_start","s+":"2E"}}}
```
useradm log
```auto
mender-useradm_1 | time="2021-07-23T21:23:18Z" level=info msg="User Administration Service, version unknown starting up"
mender-useradm_1 | failed to connect to db: server selection error: context deadline exceeded, current topology: { Type: Unknown, Servers: [{ Addr: mongo-useradm:27017, Type: Unknown, Average RTT: 0, Last error: connection() error occured during connection handshake: dial tcp: lookup mongo-useradm on 127.0.0.11:53: server misbehaving },] }
mender-useradm_1 | time="2021-07-23T21:23:28Z" level=info msg="User Administration Service, version unknown starting up"
mender-useradm_1 | failed to connect to db: server selection error: context deadline exceeded, current topology: { Type: Unknown, Servers: [{ Addr: mongo-useradm:27017, Type: Unknown, Average RTT: 0, Last error: connection() error occured during connection handshake: dial tcp: lookup mongo-useradm on 127.0.0.11:53: server misbehaving },] }
mender-useradm_1 | time="2021-07-23T21:23:39Z" level=info msg="User Administration Service, version unknown starting up"
mender-useradm_1 | failed to connect to db: server selection error: context deadline exceeded, current topology: { Type: Unknown, Servers: [{ Addr: mongo-useradm:27017, Type: Unknown, Average RTT: 0, Last error: connection() error occured during connection handshake: dial tcp: lookup mongo-useradm on 127.0.0.11:53: server misbehaving },] }
mender-useradm_1 | time="2021-07-23T21:23:50Z" level=info msg="User Administration Service, version unknown starting up"
mender-useradm_1 | failed to connect to db: server selection error: context deadline exceeded, current topology: { Type: Unknown, Servers: [{ Addr: mongo-useradm:27017, Type: Unknown, Average RTT: 0, Last error: connection() error occured during connection handshake: dial tcp: lookup mongo-useradm on 127.0.0.11:53: server misbehaving },] }
mender-useradm_1 | time="2021-07-23T21:24:01Z" level=info msg="User Administration Service, version unknown starting up"
mender-useradm_1 | failed to connect to db: server selection error: context deadline exceeded, current topology: { Type: Unknown, Servers: [{ Addr: mongo-useradm:27017, Type: Unknown, Average RTT: 0, Last error: connection() error occured during connection handshake: dial tcp: lookup mongo-useradm on 127.0.0.11:53: server misbehaving },] }
mender-useradm_1 | time="2021-07-23T21:24:15Z" level=info msg="User Administration Service, version unknown starting up"
```
deployments log
```auto
mender-deployments_1 | time="2021-07-23T21:24:30Z" level=info msg="'presign.secret' not configured. Generating a random secret." file=main.go func=main.doMain.func1 line=99
mender-deployments_1 | time="2021-07-23T21:24:30Z" level=info msg="Deployments Service, version unknown starting up"
mender-deployments_1 | failed to connect to db: Error reaching mongo server: server selection error: context deadline exceeded, current topology: { Type: Unknown, Servers: [{ Addr: mongo-deployments:27017, Type: Unknown, Average RTT: 0, Last error: connection() error occured during connection handshake: dial tcp: lookup mongo-deployments: Try again },] }
mender-deployments_1 | WARNING: ca-certificates.crt does not contain exactly one certificate or CRL: skipping
mender-deployments_1 | time="2021-07-23T21:24:41Z" level=info msg="'presign.secret' not configured. Generating a random secret." file=main.go func=main.doMain.func1 line=99
mender-deployments_1 | time="2021-07-23T21:24:41Z" level=info msg="Deployments Service, version unknown starting up"
mender-deployments_1 | failed to connect to db: Error reaching mongo server: server selection error: context deadline exceeded, current topology: { Type: Unknown, Servers: [{ Addr: mongo-deployments:27017, Type: Unknown, Average RTT: 0, Last error: connection() error occured during connection handshake: dial tcp: lookup mongo-deployments: Try again },] }
mender-deployments_1 | WARNING: ca-certificates.crt does not contain exactly one certificate or CRL: skipping
mender-deployments_1 | time="2021-07-23T21:24:52Z" level=info msg="'presign.secret' not configured. Generating a random secret." file=main.go func=main.doMain.func1 line=99
mender-deployments_1 | time="2021-07-23T21:24:52Z" level=info msg="Deployments Service, version unknown starting up"
mender-deployments_1 | failed to connect to db: Error reaching mongo server: server selection error: context deadline exceeded, current topology: { Type: Unknown, Servers: [{ Addr: mongo-deployments:27017, Type: Unknown, Average RTT: 0, Last error: connection() error occured during connection handshake: dial tcp: lookup mongo-deployments: Try again },] }
mender-deployments_1 | WARNING: ca-certificates.crt does not contain exactly one certificate or CRL: skipping
mender-deployments_1 | time="2021-07-23T21:25:04Z" level=info msg="'presign.secret' not configured. Generating a random secret." file=main.go func=main.doMain.func1 line=99
mender-deployments_1 | time="2021-07-23T21:25:04Z" level=info msg="Deployments Service, version unknown starting up"
```
If there is any info that is missing or additional details needed, let me know, I’d be happy to provide.
---
### Author:  [@Kris](https://hub.mender.io/u/Kris)
#### Post date: [July 23, 2021, 10:36pm UTC](https://hub.mender.io/t/self-hosted-3-0-0-fails-to-connect-to-mongo-demo-runs-fine/3921/2 "2021-07-23T22:36:30Z")
I’ve also tried downgrading to 2.7, following the 2.7 installation instructions verbatim, from a fresh clone of the repo, and the same failures seem to occur.
---
### Author:  [@tranchitella](https://hub.mender.io/u/tranchitella)
#### Post date: [July 24, 2021, 3:35am UTC](https://hub.mender.io/t/self-hosted-3-0-0-fails-to-connect-to-mongo-demo-runs-fine/3921/3 "2021-07-24T03:35:31Z")
@Kris from the error I see the different microservices cannot resolve the mongo aliases (e.g. mongo-useradm, mongo-deployments). These aliases are defined in docker-compose.yml:
>
Can you please check those alias are there in your configuration, and you can successfully resolve them from the containers running in your docker composition?
---
### Author:  [@Kris](https://hub.mender.io/u/Kris)
#### Post date: [July 26, 2021, 4:44pm UTC](https://hub.mender.io/t/self-hosted-3-0-0-fails-to-connect-to-mongo-demo-runs-fine/3921/4 "2021-07-26T16:44:34Z")
Thanks for the feedback.
`docker-compose.yml` is unmodified and indeed has all of those aliases defined.
I’m not sure if the best way to go about testing resolution from within containers, so if the following is not sufficient, let me know what a better way is:
```auto
kris@mender:~/mender-server/production$ docker exec -it menderproduction_mender-gui_1 sh
/var/www/mender-gui/dist # ping mongo-useradm
ping: bad address 'mongo-useradm'
/var/www/mender-gui/dist # ping mongo-workflows
ping: bad address 'mongo-workflows'
/var/www/mender-gui/dist # ping mender-gui
PING mender-gui (172.21.0.3): 56 data bytes
64 bytes from 172.21.0.3: seq=0 ttl=64 time=0.062 ms
64 bytes from 172.21.0.3: seq=1 ttl=64 time=0.113 ms
^C
--- mender-gui ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 0.062/0.087/0.113 ms
/var/www/mender-gui/dist # ping mender-useradm
PING mender-useradm (172.21.0.7): 56 data bytes
64 bytes from 172.21.0.7: seq=0 ttl=64 time=0.148 ms
64 bytes from 172.21.0.7: seq=1 ttl=64 time=0.102 ms
^C
--- mender-useradm ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 0.102/0.125/0.148 ms
/var/www/mender-gui/dist # ping mongo-deployments
ping: bad address 'mongo-deployments'
```
It would appear none of the mongo aliases are resolvable. Any clues as to where to look at next?
---
### Author:  [@tranchitella](https://hub.mender.io/u/tranchitella)
#### Post date: [July 28, 2021, 7:32am UTC](https://hub.mender.io/t/self-hosted-3-0-0-fails-to-connect-to-mongo-demo-runs-fine/3921/5 "2021-07-28T07:32:51Z")
@Kris this is definitely strange, as you pointed out none of the aliases is working. I suppose something is wrong in the networking setup of your Docker daemon. To fix it, from Mender’s PoV, you can override the configuration settings for each of the services.
For example, for deployments:
>
As you can see, you can set the DEPLOYMENTS\_MONGO\_URL env for deployments to use `mender-mongo` instead of `mongo-deployments`. The same configuration must be done for the other services.
---
### Author:  [@Kris](https://hub.mender.io/u/Kris)
#### Post date: [July 28, 2021, 5:43pm UTC](https://hub.mender.io/t/self-hosted-3-0-0-fails-to-connect-to-mongo-demo-runs-fine/3921/6 "2021-07-28T17:43:31Z")
Setting that env var before starting services seems to have no effect:
```auto
kris@mender:~/mender-server/production$ export
declare -x DEPLOYMENTS_MONGO_URL="mongodb://mender-mongo:27017"
...
```
```auto
kris@mender:~/mender-server/production$ ./run up -d
```
```auto
kris@mender:~/mender-server/production$ ./run logs mender-deployments
Attaching to menderproduction_mender-deployments_1
mender-deployments_1 | WARNING: ca-certificates.crt does not contain exactly one certificate or CRL: skipping
mender-deployments_1 | time="2021-07-28T17:18:23Z" level=info msg="'presign.secret' not configured. Generating a random secret." file=main.go func=main.doMain.func1 line=99
mender-deployments_1 | time="2021-07-28T17:18:23Z" level=info msg="Deployments Service, version unknown starting up"
mender-deployments_1 | failed to connect to db: Error reaching mongo server: server selection error: context deadline exceeded, current topology: { Type: Unknown, Servers: [{ Addr: mongo-deployments:27017, Type: Unknown, Average RTT: 0, Last error: connection() error occured during connection handshake: dial tcp: lookup mongo-deployments: Try again },] }
...
```
Any thoughts on the root cause of this? Like I said, this is standard Ubuntu. I suppose I could try spinning it up again from scratch, but this VM is otherwise a base install of 18.04 that was only used for an older 2.2 Mender server test that I’m unable to continue to use due to minio not liking Mender being so old.
---
### Author:  [@tranchitella](https://hub.mender.io/u/tranchitella)
#### Post date: [July 28, 2021, 7:30pm UTC](https://hub.mender.io/t/self-hosted-3-0-0-fails-to-connect-to-mongo-demo-runs-fine/3921/7 "2021-07-28T19:30:11Z")
@kris the env variable must be set in the docker-compose.yaml file, not in the shell.
As an example:
>
You need to set the deployments environment in the corresponding section:
>
---
### Author:  [@Kris](https://hub.mender.io/u/Kris)
#### Post date: [July 28, 2021, 8:38pm UTC](https://hub.mender.io/t/self-hosted-3-0-0-fails-to-connect-to-mongo-demo-runs-fine/3921/8 "2021-07-28T20:38:15Z")
Ah, my mistake. This does show trying to reach the correct name, but same error:
```auto
kris@mender:~/mender-server/production$ ./run logs mender-deployments
Attaching to menderproduction_mender-deployments_1
mender-deployments_1 | WARNING: ca-certificates.crt does not contain exactly one certificate or CRL: skipping
mender-deployments_1 | time="2021-07-28T19:57:23Z" level=info msg="'presign.secret' not configured. Generating a random secret." file=main.go func=main.doMain.func1 line=99
mender-deployments_1 | time="2021-07-28T19:57:23Z" level=info msg="Deployments Service, version unknown starting up"
mender-deployments_1 | failed to connect to db: Error reaching mongo server: server selection error: context deadline exceeded, current topology: { Type: Unknown, Servers: [{ Addr: mender-mongo:27017, Type: Unknown, Average RTT: 0, Last error: connection() error occured during connection handshake: dial tcp: lookup mender-mongo: Try again },] }
...
```
I can get in to mender-deployments container for a few seconds, enough to ping:
```auto
kris@mender:~/mender-server/production$ docker exec -it menderproduction_mender-deployments_1 sh
/ # ping mender-mongo
ping: bad address 'mender-mongo'
/ # ping mender-gui
PING mender-gui (172.26.0.4): 56 data bytes
64 bytes from 172.26.0.4: seq=0 ttl=64 time=0.173 ms
64 bytes from 172.26.0.4: seq=1 ttl=64 time=0.083 ms
```
I am unable to enter mender-mongo in any meaningful way, it remains in the “Restarting” state for so long:
```auto
Error response from daemon: Container 1ea664a92ecc3e02d078f15726dceb41746fd5f9fb2d2b0f09662f92657b8cf7 is restarting, wait until the container is running
```
Looking at the logs for mongo, I noticed this line this time around, not sure if this has significance or what do to about it:
```auto
mender-mongo_1 | {"t":{"$date":"2021-07-28T20:04:43.007+00:00"},"s":"I", "c":"STORAGE", "id":22315, "ctx":"initandlisten","msg":"Opening WiredTiger","attr":{"config":"create,cache_size=1460M,session_max=33000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000,close_scan_interval=10,close_handle_minimum=250),statistics_log=(wait=0),verbose=[recovery_progress,checkpoint_progress,compact_progress],"}}
mender-mongo_1 | {"t":{"$date":"2021-07-28T20:04:43.603+00:00"},"s":"F", "c":"STORAGE", "id":4671205, "ctx":"initandlisten","msg":"This version of MongoDB is too recent to start up on the existing data files. Try MongoDB 4.2 or earlier."}
mender-mongo_1 | {"t":{"$date":"2021-07-28T20:04:43.603+00:00"},"s":"F", "c":"-", "id":23089, "ctx":"initandlisten","msg":"Fatal assertion","attr":{"msgid":4671205,"file":"src/mongo/db/storage/wiredtiger/wiredtiger_kv_engine.cpp","line":919}}
mender-mongo_1 | {"t":{"$date":"2021-07-28T20:04:43.603+00:00"},"s":"F", "c":"-", "id":23090, "ctx":"initandlisten","msg":"\n\n***aborting after fassert() failure\n\n"}
mender-mongo_1 | {"t":{"$date":"2021-07-28T20:04:43.603+00:00"},"s":"F", "c":"CONTROL", "id":4757800, "ctx":"initandlisten","msg":"Writing fatal message","attr":{"message":"Got signal: 6 (Aborted).\n"}}
```
So, I blasted away all of `/var/lib/docker/volumes/` re-installed all of the docker components, re-ran the pulls and setup, and, everything is happy now. It looks like mongo was trying to use the old database despite being a different container, tags, etc. So that was a bit of a pain.
```auto
kris@mender:~$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
a18fc8d330de mendersoftware/deployments:mender-3.0.0 "/entrypoint.sh --co…" 3 minutes ago Up 3 minutes 8080/tcp menderproduction_mender-deployments_1
ee8edd5aec5f traefik:v2.4 "/entrypoint.sh --ac…" 4 minutes ago Up 4 minutes 80/tcp, 0.0.0.0:443->443/tcp, :::443->443/tcp menderproduction_mender-api-gateway_1
e618e45be50e mendersoftware/deviceauth:mender-3.0.0 "/usr/bin/deviceauth…" 4 minutes ago Up 4 minutes 8080/tcp menderproduction_mender-device-auth_1
6380c9c7fa6c mendersoftware/create-artifact-worker:mender-3.0.0 "/usr/bin/workflows …" 4 minutes ago Up 4 minutes 8080/tcp menderproduction_mender-create-artifact-worker_1
147d7df37751 mendersoftware/useradm:mender-3.0.0 "/usr/bin/useradm --…" 4 minutes ago Up 4 minutes 8080/tcp menderproduction_mender-useradm_1
5f515c28c6d2 mendersoftware/workflows-worker:mender-3.0.0 "/usr/bin/workflows …" 4 minutes ago Up 4 minutes menderproduction_mender-workflows-worker_1
2cef524b48eb mendersoftware/inventory:mender-3.0.0 "/usr/bin/inventory …" 4 minutes ago Up 4 minutes 8080/tcp menderproduction_mender-inventory_1
11d235c3f0b2 mendersoftware/deviceconnect:mender-3.0.0 "/usr/bin/deviceconn…" 4 minutes ago Up 4 minutes 8080/tcp menderproduction_mender-deviceconnect_1
cc39c18c230e mendersoftware/workflows:mender-3.0.0 "/usr/bin/workflows …" 4 minutes ago Up 4 minutes 8080/tcp menderproduction_mender-workflows-server_1
88026b4fc785 mendersoftware/deviceconfig:mender-3.0.0 "/usr/bin/deviceconf…" 4 minutes ago Up 4 minutes 8080/tcp menderproduction_mender-deviceconfig_1
e700c663e6da mongo:4.4 "docker-entrypoint.s…" 4 minutes ago Up 4 minutes 27017/tcp menderproduction_mender-mongo_1
6d06c563cfb3 mendersoftware/gui:mender-3.0.0 "/entrypoint.sh nginx" 7 minutes ago Up 4 minutes (healthy) 80/tcp, 8080/tcp menderproduction_mender-gui_1
125eba9430f7 nats:2.1.9-alpine3.12 "docker-entrypoint.s…" 7 minutes ago Up 4 minutes 4222/tcp, 6222/tcp, 8222/tcp menderproduction_mender-nats_1
0fd4f62450d0 minio/minio:RELEASE.2019-04-23T23-50-36Z "/usr/bin/docker-ent…" 7 minutes ago Up 4 minutes (healthy) 9000/tcp menderproduction_minio_1
```
Thanks for your suggestions! Hope this helps someone else coming from an older install.