# MongoDB pod stopped

**URL:** <https://hub.mender.io/t/mongodb-pod-stopped/5352>\
**Category:** General Discussions\
**Created:** [October 31, 2022, 1:16pm UTC](https://hub.mender.io/t/mongodb-pod-stopped/5352 "2022-10-31T13:16:00Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![amgedr](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/amgedr/32/642_2.png) [@amgedr](https://hub.mender.io/u/amgedr)\
**Post date:** [October 31, 2022, 1:16pm UTC](https://hub.mender.io/t/mongodb-pod-stopped/5352/1 "2022-10-31T13:16:00Z")

</div>

Hello,  
I need some help with this issue I’m facing? I deployed the server to GCP by following the tutorial a few days ago. The only change I did was used Storage instead of MinIO. Today I the MongoDB pod failed and I found the following logs:

```auto
{"t":{"$date":"2022-10-31T12:50:52.663+00:00"},"s":"I", "c":"STORAGE", "id":22430, "ctx":"WTCheckpointThread","msg":"WiredTiger message","attr":{"message":"[1667220652:663673][1:0x7f4e9c24c700], WT_SESSION.checkpoint: [WT_VERB_CHECKPOINT_PROGRESS] saving checkpoint snapshot min: 99854, snapshot max: 99854 snapshot count: 0, oldest timestamp: (1667220640, 1) , meta checkpoint timestamp: (1667220645, 1) base write gen: 261156"}}
{"t":{"$date":"2022-10-31T12:50:57.373+00:00"},"s":"I", "c":"NETWORK", "id":22943, "ctx":"listener","msg":"Connection accepted","attr":{"remote":"10.18.0.134:60190","connectionId":80023,"connectionCount":51}}
{"t":{"$date":"2022-10-31T12:50:57.379+00:00"},"s":"I", "c":"NETWORK", "id":51800, "ctx":"conn80023","msg":"client metadata","attr":{"remote":"10.18.0.134:60190","client":"conn80023","doc":{"driver":{"name":"nodejs|mongosh","version":"4.4.0"},"os":{"type":"Linux","name":"linux","architecture":"x64","version":"5.10.133+"},"platform":"Node.js v14.19.1, LE (unified)","version":"4.4.0|1.3.1","application":{"name":"mongosh 1.3.1"}}}}
{"t":{"$date":"2022-10-31T12:50:57.391+00:00"},"s":"I", "c":"NETWORK", "id":22943, "ctx":"listener","msg":"Connection accepted","attr":{"remote":"10.18.0.134:60194","connectionId":80024,"connectionCount":52}}
{"t":{"$date":"2022-10-31T12:50:57.393+00:00"},"s":"I", "c":"NETWORK", "id":51800, "ctx":"conn80024","msg":"client metadata","attr":{"remote":"10.18.0.134:60194","client":"conn80024","doc":{"driver":{"name":"nodejs|mongosh","version":"4.4.0"},"os":{"type":"Linux","name":"linux","architecture":"x64","version":"5.10.133+"},"platform":"Node.js v14.19.1, LE (unified)","version":"4.4.0|1.3.1","application":{"name":"mongosh 1.3.1"}}}}
{"t":{"$date":"2022-10-31T12:50:57.461+00:00"},"s":"I", "c":"ACCESS", "id":20249, "ctx":"conn80024","msg":"Authentication failed","attr":{"mechanism":"SCRAM-SHA-256","speculative":true,"principalName":"root","authenticationDatabase":"admin","remote":"10.18.0.134:60194","extraInfo":{},"error":"AuthenticationFailed: SCRAM authentication failed, storedKey mismatch"}}

```

Thanks.

---

<div class="post-metadata">

**Author:** ![alfrunes](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/alfrunes/32/703_2.png) [@alfrunes](https://hub.mender.io/u/alfrunes)\
**Post date:** [October 31, 2022, 3:31pm UTC](https://hub.mender.io/t/mongodb-pod-stopped/5352/2 "2022-10-31T15:31:22Z")

</div>

Hello,

It looks like the MongoDB credentials configured on the server is not matching the connection string configured for the Mender deployment.  
Assuming you have followed the [production installation using helm](https://docs.mender.io/3.4/server-installation/production-installation-with-kubernetes), could you compare the output of the following commands:

```bash
kubectl get secret mongodb -o jsonpath="{.data.mongodb-root-password}" | base64 -d && echo
kubectl get secret mongodb-common -o jsonpath="{.data.MONGO_URL}" | base64 -d

```

Example output:

> ```plaintext
> password123
> mongodb://root:password123@mongodb-0.mongodb-headless.default.svc.cluster.local:27017,mongodb-1.mongodb-headless.default.svc.cluster.local:27017
> 
> ```

Can you verify that the password in the two secrets matches for your deployment?

---

<div class="post-metadata">

**Author:** ![amgedr](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/amgedr/32/642_2.png) [@amgedr](https://hub.mender.io/u/amgedr)\
**Post date:** [October 31, 2022, 4:53pm UTC](https://hub.mender.io/t/mongodb-pod-stopped/5352/3 "2022-10-31T16:53:40Z")

</div>

Thank you for responding. Yeah they don’t match. That’s strange because the UI was working for a few day.

---

<div class="post-metadata">

**Author:** ![amgedr](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/amgedr/32/642_2.png) [@amgedr](https://hub.mender.io/u/amgedr)\
**Post date:** [November 1, 2022, 4:13am UTC](https://hub.mender.io/t/mongodb-pod-stopped/5352/4 "2022-11-01T04:13:18Z")

</div>

If I modify the password in the mongodb connections string using:

```auto
helm upgrade mender mender/mender -f values.yaml

```

Will that solve it?

---

<div class="post-metadata">

**Author:** ![amgedr](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/amgedr/32/642_2.png) [@amgedr](https://hub.mender.io/u/amgedr)\
**Post date:** [November 2, 2022, 12:38pm UTC](https://hub.mender.io/t/mongodb-pod-stopped/5352/5 "2022-11-02T12:38:39Z")

</div>

@alfrunes updating the connection string didn’t work. What could I do to get the passwords synchronized and the MongoDB pod up again?

---

<div class="post-metadata">

**Author:** ![alfrunes](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/alfrunes/32/703_2.png) [@alfrunes](https://hub.mender.io/u/alfrunes)\
**Post date:** [November 3, 2022, 4:45pm UTC](https://hub.mender.io/t/mongodb-pod-stopped/5352/6 "2022-11-03T16:45:26Z")

</div>

Yes, that should solve it. You might need to recreate the pods afterwards to load the new connection string.

```auto
for deploy in $(kubectl get deploy -l 'app.kubernetes.io/part-of=mender' -o name); do kubectl rollout restart $deploy; done

```

Note that if you re-apply the [MongoDB instructions](https://docs.mender.io/3.4/server-installation/production-installation-with-kubernetes/mongodb) you also generate a new password.

---

<div class="post-metadata">

**Author:** ![amgedr](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/amgedr/32/642_2.png) [@amgedr](https://hub.mender.io/u/amgedr)\
**Post date:** [November 9, 2022, 10:14am UTC](https://hub.mender.io/t/mongodb-pod-stopped/5352/7 "2022-11-09T10:14:43Z")

</div>

@alfrunes very sorry to keep dragging this post. I modified the yaml file from the tutorial to make the passwords match, deleted all releases and PVCs, and then reinstalled them in the same order as the tutorial. However, all the pods are ready again except for the mongodb-0 one and there are no errors in the pod’s log.
