# Deployment of AKS based mender-server Version 3.6.2 not completely successful

**URL:** https://hub.mender.io/t/deployment-of-aks-based-mender-server-version-3-6-2-not-completely-successful/6190
**Category:** General Discussions
**Tags:** mender-server
**Created:** [September 8, 2023, 7:57am UTC](https://hub.mender.io/t/deployment-of-aks-based-mender-server-version-3-6-2-not-completely-successful/6190 "2023-09-08T07:57:29Z")
**Posts on this page:** 12
**Page:** 2

<div class="post-metadata">

### Author: ![oldev](https://avatars.discourse-cdn.com/v4/letter/o/3be4f8/32.png) [@oldev](https://hub.mender.io/u/oldev)
#### Post date: [September 8, 2023, 10:48am UTC](https://hub.mender.io/t/deployment-of-aks-based-mender-server-version-3-6-2-not-completely-successful/6190/21 "2023-09-08T10:48:00Z")

</div>

> [@robgio](#):
>
> ```auto
> kubectl get secret mongodb-common -o jsonpath='{.data.MONGO}' | base64 -d
> 
> ```

The password from the Connection string in mongodb-common matches exactly my value from cofig param `mongodb.auth.rootPassword`

During the helm upgrade flow, the mongodb-common-prerelease is still having a broken connection string

```auto
kubectl get secret mongodb-common-prerelease -o jsonpath='{.data.MONGO}' | base64 -d
mongodb://root:<rootPW>@mender-mongodb-headless

```

here are the logs of the workflows-server-migration containers:

```auto
$ kubectl logs mender-db-data-migration-h498l -c workflows-server-migration
2023/09/08 10:41:09 failed to connect to db: Error reaching mongo server: connection() error occurred during connection handshake: auth error: unable to authenticate using mechanism "SCRAM-SHA-256": (AuthenticationFailed) Authentication failed.
$ kubectl logs mender-db-data-migration-kx4b5 -c workflows-server-migration
2023/09/08 10:40:50 failed to connect to db: Error reaching mongo server: connection() error occurred during connection handshake: auth error: unable to authenticate using mechanism "SCRAM-SHA-256": (AuthenticationFailed) Authentication failed.
$ kubectl logs mender-db-data-migration-7wql4 -c workflows-server-migration
2023/09/08 10:40:40 failed to connect to db: Error reaching mongo server: connection() error occurred during connection handshake: auth error: unable to authenticate using mechanism "SCRAM-SHA-256": (AuthenticationFailed) Authentication failed.

```

---

<div class="post-metadata">

### Author: ![robgio](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/robgio/32/1760_2.png) [@robgio](https://hub.mender.io/u/robgio)
#### Post date: [September 8, 2023, 10:51am UTC](https://hub.mender.io/t/deployment-of-aks-based-mender-server-version-3-6-2-not-completely-successful/6190/22 "2023-09-08T10:51:26Z")

</div>

Maybe a special character in the mongodb password? If so, could you please retry a fresh install with a complex password composed by only numbers and letters?

---

<div class="post-metadata">

### Author: ![oldev](https://avatars.discourse-cdn.com/v4/letter/o/3be4f8/32.png) [@oldev](https://hub.mender.io/u/oldev)
#### Post date: [September 8, 2023, 10:58am UTC](https://hub.mender.io/t/deployment-of-aks-based-mender-server-version-3-6-2-not-completely-successful/6190/23 "2023-09-08T10:58:08Z")

</div>

not in the secret

 ![secret_contents](https://canada1.discourse-cdn.com/flex036/uploads/mender/original/2X/b/b712ff1aff9815c40a000a8d7e2fcb2eaa89ba7d.png)

---

<div class="post-metadata">

### Author: ![oldev](https://avatars.discourse-cdn.com/v4/letter/o/3be4f8/32.png) [@oldev](https://hub.mender.io/u/oldev)
#### Post date: [September 8, 2023, 10:59am UTC](https://hub.mender.io/t/deployment-of-aks-based-mender-server-version-3-6-2-not-completely-successful/6190/24 "2023-09-08T10:59:53Z")

</div>

so the temporary resource is the mongodb-common-prerelease right? This one is not deleted and has **not** the same content as mongodb-common.

P.S.

I did run:

```auto
kubectl delete mongodb-common-prerelease
helm upgrade mender mender/mender -f mender-3.6.2.yml

```

This is how mongodb-common-prerelease got created:

```auto
\o$ kubectl get secrets mongodb-common-prerelease
NAME TYPE DATA AGE
mongodb-common-prerelease Opaque 2 24s
\o$ kubectl get secret mongodb-common-prerelease -o jsonpath='{.data.MONGO}' | base64 -d
mongodb://root:<rootPW>@mender-mongodb-headless

```

---

<div class="post-metadata">

### Author: ![oldev](https://avatars.discourse-cdn.com/v4/letter/o/3be4f8/32.png) [@oldev](https://hub.mender.io/u/oldev)
#### Post date: [September 8, 2023, 11:15am UTC](https://hub.mender.io/t/deployment-of-aks-based-mender-server-version-3-6-2-not-completely-successful/6190/25 "2023-09-08T11:15:11Z")

</div>

I will re-install. I dug into the chart and have seen quite some logic around the mongo uri generation.

I had a request to scale down pods in the cluster and did in fact change the mongodb architecture at one helm revision.

> <https://github.com/mendersoftware/mender-helm/blob/a5ca19745a91fb49adfbe5b8d56f802104de1271/mender/templates/_helpers.tpl#L61C9-L61C9>

---

<div class="post-metadata">

### Author: ![oldev](https://avatars.discourse-cdn.com/v4/letter/o/3be4f8/32.png) [@oldev](https://hub.mender.io/u/oldev)
#### Post date: [September 8, 2023, 11:21am UTC](https://hub.mender.io/t/deployment-of-aks-based-mender-server-version-3-6-2-not-completely-successful/6190/26 "2023-09-08T11:21:39Z")

</div>

Should I keep these properties in my config for fresh installation regarding the NATS issue?

```auto
workflows:
  automigrate: false

create_artifact_worker:
  automigrate: false

```

---

<div class="post-metadata">

### Author: ![robgio](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/robgio/32/1760_2.png) [@robgio](https://hub.mender.io/u/robgio)
#### Post date: [September 8, 2023, 11:33am UTC](https://hub.mender.io/t/deployment-of-aks-based-mender-server-version-3-6-2-not-completely-successful/6190/27 "2023-09-08T11:33:19Z")

</div>

Yes, you can keep the `automigrate: false` flags.  
The `mongodb-common-prerelease` could be originating from the old setup, then. When doing a fresh install you also have to take care of old volumes, that usually are kept even after a helm delete.

Tldr: for a real fresh install you have to delete all the secrets, pvc, pv, jobs, statefulset resources that could be kept even after an `helm delete`

---

<div class="post-metadata">

### Author: ![oldev](https://avatars.discourse-cdn.com/v4/letter/o/3be4f8/32.png) [@oldev](https://hub.mender.io/u/oldev)
#### Post date: [September 8, 2023, 12:02pm UTC](https://hub.mender.io/t/deployment-of-aks-based-mender-server-version-3-6-2-not-completely-successful/6190/28 "2023-09-08T12:02:49Z")

</div>

The cluster is cleaned and ready.

Now I get an yaml error when I set  
workflows.automigrate=false

```bash
\o$ helm template mender mender/mender -f mender-3.6.2.yml --set workflows.automigrate=true > /dev/null
\o$ helm template mender mender/mender -f mender-3.6.2.yml --set workflows.automigrate=false > /dev/null
Error: YAML parse error on mender/templates/create-artifact-worker-deploy.yaml: error converting YAML to JSON: yaml: line 55: mapping values are not allowed in this context

Use --debug flag to render out invalid YAML

```

---

<div class="post-metadata">

### Author: ![robgio](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/robgio/32/1760_2.png) [@robgio](https://hub.mender.io/u/robgio)
#### Post date: [September 8, 2023, 12:28pm UTC](https://hub.mender.io/t/deployment-of-aks-based-mender-server-version-3-6-2-not-completely-successful/6190/29 "2023-09-08T12:28:21Z")

</div>

Checking… Can you please try without the `automigrate` flags meanwhile?

---

<div class="post-metadata">

### Author: ![oldev](https://avatars.discourse-cdn.com/v4/letter/o/3be4f8/32.png) [@oldev](https://hub.mender.io/u/oldev)
#### Post date: [September 8, 2023, 12:33pm UTC](https://hub.mender.io/t/deployment-of-aks-based-mender-server-version-3-6-2-not-completely-successful/6190/30 "2023-09-08T12:33:56Z")

</div>

I did run 2 template of the specific file:

```auto
helm template mender ./mender -f mender-3.6.2-2.yml -s templates/create-artifact-worker-deploy.yaml --debug --version 5.2.3 --set workflows.automigrate=true > good_create-artifact-worker-deploy.yaml
helm template mender ./mender -f mender-3.6.2-2.yml -s templates/create-artifact-worker-deploy.yaml --debug --version 5.2.3 --set workflows.automigrate=false > cursed_create-artifact-worker-deploy.yaml

diff cursed_create-artifact-worker-deploy.yaml good_create-artifact-worker-deploy.yaml 
57c57,58
< memory: 128Mcommand: ["workflows", "--config", "/etc/workflows/config.yaml", "worker"]
---
> memory: 128M
> command: ["workflows", "--config", "/etc/workflows/config.yaml", "worker", "--automigrate"]

```

There’s no linebreak after memory: 128M 😵‍💫

---

<div class="post-metadata">

### Author: ![oldev](https://avatars.discourse-cdn.com/v4/letter/o/3be4f8/32.png) [@oldev](https://hub.mender.io/u/oldev)
#### Post date: [September 8, 2023, 12:58pm UTC](https://hub.mender.io/t/deployment-of-aks-based-mender-server-version-3-6-2-not-completely-successful/6190/31 "2023-09-08T12:58:23Z")

</div>

The clean setup even without `automigrate` flags was successful. I can’t tell what the difference was to the previous installation. Maybe some special character as I’m not working 100% with linux ☹

Thanks for the quick and excellent support!

```auto
kubectl get pods
NAME READY STATUS RESTARTS AGE
mender-api-gateway-7b75c59bdd-bpsf8 1/1 Running 0 8m44s
mender-create-artifact-worker-6b9f44d949-wd4wz 1/1 Running 3 (8m3s ago) 8m44s
mender-deployments-6f6764995c-pk28g 1/1 Running 0 8m44s
mender-device-auth-989c8bd96-jdzvk 1/1 Running 0 8m44s
mender-deviceconfig-67b9779b8d-qhhl8 1/1 Running 0 8m44s
mender-deviceconnect-5b67d847f-58qtx 1/1 Running 0 8m44s
mender-gui-5d999776c4-sr22k 1/1 Running 0 8m44s
mender-inventory-8899485fb-4c85p 1/1 Running 0 8m44s
mender-iot-manager-587ff944d-sshmr 1/1 Running 0 8m44s
mender-mongodb-0 1/1 Running 0 9m42s
mender-mongodb-1 1/1 Running 0 9m
mender-mongodb-arbiter-0 1/1 Running 0 9m41s
mender-nats-0 3/3 Running 0 8m44s
mender-nats-1 3/3 Running 0 8m44s
mender-nats-2 3/3 Running 0 8m44s
mender-nats-box-7bbc67486c-hzrcm 1/1 Running 0 8m44s
mender-redis-master-0 1/1 Running 0 9m39s
mender-redis-replicas-0 1/1 Running 0 9m38s
mender-redis-replicas-1 1/1 Running 0 8m52s
mender-redis-replicas-2 1/1 Running 0 8m27s
mender-useradm-7d9d56b6fd-nmn5z 1/1 Running 0 8m44s
mender-workflows-server-5cc4b8b965-j78lz 1/1 Running 2 (8m21s ago) 8m44s
mender-workflows-worker-859b66449-zgzbt 1/1 Running 3 (8m7s ago) 8m44s

```

---

<div class="post-metadata">

### Author: ![robgio](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/robgio/32/1760_2.png) [@robgio](https://hub.mender.io/u/robgio)
#### Post date: [September 8, 2023, 1:01pm UTC](https://hub.mender.io/t/deployment-of-aks-based-mender-server-version-3-6-2-not-completely-successful/6190/32 "2023-09-08T13:01:10Z")

</div>

Thanks for the debug! FYI, here’s a PR for the fix you reported: [fix: automigrate command newline fix by oldgiova · Pull Request #196 · mendersoftware/mender-helm · GitHub](https://github.com/mendersoftware/mender-helm/pull/196)

Have a great weekend.

[Previous page](https://hub.mender.io/t/deployment-of-aks-based-mender-server-version-3-6-2-not-completely-successful/6190.md?page=1)
