# Open Source Mender Server Tutorial

**URL:** <https://hub.mender.io/t/open-source-mender-server-tutorial/7477>\
**Category:** General Discussions\
**Tags:** mender, mender-server\
**Created:** [January 22, 2025, 7:42pm UTC](https://hub.mender.io/t/open-source-mender-server-tutorial/7477 "2025-01-22T19:42:02Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![mld](https://avatars.discourse-cdn.com/v4/letter/m/8797f3/32.png) [@mld](https://hub.mender.io/u/mld)\
**Post date:** [January 22, 2025, 7:42pm UTC](https://hub.mender.io/t/open-source-mender-server-tutorial/7477/1 "2025-01-22T19:42:02Z")

</div>

I am working on getting the open source mender tutorial up and running but I keep running into errors. [Mender Server | Mender documentation](https://docs.mender.io/3.7/server-installation/production-installation-with-kubernetes/mender-server)

I have started a MinIO pod, which seems to be up and running just fine:

```auto
sudo kubectl get all
NAME READY STATUS RESTARTS AGE
pod/minio 1/1 Running 0 6s

```

I have set up an access key, created a bucket (“test-bucket”), and also set environment variable MINIO\_DOMAIN=[minio.example.net](http://minio.example.net)

Next, I followed the instructions in the link above to set up the mender helm chart. Please see my helm chart below, with tokens and secrets commented out

```auto
global:
  enterprise: false
  image:
    tag: mender-3.7.8
  mongodb:
    URL: ""
  nats:
    URL: ""
  s3:
    AWS_URI: "https://mender.example.com"
    AWS_BUCKET: test-bucket
    AWS_ACCESS_KEY_ID: <removed for security reasons>
    AWS_SECRET_ACCESS_KEY: <removed for security reasons>
  url: "https://mender.example.com"

# This enables bitnami/mongodb sub-chart
mongodb:
  enabled: true
  auth:
    enabled: true
    rootPassword: <removed for security reasons>
    replicaSetKey: <removed for security reasons>

# This enabled nats sub-chart
nats:
  enabled: true

api_gateway:
  env:
    SSL: false

device_auth:
  certs:
    key: |-
      -----BEGIN PRIVATE KEY-----
      <removed for security reasons>
      -----END PRIVATE KEY-----

useradm:
  certs:
    key: |-
      -----BEGIN PRIVATE KEY-----
      <removed for security reasons>
      -----END PRIVATE KEY-----
ingress:
  enabled: true
  annotations:
    nginx.ingress.kubernetes.io/proxy-body-size: "0"
    nginx.ingress.kubernetes.io/proxy-buffering: "off"
    nginx.ingress.kubernetes.io/proxy-read-timeout: "600"
    nginx.ingress.kubernetes.io/proxy-send-timeout: "600"
  path: /
  ingressClassName: nginx
  hosts:
    - mender.example.com
  tls:
  # this secret must exists or it can be created from a working cert-manager instance
   - secretName: mender-ingress-tls
     hosts:
       - mender.example.com

```

When I do the command below via the tutorial, I get the errors below related to nats, and I never see the nats pods listed when I do “sudo kubectl get all”

```auto
helm upgrade --install mender mender/mender --version 5.11.2 -f mender-3.7.8.yml 

```

Warnings about nats:

```auto
Release "mender" does not exist. Installing it now.
coalesce.go:286: warning: cannot overwrite table with non table for mender.nats.nats.image (map[pullPolicy:IfNotPresent repository:nats tag:2.9.20-alpine])
coalesce.go:286: warning: cannot overwrite table with non table for mender.nats.nats.image (map[pullPolicy:IfNotPresent repository:nats tag:2.9.20-alpine])
coalesce.go:286: warning: cannot overwrite table with non table for mender.nats.nats.image (map[pullPolicy:IfNotPresent repository:nats tag:2.9.20-alpine])

```

Error

```auto
sudo kubectl get all
NAME READY STATUS RESTARTS AGE
pod/mender-deployments-migration-5cjxq 0/1 Error 0 33s
pod/mender-mongodb-0 1/1 Running 0 17s
pod/mender-mongodb-1 0/1 Running 0 5s
pod/mender-redis-master-0 1/1 Running 0 34s
pod/mender-redis-replicas-0 1/1 Running 0 34s
pod/mender-redis-replicas-1 0/1 Running 0 3s
pod/minio 1/1 Running 0 45s

```

```auto
sudo kubectl logs pod/mender-deployments-migration-5cjxq 
time="2025-01-22T19:43:53Z" level=warning msg="'presign.secret' not configured. Generating a random secret." caller="config.Setup@config.go:246"
failed to connect to db: Error reaching mongo server: server selection error: context deadline exceeded, current topology: { Type: ReplicaSetNoPrimary, Servers: [{ Addr: mender-mongodb-0.mender-mongodb-headless.default.svc.cluster.local:27017, Type: RSSecondary, Average RTT: 672594 }, { Addr: mender-mongodb-1.mender-mongodb-headless.default.svc.cluster.local:27017, Type: RSSecondary, Average RTT: 1089670 }, { Addr: mender-mongodb-arbiter-0.mender-mongodb-arbiter-headless.default.svc.cluster.local:27017, Type: Unknown, Last error: dial tcp: lookup mender-mongodb-arbiter-0.mender-mongodb-arbiter-headless.default.svc.cluster.local on 10.43.0.10:53: no such host },] }

```

Any and all help would be really appreciated, thank you!

---

<div class="post-metadata">

**Author:** ![robgio](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/robgio/32/1760_2.png) [@robgio](https://hub.mender.io/u/robgio)\
**Post date:** [January 23, 2025, 9:35am UTC](https://hub.mender.io/t/open-source-mender-server-tutorial/7477/2 "2025-01-23T09:35:33Z")

</div>

Hello @mld  
it looks like both MongoDB and Redis are not completely running; could you please check the logs for the MongoDB statefulset? In particular the pod which is not ready.

Thanks

---

<div class="post-metadata">

**Author:** ![mld](https://avatars.discourse-cdn.com/v4/letter/m/8797f3/32.png) [@mld](https://hub.mender.io/u/mld)\
**Post date:** [January 23, 2025, 5:10pm UTC](https://hub.mender.io/t/open-source-mender-server-tutorial/7477/3 "2025-01-23T17:10:28Z")

</div>

```auto
NAME READY STATUS RESTARTS AGE
pod/mender-api-gateway-75f8dff5b5-p9q9w 1/1 Running 0 28m
pod/mender-create-artifact-worker-7cf6d444d7-rmj5x 1/1 Running 0 28m
pod/mender-deployments-577bfdf64d-wwpcx 0/1 CrashLoopBackOff 10 (2m31s ago) 28m
pod/mender-deployments-86c987878-j7gpk 0/1 Running 0 3s
pod/mender-device-auth-78bcdd687-hpzr4 1/1 Running 0 28m
pod/mender-deviceconfig-66967cd7c7-gjvxd 1/1 Running 0 28m
pod/mender-deviceconnect-559f4496f4-7m9bb 1/1 Running 0 28m
pod/mender-gui-7944884849-pqmpl 1/1 Running 0 28m
pod/mender-inventory-6f7df89889-7kwjq 1/1 Running 0 28m
pod/mender-iot-manager-7cc67d98bc-nswgc 1/1 Running 0 28m
pod/mender-redis-master-0 1/1 Running 0 30m
pod/mender-redis-replicas-0 1/1 Running 0 30m
pod/mender-redis-replicas-1 1/1 Running 0 29m
pod/mender-redis-replicas-2 1/1 Running 0 29m
pod/mender-useradm-7775ccd87f-jqdkm 1/1 Running 0 28m
pod/mender-workflows-server-6494d5c5f9-zhhdj 1/1 Running 0 28m
pod/mender-workflows-worker-765b5bc789-tcdpf 1/1 Running 0 28m
pod/mongodb-0 1/1 Running 0 34m
pod/mongodb-arbiter-0 1/1 Running 0 34m
pod/nats-0 3/3 Running 0 33m
pod/nats-1 3/3 Running 0 33m
pod/nats-box-5fb7f4979f-z5d2s 1/1 Running 0 33m
pod/seaweedfs-filer-0 1/1 Running 0 78m
pod/seaweedfs-master-0 1/1 Running 0 78m
pod/seaweedfs-s3-57ffbb7694-78hjw 1/1 Running 0 78m
pod/seaweedfs-volume-0 1/1 Running 0 78m

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/external-nginx ExternalName <none> <host_ip_address_running_nginx_container> <none> 86m
service/kubernetes ClusterIP 10.43.0.1 <none> 443/TCP 123m
service/mender-api-gateway ClusterIP 10.43.207.141 <none> 80/TCP 28m
service/mender-deployments ClusterIP 10.43.149.87 <none> 8080/TCP 28m
service/mender-device-auth ClusterIP 10.43.113.82 <none> 8080/TCP 28m
service/mender-deviceconfig ClusterIP 10.43.94.65 <none> 8080/TCP 28m
service/mender-deviceconnect ClusterIP 10.43.248.232 <none> 8080/TCP 28m
service/mender-gui ClusterIP 10.43.215.185 <none> 80/TCP,8080/TCP 28m
service/mender-inventory ClusterIP 10.43.234.50 <none> 8080/TCP 28m
service/mender-iot-manager ClusterIP 10.43.29.206 <none> 8080/TCP 28m
service/mender-redis-headless ClusterIP None <none> 6379/TCP 30m
service/mender-redis-master ClusterIP 10.43.70.37 <none> 6379/TCP 30m
service/mender-redis-replicas ClusterIP 10.43.39.5 <none> 6379/TCP 30m
service/mender-useradm ClusterIP 10.43.150.1 <none> 8080/TCP 28m
service/mender-workflows-server ClusterIP 10.43.144.29 <none> 8080/TCP 28m
service/mongodb-arbiter-headless ClusterIP None <none> 27017/TCP 34m
service/mongodb-headless ClusterIP None <none> 27017/TCP 34m
service/nats ClusterIP None <none> 4222/TCP,6222/TCP,8222/TCP,7777/TCP,7422/TCP,7522/TCP 33m
service/seaweedfs-filer ClusterIP None <none> 8888/TCP,18888/TCP,8333/TCP,9327/TCP 78m
service/seaweedfs-filer-client ClusterIP None <none> 8888/TCP,18888/TCP,9327/TCP 78m
service/seaweedfs-master ClusterIP None <none> 9333/TCP,19333/TCP,9327/TCP 78m
service/seaweedfs-s3 ClusterIP 10.43.110.67 <none> 8333/TCP,9327/TCP 78m
service/seaweedfs-volume ClusterIP None <none> 8080/TCP,18080/TCP,9327/TCP 78m

NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/mender-api-gateway 1/1 1 1 28m
deployment.apps/mender-create-artifact-worker 1/1 1 1 28m
deployment.apps/mender-deployments 0/1 1 0 28m
deployment.apps/mender-device-auth 1/1 1 1 28m
deployment.apps/mender-deviceconfig 1/1 1 1 28m
deployment.apps/mender-deviceconnect 1/1 1 1 28m
deployment.apps/mender-gui 1/1 1 1 28m
deployment.apps/mender-inventory 1/1 1 1 28m
deployment.apps/mender-iot-manager 1/1 1 1 28m
deployment.apps/mender-useradm 1/1 1 1 28m
deployment.apps/mender-workflows-server 1/1 1 1 28m
deployment.apps/mender-workflows-worker 1/1 1 1 28m
deployment.apps/nats-box 1/1 1 1 33m
deployment.apps/seaweedfs-s3 1/1 1 1 78m

NAME DESIRED CURRENT READY AGE
replicaset.apps/mender-api-gateway-75f8dff5b5 1 1 1 28m
replicaset.apps/mender-create-artifact-worker-7cf6d444d7 1 1 1 28m
replicaset.apps/mender-deployments-577bfdf64d 1 1 0 28m
replicaset.apps/mender-deployments-86c987878 1 1 0 2m4s
replicaset.apps/mender-device-auth-78bcdd687 1 1 1 28m
replicaset.apps/mender-deviceconfig-66967cd7c7 1 1 1 28m
replicaset.apps/mender-deviceconnect-559f4496f4 1 1 1 28m
replicaset.apps/mender-gui-7944884849 1 1 1 28m
replicaset.apps/mender-inventory-6f7df89889 1 1 1 28m
replicaset.apps/mender-iot-manager-7cc67d98bc 1 1 1 28m
replicaset.apps/mender-useradm-7775ccd87f 1 1 1 28m
replicaset.apps/mender-workflows-server-6494d5c5f9 1 1 1 28m
replicaset.apps/mender-workflows-worker-765b5bc789 1 1 1 28m
replicaset.apps/nats-box-5fb7f4979f 1 1 1 33m
replicaset.apps/seaweedfs-s3-57ffbb7694 1 1 1 78m

NAME READY AGE
statefulset.apps/mender-redis-master 1/1 30m
statefulset.apps/mender-redis-replicas 3/3 30m
statefulset.apps/mongodb 1/1 34m
statefulset.apps/mongodb-arbiter 1/1 34m
statefulset.apps/nats 2/2 33m
statefulset.apps/seaweedfs-filer 1/1 78m
statefulset.apps/seaweedfs-master 1/1 78m
statefulset.apps/seaweedfs-volume 1/1 78m

NAME SCHEDULE TIMEZONE SUSPEND ACTIVE LAST SCHEDULE AGE
cronjob.batch/mender-deployments-storage-daemon 15 * * * * <none> False 0 <none> 28m

```

I switched over to using SeaweedFS as I see MinIO is no longer compatible.

I now get this error in pod/mender-deployments-577bfdf64d-wwpcx

```auto
kubectl logs pod/mender-deployments-577bfdf64d-wwpcx   
time="2025-01-23T17:02:47Z" level=warning msg="'presign.secret' not configured. Generating a random secret." caller="config.Setup@config.go:246"
time="2025-01-23T17:02:47Z" level=info msg="Deployments Service starting up" caller="main.cmdServer@main.go:158"
time="2025-01-23T17:02:47Z" level=info msg="automigrate is ON, will apply migrations" caller="mongo.Migrate@migrations.go:49"
time="2025-01-23T17:02:47Z" level=info msg="migrating deployment_service" caller="mongo.MigrateSingle@migrations.go:71"
time="2025-01-23T17:02:47Z" level=info msg="migration to version 1.2.1 skipped" caller="migrate.(*SimpleMigrator).Apply@migrator_simple.go:124" db=deployment_service
time="2025-01-23T17:02:47Z" level=info msg="migration to version 1.2.2 skipped" caller="migrate.(*SimpleMigrator).Apply@migrator_simple.go:124" db=deployment_service
time="2025-01-23T17:02:47Z" level=info msg="migration to version 1.2.3 skipped" caller="migrate.(*SimpleMigrator).Apply@migrator_simple.go:124" db=deployment_service
time="2025-01-23T17:02:47Z" level=info msg="migration to version 1.2.4 skipped" caller="migrate.(*SimpleMigrator).Apply@migrator_simple.go:124" db=deployment_service
time="2025-01-23T17:02:47Z" level=info msg="migration to version 1.2.5 skipped" caller="migrate.(*SimpleMigrator).Apply@migrator_simple.go:124" db=deployment_service
time="2025-01-23T17:02:47Z" level=info msg="migration to version 1.2.6 skipped" caller="migrate.(*SimpleMigrator).Apply@migrator_simple.go:124" db=deployment_service
time="2025-01-23T17:02:47Z" level=info msg="migration to version 1.2.7 skipped" caller="migrate.(*SimpleMigrator).Apply@migrator_simple.go:124" db=deployment_service
time="2025-01-23T17:02:47Z" level=info msg="migration to version 1.2.9 skipped" caller="migrate.(*SimpleMigrator).Apply@migrator_simple.go:124" db=deployment_service
time="2025-01-23T17:02:47Z" level=info msg="migration to version 1.2.10 skipped" caller="migrate.(*SimpleMigrator).Apply@migrator_simple.go:124" db=deployment_service
time="2025-01-23T17:02:47Z" level=info msg="migration to version 1.2.11 skipped" caller="migrate.(*SimpleMigrator).Apply@migrator_simple.go:124" db=deployment_service
time="2025-01-23T17:02:47Z" level=info msg="migration to version 1.2.13 skipped" caller="migrate.(*SimpleMigrator).Apply@migrator_simple.go:124" db=deployment_service
time="2025-01-23T17:02:47Z" level=info msg="migration to version 1.2.14 skipped" caller="migrate.(*SimpleMigrator).Apply@migrator_simple.go:124" db=deployment_service
time="2025-01-23T17:02:47Z" level=info msg="migration to version 1.2.15 skipped" caller="migrate.(*SimpleMigrator).Apply@migrator_simple.go:124" db=deployment_service
time="2025-01-23T17:02:47Z" level=info msg="DB migrated to version 1.2.15" caller="migrate.(*SimpleMigrator).Apply@migrator_simple.go:139" db=deployment_service
main: failed to setup storage client: s3: failed to check bucket preconditions: operation error S3: HeadBucket, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Head "https://mender.example.com/test-bucket": dial tcp: lookup mender.example.com on 10.43.0.10:53: no such host

```

```auto
kubectl get ingress
NAME CLASS HOSTS ADDRESS PORTS AGE
external-nginx-ingress traefik mender.example.com <host_ip_address_running_nginx_container> 80, 443 92m

```

Why does that error make it seem like mongoDB and redis are not running? Maybe my most recent attempt shown in this have it running correctly now?

I decided to use external mongoDB and nats, as I kept running into issues with having mender helm bundle them. The error I’m getting now makes it seem like it cannot connect to the SeaweedFS test bucket. I’m still confused by the presign.secret error though too, as I keep getting that on every attempt I do
