Kubernetes installation fail

I followed the instructions at Production installation with Kubernetes | Mender documentation to install a k3s cluster with a mender opensource production server.

MongoDB:

Using the command
helm upgrade --install mongodb bitnami/mongodb --version 10.21.1 -f mongodb.yml
quits with an error
Error: failed to download "bitnami/mongodb" at version "10.21.1"
If I remove the version instruction --version 10.21.1 and the installation works.
Is there any version definitely needed?

Mender Server:

In the last step “Create the admin user” I get the following error:
error: unable to upgrade connection: container not found ("useradm")

CrashLoopBackOff

I get many CrashLoopBackOff status:

root@deployment:/var/lib/rancher/k3s/server/manifests# kubectl get pod -o wide
NAME                                       READY   STATUS             RESTARTS       AGE    IP           NODE         NOMINATED NODE   READINESS GATES
cert-manager-cainjector-5bb9bfbb5c-nz6l7   1/1     Running            0              109m   10.42.0.10   deployment   <none>           <none>
cert-manager-798f8bb594-pstp5              1/1     Running            0              109m   10.42.0.9    deployment   <none>           <none>
cert-manager-webhook-bf48877d4-v2gzn       1/1     Running            0              109m   10.42.0.11   deployment   <none>           <none>
mongodb-arbiter-0                          1/1     Running            0              103m   10.42.0.12   deployment   <none>           <none>
mongodb-0                                  1/1     Running            0              103m   10.42.0.14   deployment   <none>           <none>
mongodb-1                                  1/1     Running            0              102m   10.42.0.16   deployment   <none>           <none>
nats-box-68dd458c5d-5nd6b                  1/1     Running            0              101m   10.42.0.17   deployment   <none>           <none>
nats-0                                     3/3     Running            0              101m   10.42.0.20   deployment   <none>           <none>
nats-1                                     3/3     Running            0              101m   10.42.0.21   deployment   <none>           <none>
minio-operator-fc8bbbc9b-wtk9m             1/1     Running            0              98m    10.42.0.23   deployment   <none>           <none>
minio-operator-console-7d5db9fdd4-2lxrz    1/1     Running            0              98m    10.42.0.22   deployment   <none>           <none>
minio-ss-0-1                               1/1     Running            0              94m    10.42.0.30   deployment   <none>           <none>
minio-ss-0-0                               1/1     Running            0              94m    10.42.0.31   deployment   <none>           <none>
api-gateway-7bcb79c8d4-t5lh8               1/1     Running            0              83m    10.42.0.32   deployment   <none>           <none>
gui-6b6c9dcc9-vrbft                        1/1     Running            0              83m    10.42.0.41   deployment   <none>           <none>
deployments-8f97bb89b-xrcqb                0/1     CrashLoopBackOff   21 (93s ago)   83m    10.42.0.36   deployment   <none>           <none>
deviceconnect-85854c47fb-n6bbs             0/1     CrashLoopBackOff   21 (88s ago)   83m    10.42.0.37   deployment   <none>           <none>
deviceconfig-655bc99fc-lxg94               0/1     CrashLoopBackOff   21 (80s ago)   83m    10.42.0.38   deployment   <none>           <none>
workflows-worker-864756b6d4-xlxks          0/1     CrashLoopBackOff   21 (85s ago)   83m    10.42.0.33   deployment   <none>           <none>
create-artifact-worker-5f4dd85f4d-g76q8    0/1     CrashLoopBackOff   21 (82s ago)   83m    10.42.0.35   deployment   <none>           <none>
workflows-server-5668fc8776-9blx7          0/1     CrashLoopBackOff   21 (70s ago)   83m    10.42.0.39   deployment   <none>           <none>
iot-manager-c658c9869-flltj                0/1     CrashLoopBackOff   21 (59s ago)   83m    10.42.0.42   deployment   <none>           <none>
useradm-78fc76586f-5b8x2                   0/1     CrashLoopBackOff   21 (35s ago)   83m    10.42.0.40   deployment   <none>           <none>
device-auth-59d8744b7b-txlcj               0/1     CrashLoopBackOff   21 (30s ago)   83m    10.42.0.43   deployment   <none>           <none>
inventory-5589f4df8f-zzps6                 0/1     CrashLoopBackOff   21 (23s ago)   83m    10.42.0.34   deployment   <none>           <none>

What can I do to find the mistake? Are there any steps additionally needed?

The chart version 10.21.1 for bitnami/mongodb no longer exists. Could you replace the --version argument with 11.2.0? Remember to update the Mender chart values. Note that the snippet deploying the Mender helm chart depends on the $MONGODB_ROOT_PASSWORD env variable from the MongoDB deployment step.
In the meantime I will make sure to update the documentation. All the services that are in CrashLoopBackOff state depends on MongoDB.

Thank you for the fast reply.

I deleted/uninstalled my previous k3s installation and work through all the installation steps with the different mongodb version.

All installation steps succed successfully but one pod shows the status CrashLoopBackOff:

root@deployment:/var/lib/rancher/k3s/server/manifests# kubectl get pod -o wide
NAME                                       READY   STATUS             RESTARTS        AGE   IP           NODE         NOMINATED NODE   READINESS GATES
cert-manager-cainjector-5bb9bfbb5c-zgk25   1/1     Running            0               29m   10.42.0.7    deployment   <none>           <none>
cert-manager-798f8bb594-cp28k              1/1     Running            0               29m   10.42.0.9    deployment   <none>           <none>
cert-manager-webhook-bf48877d4-k4hlp       1/1     Running            0               29m   10.42.0.8    deployment   <none>           <none>
mongodb-arbiter-0                          1/1     Running            0               27m   10.42.0.13   deployment   <none>           <none>
mongodb-0                                  1/1     Running            0               27m   10.42.0.14   deployment   <none>           <none>
mongodb-1                                  1/1     Running            0               27m   10.42.0.16   deployment   <none>           <none>
nats-box-5448cbc897-vnk74                  1/1     Running            0               26m   10.42.0.17   deployment   <none>           <none>
nats-0                                     3/3     Running            0               26m   10.42.0.19   deployment   <none>           <none>
nats-1                                     3/3     Running            0               26m   10.42.0.21   deployment   <none>           <none>
minio-operator-console-7d5db9fdd4-brp5z    1/1     Running            0               25m   10.42.0.23   deployment   <none>           <none>
minio-operator-fc8bbbc9b-5fxtm             1/1     Running            0               25m   10.42.0.22   deployment   <none>           <none>
minio-ss-0-0                               1/1     Running            0               24m   10.42.0.28   deployment   <none>           <none>
minio-ss-0-1                               1/1     Running            0               24m   10.42.0.29   deployment   <none>           <none>
workflows-worker-864756b6d4-5khfc          1/1     Running            0               14m   10.42.0.31   deployment   <none>           <none>
gui-6b6c9dcc9-8nv4v                        1/1     Running            0               14m   10.42.0.36   deployment   <none>           <none>
iot-manager-c658c9869-5hhkt                1/1     Running            0               14m   10.42.0.32   deployment   <none>           <none>
create-artifact-worker-5f4dd85f4d-277jv    1/1     Running            0               14m   10.42.0.35   deployment   <none>           <none>
inventory-5589f4df8f-n9shm                 1/1     Running            0               14m   10.42.0.37   deployment   <none>           <none>
useradm-78fc76586f-wwsp4                   1/1     Running            0               14m   10.42.0.41   deployment   <none>           <none>
api-gateway-7bcb79c8d4-g7qqv               1/1     Running            0               14m   10.42.0.33   deployment   <none>           <none>
workflows-server-5668fc8776-66gtw          1/1     Running            0               14m   10.42.0.40   deployment   <none>           <none>
deviceconnect-85854c47fb-d72hf             1/1     Running            0               14m   10.42.0.39   deployment   <none>           <none>
deviceconfig-655bc99fc-9s5mv               1/1     Running            0               14m   10.42.0.38   deployment   <none>           <none>
device-auth-59d8744b7b-5nvth               1/1     Running            0               14m   10.42.0.34   deployment   <none>           <none>
deployments-8f97bb89b-8ktgj                0/1     CrashLoopBackOff   7 (3m12s ago)   14m   10.42.0.30   deployment   <none>           <none>

This looks like the minio credentials in the Mender helm chart values does not match the generated access key ($MINIO_ACCESS_KEY and $MINIO_SECRET_KEY) from this step.

Tip: you don’t have to redeploy everything from scratch to update the values, you can run:

helm get values mender > values.yml
vim values.yml # Update values in section global.s3.*
helm upgrade mender -f values.yml

If this doesn’t help, could you share the logs from the deployments-* pod?

2022-07-14T14:37:01.525790194Z stderr F RequestError: send request failed
2022-07-14T14:37:01.525797681Z stderr F caused by: Put "https://artifacts.interelectronix.com/mender-artifact-storage": x509: certificate is valid for d2bc123fc7ac1ea25c3581b93e5a12e6.cf3bcb6853f16c11ffa66b5e1d33adda.traefik.default, not artifacts.interelectronix.com