Our journey to mender

General Discussions
1

Hey together,

for our product a battery and charger management solution we were looking for an OTA-Update solution and finally got mender successfully to run end-to-end.

While doing the testing with hosted Mender and a base image it worked instant. The final migration to Mender on AWS EKS and our own golden image had some pain points.

  • U-Boot configuration for Raspberry PI3 didn’t work out-of-the box, it was hard to figure out why
    • dtoverlay=disable-bt in config.txt fails because RPI3 firmware and U-Boot are doing somehow same stuff
      Configure U-Boot to load device tree from RPI only by replacing CONFIG_OF_EMBED=y by CONFIG_OF_BOARD=Y in the defconfig
    • Using a PI hat or anything on the uart interrupts U-Boot so that is doesn’t proceed booting the kernel
      Update U-Boot to a more recent version and disable U-Boot to wait for serial port. Add CONFIG_BOOTDELAY=-2 to the defconfig
  • Mender Convert worked like a charm
    • It still needs a check, if mender client is installed on the golden image, mender convert will just fail
    • Documentation and source code are a little bit out of sync, but the source is still good to read
    • The variable for passing a custom U-Boot should be configurable
  • Mender Helm Chart was also easy to deploy to AWS EKS via AWS CDK
    • I was a little bit sad, that the Helm Chart doesn’t support existing secrets. I just had to install ESO to load the certs from AWS SecretsManager. It only took some time
    • I still haven’t figured out, how to pass mongo url, username and password as separate values to the pods. But this only a matter of style
    • The mender deployments service has a bug where it doesn’t use our s3 service account to connect to AWS S3 (have forgotten at which point, I assume it’s related to the file storage factory in the golang code of that service, which is also doing a list of buckets, idk what’s purpose it has). Setting up an IAM user with AWS Access Key is a little bit annoying, since one has to maintain a key rotation policy
  • API Gateway
    • I haven’t figured out, what are the desired values for the websocket. ALB idle timeout and stickiness duration of the target group?
  • Contributing
    • I’m a bit sad, while solving the above problems, I didn’t found any time to invest into the custom contributing process, which is not just open a GitHub Issue or Pull Request

Many many thanks for this great product and the high quality source code :heart_eyes: and big cudos :rocket: