“menderproduction_mender-deployments_1” keeps restarting on self-hosted mender server

Hello everyone,

I have been trying to set up Mender.io server self-host for the past couple of days and has been hitting some issues. I would appreciate if anyone could look into this and offer a possible solution.

I have followed this guide to setup a Mender Server for production environments using Docker compose. The host machine is an an ODYSSEY - X86 board and running OpenWrt System. Here I want to achieve a local Mender Server that could do OTA updates locally to other devices on the same network.

After following the above guide, when I got to verification of the services, “menderproduction_mender-deployments_1” keeps restarting. Please check the below photo.

Initially, I thought this maybe because of system incompatibility since I am not running Mender server on Ubuntu. However, it seems that the issue is with a configuration.

Please check the logs below:

The IP address of the X86 server above is set to 192.168.2.1, and therefore the mender server access IP is assigned as 192.168.2.1 as well.

Eventhough the above says restarting, I can successfully login to the Mender Server UI by 192.168.2.1/ui.

Furthermore, I will attach the “prod.yml” file below so that anyone can check if I have done something wrong.

Appreciate the help.

Best Regards,
Lakshantha

looks like your cert is missing “Subject Alt Names” field set on it. By using IP addresses rather than domain names you are going to face some challenges. Will this self hosted server only be on LAN or will it be on the internet at some point?

Thank you for the response.

It will only be on LAN.

Basically, I want to deploy this as a LAN server and push OTA updates to the devices connected to the same network as the server.

I would appreciate all the help!

Ok, so the way i have tackled this in the past is then to create your own https certificate using a unused made-up subdomain (usually a subdomain of my company). Ensure when creating the cert that you fill all the necessary fields including Subject Alternative name. Then in my mender config use the certs domain name and not the ip address. Then add an entry in your LAN’ DNS server to resolve the certs domain name to its ip address so you can connect to your mender server by its domain name.

For testing-only instead of the DNS Server you can update your /etc/hosts file on your mender device and dev laptop to resolve the domain name for you.

In actual fact, we go one step further for LAN based servers, we are our own certificate authority and issue our own certs signed by our own certificate authority key to ensure a full chain of trust, but this will require you to add your public CA file to your devices so that they can trust certs signed by it.

Really appreciate the detailed response.

I will try to follow according to your response.

Best Regards,
Lakshantha

Hello @dellgreen ,

I have tried to login to the Mender Server UI with the previous configuration (the state I was in when I asked the question in the very beginning), and it says the connection is secure and I am able to login to the server.

1635965572(1)

Does this mean there is no issue with certificates?

This configuration is using IP address (192.168.2.1). However, according to your explanation before, when IP addresses are used rather than domain names, the above website is not supposed to show as a secure connection right?

I just want to confirm that my problem is really a certificate issue.

Also, sorry for too much questioning. I am not very familiar with this.

Thank you.

Best Regards,
Lakshantha

Which web browser is this? If not chrome try with that as it’s usually pretty strict. Have you accidentally added an exception to your browser for that cert?

I used Microsoft Edge. I also tried chrome now.

Sorry. I missed something before. When I typed 192.168.2.1 for the first time, it showed me this…

After that I clicked on “I understand the risks and wish to continue”

Then I can see this…

1635965572(1)

On a side note, I also want to point out that the X86 board that I am running the Mender Server on has 2 Ethernet ports. As I have mentioned before, I am running OpenWrt on it which is a router software. So one port is configured as WAN where it is connected to the internet and the other port is configured as LAN. The LAN port is configured as DHCP server with static IP 192.168.2.1. So I have connected my PC to this LAN port. The PC is used to access the Mender Server UI.

Also , I have generated the certificates following this:
https://docs.mender.io/3.1/server-installation/installation-with-docker-compose

With the following change:

API_GATEWAY_DOMAIN_NAME=“192.168.2.1”

ah yes that’s what i expected clicking “I understand the risks and wish to continue” often adds an exception to your browser for that site.

for API_GATEWAY_DOMAIN_NAME use an unused subdomain that you own and not an IP address, otherwise you will run into problems latter on when you configure the mender client on your board.

For the easiest way to pass strict TLS compliance you need to connect to your mender server with the domain name that your embed in the certificate. Otherwise you are going to get errors. For testing you can stick with “mender.example.com” and when happy with testing change it to something more appropriate for your companies LAN.

I have just used an unused subdomain for API_GATEWAY_DOMAIN_NAME. The domain is already in use by me.

After I run “./run ps” at the end, all docker containers are running with no issue!

So happy to see everything working as expected now! Thanks a lot for your detailed guidance!

Just to make sure I did things right, I have added “192.168.2.1 <subdomain.domain>” in “hosts” file of windows and used the browser to access mender server by using subdomain.domain. The UI shows up saying that the connection is secure.

Is that correct?

Also, is it a must to choose an already existing domain that I own and attach the unused subdomain? Since this is a local server, can I just go with a random name for the domain and attach a subdomain?

yep thats correct.

When you scale this up, it becomes a pain to edit every machines hosts file, and that’s when you would normally add it to your LAN’S DNS server so any machine can connect to your mender server via the domain name without any further admin overhead.

with respects to the domain name, it can be random, however a subdomain of something you own means your are not going to accidentally clash with an internet domain owned by someone else and hence preventing people from accessing that domain from your LAN.

as an example i usually use something like

mender.mycompanyname.com
mender.mycompanyname.co.uk

depending on what my company already owns

I understand.

Your response is very clear to me.

You have been very helpful. Thank you again.