TLS Handshake Verification Failure in Mender 3.4 Production Installation

I’m using Mender 3.4 production installation with Docker. I was able to set up the Mender server and log into the dashboard successfully. However, when attempting to connect a device using the Mender client, I encountered a TLS handshake verification failure. While running the client, I provided the same TLS certificate used for the server. I need help troubleshooting and resolving this issue.

Here is the logs

Setting up the Mender client: The client will regularly poll the server to check for updates and report its inventory data.
Get started by first configuring the device type and settings for communicating with the server.

Are you connecting this device to hosted.mender.io? [Y/n] n

Demo server uses a self-signed certifcate for “docker.mender.io” and modifies device’s /etc/hosts with the server’s IP address (Required if using Mender demo server.)
Do you want to configure the client for a demo server? [Y/n] n

Set the URL of the Mender Server: [https://docker.mender.io] https://10.20.30.130

Set the location of the certificate of the server; leave blank if using http (not recommended) or a certificate from a known authority (filepath, for example /etc/mender/server.crt): /etc/mender/server.crt

Demo intervals uses short poll and retry intervals (Recommended for testing.)
Do you want to run the client in demo mode? [Y/n] n

Set the update poll interval - the frequency with which the client will send an update check request to the server, in seconds: [1800]
Set the inventory poll interval - the frequency with which the client will send inventory data to the server, in seconds: [28800]

Set the retry poll interval - the frequency with which the client tries to communicate with the server (note: the client may attempt more often initially based on the previous intervals, but will fall back to this value if theserver is busy) [300]
Mender setup successfully.
2024-09-11T10:27:38+0530 level=INFO Mender setup completed successfully.
root@PT-LP-0083:/home/vadivel/automation/Gateway/scripts# systemctl restart mender-authd
systemctl restart mender-updated
root@PT-LP-0083:/home/vadivel/automation/Gateway/scripts# sudo systemctl status mender-authd
● mender-authd.service - Mender authentication service
Loaded: loaded (/lib/systemd/system/mender-authd.service; enabled; vendor preset: enabled)
Active: active (running) since Wed 2024-09-11 10:27:54 IST; 11s ago
Main PID: 17616 (mender-auth)
Tasks: 2 (limit: 9331)
Memory: 2.3M
CGroup: /system.slice/mender-authd.service
└─17616 /usr/bin/mender-auth daemon

Sep 11 10:27:54 PT-LP-0083 mender-auth[17719]: using interface /sys/class/net/eth0
Sep 11 10:27:54 PT-LP-0083 mender-auth[17616]: record_id=3 severity=info time=“2024-Sep-11 10:27:54.732682” name=“Global” msg=“Signing with: /var/lib/mender/mender-agent.pem”
Sep 11 10:27:54 PT-LP-0083 mender-auth[17616]: record_id=4 severity=error time=“2024-Sep-11 10:27:54.751779” name=“http_client” url=“https://10.20.30.130/api/devices/v1/authentication/auth_requests” msg=“https: Failed to perform the SSL handshake: certificate verify failed”
Sep 11 10:27:54 PT-LP-0083 mender-auth[17616]: record_id=5 severity=info time=“2024-Sep-11 10:27:54.751933” name=“Global” msg="Authentication error trying server ‘https://10.20.30.130’: certificate verify failed: POST https://10.20.30.130/api/devices/v1/authentication/auth_requests: "
Sep 11 10:27:54 PT-LP-0083 mender-auth[17616]: record_id=6 severity=error time=“2024-Sep-11 10:27:54.751951” name=“Global” msg=“Failed to fetch new token: Authentication error: No more servers to try for authentication”

Hi @Vadivel_K,

With 3.4 being not exactly up to date, how long has this been running? Is the certificate possibly expired?

The next thing to check would be, can you manually query the API using curl, as in one of the examples at Mender API docs

Greets,
Josef

Hi @TheYoctoJester

Thanks for the quick response. I am running this for past 2 days. The reason for using 3.4 version is it is the latest version which supports production installation using docker. Above this needs Kubernetes based deployment. I am using mender server 3.4 but the client is mender-client4. Will there be any issues due to this? I hope the latest client is backward compatible. Regarding the certificate - The server certificate I am using is same. I cross-checked this by exporting it from Mender-ui.

Hi @Vadivel_K,

Ok, I see. So to be very, very clear about this:

Using the 3.4 release is strongly discouraged, as it will not come with any bug or security fix of the last two years.

Sticking with it because your operations are not capable of handling a helm chart, which is the installation method we support for current releases, is outright irresponsible behaviour. There is no way to be nice about this.

The Mender Client 4.x is constructed as backwards compatible, so this should not be the cause of any problems.

As already pointed out, I suggest to check if the server certificate is valid and correctly applied. Using curl to query one of the API endpoints should give you either a success, or a more useful error message.

Greets,
Josef