Mender-auth bootstrap "Failed to perform the SSL handshake"

General Discussions
1

Hi dear mender community,

I’m trying to move a device from one self-hosted mender server to a different one and I’m using the
mender-auth bootstrap command for that.
I’m having some trouble and was hoping one of you could comment on that.

This is the error I’m facing:

admin@myMenderDevice~ $ sudo mender-auth bootstrap
record_id=1 severity=info time="2025-Jan-16 06:45:11.632907" name="Global" msg="Successfully loaded private key from /var/lib/mender/mender-agent.pem"
using interface /sys/class/net/eth0
record_id=2 severity=info time="2025-Jan-16 06:45:11.774078" name="Global" msg="Signing with: /var/lib/mender/mender-agent.pem"
record_id=3 severity=error time="2025-Jan-16 06:45:11.863050" name="http_client" url="https://myMender2.com/api/devices/v1/authentication/auth_requests" msg="https: Failed to perform the SSL handshake: certificate verify failed (SSL routines)"
record_id=4 severity=info time="2025-Jan-16 06:45:11.863817" name="Global" msg="Authentication error trying server 'https://myMender2.com': certificate verify failed (SSL routines): POST https://myMender2.com/api/devices/v1/authentication/auth_requests: "
record_id=5 severity=info time="2025-Jan-16 06:45:11.864097" name="Global" msg="Got Auth response"
record_id=6 severity=error time="2025-Jan-16 06:45:11.864208" name="Global" msg="Authentication error: No more servers to try for authentication"

So basically it can’t verify the cert for my mender server.

But curl for example is working:

admin@myMenderDevice~ $ curl -o /dev/null -s -w "%{http_code}\n" https://myMender2.com/ui/devices
200

I’m sure the cert, full chain and CA is in the store. What I was thinking is maybe mender-auth uses a different store where it looks up certificates?

Interestingly for my old server I didn’t need to put my cert anywhere else except for /usr/share/ca-certificates. I’m wondering why that’s the case now or if my method of switching the mender server is wrong?

Thanks for taking a look and happy new year to all :slight_smile:

2

Hi @bztry,

Moving devices across backend instances is a somewhat involved process, have you followed the documentation at Hosted Mender - tenant migration | Mender documentation?

Greetz,
Josef

3

Hi Josef,

I’ve stumbled across this doc but thought it would not apply to me since it says

Please note that this method only applies for hosted Mender instances and isn’t applicable for on-premise migrations.

Should I still follow the process even though I’m moving from self-hosted onprem to self-hosted onprem location?

Greetings

4

Hi @bztry,

Except for the “clone of old server” stage, I would think it applies.

Greetz,
Josef

5

Hi Josef,

I found a solution. The tenant migration process works as you described, but was not the culprit.

The issue was with Yocto, where for some reason the default openssl path does not include /etc/ssl/certs/ca-certificates.crt

I kind of went down the rabbit hole, found the mender client uses the Boost/Beast library for http communication and tried it out manually on yocto. For some reason, it does not use the file mentioned above while curl, wget and many other utilities do.

The solution is to exactly specify the file /etc/ssl/certs/ca-certificates.crt in mender.conf where you explicitly included the option ServerCertificate.

I think, just for future reference, what would’ve been helpful for me would be to actually show in the debug logs where mender-auth is looking for the cert, because even with --log-level trace, it just shows that the verification failed. But it’s not super important, in the end, I was able to find where the problem was :slight_smile:

Greetings and have a nice day

1 Like
6

Hi @bztry,

Thanks a lot for sharing! Yes that sounds like a tough one to crack, glad to hear you solved it.

Greetz,
Josef