Mender version 2.6 - Devices not shown on pending

Hello!

I’ve just installed new version of MENDER (2.6) on an AWS VM.
Mender client is installed on a RaspberryPi3.

Everything is instaleld correctly I think.
A take server certificate generated on server and copied to mender client.
I see tries of mender.client to register on server and “auth-error” result. And this is ok.
However I cannot see device on pending devices.

Client log

-- Logs begin at Mon 2021-03-29 11:13:22 CEST. --
mar 29 11:47:43 raspberrypi systemd[1]: Started Mender OTA update service.
mar 29 11:47:43 raspberrypi mender[1182]: time="2021-03-29T11:47:43+02:00" level=info msg="Loaded configuration file: /etc/mender/mender.conf"
mar 29 11:47:43 raspberrypi mender[1182]: time="2021-03-29T11:47:43+02:00" level=info msg="No dual rootfs configuration present"
mar 29 11:47:44 raspberrypi mender[1182]: time="2021-03-29T11:47:44+02:00" level=info msg="State transition: init [none] -> init [none]"
mar 29 11:47:44 raspberrypi mender[1182]: time="2021-03-29T11:47:44+02:00" level=info msg="State transition: init [none] -> idle [Idle]"
mar 29 11:47:44 raspberrypi mender[1182]: time="2021-03-29T11:47:44+02:00" level=info msg="State transition: idle [Idle] -> authorize-wait [Idle]"
mar 29 11:47:44 raspberrypi mender[1182]: time="2021-03-29T11:47:44+02:00" level=info msg="State transition: authorize-wait [Idle] -> authorize [Sync]"
mar 29 11:47:44 raspberrypi mender[1182]: time="2021-03-29T11:47:44+02:00" level=error msg="Authorize failed: transient error: authorization request failed: transient error: authorization request failed: (request_id: ): authentication request rejected server error message: dev auth: unauthorized"
mar 29 11:47:44 raspberrypi mender[1182]: time="2021-03-29T11:47:44+02:00" level=error msg="Authorize failed: transient error: authorization request failed: transient error: authorization request failed: (request_id: ): authentication request rejected server error message: dev auth: unauthorized"
mar 29 11:47:44 raspberrypi mender[1182]: time="2021-03-29T11:47:44+02:00" level=info msg="State transition: authorize [Sync] -> authorize-wait [Idle]"
mar 29 11:52:44 raspberrypi mender[1182]: time="2021-03-29T11:52:44+02:00" level=info msg="State transition: authorize-wait [Idle] -> authorize [Sync]"
mar 29 11:52:44 raspberrypi mender[1182]: time="2021-03-29T11:52:44+02:00" level=error msg="Authorize failed: transient error: authorization request failed: transient error: authorization request failed: (request_id: ): authentication request rejected server error message: dev auth: unauthorized"
mar 29 11:52:44 raspberrypi mender[1182]: time="2021-03-29T11:52:44+02:00" level=error msg="Authorize failed: transient error: authorization request failed: transient error: authorization request failed: (request_id: ): authentication request rejected server error message: dev auth: unauthorized"
mar 29 11:52:44 raspberrypi mender[1182]: time="2021-03-29T11:52:44+02:00" level=info msg="State transition: authorize [Sync] -> authorize-wait [Idle]"
mar 29 11:57:44 raspberrypi mender[1182]: time="2021-03-29T11:57:44+02:00" level=info msg="State transition: authorize-wait [Idle] -> authorize [Sync]"
mar 29 11:57:44 raspberrypi mender[1182]: time="2021-03-29T11:57:44+02:00" level=error msg="Authorize failed: transient error: authorization request failed: transient error: authorization request failed: (request_id: ): authentication request rejected server error message: dev auth: unauthorized"
mar 29 11:57:44 raspberrypi mender[1182]: time="2021-03-29T11:57:44+02:00" level=error msg="Authorize failed: transient error: authorization request failed: transient error: authorization request failed: (request_id: ): authentication request rejected server error message: dev auth: unauthorized"
mar 29 11:57:44 raspberrypi mender[1182]: time="2021-03-29T11:57:44+02:00" level=info msg="State transition: authorize [Sync] -> authorize-wait [Idle]"

Server log

mender-device-auth_1             | time="2021-03-29T09:36:37Z" level=info msg="401 2920μs POST /api/devices/v1/authentication/auth_requests HTTP/1.1 - Go-http-client/1.1" byteswritten=111 file=middleware.go func="accesslog.(*AccessLogMiddleware).MiddlewareFunc.func1" line=71 method=POST path=/api/devices/v1/authentication/auth_requests qs= request_id=7ef1c1c5-5417-4fe5-a6a7-b8ea0fbd667f responsetime=0.002920888 status=401 ts="2021-03-29 09:36:37.370050347 +0000 UTC" type=http
mender-device-auth_1             | time="2021-03-29T09:41:37Z" level=warning msg="dev auth: unauthorized: dev auth: unauthorized" file=response_helpers.go func=rest_utils.restErrWithLogMsg line=108 request_id=b2bcb1f6-9c89-4ebd-b680-f8160065eeaf
mender-device-auth_1             | time="2021-03-29T09:41:37Z" level=info msg="401 3445μs POST /api/devices/v1/authentication/auth_requests HTTP/1.1 - Go-http-client/1.1" byteswritten=111 file=middleware.go func="accesslog.(*AccessLogMiddleware).MiddlewareFunc.func1" line=71 method=POST path=/api/devices/v1/authentication/auth_requests qs= request_id=b2bcb1f6-9c89-4ebd-b680-f8160065eeaf responsetime=0.003445029 status=401 ts="2021-03-29 09:41:37.327445265 +0000 UTC" type=http
mender-device-auth_1             | time="2021-03-29T09:42:53Z" level=warning msg="dev auth: unauthorized: dev auth: unauthorized" file=response_helpers.go func=rest_utils.restErrWithLogMsg line=108 request_id=14aa7815-9dde-4911-b559-3f9fb58ec628
mender-device-auth_1             | time="2021-03-29T09:42:53Z" level=info msg="401 2732μs POST /api/devices/v1/authentication/auth_requests HTTP/1.1 - Go-http-client/1.1" byteswritten=111 file=middleware.go func="accesslog.(*AccessLogMiddleware).MiddlewareFunc.func1" line=71 method=POST path=/api/devices/v1/authentication/auth_requests qs= request_id=14aa7815-9dde-4911-b559-3f9fb58ec628 responsetime=0.002732346 status=401 ts="2021-03-29 09:42:53.717053471 +0000 UTC" type=http
mender-device-auth_1             | time="2021-03-29T09:47:44Z" level=warning msg="dev auth: unauthorized: dev auth: unauthorized" file=response_helpers.go func=rest_utils.restErrWithLogMsg line=108 request_id=38048f1d-d3c3-4c01-90bc-237bd930d1eb
mender-device-auth_1             | time="2021-03-29T09:47:44Z" level=info msg="401 2950μs POST /api/devices/v1/authentication/auth_requests HTTP/1.1 - Go-http-client/1.1" byteswritten=109 file=middleware.go func="accesslog.(*AccessLogMiddleware).MiddlewareFunc.func1" line=71 method=POST path=/api/devices/v1/authentication/auth_requests qs= request_id=38048f1d-d3c3-4c01-90bc-237bd930d1eb responsetime=0.002950922 status=401 ts="2021-03-29 09:47:44.509152474 +0000 UTC" type=http
mender-device-auth_1             | time="2021-03-29T09:52:44Z" level=warning msg="dev auth: unauthorized: dev auth: unauthorized" file=response_helpers.go func=rest_utils.restErrWithLogMsg line=108 request_id=4553c4dd-724d-45ac-ae25-038d703fb604
mender-device-auth_1             | time="2021-03-29T09:52:44Z" level=info msg="401 3129μs POST /api/devices/v1/authentication/auth_requests HTTP/1.1 - Go-http-client/1.1" byteswritten=109 file=middleware.go func="accesslog.(*AccessLogMiddleware).MiddlewareFunc.func1" line=71 method=POST path=/api/devices/v1/authentication/auth_requests qs= request_id=4553c4dd-724d-45ac-ae25-038d703fb604 responsetime=0.003129707 status=401 ts="2021-03-29 09:52:44.562165479 +0000 UTC" type=http

Thank you for you time
Best regards

Hi @zumiani

Is it possible this device is somehow in the “rejected” devices tab?

Alternatively, can you use the API to enumerate the devices? I think there was an issue recently where the web UI was not showing everything. @tranchitella is this possibly the same issue in an on-prem install?

Drew

Dear @drewmoseley ,

it was due to not completely supported version of MongoDB.
Following guide on Mender docs it install MongoDB 4.4.4.

I’ve deleted installation, included docker volumes, and start again installation.
Before ./run pull command I’ve edited docker-compose.yml file changing line from

mongo:4.4.4

with

mongo:4.4.3

Finished installation and it works like a charm.

Best regards

Excellent. Thanks for reporting back.

I confirm there seems to be an issue with Mender 2.6 and MongoDB 4.4.4, released a few days ago.
I suggest downgrading to MongoDB 4.4.3 if you are encountering issues.

Thank you very much @tranchitella for your help!
Your support is always very helpful and kind

@tranchitella does this problem still exist for MongoDB 4.4.6? We’re seeing this issue - mender-cli shows a device as pending, but the UI doesn’t show any (and I dug through all the device filter tabs).

@mho which version of Mender are you using? Latest versions (3.0, 3.1) are not affected by this issue.

We’re using 3.0.0 with MongoDB 4.4.6. We had to authorize a new device using the REST API. Could that be something different?

@gigilibala for more details

@mho I have same issue right now(mender 3.0.0).
Workaround:
go to devices tab
“Quick find Device” your device(I am using eeprom-stored SerialNumber as identy)
change Status filter to any

1 Like

Yeah, the device shows up when listing it through REST API or using mender-cli, but it won’t show up anywhere in the UI. I had to accept the device using the REST API manually, then it showed up in the UI. So basically pending devices won’t show up in the GUI.

@mho and @gigilibala based on the description I could think that the steps listed in here: Mender 2.5.0 Problems with Filters, Hostnames and More - #2 by mzedel might also apply in your case.

@gigilibala Im experiencing the same Problems with Mender 3.0 and Mongo DB 4.4.8. Could you solve the problem already?

Hi,
Is there a ticket tracking this issue? Has it been resolved in a recent release? We are still seeing this with Mender 3.1.0.

Dear Tranchitella,

We are observing a weird behaviour of devices not showing up in the Mender UI if there are more than 10 devices pending for acceptance. We have tested basic functionality with one device(registered/accepted installaed/updated via Mender), took an image-dump and provisioned a set of 15 devices. Added one after another but once we reach 10 devices, even if we add new device they are not showing in the UI.
The no of pending devices count got stuck at 10
Just wondering what are the basic things we need to check in these kind of scenarios.

Are you experiencing this behavior on Hosted Mender or running your own server?

@tranchitella Sorry for a late reply. We are facing the issue with self-hosted mender running on GKE. API response api/management/v2/devauth/devices?status=pending is giving a count of 20, but UI is showing only 10 pending devices.