i.MX 6ULL EVK vs demo server

sjuengst · June 1, 2020, 4:00pm

Hi, I’m trying to bring up the i.MX 6ULL EVK client using Brian Lee’s PR found here: meta-mender-community/meta-mender-nxp-imx6ull

I’ve got it built and running, but when I go to add the device to the demo server, the server says to run this command on the device: mender setup --device-type $DEVICE_TYPE --quiet --demo --server-ip 192.168.2.77. When I try this on the device, I get this error message: ERRO[0000] Must give one of -rootfs, -commit, -bootstrap or -daemon arguments module=main

I noticed that the version of the client in that PR is 1.7.1 and the server is 2.3.0. Have the mender commandline options changed between these two versions? I tried changing the client .bb files to use the latest client that the PR supports (2.1.0b1), but get the same error.

Anyone have a suggestion on how to proceed?

Thanks,

Stu

drewmoseley · June 1, 2020, 9:05pm

Hi @sjuengst welcome to Mender hub and good to see you again.

The “mender setup” subcommand is definitely newer than 1.7. I think it was added in 2.2.

But if you are integrating with Yocto and setting up the device type and server URL using standard bitbake variables then you should not need to run “mender setup”.

In @brianPTL’s post search for “Configure Mender Server URL” to see how to do that if you don’t have it setup yet.

Drew

sjuengst · June 5, 2020, 9:31pm

Hi @drewmoseley, I tried setting the server URL and device type with bitbake variables, but still didn’t get any interaction between the client and server. I went ahead and created a .bb file to pull in the latest v2.2.0 client code and that worked better. After doing client setup, the server saw the device and I was able to authorize it, add it to a group, create a release, and a deployment. Now the server is just sitting there with the deployment in “pending” state. Some other symptoms: The device detail screen says “Last check-in” was two hours ago, and it seems to be unable to fetch inventory data from device (just shows cycling dots). The device can ping the server. Any suggestions?

Stu

drewmoseley · June 8, 2020, 3:18pm

Do you have a login to the client? What does “journalctl -u mender” show?

Drew

sjuengst · June 8, 2020, 3:48pm

It looks like a problem with the server’s X.509 cert?

Jun 08 15:42:35 imx6ull14x14evk mender[6779]: time=“2020-06-08T15:42:35Z” level=info msg=“State transition: authorize [Sync] -> authorize-wait [Idle]” module=mender

Jun 08 15:43:04 imx6ull14x14evk mender[6779]: time=“2020-06-08T15:43:04Z” level=info msg=“State transition: authorize-wait [Idle] -> authorize [Sync]” module=mender

Jun 08 15:43:05 imx6ull14x14evk mender[6779]: time=“2020-06-08T15:43:05Z” level=error msg=“Failure occurred while executing authorization request: &url.Error{Op:“Post”, URL:“https://192.168.2.77/api/devices/v1/authentication/auth_requests”, Err:x509.HostnameError{Certificate:(*x509.Certificate)(0x1208c280), Host:“192.168.2.77”}}” module=client_auth

Jun 08 15:43:05 imx6ull14x14evk mender[6779]: time=“2020-06-08T15:43:05Z” level=error msg=“authorization request error: x509: cannot validate certificate for 192.168.2.77 because it doesn’t contain any IP SANs” module=client_auth

Jun 08 15:43:05 imx6ull14x14evk mender[6779]: time=“2020-06-08T15:43:05Z” level=error msg=“authorize failed: transient error: authorization request failed: generic error occurred while executing authorization request: Post https://192.168.2.77/api/devices/v1/authentication/auth_requests: x509: cannot validate certificate for 192.168.2.77 because it doesn’t contain any IP SANs” module=state

drewmoseley · June 8, 2020, 6:09pm

@kacf @lluiscampos any thoughts here?

sjuengst · June 8, 2020, 8:44pm

Googling this error reveals that the cert needs to include the IP address if you’re using an IP address as the URL. I can rebuild a cert to support this, but am confused by your documentation on this (https://docs.mender.io/2.3/administration/certificates-and-keys#replacing-keys-and-certificates) where it says

"With this knowledge, all the required keys and certificates can be generated by running:

CERT_API_CN=docker.mender.io CERT_STORAGE_CN=s3.docker.mender.io ./keygen

When I’m running your demo server on my own Ubuntu box at 192.178.2.77, what URLs should I use for CERT_API_CN and CERT_STORAGE_CN? 192.168.2.77 for both?

kacf · June 9, 2020, 6:25am

@sjuengst, can you post your /etc/mender/mender.conf file? I suspect that ServerURL is set to an IP instead of docker.mender.io, which is what it should be. This also means that there needs to be an entry in /etc/hosts though:

192.168.2.77 docker.mender.io

The --server-ip is supposed to take care of all this, but maybe something got messed up along the way. I think that creating your own keys and certificates is not necessary, and will in fact make it more difficult to run the demo.

sjuengst · June 9, 2020, 7:46pm

@kacf Yes, I had changed the ServerURL to the IP address thinking that was correct. After I changed it back to docker.mender.io, pointed the ServerCertificate to /etc/mender/server.crt, and added both docker.mender.io and s3.docker.mender.io to my /etc/hosts file, it began to work. It downloaded my deployment ok, but something failed in the Commit stage and it rolled back to the prior release. It looks like it tried to talk to somebody at IP 23.202.231.169:

2020-06-09 18:43:02 +0000 UTC info: Running Mender version 8a3bc7f
2020-06-09 18:43:02 +0000 UTC debug: handle update fetch state
2020-06-09 18:43:02 +0000 UTC debug: status reported, response 204 No Content
2020-06-09 18:43:02 +0000 UTC debug: Received fetch update response &{200 OK 200 HTTP/1.1 1 1 map[X-Xss-Protection:[1; mode=block] Content-Length:[26274816] Vary:[Origin] Last-Modified:[Tue, 09 Jun 2020 18:13:18 GMT] Content-Type:[application/vnd.mender-artifact] Etag:["80979f5f3f21e090db7445a6b17acd7a"] X-Amz-Request-Id:[1616F4593B75E454] X-Content-Type-Options:[nosniff] Date:[Tue, 09 Jun 2020 18:43:02 GMT] Connection:[keep-alive] Accept-Ranges:[bytes] Content-Security-Policy:[block-all-mixed-content] Strict-Transport-Security:[max-age=0; includeSubdomains; preload] X-Frame-Options:[DENY] Server:[openresty/1.13.6.2]] 0x1125e620 26274816 [] false false map[] 0x1117ae00 0x111b98c0}+
2020-06-09 18:43:02 +0000 UTC info: State transition: update-fetch [Download_Enter] -> update-store [Download_Enter]
2020-06-09 18:43:02 +0000 UTC debug: handle update install state
2020-06-09 18:43:02 +0000 UTC debug: status reported, response 204 No Content
2020-06-09 18:43:02 +0000 UTC debug: Read data from device manifest file: device_type=imx6ull14x14evk
2020-06-09 18:43:02 +0000 UTC debug: Current manifest data: imx6ull14x14evk
2020-06-09 18:43:02 +0000 UTC info: no public key was provided for authenticating the artifact
2020-06-09 18:43:02 +0000 UTC debug: checking if device [imx6ull14x14evk] is on compatible device list: [imx6ull14x14evk]
2020-06-09 18:43:02 +0000 UTC debug: installer: successfully read artifact [name: release-2; version: 2; compatible devices: [imx6ull14x14evk]]
2020-06-09 18:43:02 +0000 UTC debug: Active partition: /dev/mmcblk1p2
2020-06-09 18:43:02 +0000 UTC debug: Detected inactive partition /dev/mmcblk1p3, based on active partition /dev/mmcblk1p2
2020-06-09 18:43:02 +0000 UTC info: opening device /dev/mmcblk1p3 for writing
2020-06-09 18:43:02 +0000 UTC debug: Open block-device for installing update of size: 3867148288
2020-06-09 18:43:02 +0000 UTC debug: Device: /dev/mmcblk1p3 is a ubi device: false
2020-06-09 18:43:02 +0000 UTC info: native sector size of block device /dev/mmcblk1p3 is 512, we will write in chunks of 1048576
2020-06-09 18:43:02 +0000 UTC debug: Opening device: /dev/mmcblk1p3 for writing with flag: 2
2020-06-09 18:58:09 +0000 UTC info: 0 bytes remaining to be written
2020-06-09 18:58:10 +0000 UTC info: The optimized block-device writer wrote a total of 3689 frames, where 119 frames did need to be rewritten
2020-06-09 18:58:10 +0000 UTC info: Wrote 3867148288/3867148288 bytes to the inactive partition
2020-06-09 18:58:10 +0000 UTC debug: status reported, response 204 No Content
2020-06-09 18:58:10 +0000 UTC info: State transition: update-store [Download_Enter] -> update-after-store [Download_Leave]
2020-06-09 18:58:10 +0000 UTC info: State transition: update-after-store [Download_Leave] -> update-install [ArtifactInstall]
2020-06-09 18:58:10 +0000 UTC debug: statescript: timeout for executing scripts is not defined; using default of 1h0m0s seconds
2020-06-09 18:58:11 +0000 UTC debug: statescript: timeout for executing scripts is not defined; using default of 1h0m0s seconds
2020-06-09 18:58:11 +0000 UTC debug: status reported, response 204 No Content
2020-06-09 18:58:11 +0000 UTC debug: Inactive partition: /dev/mmcblk1p3
2020-06-09 18:58:11 +0000 UTC debug: Marking inactive partition (/dev/mmcblk1p3) as the new boot candidate.
2020-06-09 18:58:11 +0000 UTC info: Enabling partition with new image installed to be a boot candidate: 3
2020-06-09 18:58:11 +0000 UTC debug: Have U-Boot variable: mender_check_saveenv_canary=1
2020-06-09 18:58:11 +0000 UTC debug: List of U-Boot variables:map[mender_check_saveenv_canary:1]
2020-06-09 18:58:11 +0000 UTC debug: Have U-Boot variable: mender_saveenv_canary=1
2020-06-09 18:58:11 +0000 UTC debug: List of U-Boot variables:map[mender_saveenv_canary:1]
2020-06-09 18:58:11 +0000 UTC debug: Marking inactive partition as a boot candidate successful.
2020-06-09 18:58:11 +0000 UTC info: State transition: update-install [ArtifactInstall] -> reboot [ArtifactReboot_Enter]
2020-06-09 18:58:11 +0000 UTC debug: statescript: timeout for executing scripts is not defined; using default of 1h0m0s seconds
2020-06-09 18:58:11 +0000 UTC debug: statescript: timeout for executing scripts is not defined; using default of 1h0m0s seconds
2020-06-09 18:58:11 +0000 UTC debug: handling reboot state
2020-06-09 18:58:11 +0000 UTC debug: status reported, response 204 No Content
2020-06-09 18:58:11 +0000 UTC info: rebooting device(s)
2020-06-09 18:58:11 +0000 UTC info: Mender rebooting from active partition: /dev/mmcblk1p2
2020-06-09 18:58:35 +0000 UTC info: Running Mender version 8a3bc7f
2020-06-09 18:58:35 +0000 UTC info: State transition: init [none] -> after-reboot [ArtifactReboot_Leave]
2020-06-09 18:58:35 +0000 UTC debug: Have U-Boot variable: mender_check_saveenv_canary=1
2020-06-09 18:58:35 +0000 UTC debug: List of U-Boot variables:map[mender_check_saveenv_canary:1]
2020-06-09 18:58:35 +0000 UTC debug: Have U-Boot variable: mender_saveenv_canary=1
2020-06-09 18:58:35 +0000 UTC debug: List of U-Boot variables:map[mender_saveenv_canary:1]
2020-06-09 18:58:35 +0000 UTC debug: Have U-Boot variable: upgrade_available=1
2020-06-09 18:58:35 +0000 UTC debug: List of U-Boot variables:map[upgrade_available:1]
2020-06-09 18:58:35 +0000 UTC info: State transition: after-reboot [ArtifactReboot_Leave] -> after-reboot [ArtifactReboot_Leave]
2020-06-09 18:58:35 +0000 UTC debug: handling state after reboot
2020-06-09 18:58:35 +0000 UTC info: State transition: after-reboot [ArtifactReboot_Leave] -> update-commit [ArtifactCommit_Enter]
2020-06-09 18:58:35 +0000 UTC debug: statescript: timeout for executing scripts is not defined; using default of 1h0m0s seconds
2020-06-09 18:58:35 +0000 UTC debug: statescript: timeout for executing scripts is not defined; using default of 1h0m0s seconds
2020-06-09 18:58:35 +0000 UTC debug: handle update commit state
2020-06-09 18:58:40 +0000 UTC error: failed to report status: Put https://docker.mender.io/api/devices/v1/deployments/device/deployments/e004b379-b50d-4121-84a5-fb4c1ef36bf8/status: dial tcp 23.217.138.110:443: getsockopt: connection refused
2020-06-09 18:58:40 +0000 UTC error: error reporting update status: reporting status failed: Put https://docker.mender.io/api/devices/v1/deployments/device/deployments/e004b379-b50d-4121-84a5-fb4c1ef36bf8/status: dial tcp 23.217.138.110:443: getsockopt: connection refused
2020-06-09 18:58:40 +0000 UTC error: Failed to send status report to server: transient error: reporting status failed: Put https://docker.mender.io/api/devices/v1/deployments/device/deployments/e004b379-b50d-4121-84a5-fb4c1ef36bf8/status: dial tcp 23.217.138.110:443: getsockopt: connection refused
2020-06-09 18:58:40 +0000 UTC info: State transition: update-commit [ArtifactCommit_Enter] -> update-pre-commit-status-report-retry [ArtifactCommit_Enter]
2020-06-09 18:59:10 +0000 UTC debug: wait complete
2020-06-09 18:59:10 +0000 UTC info: State transition: update-pre-commit-status-report-retry [ArtifactCommit_Enter] -> update-commit [ArtifactCommit_Enter]
2020-06-09 18:59:10 +0000 UTC debug: handle update commit state
2020-06-09 18:59:10 +0000 UTC error: failed to report status: Put https://docker.mender.io/api/devices/v1/deployments/device/deployments/e004b379-b50d-4121-84a5-fb4c1ef36bf8/status: dial tcp 23.202.231.169:443: getsockopt: connection refused
2020-06-09 18:59:10 +0000 UTC error: error reporting update status: reporting status failed: Put https://docker.mender.io/api/devices/v1/deployments/device/deployments/e004b379-b50d-4121-84a5-fb4c1ef36bf8/status: dial tcp 23.202.231.169:443: getsockopt: connection refused
2020-06-09 18:59:10 +0000 UTC error: Failed to send status report to server: transient error: reporting status failed: Put https://docker.mender.io/api/devices/v1/deployments/device/deployments/e004b379-b50d-4121-84a5-fb4c1ef36bf8/status: dial tcp 23.202.231.169:443: getsockopt: connection refused
2020-06-09 18:59:10 +0000 UTC info: State transition: update-commit [ArtifactCommit_Enter] -> update-pre-commit-status-report-retry [ArtifactCommit_Enter]
2020-06-09 18:59:40 +0000 UTC debug: wait complete
2020-06-09 18:59:40 +0000 UTC info: State transition: update-pre-commit-status-report-retry [ArtifactCommit_Enter] -> update-commit [ArtifactCommit_Enter]
2020-06-09 18:59:40 +0000 UTC debug: handle update commit state
2020-06-09 18:59:40 +0000 UTC error: failed to report status: Put https://docker.mender.io/api/devices/v1/deployments/device/deployments/e004b379-b50d-4121-84a5-fb4c1ef36bf8/status: dial tcp 23.217.138.110:443: getsockopt: connection refused
2020-06-09 18:59:40 +0000 UTC error: error reporting update status: reporting status failed: Put https://docker.mender.io/api/devices/v1/deployments/device/deployments/e004b379-b50d-4121-84a5-fb4c1ef36bf8/status: dial tcp 23.217.138.110:443: getsockopt: connection refused
2020-06-09 18:59:40 +0000 UTC error: Failed to send status report to server: transient error: reporting status failed: Put https://docker.mender.io/api/devices/v1/deployments/device/deployments/e004b379-b50d-4121-84a5-fb4c1ef36bf8/status: dial tcp 23.217.138.110:443: getsockopt: connection refused
2020-06-09 18:59:40 +0000 UTC info: State transition: update-commit [ArtifactCommit_Enter] -> update-pre-commit-status-report-retry [ArtifactCommit_Enter]
2020-06-09 19:00:10 +0000 UTC debug: wait complete
2020-06-09 19:00:11 +0000 UTC info: State transition: update-pre-commit-status-report-retry [ArtifactCommit_Enter] -> update-commit [ArtifactCommit_Enter]
2020-06-09 19:00:11 +0000 UTC debug: handle update commit state
2020-06-09 19:00:11 +0000 UTC error: failed to report status: Put https://docker.mender.io/api/devices/v1/deployments/device/deployments/e004b379-b50d-4121-84a5-fb4c1ef36bf8/status: dial tcp 23.202.231.169:443: getsockopt: connection refused
2020-06-09 19:00:11 +0000 UTC error: error reporting update status: reporting status failed: Put https://docker.mender.io/api/devices/v1/deployments/device/deployments/e004b379-b50d-4121-84a5-fb4c1ef36bf8/status: dial tcp 23.202.231.169:443: getsockopt: connection refused
2020-06-09 19:00:12 +0000 UTC error: Failed to send status report to server: transient error: reporting status failed: Put https://docker.mender.io/api/devices/v1/deployments/device/deployments/e004b379-b50d-4121-84a5-fb4c1ef36bf8/status: dial tcp 23.202.231.169:443: getsockopt: connection refused
2020-06-09 19:00:12 +0000 UTC info: State transition: update-commit [ArtifactCommit_Enter] -> update-pre-commit-status-report-retry [ArtifactCommit_Enter]
2020-06-09 19:00:12 +0000 UTC error: transient error: Tried sending status report maximum number of times.
2020-06-09 19:00:12 +0000 UTC info: State transition: update-pre-commit-status-report-retry [ArtifactCommit_Enter] -> rollback [ArtifactRollback]
2020-06-09 19:00:12 +0000 UTC debug: transitioning to error state
2020-06-09 19:00:12 +0000 UTC debug: statescript: timeout for executing scripts is not defined; using default of 1h0m0s seconds
2020-06-09 19:00:12 +0000 UTC debug: statescript: timeout for executing scripts is not defined; using default of 1h0m0s seconds
2020-06-09 19:00:12 +0000 UTC info: performing rollback
2020-06-09 19:00:12 +0000 UTC debug: Have U-Boot variable: mender_check_saveenv_canary=1
2020-06-09 19:00:12 +0000 UTC debug: List of U-Boot variables:map[mender_check_saveenv_canary:1]
2020-06-09 19:00:12 +0000 UTC debug: Have U-Boot variable: mender_saveenv_canary=1
2020-06-09 19:00:12 +0000 UTC debug: List of U-Boot variables:map[mender_saveenv_canary:1]
2020-06-09 19:00:12 +0000 UTC debug: Have U-Boot variable: upgrade_available=1
2020-06-09 19:00:12 +0000 UTC debug: List of U-Boot variables:map[upgrade_available:1]
2020-06-09 19:00:12 +0000 UTC debug: Active partition: /dev/mmcblk1p3
2020-06-09 19:00:12 +0000 UTC debug: Detected inactive partition /dev/mmcblk1p2, based on active partition /dev/mmcblk1p3
2020-06-09 19:00:12 +0000 UTC debug: Marking inactive partition (/dev/mmcblk1p2) as the new boot candidate.
2020-06-09 19:00:12 +0000 UTC info: setting partition for rollback: 2
2020-06-09 19:00:12 +0000 UTC debug: Have U-Boot variable: mender_check_saveenv_canary=1
2020-06-09 19:00:12 +0000 UTC debug: List of U-Boot variables:map[mender_check_saveenv_canary:1]
2020-06-09 19:00:12 +0000 UTC debug: Have U-Boot variable: mender_saveenv_canary=1
2020-06-09 19:00:12 +0000 UTC debug: List of U-Boot variables:map[mender_saveenv_canary:1]
2020-06-09 19:00:12 +0000 UTC debug: Marking inactive partition as a boot candidate successful.
2020-06-09 19:00:12 +0000 UTC debug: will try to rollback reboot the device
2020-06-09 19:00:12 +0000 UTC info: State transition: rollback [ArtifactRollback] -> rollback-reboot [ArtifactRollbackReboot_Enter]
2020-06-09 19:00:12 +0000 UTC debug: statescript: timeout for executing scripts is not defined; using default of 1h0m0s seconds
2020-06-09 19:00:12 +0000 UTC debug: statescript: timeout for executing scripts is not defined; using default of 1h0m0s seconds
2020-06-09 19:00:12 +0000 UTC info: rebooting device(s) after rollback
2020-06-09 19:00:12 +0000 UTC info: Mender rebooting from inactive partition: /dev/mmcblk1p3
2020-06-09 19:00:33 +0000 UTC info: Running Mender version 8a3bc7f
2020-06-09 19:00:33 +0000 UTC info: Update was interrupted in state: rollback-reboot
2020-06-09 19:00:33 +0000 UTC info: State transition: init [none] -> verify-rollback-reboot [ArtifactRollbackReboot_Leave]
2020-06-09 19:00:33 +0000 UTC debug: transitioning to error state
2020-06-09 19:00:33 +0000 UTC debug: Have U-Boot variable: mender_check_saveenv_canary=1
2020-06-09 19:00:33 +0000 UTC debug: List of U-Boot variables:map[mender_check_saveenv_canary:1]
2020-06-09 19:00:34 +0000 UTC debug: Have U-Boot variable: mender_saveenv_canary=1
2020-06-09 19:00:34 +0000 UTC debug: List of U-Boot variables:map[mender_saveenv_canary:1]
2020-06-09 19:00:34 +0000 UTC debug: Have U-Boot variable: upgrade_available=0
2020-06-09 19:00:34 +0000 UTC debug: List of U-Boot variables:map[upgrade_available:0]
2020-06-09 19:00:34 +0000 UTC info: State transition: verify-rollback-reboot [ArtifactRollbackReboot_Leave] -> after-rollback-reboot [ArtifactRollbackReboot_Leave]
2020-06-09 19:00:34 +0000 UTC debug: handling state after rollback reboot
2020-06-09 19:00:34 +0000 UTC info: State transition: after-rollback-reboot [ArtifactRollbackReboot_Leave] -> update-error [ArtifactFailure]
2020-06-09 19:00:34 +0000 UTC debug: statescript: timeout for executing scripts is not defined; using default of 1h0m0s seconds
2020-06-09 19:00:34 +0000 UTC debug: statescript: timeout for executing scripts is not defined; using default of 1h0m0s seconds
2020-06-09 19:00:34 +0000 UTC debug: handle update error state
2020-06-09 19:00:34 +0000 UTC info: State transition: update-error [ArtifactFailure] -> cleanup [Error]
2020-06-09 19:00:34 +0000 UTC debug: statescript: timeout for executing scripts is not defined; using default of 1h0m0s seconds
2020-06-09 19:00:34 +0000 UTC debug: statescript: timeout for executing scripts is not defined; using default of 1h0m0s seconds
2020-06-09 19:00:34 +0000 UTC debug: Handling Cleanup state
2020-06-09 19:00:34 +0000 UTC info: State transition: cleanup [Error] -> update-status-report [none]
2020-06-09 19:00:34 +0000 UTC debug: statescript: timeout for executing scripts is not defined; using default of 1h0m0s seconds
2020-06-09 19:00:34 +0000 UTC debug: handle update status report state
2020-06-09 19:00:34 +0000 UTC debug: status reported, response 204 No Content
2020-06-09 19:00:34 +0000 UTC debug: attempting to upload deployment logs for failed update

kacf · June 10, 2020, 7:12am

Are you sure that both /etc/mender/mender.conf and /etc/hosts are correct in your update file as well? You can check with mender-artifact cat <UPDATE>.mender:/etc/hosts.

sjuengst · June 10, 2020, 8:05pm

@kacf I should have thought of that! It’s working now. Thanks for your help! You too @drewmoseley!

drewmoseley · June 10, 2020, 8:16pm

Excellent. Let us know if you have any more issues.

Topic		Replies	Views
[INCOMPLETE] i.MX 6ULL EVK Yocto Project rocko , nxp	4	4005	May 13, 2020
Device type compatibility Yocto Project	6	1171	April 29, 2022
Mender-connect unable to connect to server General Discussions yocto , mender-client	2	420	January 26, 2024
Mender server authorization error: demo.crt is missing in iMX 6UL EVK Yocto build General Discussions yocto , demo-server , imx6ul-evk	3	1498	May 29, 2020
i.MX6ULL Integration woes General Discussions rocko , nxp	46	10147	May 30, 2019

i.MX 6ULL EVK vs demo server

Related topics