Firmware deployment failed due to failed signature verification

General Discussions
, ,
1

Hello, I’m trying to deploy a recently built and signed firmware image to a imx6ull embedded device, but am running into issues during deployment through Mender and which looks to be a case of a signature verification error. This only seems to be the issue with firmware images built recently, as I can deploy images that were built back in June/July with no issue.

Here is a log file of a failed deployment: gist

I have attempted to verify the recently built/signed artifacts using the mender-artifact utility which is version 2.4.0, with the public.key file:

./mender-artifact read artifact-signed-image.mender -k public.key
Mender artifact:
  Name: image
  Format: mender
  Version: 2
  Signature: signed and verified correctly
  Compatible devices: '[device]'
  State scripts:

Updates:
    0:
    Type:   rootfs-image
    Files:
      name:     image-imx6ull_device.ext4
      size:     369098752
      modified: 2021-10-15 04:49:14 -0700 PDT
      checksum: 56db8d59da430e7ff2adad855105ac2f0b1ecfa421297c04eb3bc95382ead691

and with artifact-verify-key.pem (I’m actually not sure if this is supposed to work as it is a different file format and all, thought I would include it):

$ ./mender-artifact read artifact-signed-image.mender -k artifact-verify-key.pem
Mender artifact:
  Name: image
  Format: mender
  Version: 2
  Signature: signed; verification using provided key failed
  Compatible devices: '[device]'
  State scripts:

Updates:
    0:
    Type:   rootfs-image
    Files:
      name:     image-imx6ull_device.ext4
      size:     369098752
      modified: 2021-10-05 14:59:45 -0700 PDT
      checksum: 5c09bcdd6428672d0ad5b47c3f23690c62d785bfec2c0a6222415658441770a6

You can see that signature verification fails with the artifact-verify-key.pem which is present on the device but looking into both the public.key file and artifact-verify-key.pem file they are exactly the same (besides file formats). And from what I understand, the artifact-verify-key.pem is all that is used to verify firmware images on the device.

Is there something I am missing in regards to why this isn’t working?

Thank you.

Resources:
https://docs.mender.io/artifact-creation/sign-and-verify
https://docs.mender.io/artifact-creation/modify-an-artifact

2

These files are supposed to copies of each other, in other words identical. If the artifact-verify-key.pem is a different format, then a mistake has been made along the way. You should be able to simply copy public.key to artifact-verify-key.pem.

3

Hi kacf, thank you for the reply. Other than the differing file extensions the contents of both public.key and artifact-verify-key.pem are exactly the same.

I’ve checked with sha256:

$ sha256sum public.key > publickey.sha256
$ sha256sum artifact-verify-key.pem | sha256sum --check publickey.sha256
public.key: OK
4

Then I don’t understand your initial comment:

You said it was a different format. If you try with the keys you have now, does it still not work?

For the record, I tried this locally, and for me both keys work.

5

Sorry, I’m not super well-versed with verification/signing/cryptographic keys, didn’t realize that .key and .pem files extensions are the same. I have verified that the contents of both artifact-verify-key.pem and public.key are the same as well as checked the hashes with sha256

Can you think of any reason why recently built/signed images don’t deploy when images built/signed a few months earlier in the same manner with the same keys would work?

6

Let’s verify a few things here:

  1. The old images that you can still deploy, do they verify correctly with mender-artifact read -k public.key?
  2. The new images that you are producing which fail to deploy, do they verify correctly with the same command?
  3. Have you checked that the file which is mentioned in the ArtifactVerifyKey setting in /etc/mender/mender.conf has the same sha256sum?