Can not install Payload with SignatureDoesNotMatch

Im having a problem with mender-deployment uploading artefacts to Minio storage. From the Mender web GUI Im seeing the following error.

“Artifact couldn’t be uploaded. reading artifact error: Payload: can not install Payload: .ext4: io: read/write on closed pipe… [Request ID: 9df1ee4e]”

mender-deployment log shows the following
mender-deployments@mender-server | time=“2020-03-11T11:12:28Z” level=error msg=“reading artifact error: Payload: can not install Payload: .ext4: io: read/write on closed pipe: Cannot parse artifact file” file=api_deployments.go func=“http.(*DeploymentsApiHandlers).newImageWithContext” line=330 request_id=e7ac735a-41b2-4185-be1b-cb36a370d0f2 user_id=8cfd8d07-####-####-####-############
mender-deployments.@mender-server | time=“2020-03-11T11:12:28Z” level=error msg=“reading artifact error: Payload: can not install Payload: .ext4: io: read/write on closed pipe” file=view.go func=“view.(*RESTView).RenderError” line=53 request_id=e7ac735a-41b2-4185-be1b-cb36a370d0f2 user_id=8cfd8d07-####-####-####-############

And Minio shows
[REQUEST s3.PutObject] 12:14:20.461
PUT //b8570467-fc72-4061-85e9-8809d01bbb54?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=member1%2F20200311%2FLondon%2Fs3%2Faws4_request&X-Amz-Date=20200311T121419Z&X-Amz-Expires=300&X-Amz-SignedHeaders=host&X-Amz-Signature=########################################################
Host: :443
Content-Type: application/vnd.mender-artifact
User-Agent: Go-http-client/1.1
Accept-Encoding: gzip
Content-Length: 221542400

[RESPONSE] [12:14:20.461] [ Duration 577.578ms ↑ 60 B ↓ 785 B ]
403 Forbidden
Content-Security-Policy: block-all-mixed-content
Content-Type: application/xml
X-Xss-Protection: 1; mode=block
Accept-Ranges: bytes
Server: MinIO/RELEASE.2020-03-09T18-26-53Z
Vary: Origin
X-Amz-Bucket-Region: London
X-Amz-Request-Id: 15FB3EE7F2D08F18
Content-Length: 500
<?xml version="1.0" encoding="UTF-8"?>
SignatureDoesNotMatchThe request signature we calculated does not match the signature you provided. Check your key and signing method.b8570467-fc72-4061-85e9-8809d01bbb54//b8570467-fc72-4061-85e9-8809d01bbb54London15FB3EE7F2D08F183ff345bb-2115-4e11-a589-11ef50801487

[REQUEST s3.DeleteObject] 12:14:20.497
DELETE //b8570467-fc72-4061-85e9-8809d01bbb54
Host:
Accept-Encoding: gzip
Authorization: AWS4-HMAC-SHA256 Credential=s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date;x-amz-request-payer, Signature=########################################################
User-Agent: aws-sdk-go/1.12.27 (go1.11.4; linux; amd64)
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20200311T121420Z0
X-Amz-Request-Payer: requester

[RESPONSE] [12:14:20.497] [ Duration 590µs ↑ 113 B ↓ 254 B ]
204 No Content
Accept-Ranges: bytes
Content-Length: 0
Content-Security-Policy: block-all-mixed-content
Server: MinIO/RELEASE.2020-03-09T18-26-53Z
Vary: Origin
X-Amz-Bucket-Region: London
X-Amz-Request-Id: 15FB3EE81760AF82
X-Xss-Protection: 1; mode=block

Minio is a binary install on a separate server with external IP address and valid certificate. A user and group have been created on Minio for Mender to use. The policy for the group allows create permissions for mender artifact bucket and read and write permissions in the bucket.

The full Mender stack is installed with the exception of Minio and storage gateway, as minio is being treated like standard S3.

Hi @deligatedgeek,

Just to make sure that there is no errors in how the Mender Artifact was created, can you try running ,

mender-artifact read <the artifact you are uploading>

Edit:

and

mender-artifact validate <the artifact you are uploading>

Hi Mirzak,

3 days of yak shaving with my only success being to create a working “S3” bucket access policy.

I raised the ticket and immediately after my next test worked. The solution was to remove the DEPLOYMENTS_AWS_REGION variable from mender-deployment and the Minio server.

Artefacts now upload correctly and the meta data is displayed in the releases window.