Mender-Deployments and S3 Bucket

Dear Mender Community,

I’ve been trying to setup a mender service managed by rancher following this https://hackernoon.com/deploy-mender-on-digitalocean-using-rancher-container-management-platform-5d844a8cdc95 tutorial on DO.

After solving a lot issues with SSL certificates and config setup files, going through the documentation and git repo’s to find the information needed, I’m finding myself in need of some like-minded people who could help me.

Our mender-deployments containers keep failing to start with the following error:
5-7-2019 16:27:47FATA[2019-07-05T14:27:47Z] BucketAlreadyExists: Bucket already exists
5-7-2019 16:27:47 status code: 409, request id: file=proc.go func=runtime.main line=185

The error itself is obvious and setting everything up with a non existing bucket name creates the new bucket on startup and the container is able to run.

The problem arises as we already have a existing bucket! We don’t want to create new bucket.
Also we tested taking the container down as a failure test and then restart and we get the same error again.

I have been looking if there was a flag that would make the container create a new bucket when it was non existant but it seems to always create a new one by default.

Can somebody give me some more information about this case or point me in the right direction?

Kind regards,
MazeFX

1 Like

I’m seeing a very similar issue. I’ve deployed mender 3.1 self-hosted to a kubernetes cluster using the helm charts you provided. Its running on an openstack cloud with ceph object storage in place of s3.

If the s3 bucket does not already exist then mender server starts up okay and it is possible to upload artifacts and use them successfully in deployments.

However if the mender server is uninstalled form the cluster and then re-installed again; or if the deployment service is restarted for any other reason; then it fails to restart, generating this log:

$ kubectl logs deployments-6cbc8fb5c7-r277t
time="2021-11-02T15:52:29Z" level=info msg="'presign.secret' not configured. Generating a random secret." file=main.go func=main.doMain.func1 line=98
time="2021-11-02T15:52:29Z" level=info msg="Deployments Service starting up" file=main.go func=main.cmdServer line=137
time="2021-11-02T15:52:29Z" level=info msg="automigrate is ON, will apply migrations" file=migrations.go func=mongo.Migrate line=48
time="2021-11-02T15:52:29Z" level=info msg="migrating deployment_service" file=migrations.go func=mongo.MigrateSingle line=70
time="2021-11-02T15:52:29Z" level=info msg="migration to version 1.2.1 skipped" db=deployment_service file=migrator_simple.go func="migrate.(*SimpleMigrator).Apply" line=125
time="2021-11-02T15:52:29Z" level=info msg="migration to version 1.2.2 skipped" db=deployment_service file=migrator_simple.go func="migrate.(*SimpleMigrator).Apply" line=125
time="2021-11-02T15:52:29Z" level=info msg="migration to version 1.2.3 skipped" db=deployment_service file=migrator_simple.go func="migrate.(*SimpleMigrator).Apply" line=125
time="2021-11-02T15:52:29Z" level=info msg="migration to version 1.2.4 skipped" db=deployment_service file=migrator_simple.go func="migrate.(*SimpleMigrator).Apply" line=125
time="2021-11-02T15:52:29Z" level=info msg="migration to version 1.2.5 skipped" db=deployment_service file=migrator_simple.go func="migrate.(*SimpleMigrator).Apply" line=125
time="2021-11-02T15:52:29Z" level=info msg="migration to version 1.2.6 skipped" db=deployment_service file=migrator_simple.go func="migrate.(*SimpleMigrator).Apply" line=125
time="2021-11-02T15:52:29Z" level=info msg="DB migrated to version 1.2.6" db=deployment_service file=migrator_simple.go func="migrate.(*SimpleMigrator).Apply" line=140
BucketAlreadyExists: 
	status code: 409, request id: ****************-default, host id: 

It then continuously restarts, entering a crash backoff loop.

Hi, I’ve also had this issue and I’ve just sent a pull request with a fix.

It should be fixed when they merge it.

1 Like