# Mender-Deployments and S3 Bucket

**URL:** <https://hub.mender.io/t/mender-deployments-and-s3-bucket/796>\
**Category:** General Discussions\
**Created:** [July 5, 2019, 2:45pm UTC](https://hub.mender.io/t/mender-deployments-and-s3-bucket/796 "2019-07-05T14:45:28Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![MazeFX](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/mazefx/32/291_2.png) [@MazeFX](https://hub.mender.io/u/MazeFX)\
**Post date:** [July 5, 2019, 2:45pm UTC](https://hub.mender.io/t/mender-deployments-and-s3-bucket/796/1 "2019-07-05T14:45:28Z")

</div>

Dear Mender Community,

I’ve been trying to setup a mender service managed by rancher following this [https://hackernoon.com/deploy-mender-on-digitalocean-using-rancher-container-management-platform-5d844a8cdc95](https://hackernoon.com/deploy-mender-on-digitalocean-using-rancher-container-management-platform-5d844a8cdc95) tutorial on DO.

After solving a lot issues with SSL certificates and config setup files, going through the documentation and git repo’s to find the information needed, I’m finding myself in need of some like-minded people who could help me.

Our mender-deployments containers keep failing to start with the following error:  
5-7-2019 16:27:47FATA[2019-07-05T14:27:47Z] BucketAlreadyExists: Bucket already exists  
5-7-2019 16:27:47 status code: 409, request id: file=proc.go func=runtime.main line=185

The error itself is obvious and setting everything up with a non existing bucket name creates the new bucket on startup and the container is able to run.

The problem arises as we already have a existing bucket! We don’t want to create new bucket.  
Also we tested taking the container down as a failure test and then restart and we get the same error again.

I have been looking if there was a flag that would make the container create a new bucket when it was non existant but it seems to always create a new one by default.

Can somebody give me some more information about this case or point me in the right direction?

Kind regards,  
MazeFX

---

<div class="post-metadata">

**Author:** ![matt](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/matt/32/998_2.png) [@matt](https://hub.mender.io/u/matt)\
**Post date:** [November 2, 2021, 4:18pm UTC](https://hub.mender.io/t/mender-deployments-and-s3-bucket/796/2 "2021-11-02T16:18:15Z")

</div>

I’m seeing a very similar issue. I’ve deployed mender 3.1 self-hosted to a kubernetes cluster using the helm charts you provided. Its running on an openstack cloud with ceph object storage in place of s3.

If the s3 bucket does not already exist then mender server starts up okay and it is possible to upload artifacts and use them successfully in deployments.

However if the mender server is uninstalled form the cluster and then re-installed again; or if the deployment service is restarted for any other reason; then it fails to restart, generating this log:

```auto
$ kubectl logs deployments-6cbc8fb5c7-r277t
time="2021-11-02T15:52:29Z" level=info msg="'presign.secret' not configured. Generating a random secret." file=main.go func=main.doMain.func1 line=98
time="2021-11-02T15:52:29Z" level=info msg="Deployments Service starting up" file=main.go func=main.cmdServer line=137
time="2021-11-02T15:52:29Z" level=info msg="automigrate is ON, will apply migrations" file=migrations.go func=mongo.Migrate line=48
time="2021-11-02T15:52:29Z" level=info msg="migrating deployment_service" file=migrations.go func=mongo.MigrateSingle line=70
time="2021-11-02T15:52:29Z" level=info msg="migration to version 1.2.1 skipped" db=deployment_service file=migrator_simple.go func="migrate.(*SimpleMigrator).Apply" line=125
time="2021-11-02T15:52:29Z" level=info msg="migration to version 1.2.2 skipped" db=deployment_service file=migrator_simple.go func="migrate.(*SimpleMigrator).Apply" line=125
time="2021-11-02T15:52:29Z" level=info msg="migration to version 1.2.3 skipped" db=deployment_service file=migrator_simple.go func="migrate.(*SimpleMigrator).Apply" line=125
time="2021-11-02T15:52:29Z" level=info msg="migration to version 1.2.4 skipped" db=deployment_service file=migrator_simple.go func="migrate.(*SimpleMigrator).Apply" line=125
time="2021-11-02T15:52:29Z" level=info msg="migration to version 1.2.5 skipped" db=deployment_service file=migrator_simple.go func="migrate.(*SimpleMigrator).Apply" line=125
time="2021-11-02T15:52:29Z" level=info msg="migration to version 1.2.6 skipped" db=deployment_service file=migrator_simple.go func="migrate.(*SimpleMigrator).Apply" line=125
time="2021-11-02T15:52:29Z" level=info msg="DB migrated to version 1.2.6" db=deployment_service file=migrator_simple.go func="migrate.(*SimpleMigrator).Apply" line=140
BucketAlreadyExists: 
	status code: 409, request id: **************** -default, host id: 

```

It then continuously restarts, entering a crash backoff loop.

---

<div class="post-metadata">

**Author:** ![emrodre](https://avatars.discourse-cdn.com/v4/letter/e/57b2e6/32.png) [@emrodre](https://hub.mender.io/u/emrodre)\
**Post date:** [December 7, 2021, 12:48pm UTC](https://hub.mender.io/t/mender-deployments-and-s3-bucket/796/3 "2021-12-07T12:48:43Z")

</div>

Hi, I’ve also had this issue and I’ve just sent a pull request with a fix.

> <https://github.com/mendersoftware/deployments/pull/653>
>
> Object storage platforms based on Ceph (https://ceph.io), like DigitalOcean Spac…es, have an issue (https://tracker.ceph.com/issues/17398) where the function CreateBucket() does not return a ErrCodeBucketAlreadyOwnedByYou code.
> 
> This causes the following behaviour:
> 1. If the bucket does not exists it is created successfully.
> 2. If the bucket already exists it returns a 409 BucketAlreadyExists response and crashes so it's impossible to run the service.
> 
> The function HeadBucket() returns successfully if the bucket exists and the user has permissions so it can be used to check if the bucket exists and do an early return to avoid calling CreateBucket() when it already exists.
> 
> Also, there is a second possible corner case with calling CreateBucket() without checking if the bucket exists with HeadBucket() first as stated in the AWS SDK docs (https://pkg.go.dev/github.com/aws/aws-sdk-go@v1.40.37/service/s3?utm\_source=gopls#S3.CreateBucket):
> 
> \> \* ErrCodeBucketAlreadyOwnedByYou "BucketAlreadyOwnedByYou"
> \> The bucket you tried to create already exists, and you own it. Amazon S3
> \> returns this error in all Amazon Web Services Regions except in the North
> \> Virginia Region. For legacy compatibility, if you re-create an existing bucket
> \> that you already own in the North Virginia Region, Amazon S3 returns 200
> \> OK and resets the bucket access control lists (ACLs).

It should be fixed when they merge it.
