# Server upgrade from 1.7 to 2.0, deployments container doesn't start

**URL:** https://hub.mender.io/t/server-upgrade-from-1-7-to-2-0-deployments-container-doesnt-start/627
**Category:** General Discussions
**Created:** [May 25, 2019, 10:02am UTC](https://hub.mender.io/t/server-upgrade-from-1-7-to-2-0-deployments-container-doesnt-start/627 "2019-05-25T10:02:02Z")
**Posts on this page:** 7
**Page:** 1

<div class="post-metadata">

### Author: ![mterwoord](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/mterwoord/32/100_2.png) [@mterwoord](https://hub.mender.io/u/mterwoord)
#### Post date: [May 25, 2019, 10:02am UTC](https://hub.mender.io/t/server-upgrade-from-1-7-to-2-0-deployments-container-doesnt-start/627/1 "2019-05-25T10:02:02Z")

</div>

Today, I decided was a good time to upgrade my production server from 1.7 to 2.0.0  
Everything went smooth, except for the deployments container not starting.

The logs show:

> mender-deployments\_1 | time=“2019-05-25T09:57:32Z” level=info msg=“DB migrated to version 1.2.1” db=deployment\_service file=migrator\_simple.go func=“migrate.(\*SimpleMigrator).Apply” line=140  
> mender-deployments\_1 | RequestError: send request failed  
> mender-deployments\_1 | caused by: Put https://[domain]:9000/mender-artifact-storage: dial tcp 172.24.0.7:9000: connect: connection refused  
> mender-deployments\_1 | time=“2019-05-25T09:58:33Z” level=info msg=“Deployments Service, version unknown starting up”  
> mender-deployments\_1 | time=“2019-05-25T09:58:33Z” level=info msg=“automigrate is ON, will apply migrations” file=migrations.go func=migrations.Migrate line=48  
> mender-deployments\_1 | time=“2019-05-25T09:58:33Z” level=info msg=“migrating deployment\_service” file=migrations.go func=migrations.MigrateSingle line=70  
> mender-deployments\_1 | time=“2019-05-25T09:58:33Z” level=info msg=“migration to version 1.2.1 skipped” db=deployment\_service file=migrator\_simple.go func=“migrate.(\*SimpleMigrator).Apply” line=125  
> mender-deployments\_1 | time=“2019-05-25T09:58:33Z” level=info msg=“DB migrated to version 1.2.1” db=deployment\_service file=migrator\_simple.go func=“migrate.(\*SimpleMigrator).Apply” line=140  
> mender-deployments\_1 | RequestError: send request failed  
> mender-deployments\_1 | caused by: Put https://[domain]:9000/mender-artifact-storage: dial tcp 172.24.0.7:9000: connect: connection refused

The ip address 172.24.0.7 points to the storage-proxy.

What am I missing here?

---

<div class="post-metadata">

### Author: ![0lmi](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/0lmi/32/133_2.png) [@0lmi](https://hub.mender.io/u/0lmi)
#### Post date: [May 27, 2019, 10:03am UTC](https://hub.mender.io/t/server-upgrade-from-1-7-to-2-0-deployments-container-doesnt-start/627/2 "2019-05-27T10:03:57Z")

</div>

Do you use Minio + storage proxy or S3? If you use Minio then can you please share logs from storage proxy and Minio containers?

---

<div class="post-metadata">

### Author: ![mterwoord](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/mterwoord/32/100_2.png) [@mterwoord](https://hub.mender.io/u/mterwoord)
#### Post date: [May 27, 2019, 10:24am UTC](https://hub.mender.io/t/server-upgrade-from-1-7-to-2-0-deployments-container-doesnt-start/627/3 "2019-05-27T10:24:31Z")

</div>

The storage proxy container doesn’t hae any logs available. minio has:

```
2019-05-27T00:07:04.499934018Z 
2019-05-27T00:07:04.650988345Z You are running an older version of Minio released 7 months ago 
2019-05-27T00:07:04.651012972Z Update: docker pull minio/minio:RELEASE.2019-05-23T00-29-34Z 
2019-05-27T00:07:04.651017870Z 
2019-05-27T00:07:04.651021666Z 
2019-05-27T00:07:05.084630594Z Endpoint: http://172.24.0.6:9000 http://127.0.0.1:9000
2019-05-27T00:07:05.084752289Z AccessKey: mender-deployments 
2019-05-27T00:07:05.084794493Z SecretKey: chaereegh0fohPoo6agh0phaiNg5hoox 
2019-05-27T00:07:05.084873066Z 
2019-05-27T00:07:05.084883511Z Browser Access:
2019-05-27T00:07:05.084961962Z http://172.24.0.6:9000 http://127.0.0.1:9000
2019-05-27T00:07:05.085022124Z 
2019-05-27T00:07:05.085031453Z Command-line Access: https://docs.minio.io/docs/minio-client-quickstart-guide
2019-05-27T00:07:05.085078979Z $ mc config host add myminio http://172.24.0.6:9000 mender-deployments chaereegh0fohPoo6agh0phaiNg5hoox
2019-05-27T00:07:05.085156152Z 
2019-05-27T00:07:05.085164865Z Object API (Amazon S3 compatible):
2019-05-27T00:07:05.085216262Z Go: https://docs.minio.io/docs/golang-client-quickstart-guide
2019-05-27T00:07:05.085276469Z Java: https://docs.minio.io/docs/java-client-quickstart-guide
2019-05-27T00:07:05.085325010Z Python: https://docs.minio.io/docs/python-client-quickstart-guide
2019-05-27T00:07:05.085340128Z JavaScript: https://docs.minio.io/docs/javascript-client-quickstart-guide
2019-05-27T00:07:05.085393892Z .NET: https://docs.minio.io/docs/dotnet-client-quickstart-guide
```

---

<div class="post-metadata">

### Author: ![0lmi](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/0lmi/32/133_2.png) [@0lmi](https://hub.mender.io/u/0lmi)
#### Post date: [May 27, 2019, 11:29am UTC](https://hub.mender.io/t/server-upgrade-from-1-7-to-2-0-deployments-container-doesnt-start/627/4 "2019-05-27T11:29:32Z")

</div>

Storage proxy should write access logs to stdout, so, if you don’t see any logs there it means it’s something wrong with network connection and `dial tcp 172.24.0.7:9000: connect: connection refused` log message says the same. Do you have firewalls configured?

---

<div class="post-metadata">

### Author: ![mterwoord](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/mterwoord/32/100_2.png) [@mterwoord](https://hub.mender.io/u/mterwoord)
#### Post date: [May 27, 2019, 11:51am UTC](https://hub.mender.io/t/server-upgrade-from-1-7-to-2-0-deployments-container-doesnt-start/627/5 "2019-05-27T11:51:02Z")

</div>

I dont have any firewalls configured inside the mender docker network. Of course there’s an external firewall in play, which runs outside of my host.

Any ideas on how to track this further?

I’ll try (will be few days, likely) to access from a different container inside the same network.

---

<div class="post-metadata">

### Author: ![0lmi](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/0lmi/32/133_2.png) [@0lmi](https://hub.mender.io/u/0lmi)
#### Post date: [May 27, 2019, 12:13pm UTC](https://hub.mender.io/t/server-upgrade-from-1-7-to-2-0-deployments-container-doesnt-start/627/6 "2019-05-27T12:13:43Z")

</div>

Further I would check from host system if storage proxy replays:

```auto
$ telnet 127.0.0.1 9000
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
^]
telnet> quit
Connection closed.

```

And it sounds weird to me that you don’t have any log entries in storage proxy logs because when nginx starts it should write at least [those](https://github.com/mendersoftware/integration/blob/master/storage-proxy/nginx.conf#L16-L17) messages about DOWNLOAD\_SPEED and MAX\_CONNECTIONS.

---

<div class="post-metadata">

### Author: ![mterwoord](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/mterwoord/32/100_2.png) [@mterwoord](https://hub.mender.io/u/mterwoord)
#### Post date: [May 28, 2019, 7:03am UTC](https://hub.mender.io/t/server-upgrade-from-1-7-to-2-0-deployments-container-doesnt-start/627/7 "2019-05-28T07:03:07Z")

</div>

I find the issue. Of course, after finding it was a stupid thing.

I missed adding the - ./storage-proxy/nginx.conf:/usr/local/openresty/nginx/conf/nginx.conf

line in the prod.yml file.

I had to do some tweaking with elasticsearch as well, as it suddenly took way more ram, but that’s all figured out as well…

@Alex, thanks for pointing me at the storage proxy and its lack of logging the initial message!
