# Mender demo server sudden slow and stuck

**URL:** <https://hub.mender.io/t/mender-demo-server-sudden-slow-and-stuck/2043>\
**Category:** General Discussions\
**Created:** [June 15, 2020, 6:36am UTC](https://hub.mender.io/t/mender-demo-server-sudden-slow-and-stuck/2043 "2020-06-15T06:36:44Z")\
**Posts on this page:** 20\
**Page:** 2

<div class="post-metadata">

**Author:** ![kacf](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/kacf/32/146_2.png) [@kacf](https://hub.mender.io/u/kacf)\
**Post date:** [June 22, 2020, 8:57am UTC](https://hub.mender.io/t/mender-demo-server-sudden-slow-and-stuck/2043/21 "2020-06-22T08:57:58Z")

</div>

There is nothing special about the “inconsistent” state, it just means that the previous update did not finish successfully, and that the Update Module does not provide rollback. Deploying with Mender works exactly the same way afterwards.

If it’s not working for you, then it’s likely that the failed deployment broke the installation somehow. Perhaps connectivity was affected, or ability to install new files. Your best bet is probably to look at the log of the failed deployment to try to figure out what went wrong in the first place.

---

<div class="post-metadata">

**Author:** ![nishad1092](https://avatars.discourse-cdn.com/v4/letter/n/5fc32e/32.png) [@nishad1092](https://hub.mender.io/u/nishad1092)\
**Post date:** [June 23, 2020, 2:36am UTC](https://hub.mender.io/t/mender-demo-server-sudden-slow-and-stuck/2043/22 "2020-06-23T02:36:57Z")

</div>

Hi @kacf,

Yes, I understand your key points, I have deployed one release and its failed\aborted, and its a new release, and now I’ll share the deployment log here, so that you can get some idea, becuase I have tried and this aint happening, Im not able to deploy any other releases becuase the device isn’t syncing anymore after a failed\aborted deployment, I(maybe the previous release was inconsistent, I dont knw)

FYI: I know its strange, but there gotto be a prblm somewhere,  
(Its .log format, but to upload I have changed to .yaml)

[deployments.yaml](https://hub.mender.io/uploads/short-url/sbSKduHUDZZWxItohFmwaZhnR14.yaml) (20.1 KB)

1. After I start my demo server (sudo ./demo up), then when I press “enter” for logs, I get storage-proxy error, Is it because of this this entire thing is happening to my server?
2. The solution which I have is to reboot the device manually, so that device is synced again to mender and redeploy or deploy new artifact, but again , a very unlikely situation in production.

I think this issue isn’t reported by anyone, But i would love to resolve this. Mender looks very promising and Im sure something is wrong which can be corrected.

---

<div class="post-metadata">

**Author:** ![nishad1092](https://avatars.discourse-cdn.com/v4/letter/n/5fc32e/32.png) [@nishad1092](https://hub.mender.io/u/nishad1092)\
**Post date:** [June 23, 2020, 3:16am UTC](https://hub.mender.io/t/mender-demo-server-sudden-slow-and-stuck/2043/23 "2020-06-23T03:16:26Z")

</div>

@kacf,

I actually faced this issue last week, so to confirm it, I reinstalled the server, but still it is the same.

---

<div class="post-metadata">

**Author:** ![kacf](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/kacf/32/146_2.png) [@kacf](https://hub.mender.io/u/kacf)\
**Post date:** [June 23, 2020, 6:56am UTC](https://hub.mender.io/t/mender-demo-server-sudden-slow-and-stuck/2043/24 "2020-06-23T06:56:30Z")

</div>

You have these error messages at the bottom of your log file:

```auto
{"level":"info","message":"Update module output: File \"/home/jetson01/edge_read_key.py\", line 5, in \u003cmodule\u003e","timestamp":"2020-06-23T07:36:09+05:30"}
{"level":"info","message":"Update module output: filekey = open(\"/home/jetson01/EdgeID/edgekey.txt\",\"r\")","timestamp":"2020-06-23T07:36:09+05:30"}
{"level":"info","message":"Update module output: FileNotFoundError: [Errno 2] No such file or directory: '/home/jetson01/EdgeID/edgekey.txt'","timestamp":"2020-06-23T07:36:09+05:30"}

```

I don’t know your script, so it’s hard for me to tell what’s happening. Could they be the cause of the error?

---

<div class="post-metadata">

**Author:** ![nishad1092](https://avatars.discourse-cdn.com/v4/letter/n/5fc32e/32.png) [@nishad1092](https://hub.mender.io/u/nishad1092)\
**Post date:** [June 23, 2020, 7:05am UTC](https://hub.mender.io/t/mender-demo-server-sudden-slow-and-stuck/2043/25 "2020-06-23T07:05:51Z")

</div>

Yes, It could be the cause of the deployment error, But In case this happens, How to we rollback to the prevous state and also reboot, so that I can redploy it again. I know script module doesn’t support rollback and reboot. But if you have any example or even how to include these two in script module, it will be great.

I would really like to have these two artifacts added to the script module?

---

<div class="post-metadata">

**Author:** ![nishad1092](https://avatars.discourse-cdn.com/v4/letter/n/5fc32e/32.png) [@nishad1092](https://hub.mender.io/u/nishad1092)\
**Post date:** [July 20, 2020, 2:14pm UTC](https://hub.mender.io/t/mender-demo-server-sudden-slow-and-stuck/2043/26 "2020-07-20T14:14:52Z")

</div>

@mzedel @oleorhagen,

Hi, I’m writing again to this issue because unfortunately, Im having same lag but now Im using a production server, It was fine until I onboarded 5 devices, Same lag between different tabs, and Im getting 300 request in 5 min when I inspect device tab.

---

<div class="post-metadata">

**Author:** ![nishad1092](https://avatars.discourse-cdn.com/v4/letter/n/5fc32e/32.png) [@nishad1092](https://hub.mender.io/u/nishad1092)\
**Post date:** [July 22, 2020, 7:44am UTC](https://hub.mender.io/t/mender-demo-server-sudden-slow-and-stuck/2043/27 "2020-07-22T07:44:18Z")

</div>

![image](https://canada1.discourse-cdn.com/flex036/uploads/mender/original/1X/32c6a88af17c6b056397f105fa759aeaf08f5f4e.png)

This is what im facing, I knw it shouldnt happen, but it is actually bit slower even in production server, the server has 16gb ram, and it isnt using any. Do u think it has anything to do with my Computers RaM performance? cuz its the chrome browser

---

<div class="post-metadata">

**Author:** ![peter](https://avatars.discourse-cdn.com/v4/letter/p/278dde/32.png) [@peter](https://hub.mender.io/u/peter)\
**Post date:** [July 22, 2020, 6:27pm UTC](https://hub.mender.io/t/mender-demo-server-sudden-slow-and-stuck/2043/28 "2020-07-22T18:27:57Z")

</div>

hello Nishad1092!

I am sorry to hear this happens again. Could you provide screen shots form the device view from your browser?  
could you also send the logs form the deviceauth, inventory and mongodb containers?  
if there are some confidential materials there, please send them via email to [support@mender.io](mailto:support@mender.io)  
Some additional questions, if you could take a look at them:

- how are you connecting to the server?
- where is it located with the relation to your desktop?
- could you try a different machine? perhaps run a web browser remotely “closer to the server”?
- what happens when you reload without cache (command/control+shift+r) and run from the incognito window? how long until you see that behaviour?
- have you tried restarting Mender? does it change anything?

peter

---

<div class="post-metadata">

**Author:** ![nishad1092](https://avatars.discourse-cdn.com/v4/letter/n/5fc32e/32.png) [@nishad1092](https://hub.mender.io/u/nishad1092)\
**Post date:** [July 24, 2020, 7:18am UTC](https://hub.mender.io/t/mender-demo-server-sudden-slow-and-stuck/2043/29 "2020-07-24T07:18:33Z")

</div>

Hi @peter,

Thank you for your response, Ill definitely send you the mail with all needed information, just give me sometime.

For now, could you tell me how to restart the mender. Right now, everytime Im doing only (sudo ./run up) , Is there any other command.

---

<div class="post-metadata">

**Author:** ![tranchitella](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/tranchitella/32/606_2.png) [@tranchitella](https://hub.mender.io/u/tranchitella)\
**Post date:** [July 28, 2020, 6:17am UTC](https://hub.mender.io/t/mender-demo-server-sudden-slow-and-stuck/2043/30 "2020-07-28T06:17:28Z")

</div>

@nishad1092 can you please verify if the issue is in the UI or in the API calls? You can use the Chrome Developer tools to get a curl command for one of the API calls (e.g. the one calling devices/count) and try to execute it from your console. In this way, we can understand if the issue you are facing is because the UI is sluggish, or if is the backend which is slow in answering your calls.

---

<div class="post-metadata">

**Author:** ![peter](https://avatars.discourse-cdn.com/v4/letter/p/278dde/32.png) [@peter](https://hub.mender.io/u/peter)\
**Post date:** [July 28, 2020, 6:24am UTC](https://hub.mender.io/t/mender-demo-server-sudden-slow-and-stuck/2043/31 "2020-07-28T06:24:15Z")

</div>

Hey Nishad!

thanks. restart: just stop with: `./run down -v` and then start, check `./run ps` or `docker ps` to see that all containers are gone before running `./run up`.  
one more thing: could you run `nice -n 8 top -d1` on the server machine, while you are experiencing “sluggish ui” effect? do you have sysstat package configured and installed? could you try to access mem/cpu/diskio statistics during the ui session that is showing slow behaviour?

peter

---

<div class="post-metadata">

**Author:** ![nishad1092](https://avatars.discourse-cdn.com/v4/letter/n/5fc32e/32.png) [@nishad1092](https://hub.mender.io/u/nishad1092)\
**Post date:** [July 28, 2020, 7:31am UTC](https://hub.mender.io/t/mender-demo-server-sudden-slow-and-stuck/2043/32 "2020-07-28T07:31:03Z")

</div>

Hi @peter ,

Thank you for your response.  
But then if I do ./run down -v , won’t this command erase the existing data from server. So instead isn’t it ./run stop ??

Sure I’ll also checck with nice -n 8 top -d1 on the server machine once.

---

<div class="post-metadata">

**Author:** ![nishad1092](https://avatars.discourse-cdn.com/v4/letter/n/5fc32e/32.png) [@nishad1092](https://hub.mender.io/u/nishad1092)\
**Post date:** [July 28, 2020, 7:33am UTC](https://hub.mender.io/t/mender-demo-server-sudden-slow-and-stuck/2043/33 "2020-07-28T07:33:32Z")

</div>

Hi @tranchitella,

Thank you for ur response…  
I don’t think there is any issue when there is API call. Calls are happening well. But the UI is sluggish. I do have only 3 devices now, But many devices I decommisioned and rejected, do u think the cache or the old data is causing the issue?

Is there a way to clear the old data of decommisioned or rejected devices?

---

<div class="post-metadata">

**Author:** ![peter](https://avatars.discourse-cdn.com/v4/letter/p/278dde/32.png) [@peter](https://hub.mender.io/u/peter)\
**Post date:** [July 28, 2020, 8:24am UTC](https://hub.mender.io/t/mender-demo-server-sudden-slow-and-stuck/2043/34 "2020-07-28T08:24:50Z")

</div>

> [@nishad1092](#):
>
> But then if I do ./run down -v , won’t this command erase the existing data from server. So instead isn’t it ./run stop ??

yes of course, stop, my bad, good that you are verifying what I write!

peter

---

<div class="post-metadata">

**Author:** ![tranchitella](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/tranchitella/32/606_2.png) [@tranchitella](https://hub.mender.io/u/tranchitella)\
**Post date:** [July 28, 2020, 2:01pm UTC](https://hub.mender.io/t/mender-demo-server-sudden-slow-and-stuck/2043/35 "2020-07-28T14:01:28Z")

</div>

You can try opening the UI in an incognito window/tab, this would start from a clean state.  
Have you tried using the UI from another computer?

---

<div class="post-metadata">

**Author:** ![nishad1092](https://avatars.discourse-cdn.com/v4/letter/n/5fc32e/32.png) [@nishad1092](https://hub.mender.io/u/nishad1092)\
**Post date:** [July 29, 2020, 11:20am UTC](https://hub.mender.io/t/mender-demo-server-sudden-slow-and-stuck/2043/36 "2020-07-29T11:20:46Z")

</div>

Hi @tranchitella, Yes I tried from Incognito and also from another computer, but the problem persist. But let me try restarting and also the top command which @peter told me while the UI hangs.

Ill get the results and get back

---

<div class="post-metadata">

**Author:** ![peter](https://avatars.discourse-cdn.com/v4/letter/p/278dde/32.png) [@peter](https://hub.mender.io/u/peter)\
**Post date:** [July 31, 2020, 9:40am UTC](https://hub.mender.io/t/mender-demo-server-sudden-slow-and-stuck/2043/37 "2020-07-31T09:40:30Z")

</div>

ok, so the top shows no load on the server. I think now we are down to two things:

1. high load on the machine that you run the browser on
2. poor network connection  
lets tackle them in order.

- first: could you check the load on the machine you are running the browser on? does it show high CPU, or mem usage?
- second: could you run ping and traceroute to the server address you are running Mender on?

peter

---

<div class="post-metadata">

**Author:** ![nishad1092](https://avatars.discourse-cdn.com/v4/letter/n/5fc32e/32.png) [@nishad1092](https://hub.mender.io/u/nishad1092)\
**Post date:** [August 3, 2020, 12:46pm UTC](https://hub.mender.io/t/mender-demo-server-sudden-slow-and-stuck/2043/38 "2020-08-03T12:46:08Z")

</div>

Hi @peter,

I’m sorry I couldn’t reply back soon.

It is a server in Azure, and it has 4 vCPUS and 16Gb RAM, so the memory usage, network usage looks fine all the time. It hasnt hit the peek even when the UI is sluggish…I’ll again loook into it but this is what I found. And then I’m my laptop’s chrome browser to login to Mender server.

---

<div class="post-metadata">

**Author:** ![mzedel](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/mzedel/32/261_2.png) [@mzedel](https://hub.mender.io/u/mzedel)\
**Post date:** [August 4, 2020, 8:14am UTC](https://hub.mender.io/t/mender-demo-server-sudden-slow-and-stuck/2043/39 "2020-08-04T08:14:02Z")

</div>

Considering the similarity with your previous description I could still use the the _full_ body for the device list request (`devices?status=accepted&per_page=20&page=1` + the ones with the UUID), better yet the entire response to say anything more specific. And the device intervals from the devices.  
If there is anything confidential/ sensitive in these responses, feel free to send them in a private message…

---

<div class="post-metadata">

**Author:** ![peter](https://avatars.discourse-cdn.com/v4/letter/p/278dde/32.png) [@peter](https://hub.mender.io/u/peter)\
**Post date:** [August 4, 2020, 8:27am UTC](https://hub.mender.io/t/mender-demo-server-sudden-slow-and-stuck/2043/40 "2020-08-04T08:27:53Z")

</div>

Hey @nishad1092, yes, I mean your laptop, what is the CPU and mem usage on your laptop during the _sluggish period_?

pg

[Previous page](https://hub.mender.io/t/mender-demo-server-sudden-slow-and-stuck/2043.md?page=1)

[Next page](https://hub.mender.io/t/mender-demo-server-sudden-slow-and-stuck/2043.md?page=3)
