# Mender-monitor and intended shutdowns

**URL:** <https://hub.mender.io/t/mender-monitor-and-intended-shutdowns/4806>\
**Category:** General Discussions\
**Tags:** monitor\
**Created:** [March 29, 2022, 2:09am UTC](https://hub.mender.io/t/mender-monitor-and-intended-shutdowns/4806 "2022-03-29T02:09:57Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![drewmoseley](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/drewmoseley/32/47_2.png) [@drewmoseley](https://hub.mender.io/u/drewmoseley)\
**Post date:** [March 29, 2022, 2:09am UTC](https://hub.mender.io/t/mender-monitor-and-intended-shutdowns/4806/1 "2022-03-29T02:09:57Z")

</div>

I’m trying to use mender-monitor to determine when some services are running or not. I’m seeing monitor triggers both at initial boot and on reboot, which feels like a race condition between intentional service shutdowns and mender-monitor detection. Is there a way to have mender-monitor ignore these service shutdowns as intentional? In my case I’m using “shutdown -r now” to force the reboot so it seems like it should be possible to detect this situation and not trigger an alert.

Drew

---

<div class="post-metadata">

**Author:** ![drewmoseley](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/drewmoseley/32/47_2.png) [@drewmoseley](https://hub.mender.io/u/drewmoseley)\
**Post date:** [April 2, 2022, 5:57pm UTC](https://hub.mender.io/t/mender-monitor-and-intended-shutdowns/4806/2 "2022-04-02T17:57:53Z")

</div>

@lluiscampos @kacf any thoughts on this?

---

<div class="post-metadata">

**Author:** ![kacf](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/kacf/32/146_2.png) [@kacf](https://hub.mender.io/u/kacf)\
**Post date:** [April 4, 2022, 10:46am UTC](https://hub.mender.io/t/mender-monitor-and-intended-shutdowns/4806/3 "2022-04-04T10:46:53Z")

</div>

@merlin: Maybe this could be detected by calling something like `systemctl is-system-running` and see if it is `stopping`, before considering the alert triggered?

---

<div class="post-metadata">

**Author:** ![drewmoseley](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/drewmoseley/32/47_2.png) [@drewmoseley](https://hub.mender.io/u/drewmoseley)\
**Post date:** [April 4, 2022, 3:33pm UTC](https://hub.mender.io/t/mender-monitor-and-intended-shutdowns/4806/4 "2022-04-04T15:33:59Z")

</div>

Interesting. I didn’t know that was a command. I’ll experiment with it.

---

<div class="post-metadata">

**Author:** ![drewmoseley](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/drewmoseley/32/47_2.png) [@drewmoseley](https://hub.mender.io/u/drewmoseley)\
**Post date:** [April 4, 2022, 4:21pm UTC](https://hub.mender.io/t/mender-monitor-and-intended-shutdowns/4806/5 "2022-04-04T16:21:05Z")

</div>

OK. Changing `/usr/share/mender-monitor/lib/service-lib.sh` to the following seems to work for me. At least in the limit of my minimal testing. I don’t know if there is an equivalent for sysvinit.

Drew

```auto
#!/usr/bin/env bash
#
# Copyright 2021 Northern.tech AS
#
# All Rights Reserved

function service_systemd_is_running() {
    case $(/bin/systemctl is-system-running) in
        running | degraded )
            # In these states we can go ahead and check service status 
            systemctl is-active --quiet "$1"
            ;;
        * )
            # In any other state, the service status is not necessarily correct
            # so we will just return true to bypass the check for now.
            true
            ;;
    esac
}

function service_sysv_is_running() {
    /etc/init.d/${1} status > /dev/null 2>&1
}

```

---

<div class="post-metadata">

**Author:** ![kacf](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/kacf/32/146_2.png) [@kacf](https://hub.mender.io/u/kacf)\
**Post date:** [April 5, 2022, 9:31am UTC](https://hub.mender.io/t/mender-monitor-and-intended-shutdowns/4806/6 "2022-04-05T09:31:02Z")

</div>

Thanks @drewmoseley! I have submitted this internally, I’ll try to get this into the next release!

---

<div class="post-metadata">

**Author:** ![peter](https://avatars.discourse-cdn.com/v4/letter/p/278dde/32.png) [@peter](https://hub.mender.io/u/peter)\
**Post date:** [April 7, 2022, 8:21am UTC](https://hub.mender.io/t/mender-monitor-and-intended-shutdowns/4806/7 "2022-04-07T08:21:47Z")

</div>

looks good, we are already discussing in the PR

pg

---

<div class="post-metadata">

**Author:** ![peter](https://avatars.discourse-cdn.com/v4/letter/p/278dde/32.png) [@peter](https://hub.mender.io/u/peter)\
**Post date:** [April 12, 2022, 12:02pm UTC](https://hub.mender.io/t/mender-monitor-and-intended-shutdowns/4806/8 "2022-04-12T12:02:04Z")

</div>

this is all strange, @drewmoseley do you see alerts coming in on every shutdown? we do have an explicit check for that: `monitor.d/service.sh`: **14** where we check if daemon is not in the process of shutting down. it however relies on the SIGTERM being delivered to the `mender-monitord`, is it possible that your systemd services are being shutdown first?

peter

---

<div class="post-metadata">

**Author:** ![drewmoseley](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/drewmoseley/32/47_2.png) [@drewmoseley](https://hub.mender.io/u/drewmoseley)\
**Post date:** [April 12, 2022, 2:44pm UTC](https://hub.mender.io/t/mender-monitor-and-intended-shutdowns/4806/9 "2022-04-12T14:44:43Z")

</div>

Hi @peter

I’m pretty sure this happens every time. I don’t see any such check in my service.sh file. Line 14 is `SERVICE_RC_OK=0`. Maybe I have an older version? I am using mender-monitor-1.1.0.tar.gz as [downloaded from here](https://docs.mender.io/system-updates-yocto-project/customize-mender#monitor).

Drew

---

<div class="post-metadata">

**Author:** ![peter](https://avatars.discourse-cdn.com/v4/letter/p/278dde/32.png) [@peter](https://hub.mender.io/u/peter)\
**Post date:** [April 12, 2022, 5:00pm UTC](https://hub.mender.io/t/mender-monitor-and-intended-shutdowns/4806/10 "2022-04-12T17:00:02Z")

</div>

Hey Drew,

ok, I am sorry, it has been fixed in master, and not yet released.  
it will be part of the next release. are you able to access master?

peter

---

<div class="post-metadata">

**Author:** ![drewmoseley](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/drewmoseley/32/47_2.png) [@drewmoseley](https://hub.mender.io/u/drewmoseley)\
**Post date:** [April 12, 2022, 5:19pm UTC](https://hub.mender.io/t/mender-monitor-and-intended-shutdowns/4806/11 "2022-04-12T17:19:38Z")

</div>

I cannot access that but I don’t really need it right now. I have just patched the file locally in my Yocto configuration so I have it working. I’ll take a look when the next release comes out and revert my local change.

Drew