PATH issue when using mender-connect

General Discussions
1

As I mentioned in this other thread: Using update module as a rootfs-image
I am currently experimenting with Mender 2.6 and noticed a strange PATH issue while using the shell provided by mender-connect. This is with Yocto, meta-mender branch dunfell.

When I run some Mender CLI commands through the remote shell, I get errors such as these:

root@osu-00e0709d97ea:~# mender show-artifact
ERRO[0000] Failed to read the current active partition: exec: "mount": executable file not found in $PATH 
dev-setup-offline-1.0.13
root@osu-00e0709d97ea:~# mender send-inventory
ERRO[0000] failed to force updateCheck: getMenderDaemonPID: Failed to run systemctl 
root@osu-00e0709d97ea:~#

But I can launch “mount” and “systemctl” manually in the same shell:

root@osu-00e0709d97ea:~# mount  
proc on /proc type proc (rw,relatime)
sysfs on /sys type sysfs (rw,relatime)
[...]
root@osu-00e0709d97ea:~# systemctl --help
systemctl [OPTIONS...] COMMAND ...
[...]

Also, the same Mender CLI commands issued on a terminal on the device (or by SSH) work fine:

root@osu-00e0709d97ea:~# mender show-artifact
dev-setup-offline-1.0.13
root@osu-00e0709d97ea:~# mender send-inventory
root@osu-00e0709d97ea:~#

After some experimentation, I noticed that PATH is set (which is why I can run mount and others), but it is not exported (doesn’t show when you type “env” in a remote shell, but does when using a local shell). Exporting PATH fixes the issue for this terminal session (the issue comes back if the remote shell is closed and reopened):

root@osu-00e0709d97ea:~# echo $PATH
/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin
root@osu-00e0709d97ea:~# export PATH
root@osu-00e0709d97ea:~# mender show-artifact
dev-setup-offline-1.0.13
root@osu-00e0709d97ea:~# mender send-inventory
root@osu-00e0709d97ea:~#

Is there something missing in the mender-connect client configuration, or systemd service?

2

Interesting. This does seem like a bug to me. @oleorhagen can you take a look?

In the meantime you may be able to cobble together a workaround in the actual mender-connect configuration.

3

This is related to a discussion we had in the original implementation about login shells. The suggestion there was to use su, but this was turned down. However. it may be possible to fix by passing the -l option to the shell. All of sh, bash, csh, ksh and zsh support this.

Pinging @tranchitella as well.

4

Manually running “/bin/bash -l” from the remote terminal does fix the issue. However, setting “/bin/bash -l” as “ShellCommand” in the config file prevents mender-connect from starting:

Mar 05 14:40:37 osu-00e0709d97ea mender-connect[530]: time="2021-03-05T14:40:37Z" level=error msg="given shell (/bin/bash -l) is not executable"

I also tried with other parameters, like “/bin/bash -i”, to check if the -l was causing the issue, but it still breaks. It seems like adding ANY parameter to “ShellCommand” breaks mender-connect.

5

@hacpa I confirm the ShellCommand doesn’t support arguments; if you want to customize it, you need to create a script (which can run bash -i or do whatever initialization you neeed) and point mender-connect to it.

6

Sorry, I should have been clearer: I meant adding -l in the Mender source code, so it was primarily a comment aimed at the developers. But you confirmed that adding the parameter worked, so I think it deserves a ticket.