Mender client HTTP proxy support

General Discussions
1

Hi everyone,

I’m trying to understand whether the mender client does support HTTP proxy to connect to a mender server. The only thing I found is this discussion saying it’s currently not supported: https://groups.google.com/a/lists.mender.io/forum/#!msg/mender/hYak9bd3rXE/8cRKSCpiBgAJ

And this reflects as well my observation. My mender client is not able to connect to a mender server inside our company network (which is using HTTP proxy). The HTTP_PROXY setting seems to be ignored.

Question: Did I miss something or is the mender client not supporting HTTP proxy? Is it planned to be supported in the future?

Thanks,
Bruno

2

Hi @freibrun,

I must admit I am not an expert on this area, but can’t you just use the “internal company IP” of the Mender server directly (instead of HTTP_PROXY)? Mender does require TLS for all communication for security reasons, is that supported with HTTP_PROXY or would HTTPS_PROXY be needed? If support for this requires plaintext communication it is unlikely we can prioritize this due to security requirements, unfortunately. But again I am not an expert. :slight_smile:

We are looking at adding support for reverse proxy (with TLS termination and initiation) for potentially different reasons (e.g. to forward requests from device to hosted Mender when there are restrictive security policies in place), not sure if this would be relevant here.

Does that make sense?

3

I also think that comment on the older thread is still valid and would try it out :slight_smile:

https://groups.google.com/a/lists.mender.io/d/msg/mender/hYak9bd3rXE/vQL_ji6rAwAJ

Please report back if you have the opportunity to test it out.

4

Unfortunately I’m also not an expert on that topic. I just noticed that I get no connection to mender server inside our company network. Same is valid for any other services (such as apt-get, docker pull, pip install, …) as long as I don’t configure HTTP_PROXY and HTTPS_PROXY with our companies proxy.

Once I set HTTP_PROXY and HTTPS_PROXY (and configure all extra portions needed for the different tools which handles proxies differently), all services are running and are able to connect “to the internet”.
All except mender client.

That’s why I wondered if there is any additional setting needed to let mender client connecting to mender server via proxy.

I don’t think reverse proxy (used by servers) helps in this scenario. I’m looking for a forward proxy (used by clients).

5

Currently I have no opportunity to test it.

As a workaround I simply switched to a “proxy free environment” for my ongoing mender tests. :smiley:

1 Like
6

@freibrun,

This might have been fixed already, do you have the chance to test with Mender 2.2.0 or newer? There is also a Mender 2.3.0 beta right around the corner.

There is a ticket here, but it looks like this issue has been fixed: https://tracker.mender.io/browse/MEN-3110

1 Like
7

What is the best way to rev the mender-client to 2.2.0 or 2.3.0 to test the HTTP/HTTPS_PROXY env variables? Menderized sdimgs were created using using Mender-convert on raspbian and have client version 2.1.1 I currently have a number of pilot units running behind a proxy.

8

2.1.2 is the latest of the Mender client, though there is a beta, 2.2.0b1 (technically released already, but not announced, very soon though). Don’t be confused by Mender 2.3, this is the whole product line, but it’s not the same as the client version. There will be a 2.0.0 Beta release of mender-convert soon as well, so just give it a few days.

9

Hi @jem2e4 there is no need for you to upgrade the client in order to test this. The Changes should be present in 2.1.1 as well.

10

@oleorhagen Thanks for the reply. Does 2.1.1 use the system proxy or is it pulling from a different configuration.

11

If by system proxy, you are referring to the internal golang http proxy implementation, then yes.

I saw you edited your post, and will refer to that now, as I noticed there was an s missing in your HTTP_PROXY. I’m pretty sure you need to set the HTTPS_PROXY variable, then this should work transparently.

To the best of my knowledge at least.

12

@oleorhagen, yes, great observation, both HTTP and HTTPS proxys are being added to the environment during boot as part of the provisioning script.

After some review/testing here is some potentially useful data:
The proxy server is also the nameserver and in reviewing HTTP and DNS traffic hosted.mender.io resolves the 3 aws addresses 52.5.180.155, 34.205.112.193, 52.20.53.236
However, following the DNS lookup the proxy appears to be ignored with destinations being set to the aws servers?Src: 10.221.1.138 Dst: 52.5.180.155

Curl notes:
curl https://hosted.mender.io or http://hosted.mender.io responds with a 301 (GET)
curl https://hosted.mender.io/api/management/v1/useradm/auth/login for a JWT with success (POST and CONNECT)
In review Curl is able to traverse the proxy from the unit, but the mender client isn’t.

Thoughts or ideas on configuration?

13

Note I downloaded the latest vanilla raspbian build and used the mender-convert master and added HTTP HTTPS proxy address into /etc/environment for testing purposes.

14

Hmm, I see. Interesting. I will set aside some time later today, to dig into this a bit further. Thanks for the detailed information.

15

@jem2e4 is there any chance that you could run the client with the -debug flag, and attach the logs here, for my perusal?

16

@oleorhagen thank you, I believe we were able to get this working on the vanilla Raspbian Lite. Will post on details.

1 Like
17

Great!

Curious to hear what the issues were!

A good write-up will probably be a good reference for the community later on also.

18

@oleorhagen I hope this information is of use: Although, both env and my shell had the HTTP/HTTPS http/https proxy variables set correctly in my Golden Image and cURL as well as chromium browser would navigate the proxy correctly, Mender would not traverse it.
The current Raspbian Buster Lite download had changes in relation to proxies. There is a file created under sudoers.d/010_proxy with the following entries:

Defaults env_keep += “http_proxy HTTP_PROXY”
Defaults env_keep += “https_proxy HTTPS_PROXY”

once the file was created on my GI build manually, mender client would check-in and download artifacts as expected.
All that being said I’m a novice, and you may have a better understanding of these changes were needed and why.

1 Like
19

I see

I might have been stupid here,

but have a second look at your environment setup for the systemd service file for the mender-client.

Good that it works now with the sudo environment (which mender needs). But the HTTPS_PROXY variable can also be set on a unit file basis through the ENVIRONMENT variable :slight_smile:

https://www.freedesktop.org/software/systemd/man/systemd.service.html

20

Yes, I was not checking the active environment for mender-client. The HTTPS_PROXY is not added to the mender-client environment during the mender-convert process nor when the service launches. Also, to speed up testing i was launching the mender -daemon which does pull the current environment HTTPS_PROXY. The Squid Proxy server setup added a little more confusion as well. Simply put the HTTPS_PROXY needed to be manually added to the Environment of mender-client.service

1 Like