Mender-client 4.0.0-1+debian+bullseye

General Discussions
1

Hello,

I noticed that the version of mender-client in your Debian bullseye repo has changed from 3.5.2-1+debian+bullseye to 4.0.0-1+debian+bullseye, but I don’t see any reference to 4.0.0 in your release notes or general documentation. When will you be publishing release notes and updating documentation?

https://docs.mender.io/release-information/release-notes-changelog/mender-client
https://docs.mender.io/overview/compatibility
https://docs.mender.io/release-information/release-schedule

Cheers,
Zach

2

Hi @bssi,

We are just executing the release process for the 4.0 client, the corresponding resources will be updated soon.

Greets,
Josef

3

Is the release process complete? When attempting to install mender on a new device via the install script, it seems to complete successfully, installing 4.0.0. However the mender binary /usr/bin/mender doesn’t exist, and causes other mender services to fail. Is there any way to install 3.5.2? We have that version working on other devices, however it doesn’t seem to be in the apt repo anymore.

4

It appears the mender binary was renamed to mender-update, I was able to get around this without changing my application with:
ln -s /usr/bin/mender-update /usr/bin/mender
However, I am unable to get standalone updates to work as mender-update commit errors with no update pending; while fw_printenv does indicate one (upgrade_avaliable=1).

Getting access to 3.5.2 for now so I can get this update shipped would be good. It appears I will need to alter my application to support 4.0.0.

5

Hi @mmetzler and @mdbellrd,

I’m really sorry for the inconvenience this is causing! Yes, the mender binary has been split and renamed into mender-auth, mender-update and mender-setup. This offers a few benefits, like not shipping the setup logic to all devices which do not need them, or being able to install mender-connect without the update component.

The release process is taking a bit longer as it is more moving parts therefore, and I want to apologise for the issues this causes while its going on.

For the install script, its source is located at mender-dist-packages/scripts/install-mender.sh at master · mendersoftware/mender-dist-packages · GitHub, and you should be able to just use the last version for now, respectively stick with the releases on the 3.5.2 tag.

We appreciate the feedback, it helps us make the new client better. Thank you for your contributions!

Greets,
Josef

6

@TheYoctoJester - Please put the 3.5.2 version back on https://downloads.mender.io/repos/debian … it is absolutely crazy that the Mender team would nuke all 3.x.x versions off their Debian repo in order to post 4.x.x that is a major rev and is incompatible with how 3.x.x was used. Great that everything was split up, but as I’m sure you know, Mender is widely used in IoT device firmware builds that heavily depend on having a controlled setup - and, for us, that setup means knowing exactly where/how the mender-client is working and being able to replace mender-device-identity, which used to be part of the mender-client deb package

… now all of our firmware builds are broken and we need to spend a bunch of time figuring out how to migrate to v4 and then do full integration testing after we make the migration before we have a usable build again… which is likely a week or more of downtime.

Here is the full list of what we now have access to:

$ apt update
$ rmadison mender-client
 mender-client | 2.1.2-2           | focal/universe          | source, amd64, arm64, armhf, ppc64el, s390x
 mender-client | 2.1.2-2ubuntu0.2  | focal-security/universe | source, amd64, arm64, armhf, ppc64el, riscv64, s390x
 mender-client | 2.1.2-2ubuntu0.2  | focal-updates/universe  | source, amd64, arm64, armhf, ppc64el, riscv64, s390x
 mender-client | 3.0.0+ds1-2       | jammy/universe          | source, amd64, arm64, armhf, ppc64el, riscv64, s390x
 mender-client | 3.4.0+ds1-5       | lunar/universe          | source, amd64, arm64, armhf, ppc64el, riscv64, s390x
 mender-client | 3.4.0+ds1-5build1 | mantic/universe         | source, amd64, arm64, armhf, ppc64el, riscv64, s390x
 mender-client | 3.4.0+ds1-5build1 | noble/universe          | source, amd64, arm64, armhf, ppc64el, riscv64, s390x

On Release version and schedule | Mender documentation under Current LTS releases there is a note that says:

Please note: 3.5 was initially released as an LTS but has since been withdrawn because of technical reasons. Due to the delays in releasing the next LTS (3.6), the support duration for 3.3 has been extended.

So, was 3.5.2 nuked because of this? If so, where is 3.6?

7

Hi @scscscsc,

I fully agree that this is a problematic situation. Already doing my best to escalate things.

Concerning the release notes, this is a bit confusing as the Mender Client version numbers move against each other. The 3.6 full release did not include a new client as there was no changes at that time, hence 3.5.2 is still the current one.

Greets,
Josef

2 Likes
8

Hi @TheYoctoJester,

I am glad you are escalating this. Much appreciated… Thank you! To give you more ammunition as you advocate on our behalf I would like to second @scscscsc’s motion to put 3.5.2 back in the repo… We are just about to start shipping a batch of production systems on a very tight timeline and absolutely the last thing we need at this juncture is to take additional precious time to migrate to and QA mender-client 4.0.0. In general we would never migrate directly to a X.0.0 release of anything if it could possibly be avoided. Especially something mission critical like our OTA update solution.

Cheers,
Zach

9

I am also seeing big issues here. I wanted to release an update yesterday, but during testing realized that the mender binary was missing, which meant the update would succeed, but the firmware would NEVER reconnect to mender. Had we not caught this, we could have nuked our entire fleet of deployed devices (the website still just says “rebooting”). Any help in the interim would be much appreciated.

10

@scscscsc @bssi I might have at least a workaround soon. Will keep you posted.

11

@bssi @scscscsc

I could get the repository situation looked into. The packages are still there, just not in the index. You can still pull them via

wget https://downloads.mender.io/repos/debian/pool/main/m/mender-client/mender-client_3.5.2-1+DIST+NAME_ARCH.deb

which for example Debian Buster on amd64 results in

wget https://downloads.mender.io/repos/debian/pool/main/m/mender-client/mender-client_3.5.2-1+debian+buster_amd64.deb

Hope this gets you going again!

Greets,
Josef

12

@TheYoctoJester Thank you. We’ll still have to adjust our build, but far preferred than having to make an unplanned migration to 4.0.0!

1 Like
13

@TheYoctoJester - Thank you, this got our builds working again and should hold us over until this is properly resolved.

14

@scscscsc @bssi @cspensky @mmetzler @mdbellrd

Please see Update and statement on the Mender Client 4.0 Debian package incident for more details.

1 Like