Getting "error: fw_setenv returned failure: exit status 2" after "Committing update" OTA update

General Discussions
, , ,
41

@kacf
As you proposed, I have added state scripts in mender_convert and created OS image & mender artifacts.

custom_config.yml (1.4 KB)
mender_convert logs
convert.log.a8E2.yml (27.6 KB)

  1. Made a new flash on target device where image having artifact name as V1 (i.e. x86_64-Testserver_StateScriptTest_v1_170621)

  2. Triggered OTA with V2 successfully log as
    deployments.0001.b542a111-8d87-4045-9230-4fb799a48f96.log1_successOTA.yml (14.3 KB)

  3. Triggered OTA again with V1 but failed as
    OTA_Fail_states_log.yml (23.3 KB)

Let me know for any further outputs for debugging issue.

42

Thanks. Unfortunately it again revealed nothing. It’s quite incredible how well hidden this is.

I have one more thing for you to try: Let’s enable full script debugging in the fw_printenv script itself. In addition to your existing changes, can you add this change to mender-convert, re-convert, and then rerun the deployment:

diff --git a/mender-convert-modify b/mender-convert-modify
index cf8f215..c064662 100755
--- a/mender-convert-modify
+++ b/mender-convert-modify
@@ -222,6 +222,7 @@ EOF
     run_and_log_cmd "make 2>&1"
     run_and_log_cmd "sudo make DESTDIR=../ BOOT_DIR=boot install-boot-files"
     run_and_log_cmd "sudo make DESTDIR=../rootfs install-tools"
+    sed -i -e '2 i set -x' ../rootfs/usr/bin/fw_printenv
     cd - > /dev/null 2>&1
 
     # Remove conflicting boot files. These files do not necessarily effect the
43

@kacf Thanks for giving me your valuable time to debug this issue.

As per your suggestion, I have enabled full script debugging in the fw_printenv script and then created again artifacts say v1 & v2.

  1. Made a new flash on the target device where the image has artifact name as v1.
  2. Triggered full OS OTA deployment v2 artifact but now it has failed in first OTA itself as below

image
image1060×298 24.3 KB

image
image959×500 31.1 KB

Failed OTA deployment log is:

Failed_logs_Testserver_StateScriptTest_enable_debug_v2_210621.yml (19.0 KB)

Let me know for any further details outputs for debugging the issue.

44

That’s weird, that last change had no effect on the output at all. Can you run this on one of the failing devices:

head /usr/bin/fw_printenv
ls -l /usr/bin/fw_setenv
45

@kacf
output of provided commands are below:

$ head /usr/bin/fw_printenv 
#!/bin/sh
set -x

# This script mimicks the behavior of fw_printenv/fw_setenv from U-Boot.

# Here is how the environment works. We've got three problems to overcome:
#   1. GRUB doesn't support checksumming the environment.
#   2. There is a hashsum command, but it can only be used to verify checksums,
#      not generate them.
#   3. We want to store the environment on the boot partition, which is FAT, and


$ ls -l /usr/bin/fw_setenv 
lrwxrwxrwx 1 root root 11 Jun 18 03:34 /usr/bin/fw_setenv -> fw_printenv
46

Hmm, could there be more than one on your system? What does which -a fw_printenv say?

47

@kacf


$ which -a fw_printenv
/usr/bin/fw_printenv
48

Ok…

At this point, I’m really clueless. I can’t see anything wrong anywhere, and still it’s failing. Your best bet at this point might be to start stepping through the code with a debugger, near the place where it fails. Or if you have the means, look into our consulting services and have someone do board integration where we look into this problem on our side, using a board you send us.

Sorry I can’t be of more assistance.