# Failing updates on Tegra TX2 due to issues with \`nvbootctrl\` **URL:** \ **Category:** General Discussions\ **Tags:** yocto, tegra\ **Created:** [November 2, 2020, 2:03pm UTC](https://hub.mender.io/t/failing-updates-on-tegra-tx2-due-to-issues-with-nvbootctrl/2765 "2020-11-02T14:03:22Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![manuel\_vps](https://avatars.discourse-cdn.com/v4/letter/m/a87d85/32.png) [@manuel\_vps](https://hub.mender.io/u/manuel_vps)\ **Post date:** [November 2, 2020, 2:03pm UTC](https://hub.mender.io/t/failing-updates-on-tegra-tx2-due-to-issues-with-nvbootctrl/2765/1 "2020-11-02T14:03:22Z")
Hello everyone, we’re having issues with failing updates, and traced the problem back to `nvbootctrl get-current-slot` returning the wrong slot in 50% of all cases. There is a thread on this problem at the NVidia support forum as well. There, I learned that [Mender uses U-Boot variables to switch between the two rootfs partitions, and that this is a separate mechanism from the A/B slot mechanism in the NVIDIA bootloader chain](https://forums.developer.nvidia.com/t/nvbootctrl-get-current-slot-returning-unexpected-slot/156982/2). Can anyone elaborate on that? I would like to learn in which way Mender interfaces and affects the A/B slot mechanism in the NVIDIA bootloader chain. Also, I would like to learn how system boots works in general on that platform. I didn’t quite grok the vanilla A/B slot system, let alone the changes due to Mender. We are observing the following symptoms: - Mender update fails, when `nvbootctrl get-current-slot` returns the wrong slot. - The values returned by the `nvbootctrl` commands change with every boot. They follow the following cyclic pattern: Boot 1: `nvbootctrl get-current-slot` : **0** ; Priority of slot 0: **14** ; Priority of slot 1: **14** ; Boot 2: `nvbootctrl get-current-slot` : **0** ; Priority of slot 0: **14** ; Priority of slot 1: **15** ; Boot 3: `nvbootctrl get-current-slot` : **1** ; Priority of slot 0: **13** ; Priority of slot 1: **15** ; Boot 4: `nvbootctrl get-current-slot` : **1** ; Priority of slot 0: **15** ; Priority of slot 1: **14** ; In each and every of these reboots, - the machine booted from slot 0 according to both `fw_printenv mender_boot_part` and `findmnt /`. - `retry_count` as reported by `nvbootctrl dumps-slots-info` was 7 for both slots - `boot_successful` as reported by `nvbootctrl dumps-slots-info` was 1 for both slots Thanks a lot, Manuel ---
**Author:** ![dwalkes](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/dwalkes/32/41_2.png) [@dwalkes](https://hub.mender.io/u/dwalkes)\ **Post date:** [November 3, 2020, 12:32am UTC](https://hub.mender.io/t/failing-updates-on-tegra-tx2-due-to-issues-with-nvbootctrl/2765/2 "2020-11-03T00:32:27Z")
> [@manuel\_vps](#): > > I would like to learn in which way Mender interfaces and affects the A/B slot mechanism in the NVIDIA bootloader chain The A/B slot mechanism interaction happens in [https://github.com/mendersoftware/meta-mender-community/tree/dunfell/meta-mender-tegra/recipes-mender/tegra-state-scripts](https://github.com/mendersoftware/meta-mender-community/tree/dunfell/meta-mender-tegra/recipes-mender/tegra-state-scripts) as a part of the mender updates, so for instance [here](https://github.com/mendersoftware/meta-mender-community/blob/dunfell/meta-mender-tegra/recipes-mender/tegra-state-scripts/files/redundant-boot-install-script-uboot#L59) is where the nv\_update\_engine command is executed and [here](https://github.com/mendersoftware/meta-mender-community/blob/dunfell/meta-mender-tegra/recipes-mender/tegra-state-scripts/tegra-state-scripts_1.0.bb#L31) is where that script is setup as the ArtifactInstall [state script](https://docs.mender.io/artifact-creation/state-scripts). I don’t really understand how Mender is going to be involved on standalone reboots, since the scripts here should only execute as a part of an update or rollback. So I think the issue to understand is why are slot numbers changing with normal reboots. It would be interesting to try to reproduce this phenomenon on a build without mender installed at all, for instance stock L4T from NVIDIA or [tegra demo distro](https://github.com/OE4T/tegra-demo-distro/tree/dunfell-l4t-r32.4.3) with `tegrademo` instead of `tegrademo-mender`. Incidentally, we just noticed this same issue on one device today after using mender updates successfully on several devices for months. ---
**Author:** ![dwalkes](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/dwalkes/32/41_2.png) [@dwalkes](https://hub.mender.io/u/dwalkes)\ **Post date:** [November 3, 2020, 1:29am UTC](https://hub.mender.io/t/failing-updates-on-tegra-tx2-due-to-issues-with-nvbootctrl/2765/3 "2020-11-03T01:29:57Z")
A bit more about the difference between u-boot and cboot interaction: With u-boot the boot partition is chosen [here](https://github.com/mendersoftware/meta-mender-community/blob/dunfell/meta-mender-tegra/recipes-bsp/u-boot/patches/0011-Jetson-TX2-mender-boot-commands.patch#L45) based on mender variables set in uboot environment in response to `fw_printenv` and `fw_setenv` from the [mender client](https://github.com/mendersoftware/mender/blob/20b165e2e82215b8a486e9fe258e9f24da1ddadc/installer/bootenv.go#L152). However, with cboot builds there’s a [fake version of libubootenv](https://github.com/mendersoftware/meta-mender-community/blob/dunfell/meta-mender-tegra/recipes-bsp/u-boot/libubootenv-fake_1.0.bb) which [uses nvboootctrl to get the current slot](https://github.com/mendersoftware/meta-mender-community/blob/dunfell/meta-mender-tegra/recipes-bsp/u-boot/libubootenv-fake/fw_printenv#L12) and uses this to communicate to mender which partition is currently running. So in a uboot based implementation if software outside mender (ie the nvidia bootloader) decides to switch between between boot slots using the logic in the [Update Engine State Machine](https://docs.nvidia.com/jetson/l4t/index.html#page/Tegra%2520Linux%2520Driver%2520Package%2520Development%2520Guide%2Fbootloader_update_agx_tx2.html%23wwpID0E0SD0HA) this will mean there will be a mismatch between the bootloader and the root filesystem and this is what the error is telling you. This would obviously be an issue if it happened across any update which attempted to update the bootloader. In a cboot based system, the bootloader and root filesystem will always be in sync, so if the [Update Engine State Machine](https://docs.nvidia.com/jetson/l4t/index.html#page/Tegra%2520Linux%2520Driver%2520Package%2520Development%2520Guide%2Fbootloader_update_agx_tx2.html%23wwpID0E0SD0HA) decides to roll back to a different boot slot it will also roll back the root filesystem. You won’t get the mismatch error you see with uboot in this case but you will be running a rootfs and bootloader version you likely didn’t expect. So in both cases it’s bad that the update engine is deciding to switch boot slots on its own, but it’s likely more problematic for uboot. ---
**Author:** ![dwalkes](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/dwalkes/32/41_2.png) [@dwalkes](https://hub.mender.io/u/dwalkes)\ **Post date:** [November 3, 2020, 3:41pm UTC](https://hub.mender.io/t/failing-updates-on-tegra-tx2-due-to-issues-with-nvbootctrl/2765/4 "2020-11-03T15:41:17Z")
Created an issue at [https://github.com/OE4T/meta-mender-community/issues/7](https://github.com/OE4T/meta-mender-community/issues/7) to track how to deal with this. I’d like to understand whether `PREFERRED_PROVIDER_virtual/bootloader = "cboot-prebuilt"` is a possible solution, or at least makes the problem easier to deal with.