Jetson-TX2 - updating device-tree

mirzak · June 30, 2020, 11:03am

@dwalkes probably knows this better, but I believe on the NVIDIA boards the nv_update_engine is used to update U-boot/kernel/dtb.

With this state-script containing the gist of it:

github.com

mendersoftware/meta-mender-community/blob/zeus-test/meta-mender-tegra/recipes-mender/tegra-state-scripts/files/redundant-boot-install-script#L62


# Run the update engine in the context of the just-installed rootfs
# to ensure that the TNSPEC in the config file it uses matches the
# TNSPEC in the update payload.  But first we have to set up the
# configuration file with the TNSPEC.
if [ -L "${mnt}/etc/nv_boot_control.conf" -a -x "${mnt}/usr/bin/setup-nv-boot-control" ]; then
    rm "${mnt}/etc/nv_boot_control.conf"
    if ! chroot "${mnt}" /usr/bin/setup-nv-boot-control; then
	echo "ERR: could not initialize nv_boot_control.conf in new rootfs" >&2
    fi
fi
if ! chroot "${mnt}" /usr/sbin/nv_update_engine --install no-reboot; then
    echo "ERR: bootloader update failed" >&2
    cleanup
    exit 1
fi
echo "Successful bootloader update"
if [ -n "$COPY_MACHINE_ID" -a -f /etc/machine-id -a -f "${mnt}/etc/machine-id" ]; then
    cp /etc/machine-id "${mnt}/etc/machine-id"
    echo "Copied current /etc/machine-id to new rootfs"
fi
cleanup

naro · June 30, 2020, 11:07am

thanks for your reply!
We tried running the nv_update_engine. It didn’t update the device tree. But my understanding is that the devicetree is not part of the BUP?

If it helps we get the following entry_table in the /opt/ota_package directory after running nv_update_engine:
root@jetson-tx2:~# cat /opt/ota_package/entry_table
[2020-06-30 13:05:58] bpmp-fw,1368,533904,2,1,1
[2020-06-30 13:05:58] sce-fw,535272,125568,2,1,1
[2020-06-30 13:05:58] cpu-bootloader,660840,284128,2,1,1
[2020-06-30 13:05:58] mb2,944968,116816,2,0,1
[2020-06-30 13:05:58] spe-fw,1061784,95984,2,0,1
[2020-06-30 13:05:58] secure-os,1157768,313152,2,1,1
[2020-06-30 13:05:58] mb1,1470920,99664,2,2,1
[2020-06-30 13:05:58] mts-bootpack,1570584,2082144,2,1,1
[2020-06-30 13:05:58] mts-preboot,3652728,63104,2,0,1
[2020-06-30 13:05:58] sc7,3715832,23632,2,1,1
[2020-06-30 13:05:58] kernel,3739464,483728,2,1,1

mirzak · June 30, 2020, 11:20am

Ok, then I am not sure how it works. So will defer to @dwalkes or @madisox who have primarily been involved in the tegra integrations…

madisox · June 30, 2020, 11:55am

Parts of the BUP payload, like the DTB, are specific to a version of the TX2 module you have in your device. Only if a copy in the payload matches the version of your module will nv_update_engine install it.

The version information it uses for matching is in /etc/nv_boot_control.conf on the TNSPEC line. Can you post that here?

naro · June 30, 2020, 12:02pm

Thanks for your help!
Here is the requested output

[2020-06-30 13:57:57] root@jetson-tx2:~# cat /etc/nv_boot_control.conf
[2020-06-30 13:57:59] TNSPEC 3310-D00-1000-F.0-1-0-jetson-tx2-mmcblk0p1
[2020-06-30 13:57:59] TEGRA_CHIPID 0x18
[2020-06-30 13:57:59] TEGRA_OTA_BOOT_DEVICE /dev/mmcblk0boot0
[2020-06-30 13:57:59] TEGRA_OTA_GPT_DEVICE /dev/mmcblk0boot1

naro · June 30, 2020, 1:05pm

and for what it’s worth I get this when I grep for 3310 in the bl_update_payload (not sure if it’s relevant):

root@jetson-tx2:~# strings /opt/ota_package/bl_update_payload | grep 3310
3310-1000-B00-A

madisox · June 30, 2020, 3:28pm

Yes, that’s the problem, then. If you’re using the zeus branch in meta-tegra (based on L4T R32.2.3), BUP payloads built only support a single board revision. The zeus-l4t-r32.3.1 branch builds multi-spec BUP payloads, and it should be including the necessary artifacts for the revision D00 FAB.

naro · June 30, 2020, 3:51pm

We are on the zeus-l4t-r32.3.1 branch, actually. Is there a specific (recent) commit that we need to be aware of?
Is is possible to misconfigure/misuse this feature somehow on our end?
Can I check something in my build to verify the generation of the payload works as expected?

Thanks!

naro · June 30, 2020, 6:09pm

I tried to dig a bit deeper. But to be honst I don’t quite understand how the different bup payloads are getting build and deployed. I see multiple payloads being deployed in the images directory (e.g. u-boot-dtb.bin.bl_only.bup-payload) but they all contain only the 3310-1000-B00-A string.

But the generated generate_bup_payload.sh file looks like this and does contain the paramenters specified in jeston-tx2.conf:

#!/bin/bash
rm -rf signed multi_signed rollback.bin payloads_t18x
export BOARDID=3310
export fuselevel=fuselevel_production
export localbootfile=boot.img
export CHIPREV=0
MACHINE=jetson-tx2 FAB=“B00” BOARDSKU="" BOARDREV="" ./tegra186-flash-helper.sh --bup ./flash.xml.in tegra186-quill-p3310-1000-c03-00-base.dtb jetson-tx2.cfg 0x7090000 “@" MACHINE=jetson-tx2 FAB="B02" BOARDSKU="" BOARDREV="" ./tegra186-flash-helper.sh --bup ./flash.xml.in tegra186-quill-p3310-1000-c03-00-base.dtb jetson-tx2.cfg 0x7090000 "@”
MACHINE=jetson-tx2 FAB=“C04” BOARDSKU="" BOARDREV="" ./tegra186-flash-helper.sh --bup ./flash.xml.in tegra186-quill-p3310-1000-c03-00-base.dtb jetson-tx2.cfg 0x7090000 “@" MACHINE=jetson-tx2 FAB="D00" BOARDSKU="" BOARDREV="" ./tegra186-flash-helper.sh --bup ./flash.xml.in tegra186-quill-p3310-1000-c03-00-base.dtb jetson-tx2.cfg 0x7090000 "@”
MACHINE=jetson-tx2 FAB=“D01” BOARDSKU="" BOARDREV="" ./tegra186-flash-helper.sh --bup ./flash.xml.in tegra186-quill-p3310-1000-c03-00-base.dtb jetson-tx2.cfg 0x7090000 “$@”

Should this build result in one BUP package that will work on all specified board revisions? Or do we need to explicitly set this information somewhere?

madisox · June 30, 2020, 9:42pm

Should this build result in one BUP package that will work on all specified board revisions?

It should, yes, once it’s done. You can verify this with the BUP_generator.py script, which you can extract from the L4T BSP (in the Linux_for_Tegra/bootloader directory).

Example from one of my builds:

$ BUP_generator.py -c u-boot-dtb.bin.bup-payload 
BLOB PATH:
/build/madison/zeus-l4t-r32.3.1/build/tmp/deploy/images/jetson-tx2/u-boot-jetson-tx2.bin.bup-payload

BLOB HEADER:
       Magic: NVIDIA__BLOB__V2
     Version: 0x00020000
   Blob Size: 10,472,000 bytes
 Header Size: 48 bytes
 Entry Count: 36 partition(s)
        Type: 0 (0 for update, 1 for BMP)
Uncompressed
   Blob Size: 10,472,000 bytes
   Accessory: 0x0000000000000000

ENTRY TABLE:
|   part_name    |  offset  | part_size | version | op_mode |                  tnspec                 |
|        bpmp-fw |     4368 |    533904 |    2    |    0    |                                         |
|         sce-fw |   538272 |    125568 |    2    |    0    |                                         |
| cpu-bootloader |   663840 |    284128 |    2    |    0    |                                         |
|            mb2 |   947968 |    116816 |    2    |    0    |                                         |
|         spe-fw |  1064784 |     95984 |    2    |    0    |                                         |
|      secure-os |  1160768 |    243520 |    2    |    0    |                                         |
|            mb1 |  1404288 |     99664 |    2    |    2    |                                         |
|   mts-bootpack |  1503952 |   2082144 |    2    |    2    |                                         |
|    mts-preboot |  3586096 |     63104 |    2    |    2    |                                         |
|            sc7 |  3649200 |     23632 |    2    |    2    |                                         |
|    bpmp-fw-dtb |  3672832 |    466240 |    2    |    0    | 3310-B00-1000--1-0-jetson-tx2-mmcblk0p1 |
|    bpmp-fw-dtb |  4139072 |    466240 |    2    |    0    | 3310-B02-1000--1-0-jetson-tx2-mmcblk0p1 |
|    bpmp-fw-dtb |  4605312 |    466240 |    2    |    0    | 3310-C04-1000--1-0-jetson-tx2-mmcblk0p1 |
|    bpmp-fw-dtb |  5071552 |    466240 |    2    |    0    | 3310-D00-1000--1-0-jetson-tx2-mmcblk0p1 |
|    bpmp-fw-dtb |  5537792 |    466240 |    2    |    0    | 3310-D01-1000--1-0-jetson-tx2-mmcblk0p1 |
| bootloader-dtb |  6004032 |    371584 |    2    |    0    | 3310-B00-1000--1-0-jetson-tx2-mmcblk0p1 |
| bootloader-dtb |  6375616 |    371584 |    2    |    0    | 3310-B02-1000--1-0-jetson-tx2-mmcblk0p1 |
| bootloader-dtb |  6747200 |    371584 |    2    |    0    | 3310-C04-1000--1-0-jetson-tx2-mmcblk0p1 |
| bootloader-dtb |  7118784 |    371584 |    2    |    0    | 3310-D00-1000--1-0-jetson-tx2-mmcblk0p1 |
| bootloader-dtb |  7490368 |    371584 |    2    |    0    | 3310-D01-1000--1-0-jetson-tx2-mmcblk0p1 |
|            BCT |  7861952 |      3584 |    2    |    2    | 3310-B00-1000--1-0-jetson-tx2-mmcblk0p1 |
|            BCT |  7865536 |      3584 |    2    |    2    | 3310-B02-1000--1-0-jetson-tx2-mmcblk0p1 |
|            BCT |  7869120 |      3584 |    2    |    2    | 3310-C04-1000--1-0-jetson-tx2-mmcblk0p1 |
|            BCT |  7872704 |      3584 |    2    |    2    | 3310-D00-1000--1-0-jetson-tx2-mmcblk0p1 |
|            BCT |  7876288 |      3584 |    2    |    2    | 3310-D01-1000--1-0-jetson-tx2-mmcblk0p1 |
|        MB1_BCT |  7879872 |     50096 |    2    |    0    | 3310-B00-1000--1-0-jetson-tx2-mmcblk0p1 |
|        MB1_BCT |  7929968 |     50096 |    2    |    0    | 3310-B02-1000--1-0-jetson-tx2-mmcblk0p1 |
|        MB1_BCT |  7980064 |     50096 |    2    |    0    | 3310-C04-1000--1-0-jetson-tx2-mmcblk0p1 |
|        MB1_BCT |  8030160 |     50096 |    2    |    0    | 3310-D00-1000--1-0-jetson-tx2-mmcblk0p1 |
|        MB1_BCT |  8080256 |     50096 |    2    |    0    | 3310-D01-1000--1-0-jetson-tx2-mmcblk0p1 |
|         kernel |  8130352 |    483728 |    2    |    0    |                                         |
|     kernel-dtb |  8614080 |    371584 |    2    |    0    | 3310-B00-1000--1-0-jetson-tx2-mmcblk0p1 |
|     kernel-dtb |  8985664 |    371584 |    2    |    0    | 3310-B02-1000--1-0-jetson-tx2-mmcblk0p1 |
|     kernel-dtb |  9357248 |    371584 |    2    |    0    | 3310-C04-1000--1-0-jetson-tx2-mmcblk0p1 |
|     kernel-dtb |  9728832 |    371584 |    2    |    0    | 3310-D00-1000--1-0-jetson-tx2-mmcblk0p1 |
|     kernel-dtb | 10100416 |    371584 |    2    |    0    | 3310-D01-1000--1-0-jetson-tx2-mmcblk0p1 |

naro · July 1, 2020, 6:38pm

Hi @madisox,

I have the feeling I am missing something very basic in my configuration.
This is the output for the same file on my build. There are no devicetrees included:

> naro@naro:~/drive_yocto/yocto_43/build/tmp/deploy/images/jetson-tx2$ ./BUP_generator.py -c u-boot-jetson-tx2.bin.bup-payload
> BLOB PATH:
> /home/naro/drive_yocto/yocto_43/build/tmp/deploy/images/jetson-tx2/u-boot-jetson-tx2.bin.bup-payload
> 
> BLOB HEADER:
>        Magic: NVIDIA__BLOB__V2
>      Version: 0x00020000
>    Blob Size: 4,223,192 bytes
>  Header Size: 48 bytes
>  Entry Count: 11 partition(s)
>         Type: 0 (0 for update, 1 for BMP)
> Uncompressed
>    Blob Size: 4,223,192 bytes
>    Accessory: 0x0000000000000000
> 
> ENTRY TABLE:
> |   part_name    |  offset | part_size | version | op_mode | tnspec |
> |        bpmp-fw |    1368 |    533904 |    2    |    0    |        |
> |         sce-fw |  535272 |    125568 |    2    |    0    |        |
> | cpu-bootloader |  660840 |    284128 |    2    |    0    |        |
> |            mb2 |  944968 |    116816 |    2    |    0    |        |
> |         spe-fw | 1061784 |     95984 |    2    |    0    |        |
> |      secure-os | 1157768 |    313152 |    2    |    0    |        |
> |            mb1 | 1470920 |     99664 |    2    |    2    |        |
> |   mts-bootpack | 1570584 |   2082144 |    2    |    2    |        |
> |    mts-preboot | 3652728 |     63104 |    2    |    2    |        |
> |            sc7 | 3715832 |     23632 |    2    |    2    |        |
> |         kernel | 3739464 |    483728 |    2    |    0    |        |

madisox · July 1, 2020, 6:57pm

Yes, that’s definitely incomplete. Check the log file for the u-boot-bup-payload do_deploy task, which should be at $BUILDDIR/tmp/work/jetson_tx2-poky-linux/u-boot-bup-payload/1.0/temp/log.do_deploy (or something similar, depending on your distro configuration). I have seen cases where the BUP generator script encounters (and logs) errors but doesn’t fail out the execution of the task. That could be happening here.

naro · July 1, 2020, 7:17pm

Ok, some of the sections look suspicious. It seems the script can’t find relevant files. Do I interpret that correctly? Is there a dependency missing?

[   7.0670 ] Copying signed file in /home/naro/drive_yocto/yocto_43/build/tmp/work/jetson_tx2-poky-linux/u-boot-bup-payload/1.0-r0/bup-payload/signed
[  13.0453 ] tegraparser_v2 --pt flash.xml.bin --generateflashindex /home/naro/drive_yocto/yocto_43/build/tmp/work/jetson_tx2-poky-linux/u-boot-bup-payload/1.0-r0/bup-payload/signed/flash.xm
l.tmp flash.idx
[  13.0504 ] File u-boot-bup-payload-jetson-tx2.dataimg open failed
[  13.1515 ]
Error: Return value 19
Command tegraparser_v2 --pt flash.xml.bin --generateflashindex /home/naro/drive_yocto/yocto_43/build/tmp/work/jetson_tx2-poky-linux/u-boot-bup-payload/1.0-r0/bup-payload/signed/flash.xml.tmp
 flash.idx

signing images failed


Creating update payloads for Jetson-3310-D00-1000--1-0-jetson-tx2-mmcblk0p1 board ...
ls: cannot access 'multi_signed/*/tegra186*bpmp*_sigheader.dtb.encrypt': No such file or directory
WARNING: Failed to find image for bpmp-fw-dtb
ls: cannot access 'multi_signed/*/tegra186-quill*_sigheader.dtb.encrypt': No such file or directory
WARNING: Failed to find image for bootloader-dtb
ls: cannot access 'multi_signed/*/tegra186-quill*_sigheader.dtb.encrypt': No such file or directory
WARNING: Failed to find image for kernel-dtb
ls: cannot access 'multi_signed/*/br_bct*.bct': No such file or directory
WARNING: Failed to find image for BCT
ls: cannot access 'multi_signed/*/mb1_cold_boot*.bct.encrypt': No such file or directory
WARNING: Failed to find image for MB1_BCT
creating bl_and_kernel payload

Thanks a lot for your help on this! I really appreciate it!

madisox · July 2, 2020, 9:33am

Yep, that’s the problem. I think this is due to some recent commits in meta-tegra that need some corresponding changes in the meta-mender-community/meta-mender-tegra layer.

I have some changes on a branch in my fork of that repository that should fix this. In particular, this commit.

naro · July 2, 2020, 8:53am

Awesome! Your patches fixed it for me, the payload contains the devicetrees now!
Many thanks again for your quick help with this issue!

@mirzak, do you have a timeframe when those patches will make it into mender?

madisox · July 2, 2020, 11:23am

I thought @dwalkes had already picked up those patches to upstream to meta-mender-community… I’ll work with him to get them there.

jgrahsl · July 2, 2020, 11:45am

I can confirm that these patches enabled me to update the dtb using nv_update_engine.

Thank you @madisox and @naro

mirzak · July 2, 2020, 11:48am

There was a big update merged from @dwalkes (link) but I can not see that the mentioned change-set was part of that PR.

dwalkes · July 2, 2020, 11:48am

@madisox looks like it didn’t make it into https://github.com/mendersoftware/meta-mender-community/pull/147/commits. I just opened a PR here with the changes that didn’t make it in.

madisox · July 2, 2020, 12:01pm

Thanks, @dwalkes!