Nvidia Jetson Orin L4T image Update

Board Integrations
, ,
1

Setup

Pre flash steps

  • Extract the BSP (will create Linux_for_Tegra folder)
  • Extract the Sample Root Filesystem with sudo into Linux_for_Tegra/rootfs (see below)
  • Extract the OTA tools in the Linux_for_Tegra folder.
  • Populate the L4T with Nvidia provided binaries for the hardware:
sudo ./apply_binaries.sh
  • Optional: Create user/password so we can ssh into Jetson after flash:
sudo ./tools/l4t_create_default_user.sh -u mender -p changeme -n jetson --accept-license

Flashing

From this point all the commands are executed in a Workstation running Ubuntu

Make sure the device is in force recovery mode

We must use the l4t_initrd_flash.sh for Orin NX and Nano or flash.sh for the others. Please visit for JetPack 5 and for JetPack 6 for more information.

Ensure to use the ROOTFS_AB=1 flag to enable the A/B partitioning.

For example:

sudo ROOTFS_AB=1 ROOTFS_RETRY_COUNT_MAX=1 ./flash.sh jetson-agx-orin-devkit mmcblk0p1

After flashing

After the flashing, boot the device and perform the following actions

Partitioning schema

From this point all the commands are executed the Jetson Device, you can connect to it by Serial connection, SSH or by using a monitor and keyboard.

We will need a place where to store the /data directory for persistent data. In this example, we are using the UDA partition for storing the Mender’s persistent /data directory.

Assuming the UDA partition is the XX, then, lets format it for storing files in there. We tested it with ext4.

sudo mkfs.ext4 /dev/mmcblk1pXX

And use it as the /data partition as required by Mender:

sudo mkdir -p /data
sudo su -c "echo '/dev/mmcblk1pXX /data ext4 defaults 0 0' >> /etc/fstab"
sudo mount -a

Install Mender

Download the Mender’s application. In this tutorial we are using the Express installation, but you can choose a different method from here. If you follow this one, you must install curl first or use wget instead.

curl -fLsS https://get.mender.io -o get-mender.sh
# Review the content of the get-mender.sh file
sudo bash get-mender.sh
# The device type is set as jetson-orin for this example

As it was installed as a userspace application, let’s move some configuration files to the /data partition

sudo systemctl stop mender-client
sudo mv /var/lib/mender /data
sudo ln -s /data/mender /var/lib/mender

Let’s create the rootfs-image-jetson Update Module

This Update Module will handle Jetson updates and interact with the Nvidia’s OTA tools and the bootloader interface cli nvbootctrl.

Please read all the comments inside the following code (Update Module), you may need to tweak it

Create the file /usr/share/mender/modules/v3/rootfs-image-jetson with the following content:

#!/bin/sh

set -ue

STATE="$1"
FILES="$2"

# Notice *ota_tools_aarch64.tbz2* and *ota_payload_package.tar.gz* 
# names should match with the name when creating the `mender-artifact`
ota_payload_package="$FILES"/files/ota_payload_package.tar.gz
nvidia_tools="$FILES"/files/ota_tools_aarch64.tbz2

# Nvidia uses the concept of "slots", they will
# be always 0 and 1 values for A and B partitions

MENDER_ROOTFS_PART_A_NUMBER="0"
MENDER_ROOTFS_PART_B_NUMBER="1"

# Some useful information for integrity check
mkdir -p "/data/fs_update/"
mender_boot_part="/data/fs_update/mender_boot_part"
active_num="$(nvbootctrl get-current-slot)"

if test $active_num -eq $MENDER_ROOTFS_PART_A_NUMBER; then
    passive_num=$MENDER_ROOTFS_PART_B_NUMBER
else
    passive_num=$MENDER_ROOTFS_PART_A_NUMBER
fi

case "$STATE" in
      Download)
        if [ "$(nvbootctrl get-number-slots)" != "2" ]; then
            echo "Your device need to be configured to use A/B partitioning"
            exit 1
        fi
        ;;

    ArtifactInstall)
        # Let's record what slot we expect to boot next time
        echo $passive_num > $mender_boot_part
        # You need a big enough DATA partition to fit the rootfs image, 
        # you can change it for any tmp folder as well
        DATA="/data"
        # Let's enable Nvidia's OTA tools to our workdir
        mkdir -p "${DATA}/workdir"
        WORKDIR="${DATA}/workdir"
        tar -jxvf $nvidia_tools -C $WORKDIR
        # Let's move the OTA payload
        mkdir -p "${DATA}/ota/"
        ln -sfn "${DATA}/ota" "/ota"
        mv $ota_payload_package "${DATA}/ota/"
        # And this folder is required for the updater
        mkdir -p "${DATA}/ota_work"
        ln -sfn "${DATA}/ota_work" "/ota_work"
        # Let's trigger the upgrade process from Nvidia's OTA tools
        cd ${WORKDIR}/Linux_for_Tegra/tools/ota_tools/version_upgrade
        # Uncomment the following one for the DevKit using an SD card. **
        # ln -sfn /ota_work/external_device/ /ota_work/internal_device
        # For Orin Nano DevKit
        ./nv_ota_start.sh /dev/mmcblk1 "/ota/ota_payload_package.tar.gz"
        # nv_ota_start.sh set the unused slot to active for next reboot
        >&2 echo "Next boot will load Slot $passive_num" 
        # Cleaning up the disk
        rm -rf "${DATA}/ota_work"
        rm -rf "${DATA}/ota"
        rm -rf "${DATA}/workdir"
        unlink /ota
        # The following one just if uncommented **
        #unlink /ota_work/external_device/
        unlink /ota_work
        ;;

    PerformsFullUpdate)
        echo "Yes"
        ;;

    NeedsArtifactReboot)
        echo "Automatic"
        ;;

    SupportsRollback)
        echo "Yes"
        ;;

    ArtifactVerifyReboot)
        # We use stderr for logging as Mender protocol uses stdout for exchanging messages with the server.
        >&2 echo "ArtifactVerifyReboot: The active partition is $active_num while the last passive was $(cat $mender_boot_part)"
        if test "$(cat $mender_boot_part)" != "$active_num"; then
            exit 1
        fi
        # Recommend calling sync at the end here as well
        sync
        ;;

    ArtifactVerifyRollbackReboot)
        >&2 echo "ArtifactVerifyRollbackReboot: The active partition is $active_num while the last passive was $(cat $mender_boot_part)"
        if test "$(cat $mender_boot_part)" = "$active_num"; then
            exit 1
        fi
        # Recommend calling sync at the end here as well
        sync
        ;;

    ArtifactCommit)
        >&2 echo "ArtifactCommit: The active partition is $active_num while the last passive was $(cat $mender_boot_part)"
        if test "$(cat $mender_boot_part)" = "$active_num"; then
             >&2 echo "Flagging the deployment as successful"
        else
            # If we get here, an upgrade in standalone mode failed to  
            # boot and the user is trying to commit from the old OS.
            # This communicates to the user that the upgrade failed.
            echo "Upgrade failed and was reverted: refusing to commit!"
            exit 1
        fi
        ;;

    ArtifactRollback)
        >&2 echo "ArtifactRollback: The active partition is $active_num while the last passive was $(cat $mender_boot_part)"
        if test "$(cat $mender_boot_part)" = "$active_num"; then
            nvbootctrl set-active-boot-slot $passive_num
        fi
        sync
        ;;
esac
exit 0

And make it executable

sudo chmod +x /usr/share/mender/modules/v3/rootfs-image-jetson

You can start the Mender service at this point

sudo systemctl start mender-client

Golden image generation

From now just modify this image as you like. We are going to take a snapshot to create a OTA package with a Mender artifact.

Snapshot creation

When you are happy with your “new image” we create a snapshot in another workstation from the Jetson device.

USER="user" #host username
HOST="host-ip" #check it is reachable from the Jetson
sudo mender snapshot dump | ssh $USER@$HOST /bin/sh -c 'cat > $HOME/my-jetson-fs.raw'

Nvidia’s OTA package creation

From this point all the commands are executed in a Workstation running Ubuntu 20.04

Assuming now you have the image dump from the previous command in your workstation in $HOME/my-jetson-fs.raw

# Let's create a temp file to mount the previous raw image
mkdir -p tmp-fs
sudo mount -o loop my-jetson-fs.raw tmp-fs
cd tmp-fs
# And we create a tarball with its content
sudo tar -cvpzf ../image_fs.tar.gz --exclude=./data --exclude=./image_fs.tar.gz --one-file-system ./
cd ..
# At this point we don't need the raw image generated by mender snapshot. Clean the folder a little bit if you want
sudo umount tmp-fs
rm my-jetson-fs.raw

Now we can create the ota_payload_package from Nvidia.

Please be aware rootfs_updater.sh is tools/ota_tools/version_upgrade/nv_ota_update.sh renamed, you can modify this file if needed.

cp ./Linux_for_Tegra/tools/ota_tools/version_upgrade/nv_ota_update.sh rootfs_updater.sh

In the following command,

  • -s Skip generating the rootfs image. We are providing one from the golden image by setting the -f option.

  • -r Generates an OTA payload package to only update rootfs. This option is meaningful only for an update without layout (re-partitioning) change.

  • Replace the /path/to from the following command.
  • You may need to use --external-device nvme0n1 if using a SD card, and that is the reason we may need the ln -sfn /ota_work/external_device/ /ota_work/internal_device in the Update Module.
  • image_fs.tar.gz is the tarball we generated at the begining of this section.
cd ./Linux_for_Tegra/tools/ota_tools/version_upgrade/
sudo l4t_generate_ota_package.sh -sr \
        -f /path/to/image_fs.tar.gz \
        -o /path/to/rootfs_updater.sh \
        jetson-agx-orin-devkit R36-3

For now, R35-5 is the current latest L4T release for JetPack 5, R36-3 for JetPack 6, you may need to change it when generating an Upgrade Image for the following releases.

Mender artifact generation

Now we can generate a Mender artifact. Previously we created a tarball that the Nvidia toolset knows how to handle, now we are adding metadata so Mender will know how to trigger all the Nvidia’s OTA tools by using the Update Module we created before

Replace the /path/to from the following command.

You can add your own artifact names. Also, modify the jetson-orin name for another one that better fits your device. Also replace the following Xs with the actual name of your device:

ARTIFACT_NAME="my-update-1.0"
DEVICE_TYPE="jetson-orin"
OUTPUT_PATH="my-update-1.0.mender"
# Make sure the names are consistent with the Update Module, 
# you may need to rename ota_tools_R3X.X.X_aarch64.tbz2 to 
# ota_tools_aarch64.tbz2 for example
IMAGE="./path/to/Linux_for_Tegra/bootloader/jetson-XXXX-devkit/ota_payload_package.tar.gz"
OTA_TOOLS="./path/to/ota_tools_R3X.X.X_aarch64.tbz2" 
mender-artifact write module-image -T rootfs-image-jetson -n ${ARTIFACT_NAME} -t ${DEVICE_TYPE} -o ${OUTPUT_PATH} -f ${IMAGE} -f ${OTA_TOOLS}

And now you can upload to the Mender Server the new my-update-1.0.mender or however you named OUTPUT_PATH in the step above.

Before deployment

From this point all the commands are executed in the Jetson Device, you can connect to it by Serial connection, SSH or by using a monitor and keyboard.

Check in what SLOT you are right now:

sudo nvbootctrl -t rootfs dump-slots-info

Current rootfs slot: A
Active rootfs slot: A
num_slots: 2
slot: 0,             retry_count: 3,             status: normal
slot: 1,             retry_count: 3,             status: normal

Create a deployment

From this point all the commands are in the Mender Web UI at https://hosted.mender.io

Upload the artifact

You will need to click on RELEASES on the left panel and then the purple UPLOAD button, there you can upload your my-update-1.0.mender file.

Deploy the artifact

Then, click on DEVICES, look for the Jetson device and click on it. Then on the + button at the right bottom corner, and then to “Create Deployment for this Device”.

You can now click on DEPLOYMENTS section from the left panel and check its progress.

The Jetson was updated

From this point all the commands are executed back in the Jetson device by using a monitor, SSH or a Serial connection.

First, check in what SLOT you are right now, as you can see, it switched to B:

Current rootfs slot: B
Active rootfs slot: B
num_slots: 2
slot: 0,             retry_count: 3,             status: normal
slot: 1,             retry_count: 3,             status: normal

And finally, you should be able to see anything new that was added as part of the golden image.