Provisioning NanoOWL into your Jetson board by using Mender to enable real-time, high-performance computer vision at the edge

[lramirez]

NanoOWL is an image analysis optimization project designed to make OWL-ViT models run in real-time on the NVIDIA Jetson Orin platforms. Its real-time capabilities and a flexible “tree detection” pipeline make it ideal for applications like robotics, autonomous vehicles, smart surveillance, industrial automation, and augmented reality experiences where quick object detection and identification are crucial.

And now, you can install a system like that by using Mender to make it available to your whole fleet

Pre-requisites

Hardware

• This guide requires one of the following Nvidia Jetson device:

  • Jetson AGX Orin (64GB)
  • Jetson AGX Orin (32GB)
  • Jetson Orin NX (16GB)
  • Jetson Orin Nano (8GB)

• A camera attached to our Jetson device.

This tutorial was tested on the Nvidia Jetson AGX Orin devkit. In this example we used a simple Logi webcam over USB.

Flash your Jetson

It is required to install at least the BSP included in the JetPack 5 (L4T r35.x).

Follow the official Nvidia documentation based on your board.

This tutorial was tested on JetPack 6 (L4T r36.x).

Install Mender

Install Mender in your Jetson board by following Mender’s docs instructions.

Also, install mender-artifact in your workstation by following this guide.

Using Mender to deploy a new NGC containerized app (NanoOwl demo)

Creating the Mender Update Module

Create a file named nanoowl-mender-module with the following content:

#!/bin/bash

set -e

STATE="$1"
FILES="$2"

WORKDIR="/opt/workdir"

case "$STATE" in
    SupportsRollback)
        echo "No"
        ;;

    ArtifactInstall)
        # Creating the WORKDIR, you can modify it above
        mkdir -p $WORKDIR
        # Clone the containers' directory
        rm -rf $WORKDIR/jetson-containers
        echo "Cloning the container's repository" 1>&2
        git clone https://github.com/dusty-nv/jetson-containers $WORKDIR/jetson-containers
  
        cd $WORKDIR/jetson-containers
    
        # Let's install some dependencies
        echo "Installing dependencies" 1>&2
        apt update
        apt install -y python3-pip
        pip3 install -r requirements.txt

        # Let's make the container deattached instead of interactive
        echo "Modifying the run.sh to make it non-interactive" 1>&2
        sed -i 's/-it/-dt/g' run.sh

        # For cleaness of stdout, autotag and run.sh executed in two steps
        echo "Defining the correct container's tag to pull and store it in ./autotag-output" 1>&2
        ./autotag -q -o $WORKDIR/jetson-containers/autotag-output nanoowl
        container_id=$(./run.sh $(cat $WORKDIR/jetson-containers/autotag-output)) > /dev/null 2>&1

        # This is executed inside the container
        # Then let's run the video camera for example, please feel free to use other by
        # modifying the following script
        echo "Running the nanoowl's camera example" 1>&2
        docker exec $container_id sh -c "cd examples/tree_demo && python3 tree_demo.py ../../data/owl_image_encoder_patch32.engine > /dev/null 2>&1 &"

        # The above example uses port 7860, change it if you use any other demo and it requires it
        port=7860

        # Try to connect repeatedly (each 5 seconds) until successful
        echo "Busy-waiting for the webserver to be ready" 1>&2
        while ! nc -z localhost $port; do 
            sleep 5
        done
       
        ip_address=$(hostname -I | awk '{print $1}')
        # Finally we want to post this:
        echo "Make sure you have a camera attached and" 1>&2
        echo "Please open a browser in $ip_address:$port" 1>&2
        ;;

    NeedsArtifactReboot)
        echo "No"
        ;;
    
esac
exit 0

Install the Update Module

Copy the Update Module from above by following this guide.

Create a Mender Artifact

Replace the device-type per the one set in the mender setup process. Ideally follow Nvidia’s like jetson-agx-orin-devkit for the Jetson AGX Orin DevKit.

Feel free to change the artifact-name as this is how this artifact will be named in the Mender ecosystem.

The type must match the name of the Update Module. In our example, we named it as nanoowl-mender-module.

mender-artifact write module-image \
    --artifact-name "nanoowl-installation" \
    --output-path "nanoowl-installation.mender"
    --device-type "jetson-agx-orin-devkit" \
    --type "nanoowl-mender-module"

This will generate a Mender’s artifact named nanoowl-installation.mender that we can upload into our Mender Server and deploy into multiple Jetson devices from our fleet.

Whats next?

After this artifact is deployed into a Jetson board, you should be able to open in your workstation’s browser the NanoOwl webapp we installed in port 7860.

This one example on how you can use it:

Have fun!