Raspberry Pi 3 Model B/B+ Raspbian

raspbian
raspberry-pi-3

#1

Board description

The Raspberry Pi 3 Model B/B+ is a popular single board computer based on Broadcom SoCs. It is the most powerful board within the Raspberry Pi family and probably the most popular with “makers”.

770A5842-462x322

URL: https://www.raspberrypi.org/products/raspberry-pi-3-model-b-plus
URL: https://www.raspberrypi.org/products/raspberry-pi-3-model-b
Wiki: https://elinux.org/RPi_Hub

Test results

The Raspbian releases in the table below have been tested by the Mender community. Please update it if you have tested this integration on other Raspbian releases:

Rasbian Build Runtime
Raspbian Stretch 2018-11-13 :test_works: :test_works:

Build Means that the image generation completes without errors and outputs images.
Runtime Means that Mender has been verified to work on the board.

Getting started

Prerequisites

  • A Linux-based laptop/workstation (Ubuntu has been verified to work)
  • You need to install Docker Engine to use this tutorial
  • You need to install the following additional packages:
    sudo apt-get install qemu-user-static git

Build Docker image for mender-convert

Open a terminal and clone the mender-convert repository, e.g.

git clone --no-checkout https://github.com/mendersoftware/mender-convert.git

Enter your mender-convert environment:

cd mender-convert

Checkout a fixed revision working revision:

git checkout 1.0.0b

Note that we will use the master branch at a specific commit in this example because the emulated device environment covered below is only in master still.

There is a utility script which can be used to generate the appropriate docker image to run mender-convert:

./docker-build

This will create a container image you can use to run mender-convert.

Download the latest stable Raspberry Pi raw disk image

Go to the official download site for Raspberry Pi images and download the latest image.

In our example we will use the 2018-11-13-raspbian-stretch-lite.zip raw disk image.

Move the binaries into your mender-convert environment

Download the raw Raspberry Pi disk image into a subdirectory input:

mkdir -p input
cd input
wget https://downloads.raspberrypi.org/raspbian_lite/images/raspbian_lite-2018-11-15/2018-11-13-raspbian-stretch-lite.zip

Extract the raw Raspberry Pi disk image:

unzip 2018-11-13-raspbian-stretch-lite.zip && cd ..

Convert the Raspberry Pi disk image to support Mender

With the raw disk image and the container configured above, we can convert the image.

You can get your Hosted Mender tenant token at the My organization page in Hosted Mender.
If you want to use a self-hosted Mender server, please do not use the --tenant-token option and adjust --server-url. Alternately, if you want to build the image for demo purposes, use the --demo flag together with --demo-host-ip option.

Setup the environment prior to running mender-convert:

DEVICE_TYPE="raspberrypi3"
RAW_DISK_IMAGE="input/2018-11-13-raspbian-stretch-lite.img"

ARTIFACT_NAME="2018-11-13-raspbian-stretch-lite"
MENDER_DISK_IMAGE="2018-11-13-raspbian-stretch-lite.sdimg"
TENANT_TOKEN="<INSERT-TOKEN-FROM Hosted Mender>"

NOTE! Update the TENANT_TOKEN with the one you got from My organization page in Hosted Mender.

Run mender-convert inside the container by running:

./docker-mender-convert from-raw-disk-image                 \
            --raw-disk-image $RAW_DISK_IMAGE                \
            --mender-disk-image $MENDER_DISK_IMAGE          \
            --device-type $DEVICE_TYPE                      \
            --mender-client /mender                         \
            --artifact-name $ARTIFACT_NAME                  \
            --bootloader-toolchain arm-linux-gnueabihf      \
            --server-url "https://hosted.mender.io"         \
            --tenant-token $TENANT_TOKEN \
            --demo

NOTE! The --demo flag configures suitable polling intervals for testing and evaluation and this is to not be used in a production environment.

Conversion will take 10-15 minutes, depending on your storage and resources available.

Use the output images

After a successful conversion, the images and artifacts are:

  • output/2018-11-13-raspbian-stretch-lite.sdimg
  • output/2018-11-13-raspbian-stretch-lite.ext4
  • output/2018-11-13-raspbian-stretch-lite.mender

The disk image (with .sdimg suffix) is used to provision the device storage for devices without Mender running already. Please proceed to the official documentation on provisioning a new device for steps to do this.

On the other hand, if you already have Mender running on your device and want to deploy a rootfs update from this conversion, you should use the Mender Artifact files, which have .mender suffix. You can either deploy this Artifact in managed mode with the Mender server as described in Deploy to physical devices or by using the Mender client only in Standalone deployments.

Boot from the SD card and connect to your Mender server

Ensure your device has Internet connectivity (e.g. through Ethernet cable with DHCP support).

After provisioning a SD card with the converted disk image (.sdimg) above, boot your device from it.

After about 10 minutes, you should see your device Pending authorization under the Devices tab in your Mender server.
Authorize your device to join your Mender server.

You can now deploy software updates to your Raspberry Pi using the Mender server!

Create your first update Artifact

In Mender the software updates are packaged as Mender Artifact files.
You already have one Mender Artifact from your initial conversion above.

For testing purposes, we will generate a new Mender Artifact using a emulated device environment.
Enter the device-image-shell subdirectory in mender-convert. If this directory does not exist, make sure you use a recent branch or tag of the mender-convert repository (e.g. master).

cd device-image-shell

Build Docker image for device-image-shell

For simplicity and portability, the emulated device environment is provided as a Docker container.
Build the Docker image:

./docker-build

Use the device-image-shell container image

You can now enter a shell in your device root file system image by running docker-device-image-shell with the desired arguments:

  1. path to your existing root file system image
  2. desired name for the generated Mender Artifact
  3. device type, which Mender uses to ensure compatibility between devices and software

Starting from your original root file system image created above, enter an emulated shell (you might need to adjust the path – the first argument):

./docker-device-image-shell ../output/2018-11-13-raspbian-stretch-lite.ext4 2018-11-13-raspbian-stretch-lite-modified raspberrypi3

You should now see a shell prompt. You are in an emulated environment, so any commands run here will behave as if you ran them on your device! In addition, any changes you make will be preserved in the output root file system image and Mender Artifact.

For example, to update to the latest packages run:

apt update
apt upgrade

When you are done, press Ctrl+D or run the exit command. Generating the Mender Artifact will take a few more minutes, depending on the size of the input image and resources available on your workstation.

After it finishes you can find your new .ext4 and .mender files in the output/ directory, e.g.

  • output/2018-11-13-raspbian-stretch-lite-modified.ext4
  • output/2018-11-13-raspbian-stretch-lite-modified.mender

You can use the Mender Artifact (.mender file) to deploy the changes you made in the shell to all your devices!

Upload and deploy your new Mender Artifact

Go to your Mender server and upload this new Artifact under the Artifacts tab.
This might take a while, as the Raspbian distribution is quite large.

After the Artifact is uploaded, go to the Deployments tab and create a deployment
using this Artifact and your Raspberry Pi device. You should shortly
see the deployment being in progress. This will usually take a bit longer than
uploading your Artifact if you are on the same network as the device.

Verify the update

After the deployment has succeeded, log in to your device and verify that all packages have been updated to the latest version.

Run the following command:

sudo apt update

It should tell you that all packages are up to date!

Now you can deploy the original system again by uploading the original Mender Artifact output
by mender-convert (e.g. 2018-11-13-raspbian-stretch-lite.mender) to your Mender server.

You can also generate more Mender Artifact files using the docker-device-image-shell tool!

An improved workflow to generate Artifacts

The workflow of using an emulated device works for testing purposes, but it might
have some limitations as we are emulating and not logged in to a real device or user.

When working with real deployments the recommended workflow is to have
one golden device, that has not been converted to support Mender.
On this device you carry out all the modifications you need, and then
use the resulting SD card to create Mender Artifact files, in summary:

  • flash vanilla Rasbian to the SD card
  • boot the SD card, log in and make any modifications needed
  • copy the SD card into an image on your workstation (e.g. using dd)
  • run mender-convert with the from-raw-disk-image option to generate a Mender Artifact (like above)
  • upload the Artifact to your Mender server
  • deploy it to your devices

Note that your golden device or SD card is not running Mender and is
not modified during deployments. It is simply the “source” for
generating the Artifacts that you deploy to the devices in the field.

References

  • The documentation on Building a Mender Debian image contains more information about using Mender with the Debian family of distributions.

  • The official Mender documentation explains how Mender works. This is simply a board-specific complement to the official documentation.


#2

Hi, I got to the step to use the device-image-shell container image but it’s giving me an error:

Entering emulated shell in device image. All commands are run as the root user of the device image.
Make changes (e.g. apt update, apt upgrade, wget …) and press Ctrl-D when done.
chroot: failed to run command ‘/bin/bash’: Exec format error

I googled this and it said something about armhf vs amd64 architecture. I’m running on Ubuntu 18.04 64 bit. How do I fix this? Thanks!


#3

I googled this and it said something about armhf vs amd64 architecture. I’m running on Ubuntu 18.04 64 bit.

Yes this is true, errors like these indicate that there is something wrong with the emulation.

Could you try installing:

apt-get install qemu-arm-static

On your Ubuntu machine.


#4

That worked, but with the package called qemu-user-static. Thanks! Now that I am trying to build the development Mender server (from here: https://docs.mender.io/1.7/administration/production-installation) so I can try deploying artifacts. After following all the steps on that page, after I login, I get a bad gateway error. I previously got the demo server working with the virtual devices and images, but the physical Raspberry Pi with the demo image wouldn’t connect to the server as is. After I did some messing around, it worked but when I tried to deploy the artifacts it got stuck at the rebooting stage, probably because there was the same problem connecting to the server. Have you experienced any of these problems? I mainly care about why I am getting the bad gateway error as now I can’t even find any pending devices after I dd’d the modified image to the SD card.


#5

That worked, but with the package called qemu-user-static. Thanks!

Thanks for reporting back, I have updated the instructions to include these steps. Please let me know if you think the updates I made sense.

After I did some messing around, it worked but when I tried to deploy the artifacts it got stuck at the rebooting stage, probably because there was the same problem connecting to the server

The demo artifacts do require some manual modification to be able to connect to the machine where you are running the Mender server. This section in the documentation should cover it:

https://docs.mender.io/1.7/getting-started/deploy-to-physical-devices

I mainly care about why I am getting the bad gateway error as now I can’t even find any pending devices after I dd’d the modified image to the SD card.

Where exactly are you getting the “bad gateway” error? In the GUI?

I would recommend checking out this section in our docs,

https://docs.mender.io/1.7/troubleshooting/mender-server

At least we would need to see some server side logs to better be able help you debug this. How to extract logs is covered in above link.

Also could we move the discussion to https://hub.mender.io/c/general-discussions, just create a post there as I would assume that this is not related to this specific post any more.


#6

I’ve seen the “Bad Gateway” in the Web UI when the server is starting. In my case, simply waiting a bit longer and refreshing the web page was all that was needed.


#7

@mirzak @corsonus did you have to install “qemu-user-static” on the docker host machine? That was a bit surprising to me.

It is installed in the container: https://github.com/mendersoftware/mender-convert/blob/master/device-image-shell/Dockerfile#L7


#8

Unrelated to the above, should we add some docs (here or somewhere else) how to configure files on the data partition? This can be done with mender-artifact cp myfile.cfg image.sdimg:/data/myfile.cfg.


#9

@mirzak @corsonus did you have to install “qemu-user-static” on the docker host machine? That was a bit surprising to me.

Well the Linux kernel on the host does the emulation, but I actually think that this package is needed " binfmt-support", which “qemu-user-static” depends on.


Raspberry Pi 3 Model B/B+
#10

I’m having issues getting the conversion to finish. It always fails on step 9. Running it directly I get the following failure building U-Boot.

I’m running:
./docker-mender-convert install-bootloader-to-mender-disk-image --mender-disk-image "2018-11-13-raspbian-stretch-lite.sdimg" --device-type "raspberrypi3" --bootloader-toolchain arm-linux-gnueabihf --keep

I’m running this from macOS 10.14.2.

Build Log:

1/1 Installing Bootloader to Mender disk image...
	Building U-Boot related files.
HEAD is now at 9214bb597e deconfig: rpi0w: enable mender requirements
D	scripts/Kconfig
input in flex scanner failed
scripts/kconfig/Makefile:121: recipe for target 'rpi_3_32b_defconfig' failed
make[1]: *** [rpi_3_32b_defconfig] Error 2
Makefile:478: recipe for target 'rpi_3_32b_defconfig' failed
make: *** [rpi_3_32b_defconfig] Error 2
input in flex scanner failed
scripts/kconfig/Makefile:46: recipe for target 'silentoldconfig' failed
make[2]: *** [silentoldconfig] Error 2
Makefile:478: recipe for target 'silentoldconfig' failed
make[1]: *** [silentoldconfig] Error 2
make: *** No rule to make target 'include/config/auto.conf', needed by 'include/config/uboot.release'.  Stop.
input in flex scanner failed
scripts/kconfig/Makefile:46: recipe for target 'silentoldconfig' failed
make[2]: *** [silentoldconfig] Error 2
Makefile:478: recipe for target 'silentoldconfig' failed
make[1]: *** [silentoldconfig] Error 2
In file included from ./tools/env/fw_env_private.h:9:0,
                 from tools/env/../../env/flags.c:14,
                 from tools/env/env_flags.c:1:
include/linux/kconfig.h:4:32: fatal error: generated/autoconf.h: No such file or directory
 #include <generated/autoconf.h>
                                ^
compilation terminated.
scripts/Makefile.host:116: recipe for target 'tools/env/env_flags.o' failed
make[1]: *** [tools/env/env_flags.o] Error 1
Makefile:1469: recipe for target 'envtools' failed
make: *** [envtools] Error 2
Error: cannot build U-Boot. Aborting

Thanks for any help


#11

This seems to be cause of an issue as it is building for rpi0w not for raspberry3. Let me check codebase.


#12

Nope false alarm :frowning: this is message where HEAD of u-boot mender is:

HEAD is now at 9214bb597e deconfig: rpi0w: enable mender requirements

I’ve tried on linux machine works perfectly fine. I’ll try later on OSx and let you know how it goes. Which version of mender-convert you’re using? Thanks.


#13

I tried with dfa0adebc9. I also tried this from two different Macs, 10.14.2 and 10.13.3. Both give the same failure.

When I run from my Ubuntu it works just as expected.


#14

Yes I have same issue on OSx system. Strange as everything runs in docker container. I’ll create an issue on mender-convert. Thanks for report and I keep you updated.


#15

Thanks for the input @adamelliot @MarekBelisko. This has actually never been tested on MacOS. I’ve updated the requirements for now and we can look at expanding the platform support.


#16

Hi @eystein, thanks a lot for this post! I didn’t manage to get it working though, by following these steps on Ubuntu 14.04 for a Raspberry Pi 3 B. I end up with a rainbow screen. If I replace Mender’s /boot/kernel7.img with the original kernel and replace root=${mender_kernel_root} with root=/dev/mmcblk0p2 in /boot/cmdline.txt, then everything is fine, but of course I lose the ability to boot the secondary partition, which defeats Mender’s purpose.

I suspected this mender bug but I don’t have any enabled dtoverlay in /boot/config.txt.

Since mender-convert is running in Docker, I suspect I’m not the only one affected. And since it was working for you, I suspect some dependencies declared in the Dockerfile (e.g. ubuntu:18.04 or u-boot-tools) must have changed meanwhile. If you delete all your Docker images and containers and follow these steps again from scratch, do you now end up as well on a rainbow screen?

Or do you have an idea of something I could try out on top of your head? Thanks a lot!


#17

@AurelienLourot do you have serial console log from device? It’s raspbaerypi board only or do you have some HATs (peripherals connected)? Thanks.


#18

What is the size of your SD card and do you have any peripherals connected to the Pi?


#19

@MarekBelisko @mirzak Thanks a lot for your quick answers: you helped me a lot. I quickly discovered that I wasn’t able to get anything on the serial console, even with a standard Raspbian image (without Mender). After a long struggle in vain, I started to suspect the hardware and indeed if I plug the SD card in another Raspberry Pi of the exact same model (3 Model B V1.2), everything works perfectly. Problem solved, thanks again!


#20

@AurelienLourot ok thanks for info. BTW to have serial output you need to add to config.txt line enable_uart=1 to have output on serial console.