Dd vs wic for raspberry-pi-3 provisioning

[Tamis]

Hello to all,

I have recently started using a raspberry pi 3 with yocto kirkstone branch and meta-mender at the same branch for constructing an image for my pi. After following all the guidelines at System updates: Yocto Project | Mender documentation I was able create the sdimg image and successfully boot my target.

I have some questions though and I would like some clarifications if possible.
The image constructed from yocto / mender if I make an fdisk I get the following:

fdisk -l core-image-raspberrypi3.sdimg
Disk core-image-raspberrypi3.sdimg: 1,18 GiB, 1262485504 bytes, 2465792 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x86882af8

Device                                   Boot   Start     End Sectors  Size Id Type
core-image-raspberrypi3.sdimg1 *      24576  106495   81920   40M  c W95 FAT32 (LBA)
core-image-raspberrypi3.sdimg2       106496 1155071 1048576  512M 83 Linux
core-image-raspberrypi3.sdimg3      1155072 2203647 1048576  512M 83 Linux
core-image-raspberrypi3.sdimg4      2203648 2465791  262144  128M 83 Linux

I see that the starting sector is the 24576 while the previous sectors are used for the uboot.env file.
So the first 8MB (uboot.env) + 4MB alignment? = 12MBs are used for the uboot.env file
As also stated from the wks file:

cat mender-sdimg.wks
part --source rawcopy --sourceparams="file=/home/<path>/uboot.env" --ondisk "mmcblk0" --align 4096 --no-table
part --source rawcopy --sourceparams="file=/home/<path>/core-image-raspberrypi3.bootimg" --ondisk "mmcblk0" --align 4096 --fixed-size 40 --active 
part --source rawcopy --sourceparams="file=/home/<path>/care-image-raspberrypi3.ext4" --ondisk "mmcblk0" --align 4096 --fixed-size 524288k 
part  --ondisk "mmcblk0" --fstype=ext4 --align 4096 --fixed-size 524288k 
part --source rawcopy --sourceparams="file=/home/<path>/core-image-raspberrypi3.dataimg" --ondisk "mmcblk0" --align 4096 --fixed-size 128

According to the provisioning instructions stated here:
https://docs.mender.io/system-updates-yocto-project/provisioning-a-new-device

Use of the dd command is the method for the first provisioning of the target.

But I also used the yocto wic command to provision the target due to the expand option it has for the data partition to take up all the remaining space.

So after using the following wic command:

wic write core-image-raspberrypi3.dataimg /dev/sdb --expand 1:0,2:0,3:0,4:26G

I get the following partitioning at the target:

fdisk -l /dev/mmcblk0
Disk /dev/mmcblk0: 28.5 GiB, 30601641984 bytes, 59768832 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x86882af8

Device         Boot   Start      End  Sectors  Size Id Type
/dev/mmcblk0p1 *       2048    83967    81920   40M  c W95 FAT32 (LBA)
/dev/mmcblk0p2        83968  1132543  1048576  512M 83 Linux
/dev/mmcblk0p3      1132544  2181119  1048576  512M 83 Linux
/dev/mmcblk0p4      2181120 59768831 57587712 27.5G 83 Linux

Again the target seems to boot and operate fine.

The starting sector now is 2048.
So my question now is:
Where is the uboot.env now is stored?

In both situations the contents of fw_env.config is:
/dev/mmcblk0 0x400000 0x4000
/dev/mmcblk0 0x800000 0x4000

and fw_printenv works normally.

Is there a change that the uboot.env being written with the wic method to the boot partition and overwrite some files there or happens to being written to an empty space and my target just happens to boot or the whole procedure even with the wic command is accepted?

Finally my target has an eMMC card which has two special partitions:
/dev/mmcblk0boot0
/dev/mmcblk0boot1

Which seems to be completely empty now.
Are there any variables in mender which enable to use those partitions in order to store there the uboot.env?
And is it possible to write to those partitions via the sdimg via the initial provisioning?

Thanks a lot in advance,
Any help is welcome!

[TheYoctoJester]

Hi @Tamis!

First, thanks a lot for the elaborate description. I would love to help, but a number of things are still not clear to me.

• the RPi3 in itself does not have an eMMC. So are we actually talking about a CM3 based thing?
• if you want to provision said eMMC through the sdimg, what do you exactly mean? Writing the image to the eMMC, or writing it to a SD card and using some mechanism to provision the eMMC additionally?
• your wic write example refers to the dataimg, which is probably an error.

I’m currently looking into the U-Boot env topic, and once I know we can start to put the puzzle pieces together.

Greetz,
Josef

[Tamis]

Hello @TheYoctoJester,

And thank you for your answer!
Sorry for my late answer. It seems the notification email somewhere slipped

About your questions now:

1. Yeah you are actually correct. This is a custom HW based on CM3 that has eMMC only onboard.
2. I am writing to the eMMC directly. I plug the target to my machine via a USB cable, I then use the rpiboot program provided from: GitHub - raspberrypi/usbboot: Raspberry Pi USB booting code, moved from tools repository
   This will give direct access to eMMC so I am doing dd or wic (this was the question if wic is OK)
3. This is a copy - paste error. I doing the correct:

wic write core-image-raspberrypi3.sdimg /dev/sdb --expand 1:0,2:0,3:0,4:26G

Actually after further investigation I am pretty sure that uboot env corrupts the boot partition and the files being residing at the below affected areas if wic is used

/dev/mmcblk0 0x400000 0x4000
/dev/mmcblk0 0x800000 0x4000

I executed a stings command to the elf files of boot partition and found in there uboot env parts which it shouldn’t exist. I compared them also with the ones produced in my host machine. So I am pretty sure now about the corruption.
The fortunate or unfortunate thing is that it seems that the files being corrupted do not play sometime a significant role so the target might boot.

wic command whatever the start sector is in sdimg always starts from 2048 sector. So this seems to be a bug in wic command, or wic is totally not accepted for writing the image to the target.
Should this be fixed in wic?
But I guess this is a rather yocto related question and not a mender related one.

With dd I always get the correct start sector as it is reported by fdisk in sdimg.
So the sections that uboot env will be written is empty and no corruption happens.

So the findings suggest that wic is not to be trusted to provision and flush the sdimg into the target.

Lastly about the mmcblk0boot partitions.

I have found that writing to the mmcblk0boot partitions directly via flashing an image is not possible but uboot can use them to save its environment there. But uboot needs to be patched.

So I changed the mender patches and I added those variables:

diff --git a/meta-mender-raspberrypi/recipes-bsp/u-boot/patches/0001-configs-rpi-enable-mender-requirements.patch b/meta-mender-raspberrypi/recipes-bsp/u-boot/patches/0001-configs-rpi-enable-mender-requirements.patch
index a84c6208..9de576a8 100644
--- a/meta-mender-raspberrypi/recipes-bsp/u-boot/patches/0001-configs-rpi-enable-mender-requirements.patch
+++ b/meta-mender-raspberrypi/recipes-bsp/u-boot/patches/0001-configs-rpi-enable-mender-requirements.patch
@@ -79,14 +79,16 @@ index f4fdb455f4..0aec385bf7 100644
  CONFIG_SYS_RELOC_GD_ENV_ADDR=y
  CONFIG_ENV_VARS_UBOOT_RUNTIME_CONFIG=y
  CONFIG_MMC_SDHCI=y
-@@ -43,3 +43,7 @@ CONFIG_SYS_WHITE_ON_BLACK=y
+@@ -43,3 +43,9 @@ CONFIG_SYS_WHITE_ON_BLACK=y
  CONFIG_CONSOLE_SCROLL_LINES=10
  CONFIG_PHYS_TO_BUS=y
  CONFIG_OF_LIBFDT_OVERLAY=y
 +CONFIG_ENV_IS_IN_MMC=y
-+CONFIG_ENV_OFFSET=0x800000
-+CONFIG_ENV_OFFSET_REDUND=0x1000000
++CONFIG_ENV_OFFSET=0x0
++CONFIG_ENV_OFFSET_REDUND=0x0
 +CONFIG_SYS_REDUNDAND_ENVIRONMENT=y
++CONFIG_SYS_MMC_ENV_PART=1
++CONFIG_SUPPORT_EMMC_BOOT=y

CONFIG_ENV_OFFSET says that the uboot env is at offset 0x0
CONFIG_ENV_OFFSET_REDUND use the same offset also 0x0
This will enable into uboot the hardware redundant so uboot will use both mmcblk0boot0 and mmcblk0boot1 eMMC partitions

CONFIG_SYS_MMC_ENV_PART=1 for the boot partitions
CONFIG_SUPPORT_EMMC_BOOT=y in order uboot to have access to those partitions

In mender layer the following might be required:

diff --git a/meta-mender-core/recipes-bsp/u-boot/u-boot-mender-helpers.inc b/meta-mender-core/recipes-bsp/u-boot/u-boot-mender-helpers.inc
index 6eb17583..16b31926 100644
--- a/meta-mender-core/recipes-bsp/u-boot/u-boot-mender-helpers.inc
+++ b/meta-mender-core/recipes-bsp/u-boot/u-boot-mender-helpers.inc
@@ -20,6 +20,7 @@ def mender_get_uboot_env_mmc_linux_device_path(d):
     return storage_device + env_part_append
 
 MENDER_UBOOT_MMC_ENV_LINUX_DEVICE_PATH ?= "${@mender_get_uboot_env_mmc_linux_device_path(d)}"
+MENDER_UBOOT_MMC_ENV_LINUX_DEVICE_PATH_REDUND ?= "${@mender_get_uboot_env_mmc_linux_device_path(d)}"
 
 # This should evaluate to the same as MENDER_RESERVED_SPACE_BOOTLOADER_DATA.
 # The only reason it's not evaluated the same way is that we don't have the
@@ -52,7 +53,7 @@ mender_create_fw_env_config_file() {
     # create fw_env.config file
     cat > $1 <<EOF
 ${MENDER_UBOOT_MMC_ENV_LINUX_DEVICE_PATH} ${MENDER_UBOOT_ENV_STORAGE_DEVICE_OFFSET_1} $HEX_BOOTENV_SIZE
-${MENDER_UBOOT_MMC_ENV_LINUX_DEVICE_PATH} ${MENDER_UBOOT_ENV_STORAGE_DEVICE_OFFSET_2} $HEX_BOOTENV_SIZE
+${MENDER_UBOOT_MMC_ENV_LINUX_DEVICE_PATH_REDUND} ${MENDER_UBOOT_ENV_STORAGE_DEVICE_OFFSET_2} $HEX_BOOTENV_SIZE
 EOF
 }

Above code will created the correct fw_env.config file in /etc/ in order for the fw_printenv command to be working correctly.

It will create this:

cat /etc/fw_env.config
/dev/mmcblk0boot0 0x0 0x4000
/dev/mmcblk0boot1 0x0 0x4000

Finally in local.conf or distro.conf you will need the following variables set:

# HW REDUND BOOT PARTITION
# /dev/mmcblk0boot0 0x0 0x4000
# /dev/mmcblk0boot1 0x0 0x4000
MENDER_PARTITION_ALIGNMENT = "4194304"
MENDER_UBOOT_ENV_STORAGE_DEVICE_OFFSET = ""
MENDER_UBOOT_CONFIG_SYS_MMC_ENV_PART = "1"
MENDER_UBOOT_MMC_ENV_LINUX_DEVICE_PATH = "${MENDER_STORAGE_DEVICE}boot0"
MENDER_UBOOT_MMC_ENV_LINUX_DEVICE_PATH_REDUND = "${MENDER_STORAGE_DEVICE}boot1"
MENDER_UBOOT_ENV_STORAGE_DEVICE_OFFSET_1 = "0x0"
MENDER_UBOOT_ENV_STORAGE_DEVICE_OFFSET_2 = "0x0"

As you see I created a new MENDER variable the

MENDER_UBOOT_MMC_ENV_LINUX_DEVICE_PATH_REDUND

in order to make MENDER to also use HW redundancy.

Would it worth above modifications a PR in order mender fully supports the placement of uboot environment in eMMC boot partitions?
It seems a partial support to exist but the HW redundancy does not exist.

Best regards,
Tamis

[TheYoctoJester]

Hi @Tamis,

Thanks a lot for sharing your elaborate analysis! Yes, I agree with the conclusion that some addition to the Mender layers could be helpful, but not really sure how and where to package it. My best guess is that adding it to the meta-mender-raspberrypi partial in meta-mender-community. The reason is that the main layers are meant to support the standard Raspis, and using CMs will always lead to a custom board to some extent - so the result would be more like an example/blueprint, not so much a full blown integration.

Having said that, the function expansions to evaluate the variables obviously would need to go into the main layer. I’ll try to find out how we can proceed there.

One observation from the Yocto side: the example set that you provided for local.conf actually serves as a perfect example of what should go into a custom derived machine.

So all in all - this is a perfect example of a custom board integration - I just am not completely clear about how to put it in a way that others can benefit from.

Let me think a bit and hopefully get back to you soon!
Josef