How to set new partition size with mender-convert

General Discussions
, ,
1

OK! I completed the “Convert the Raspberry Pi disk image to support Mender” section, flashed the .sdimg file to a SD disk, booted it on the Pi, the Mender daemon is running and I was able to authorize the device in hosted Mender.

:+1:

Next I tried creating my “golden image”. I flashed 2018-11-13-raspbian-stretch-lite.img to a second SD card, booted it on the Pi, and made a change to the filesystem (I created a file “two” in the /home/pi directory as a marker).

I then copied the SD card on my Mac to a two.img file using dd. (The resulting two.img file was 15GB, the size of the SD card). To check the img file, I flashed the file back to the SD card using Etcher and verified that I was able to boot the SD card on the Pi and see my filesystem change.

Now I ran docker-mender-convert with the raw disk image set to two.img, but it’s trying to create two 15GB partitions:

*** Data partition size set to default value: 128MB ***
*** Total storage size set to default value: 8GB ***
1/9 Repartitioning raw disk image...
	Detected raw disk image with 2 partition(s).
	Calculating partitions' sizes of the Mender image.
	Adjust Mender disk image size to the total storage size (8000MB).
	Defined total storage size of MB is too small.
	Minimal required storage is 30448MB. Aborting.
Check /mender-convert/output/build.log for details.

Is there a way to tell mender-convert that I only want 4GB partitions?

I thought maybe raw-disk-image-shrink-rootfs was what I might want to use, but I’m not sure how to use it:

./docker-mender-convert raw-disk-image-shrink-rootfs --raw-disk-image input/two.img 

IMAGE_NAME=mender-convert

MENDER_CONVERT_DIR="$(pwd)"
mkdir -p output

docker run \
       --mount type=bind,source="$MENDER_CONVERT_DIR,target=/mender-convert" \
       --privileged=true \
       $IMAGE_NAME "$@"
Running mender-convert raw-disk-image-shrink-rootfs --raw-disk-image input/two.img
*** Data partition size set to default value: 128MB ***
*** Total storage size set to default value: 8GB ***
1/1 Shrinking raw disk image root filesystem...
./mender-convert: line 191: local: `=': not a valid identifier
./mender-convert: line 191: local: `512': not a valid identifier
./mender-convert: line 201: 98304 * : syntax error: operand expected (error token is "* ")

Or should I shrink the partition on my “golden” SD card to 4GB before copying it to my .img file?

2

I moved this to a separate topic as we are now entering more generic information.

Is there a way to tell mender-convert that I only want 4GB partitions?

Yes mender-convert has an option --storage-total-size-mb where you can specify a total size in MB, and this is used to calculate the partition sizes. There is also a variable to set the data part size (data-part-size-mb), typically you do not need to set this to a very high number as this will be expanded to occupy the “rest of the disk” on first boot.

To calculate rootfs partition size there is a simple formula,

rootfs partition size = (storage-total-size-mb - data-part-size-mb - boot part size (40MB)) / 2

But there is a caveat, rootfs partition size can not be smaller then the size of the input image.

I thought maybe raw-disk-image-shrink-rootfs was what I might want to use, but I’m not sure how to use it:

And this is where raw-disk-image-shrink-rootfs comes in, to be able to shrink the expanded image to actual size + 20 % free space, and then it can be resized using --storage-total-size-mb

Also the reason why your rootfs is 15GB is because by default Raspbian will also resize the rootfs partition on first boot to fill the disk, and I am assuming that you have a 16 GB SD card.

The command that you used to run raw-disk-image-shrink-rootfs is correct.

The problem you are seeing is fixed here. The reason that you are seeing this bug is that the mender-convert used for rpi0w is not the official mender-convert repository and is instead based on a fork. A link to a very long discussion on why can be found here.

If you look at the linked commit for the raw-disk-image-shrink-rootfs fix, you can easily patch that locally for now. But I am sure that @MarekBelisko will sync up his fork when he has some time.

1 Like
3

Yes certainly. I’ll rebase on latest mender-convert and probably do new fork to also have correct repo name. Thanks. @andrew_wilcox keep you updated.

4

Thank you @mirzak, I appreciate the detailed reply!

5

Hey,
Are --storage-total-size-mb and --data-part-size-mb still supported? 'Cause my docker-mender-convert complaints that these two options are not supported.
How else can the partition sizes be modified from mender-convert?

1 Like
6

Answering the last question myself: Prepend the according yocto variables: https://docs.mender.io/2.2/artifacts/yocto-project/variables
However, the command line arguments are not support anymore, right?

7

Turns out that prepending the MENDER_ variable does not seem to have an effect (except for MENDER_ARTIFACT_NAME. cmd call:

 MENDER_ARTIFACT_NAME=qs-v0.0.1 MENDER_STORAGE_TOTAL_SIZE_MB=8000 MENDER_DATA_PART_SIZE_MB=300 ./docker-mender-convert   --disk-image input/qs-golden-base.img  --config configs/raspberrypi3_config     --overlay ./rootfs_overlay_demo

a few lines below:

2020-01-16 18:51:39 [INFO] [mender-convert-extract] PART 1: SIZE: 256M TYPE: 0xc

Or does the partition size have to be 2 to the power of something?

8

It is only possible to provide the MENDER_ARTIFACT_NAME on the CLI, the rest of the variables are take from configuration files.

You can provide your own values by creating a custom configuration file and provide it using the --config option.

More info can be found here,

https://docs.mender.io/development/artifacts/debian-family/image-configuration#configuration-files

1 Like
9

Hello, I’m new to Mender and I’ve been using the mender-convert scripts on Raspbian Buster. When I first cloned the mender-convert repository the default scripts worked great with my 16g golden images.
After pulling the master yesterday (some client updates were needed) I’m getting similar results to Andrew above and would like to use the raw-disk-image-shrink-rootfs for similar output of sdimg and artifacts if possible. Note: I’m using a PI 3b+ not a 0
New log output:

Running mender-convert --disk-image input/GI_618_1-29-20_Deploy.img --config configs/raspberrypi3_config --overlay ./rootfs_overlay_demo

2020-02-11 15:30:18 [INFO] [mender-convert-modify] Performing platform specific modifications (if any)
2020-02-11 15:30:18 [INFO] [mender-convert-modify] Performing user/local specific modifications (if any)
2020-02-11 15:30:18 [INFO] [mender-convert-modify] Applying rootfs overlay: ./rootfs_overlay_demo
mender-convert-modify has finished. Cleaning up…
Running mender-convert-package: --disk-image input/GI_618_1-29-20_Deploy.img --config configs/raspberrypi3_config --overlay ./rootfs_overlay_demo
2020-02-11 15:30:18 [INFO] [mender-convert-package] Using configuration file: configs/mender_convert_config
2020-02-11 15:30:19 [INFO] [mender-convert-package] Using configuration file: configs/raspberrypi3_config
2020-02-11 15:30:19 [WARN] [mender-convert-package] The allocated boot part size 40 MiB is too small.
2020-02-11 15:30:19 [WARN] [mender-convert-package] The actual boot part size is 256 MiB
2020-02-11 15:30:19 [WARN] [mender-convert-package] Will adjust MENDER_BOOT_PART_SIZE_MB automatically
2020-02-11 15:30:19 [WARN] [mender-convert-package] Considered adjusting the configuration file to avoid this message
2020-02-11 15:30:20 [WARN] [mender-convert-package] The calculated rootfs partition size 3892 MiB is too small.
2020-02-11 15:30:20 [WARN] [mender-convert-package] The actual rootfs image size is 4976 MiB
2020-02-11 15:30:20 [FATAL] [mender-convert-package] You can try adjusting the MENDER_STORAGE_TOTAL_SIZE_MB variable to increase available space, or modify one of the variables IMAGE_ROOTFS_SIZE, IMAGE_ROOTFS_EXTRA_SPACE or IMAGE_OVERHEAD_FACTOR to reduce the size of the root filesystem.
mender-convert-package has finished. Cleaning up…

Where can I add the raw-disk-image-shrink-rootfs option or should I work with the MENDER_STORAGE_TOTAL_SIZE_MB and other options in the mender_convert_config? Thanks for any help.

10

Hi @jem2e4, welcome to Mender hub. The specific option you mention is not part of the new iteration of mender-convert. I suspect the right approach here is to minimize the MENDER_STORAGE_TOTAL_SIZE_MB setting and then grow the size of the data partition on first boot.

@mirzak anything to add?

11

@drewmoseley Thanks for the information! I’ve moved forward with the necessary sizing changes.
However, after using the new mender-convert the unit checked in w/ hosted mender the version is odd:

mender_client_version
25491c0
mender -version reports the same on the unit. Is that current?

12

Hi @jem2e4 that is the commit has of the current head of MASTER. In my Yocto build which is explicitly using a tagged release, it reports “2.1.2”. My guess is that when mender-convert is officially released, this will change on your build as well. @lluiscampos @mirzak can either of you confirm?

13

Hi @jem2e4,

We just released mender-convert 2.0.0. Could you give it a try and see if your issues are fixed in our stable release?

You just need to checkout the correct version with:

git fetch && git checkout 2.0.0

Thanks

14

@lluiscampos, Yes, thanks! It works well.

15

Hi @mirzak ,

I think that would be really helpful to have the rootfs parition size calculation formula in the doc.
Especially on this page: Partition configuration | Mender documentation