# Speed up mender upgrade **URL:** \ **Category:** General Discussions\ **Created:** [March 1, 2019, 11:17am UTC](https://hub.mender.io/t/speed-up-mender-upgrade/268 "2019-03-01T11:17:33Z")\ **Posts on this page:** 17\ **Page:** 1
**Author:** ![DonalHEmb](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/donalhemb/32/75_2.png) [@DonalHEmb](https://hub.mender.io/u/DonalHEmb)\ **Post date:** [March 1, 2019, 11:17am UTC](https://hub.mender.io/t/speed-up-mender-upgrade/268/1 "2019-03-01T11:17:33Z")
Hi There, Is there any way to speed up the flash time of a .mender image? I’ve recently increased the size of our root file systems from about 800MB to 6.5GB, and started using bmap-tools to speed up the flash of the SD card (I can flash a full ~16GB SDIMG image file in a short few minutes). However, increasing the size of the root file system has had a dramatic negative impact on the mender upgrade time (even though the .mender file size is still about 70MB). Is there anything which can be done for this? Perhaps some way of incorporating the use of bmap file into the mender image flash? Cheers, Donal ---
**Author:** ![kacf](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/kacf/32/146_2.png) [@kacf](https://hub.mender.io/u/kacf)\ **Post date:** [March 1, 2019, 11:50am UTC](https://hub.mender.io/t/speed-up-mender-upgrade/268/2 "2019-03-01T11:50:12Z")
This has been discussed a bit in our team before, and the conclusion was that this will be much easier to implement using an update module, which was why we have focused our efforts on developing that first. It is nearing completion and will be in the upcoming Mender 2.0.0. Writing such an update module would be simple, you’d just receive a pipe you stream the bmap file from, and then apply it to the filesystem. See the update modules spec [https://github.com/mendersoftware/mender/blob/35f539c236a796a6565a3bedd192b23f3ce556aa/Documentation/update-modules-v3-file-api.md](https://github.com/mendersoftware/mender/blob/35f539c236a796a6565a3bedd192b23f3ce556aa/Documentation/update-modules-v3-file-api.md) for more information on this. ---
**Author:** ![DonalHEmb](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/donalhemb/32/75_2.png) [@DonalHEmb](https://hub.mender.io/u/DonalHEmb)\ **Post date:** [March 1, 2019, 1:55pm UTC](https://hub.mender.io/t/speed-up-mender-upgrade/268/3 "2019-03-01T13:55:03Z")
Hi Kristian, Thank you for the info. Do you know why the flash takes so long? Is it just the write time for the SD card or is there a bottleneck somewhere else (perhaps mender)? Cheers, Donal ---
**Author:** ![mirzak](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/mirzak/32/2056_2.png) [@mirzak](https://hub.mender.io/u/mirzak)\ **Post date:** [March 1, 2019, 2:12pm UTC](https://hub.mender.io/t/speed-up-mender-upgrade/268/4 "2019-03-01T14:12:10Z")
The reason it is taking long is that your file-system image is 6.5GB and it is probably 99 % zeroes because the compressed artifact is only 70 MB. The mender client currently is not able to handle sparse files, so it needs to write the whole 6.5GB of data even if it just zeroes. One optimization that I normally do to speed up things (taking your sizes as example) is to still allocate the partition size to 6.5GB but the file system size I make relative to actual content size + 15 % extra free space. That way the mender client will only write the actual content + a relative number of zero data for free space. ---
**Author:** ![mterwoord](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/mterwoord/32/100_2.png) [@mterwoord](https://hub.mender.io/u/mterwoord)\ **Post date:** [March 1, 2019, 5:32pm UTC](https://hub.mender.io/t/speed-up-mender-upgrade/268/5 "2019-03-01T17:32:09Z")
@Mirzak, do you have any links/notes on how to make that working? ---
**Author:** ![mirzak](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/mirzak/32/2056_2.png) [@mirzak](https://hub.mender.io/u/mirzak)\ **Post date:** [March 1, 2019, 6:35pm UTC](https://hub.mender.io/t/speed-up-mender-upgrade/268/6 "2019-03-01T18:35:41Z")
I typically do this in Yocto, e.g in the `local.conf` ```auto # Explicitly set this to zero as it might sneak in depending on what we include. IMAGE_ROOTFS_EXTRA_SPACE = "0" # # # If we set this to zero we rely completely on IMAGE_OVERHEAD_FACTOR. IMAGE_ROOTFS_SIZE = "0" # # # This will make sure that we a relative amount of extra free space on rootfs. # # 1.2 means that we will have 20 % extra free space added. IMAGE_OVERHEAD_FACTOR = "1.2" ``` ---
**Author:** ![mterwoord](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/mterwoord/32/100_2.png) [@mterwoord](https://hub.mender.io/u/mterwoord)\ **Post date:** [March 2, 2019, 10:45am UTC](https://hub.mender.io/t/speed-up-mender-upgrade/268/7 "2019-03-02T10:45:31Z")
Alright, so then you flash an “old” image, which has bigger size partitions, and then update using mender to a smaller update? ---
**Author:** ![kacf](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/kacf/32/146_2.png) [@kacf](https://hub.mender.io/u/kacf)\ **Post date:** [March 4, 2019, 9:08am UTC](https://hub.mender.io/t/speed-up-mender-upgrade/268/8 "2019-03-04T09:08:01Z")
> Thank you for the info. Do you know why the flash takes so long? Is it just the write time for the SD card or is there a bottleneck somewhere else (perhaps mender)? Also, we know about one bottleneck, due to small write buffers, which has already been fixed (1a0d2168fddc981212b78e172d4a18f498f04104 in the mender repository, the forum won’t let me link to it for some reason), but not released yet. ---
**Author:** ![DonalHEmb](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/donalhemb/32/75_2.png) [@DonalHEmb](https://hub.mender.io/u/DonalHEmb)\ **Post date:** [March 4, 2019, 9:55am UTC](https://hub.mender.io/t/speed-up-mender-upgrade/268/9 "2019-03-04T09:55:54Z")
> [@kacf](#): > > Also, we know about one bottleneck, due to small write buffers, which has already been fixed (1a0d2168fddc981212b78e172d4a18f498f04104 in the mender repository, the forum won’t let me link to it for some reason), but not released yet. Struggling to find that commit, is there a commit description I could search for? ---
**Author:** ![mirzak](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/mirzak/32/2056_2.png) [@mirzak](https://hub.mender.io/u/mirzak)\ **Post date:** [March 4, 2019, 10:07am UTC](https://hub.mender.io/t/speed-up-mender-upgrade/268/10 "2019-03-04T10:07:29Z")
> Struggling to find that commit, is there a commit description I could search for? > > > Testing showed that writing to the block device in increments of single > sectors …(512 bytes on my eMMC device) contributed to an OOM condition in > the kernel when allocating DMA scatter-gather descriptors. > > Rather than making a zillion small writes and expecting the kernel to > coalesce them for us when we're easily able put this data into already > (virtually) contiguous memory in userspace, we write to the block device > in "medium-size chunks" (around 1 MiB) that are a power-of-two multiple > of the sector size (obviating any concerns about sector misalignment > causing reduced performance or undue wear on the underlying storage). > > The 1 MiB size could be made configurable if need be. > > This commit introduces the following: > > \- Write to block device at some power-of-two multiple of the sector size. > \- We introduce chunkedCopy() which ensures that writes happen exactly in > increments of the specified size (except for the last write after EOF > of the input). Added unit test to verify correct behavior. > > Changelog: Work around occasional OOM bug in mmc driver. > Signed-off-by: Jeremy Trimble \ > Signed-off-by: Kristian Amlie \ ---
**Author:** ![mterwoord](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/mterwoord/32/100_2.png) [@mterwoord](https://hub.mender.io/u/mterwoord)\ **Post date:** [March 10, 2019, 2:52pm UTC](https://hub.mender.io/t/speed-up-mender-upgrade/268/11 "2019-03-10T14:52:35Z")
How does the OS know to ignore the data behind the actual newly flashed data? Is it because the partition was made small and just happens to live in a somewhat larger partition? Is that going OK all the time? ---
**Author:** ![mirzak](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/mirzak/32/2056_2.png) [@mirzak](https://hub.mender.io/u/mirzak)\ **Post date:** [March 11, 2019, 7:38am UTC](https://hub.mender.io/t/speed-up-mender-upgrade/268/12 "2019-03-11T07:38:09Z")
> How does the OS know to ignore the data behind the actual newly flashed data? > Is it because the partition was made small and just happens to live in a somewhat larger partition? Well there is a distinction between partition and filesystem, and they normally do not have any information about the other. The partition size never changes, but you do have the possibility to change the filesystem image size. What you flash with Mender is a filesystem image on to a partition, the filesystem image holds all the necessary information (size etc…) and this is what the OS interacts with. Mender will make the check if your filesystem is larger then the allocated partition, but there are no problems if it is smaller. ---
**Author:** ![HerrMuellerluedensch](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/herrmuellerluedensch/32/513_2.png) [@HerrMuellerluedensch](https://hub.mender.io/u/HerrMuellerluedensch)\ **Post date:** [December 7, 2020, 12:26am UTC](https://hub.mender.io/t/speed-up-mender-upgrade/268/13 "2020-12-07T00:26:37Z")
I just saw this thread and it fit a question I just had: I’m on `zeus` currently and I saw the `mender` artifact comes with a `mender.bmap` file. Thus, is the information in this thread outdated? Can I use the `bmaptool` to speed up flashing an artifact to the inactive partition? ---
**Author:** ![drewmoseley](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/drewmoseley/32/47_2.png) [@drewmoseley](https://hub.mender.io/u/drewmoseley)\ **Post date:** [December 7, 2020, 3:14pm UTC](https://hub.mender.io/t/speed-up-mender-upgrade/268/14 "2020-12-07T15:14:58Z")
No, the bmap file won’t work as it covers the entire artifact including the metadata. It should probably be removed. I believe the current image writing code is fairly well optimized when writing redundant data. @kacf will know more. ---
**Author:** ![dellgreen](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/dellgreen/32/85_2.png) [@dellgreen](https://hub.mender.io/u/dellgreen)\ **Post date:** [December 7, 2020, 5:22pm UTC](https://hub.mender.io/t/speed-up-mender-upgrade/268/15 "2020-12-07T17:22:59Z")
Are we certain that .mender.bmap is not just for a single partition as mine says its image size is 76.3MiB compared to my .sdimg.bmap file which has image size of 3.5GiB? ---
**Author:** ![drewmoseley](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/drewmoseley/32/47_2.png) [@drewmoseley](https://hub.mender.io/u/drewmoseley)\ **Post date:** [December 7, 2020, 6:30pm UTC](https://hub.mender.io/t/speed-up-mender-upgrade/268/16 "2020-12-07T18:30:47Z")
It definitely is just for a single image, but it’s generated against the .mender file including the metadata rather than the binary image that is wrapped in the .mender file. Drew ---
**Author:** ![kacf](https://yyz2.discourse-cdn.com/flex036/user_avatar/hub.mender.io/kacf/32/146_2.png) [@kacf](https://hub.mender.io/u/kacf)\ **Post date:** [December 8, 2020, 7:49am UTC](https://hub.mender.io/t/speed-up-mender-upgrade/268/17 "2020-12-08T07:49:41Z")
> [@drewmoseley](#): > > No, the bmap file won’t work as it covers the entire artifact including the metadata. It should probably be removed. I believe the current image writing code is fairly well optimized when writing redundant data. @kacf will know more. Yes, it is optimized, but not as optimized as bmap. Bmap requires knowledge of filesystem internals, whereas Mender writes as optimized as is possible without knowing about filesystem internals. This means it reads the blocks first (which is faster than writing), and only writes those that have changed. Bmap on the other hand, omits writing entire blocks if they are part of the free space on the filesystem, even if those blocks differ from those in the artifact.