`mender-artifact install` truncates large files

General Discussions
1

Hi all! I’m on mender-artifact v3.1.0 and I’m seeing mender-artifact install truncate a large file when I try to install it into an artifact. I have a 23MB source file, and I’m running

$ mender-artifact install -m 0755 large_src_file artifact.mender:/dst_file
$ mender-artifact cp artifact.mender:/dst_file ./copy_of_large_src_file
$ ls -sh ./copy_of_large_src_file
32K ./copy_of_large_source_file

If I instead use mender-artifact cp large_src_file artifact.mender:/dst_file and copy the file back out, it has the full 23MB file.

This led me to look at the difference in code between cp and install and noticed a difference in how mender-artifact writes the file in the artifact partition. Here’s a patch of a change I made to cli/artifact/copy.go that seems to fix it.

From b38cbd02cb822be3895028dc0582c4b05ab30b09 Mon Sep 17 00:00:00 2001
From: Sam Baxter <sbaxter@izotope.com>
Date: Fri, 25 Oct 2019 13:13:53 +0000
Subject: [PATCH] Fix mender-artifact install for large files


diff --git a/cli/mender-artifact/copy.go b/cli/mender-artifact/copy.go
index 9c20454..052030a 100644
--- a/cli/mender-artifact/copy.go
+++ b/cli/mender-artifact/copy.go
@@ -161,7 +161,7 @@ func Install(c *cli.Context) (err error) {
                        return cli.NewExitError(fmt.Sprintf("%v", err), 1)
                }
                w = f
-               if _, err = io.Copy(w, r); err != nil {
+               if err = f.CopyTo(c.Args().First()); err != nil {
                        return cli.NewExitError(fmt.Sprintf("%v", err), 1)
                }
                return nil
-- 
2.20.1

I guess I’m wondering 1) if this is a legitimate issue or if I’m missing something, and 2) does this patch look like the right way to fix this? I’m happy to make a PR and try to contribute this upstream if this looks good.

2

Tagging @oleorhagen and @kacf who are the ones familiar with these parts of the code.

3

This has been fixed in mender-artifact 3.2.0, you should try that one!

4

Thanks for the quick reply, I’ll verify that works for me. Is there somewhere I can find release notes so that I can look for this sort of thing in the future?

5

You can see the release notes here (second bullet point for mender-artifact). It was released just two days before your initial post, so it was easy to miss I guess. :slight_smile:

2 Likes
6

Ah, that’ll do it. Thanks again! Things are looking good on my end now.