Mender-artifact creation to update some files

General Discussions
1

I’m trying to build an artifact to update some files inside mender managed devices.
I’m using following commands

DEVICE_TYPE="raspberrypi3"
./mender-artifact write rootfs-image -t $DEVICE_TYPE  -o ./output/update-applications-1.0.mender -n update-applications-1.0 -u ./input/update.tar \
-s ./input/script/ArtifactInstall_Enter_01-update-applications \
-s ./input/script/ArtifactRollback_Enter_01-update-applications \
-s ./input/script/ArtifactCommit_Enter_01-update-applications

I’ve created 3 scripts to manage files inside “update.tar” archive

ArtifactInstall_Enter_01-update-applications

#!/bin/bash

set -e

STATE="$1"
FILES="$2"

UPDATE_DIR="/tmp/update"

prev_files_web=/tmp/prev_web.tar
prev_files_jar=/tmp/prev_jar.tar
prev_files_sqlite=/tmp/prev_sqlite.tar
update_files="$FILES"/files/update.tar

case "$STATE" in

	NeedsArtifactReboot)
		echo "No"
	;;
	
	SupportsRollback)
		echo "Yes"
	;;
	
	ArtifactInstall)
		if [ ! -f $update_files ]; then
			exit 1
		fi
		
		# unpack update files
		mkdir -p $UPDATE_DIR
		tar -xf ${update_files} -C $UPDATE_DIR
		update_files_web=[ -d $UPDATE_DIR/web ]
		update_files_jar=[ -d $UPDATE_DIR/jar ]
		#update_files_sqlite=[ -d $UPDATE_DIR/sqlite ]
		
		if [ -f $update_files_web ]; then
			## stop services
			# Nodejs APP
			sudo -u pi pm2 stop EASServer
			
			# save previous files
			tar -cvf ${prev_files_web} /home/pi/EASServer/web
			
			# remove Nodejs app
			rm -rf /home/pi/EASServer/web
			
			# create directory
			mkdir -p /home/pi/EASServer/web
			
			# copy new files
			cp -r $UPDATE_DIR/web/* /home/pi/EASServer/web
			
			# assign directory to user
			chown -R pi:pi /home/pi/EASServer/web
			
			# save reference for this upgrade
			touch ${prev_files_web}.lock
		fi
		
		if [ -f $update_files_sqlite ]; then
			## stop services
			# Nodejs APP
			sudo -u pi pm2 stop EASServer
			
			# save previous files
			tar -cvf ${prev_files_sqlite} /home/pi/EASServer/sqlite
			
			# remove Nodejs app
			rm -rf /home/pi/EASServer/sqlite
			
			# create directory
			mkdir -p /home/pi/EASServer/sqlite
			
			# copy new files
			tar -xf ${update_files_sqlite} -C /home/pi/EASServer/sqlite
			
			# assign directory to user
			chown -R pi:pi /home/pi/EASServer/sqlite
			
			# save reference for this upgrade
			touch ${prev_files_sqlite}.lock
		fi
		
		if [ -f $update_files_jar ]; then
			## stop services
			# Java APP
			service EASServer stop
			
			# save previous files
			tar -cvf ${prev_files_jar} /home/pi/EASServer/jar
			
			# remove Nodejs app
			rm -rf /home/pi/EASServer/jar
			
			# create directory
			mkdir -p /home/pi/EASServer/jar
			
			# copy new files
			cp -r $UPDATE_DIR/jar/* /home/pi/EASServer/jar
			
			# assign directory to user
			chown -R pi:pi /home/pi/EASServer/jar
			
			# save reference for this upgrade
			touch ${prev_files_jar}.lock
		fi
		
		## restart services
		# Nodejs APP
		sudo -u pi pm2 start EASServer
		# Java APP
		service EASServer start
		;;
esac

exit 0

ArtifactRollback_Enter_01-update-applications

#!/bin/bash

set -e

STATE="$1"
FILES="$2"

UPDATE_DIR="/tmp/update"

prev_files_web=/tmp/prev_web.tar
prev_files_jar=/tmp/prev_jar.tar
prev_files_sqlite=/tmp/prev_sqlite.tar
update_files="$FILES"/files/update.tar

case "$STATE" in

	ArtifactRollback)
		if [ -f ${prev_files_web}.lock ]; then
			## stop services
			# Nodejs APP
			sudo -u pi pm2 stop EASServer
			
			# remove Nodejs app
			rm -rf /home/pi/EASServer/web
			
			# create directory
			mkdir -p /home/pi/EASServer/web
			
			# copy new files
			tar -xf ${prev_files_web} -C /home/pi/EASServer/web
			
			# assign directory to user
			chown -R pi:pi /home/pi/EASServer/web
		fi
		
		if [ -f ${prev_files_sqlite}.lock ]; then
			## stop services
			# Nodejs APP
			sudo -u pi pm2 stop EASServer
			
			# remove Nodejs app
			rm -rf /home/pi/EASServer/sqlite
			
			# create directory
			mkdir -p /home/pi/EASServer/sqlite
			
			# copy new files
			tar -xf ${prev_files_sqlite} -C /home/pi/EASServer/sqlite
			
			# assign directory to user
			chown -R pi:pi /home/pi/EASServer/sqlite
		fi
		
		if [ -f ${prev_files_jar}.lock ]; then
			## stop services
			# Java APP
			service EASServer stop
			
			# remove Nodejs app
			rm -rf /home/pi/EASServer/jar
			
			# create directory
			mkdir -p /home/pi/EASServer/jar
			
			# copy new files
			tar -xf ${prev_files_jar} -C /home/pi/EASServer/jar
			
			# assign directory to user
			chown -R pi:pi /home/pi/EASServer/jar
		fi
		
		## restart services
		# Nodejs APP
		sudo -u pi pm2 start EASServer
		# Java APP
		service EASServer start
		;;

esac

exit 0

ArtifactCommit_Enter_01-update-applications

#!/bin/bash

set -e

STATE="$1"
FILES="$2"

UPDATE_DIR="/tmp/update"

prev_files_web=/tmp/prev_web.tar
prev_files_jar=/tmp/prev_jar.tar
prev_files_sqlite=/tmp/prev_sqlite.tar
update_files="$FILES"/files/update.tar

case "$STATE" in
	ArtifactCommit)
		# delete all temporary files
		rm -f ${prev_files_web}
		rm -f ${prev_files_sqlite}
		rm -f ${prev_files_jar}
		
		rm -f ${prev_files_web}.lock
		rm -f ${prev_files_sqlite}.lock
		rm -f ${prev_files_jar}.lock
		;;
esac

exit 0

But something went wrong: deployment ends with an error

2019-04-08 08:37:53 +0000 UTC info: Running Mender version 2.0.0b1
2019-04-08 08:37:53 +0000 UTC debug: handle update fetch state
2019-04-08 08:37:53 +0000 UTC debug: status reported, response 204 No Content
2019-04-08 08:37:54 +0000 UTC debug: Received fetch update response &{200 OK 200 HTTP/1.1 1 1 map[Accept-Ranges:[bytes] X-Amz-Request-Id:[159372F0AC2F0159] Etag:[“1ea70d11cd675a388e94908bc0072ebc”] X-Xss-Protection:[1; mode=block] Server:[openresty/1.13.6.2] Date:[Mon, 08 Apr 2019 08:37:54 GMT] Content-Length:[7233536] Connection:[keep-alive] Content-Security-Policy:[block-all-mixed-content] Last-Modified:[Mon, 08 Apr 2019 08:36:19 GMT] X-Content-Type-Options:[nosniff] Content-Type:[application/vnd.mender-artifact] Vary:[Origin] Strict-Transport-Security:[max-age=63072000; includeSubdomains; preload] X-Frame-Options:[DENY]] 0x119a040 7233536 false false map 0x1288200 0x13543c0}+
2019-04-08 08:37:54 +0000 UTC info: State transition: update-fetch [Download_Enter] → update-store [Download_Enter]
2019-04-08 08:37:54 +0000 UTC debug: handle update install state
2019-04-08 08:37:55 +0000 UTC debug: status reported, response 204 No Content
2019-04-08 08:37:55 +0000 UTC debug: Read data from device manifest file: device_type=raspberrypi3
2019-04-08 08:37:55 +0000 UTC debug: Found needed line: device_type=raspberrypi3
2019-04-08 08:37:55 +0000 UTC debug: Current manifest data: raspberrypi3
2019-04-08 08:37:55 +0000 UTC info: no public key was provided for authenticating the artifact
2019-04-08 08:37:55 +0000 UTC debug: checking if device [raspberrypi3] is on compatibile device list: [raspberrypi3]
2019-04-08 08:37:55 +0000 UTC debug: installer: processing script: ArtifactCommit_Enter_01-update-applications
2019-04-08 08:37:55 +0000 UTC debug: installer: processing script: ArtifactInstall_Enter_01-update-applications
2019-04-08 08:37:55 +0000 UTC debug: installer: processing script: ArtifactRollback_Enter_01-update-applications
2019-04-08 08:37:55 +0000 UTC debug: installer: successfully read artifact [name: update-applications-1.0; version: 2; compatible devices: [raspberrypi3]]
2019-04-08 08:37:55 +0000 UTC debug: Trying to install update of size: 7558144
2019-04-08 08:37:55 +0000 UTC debug: Active partition: /dev/mmcblk0p2
2019-04-08 08:37:55 +0000 UTC debug: Detected inactive partition /dev/mmcblk0p3, based on active partition /dev/mmcblk0p2
2019-04-08 08:37:55 +0000 UTC info: native sector size of block device /dev/mmcblk0p3 is 512, we will write in chunks of 1048576
2019-04-08 08:37:55 +0000 UTC info: opening device /dev/mmcblk0p3 for writing
2019-04-08 08:37:55 +0000 UTC info: partition /dev/mmcblk0p3 size: 3573547008
2019-04-08 08:38:01 +0000 UTC info: Running Mender version 2.0.0b1
2019-04-08 08:38:01 +0000 UTC error: Update was interrupted in state: update-store
2019-04-08 08:38:01 +0000 UTC info: State transition: init [none] → cleanup [Error]
2019-04-08 08:38:01 +0000 UTC debug: transitioning to error state
2019-04-08 08:38:01 +0000 UTC debug: statescript: timeout for executing scripts is not defined; using default of 1h0m0s seconds
2019-04-08 08:38:01 +0000 UTC debug: Handling Cleanup state
2019-04-08 08:38:01 +0000 UTC info: State transition: cleanup [Error] → update-status-report [none]
2019-04-08 08:38:01 +0000 UTC debug: statescript: timeout for executing scripts is not defined; using default of 1h0m0s seconds
2019-04-08 08:38:01 +0000 UTC debug: handle update status report state
2019-04-08 08:38:02 +0000 UTC debug: status reported, response 204 No Content
2019-04-08 08:38:02 +0000 UTC debug: attempting to upload deployment logs for failed update

Something worng in my script or artifact creation?
Thank you
Best regards

2

2019-04-08 08:38:01 +0000 UTC error: Update was interrupted in state: update-store

I see the error, but unclear to why. Are you trying to reboot the device or restart the mender service in you state-scripts?

The scripts that you have are written in the Update Module format,

https://github.com/mendersoftware/mender/blob/2.0.0b1/Documentation/update-modules-v3-file-api.md

but you are passing them as state-scripts when creating the Mender Artifact. These are two different features and in the case of state-scripts your scripts will never be called with any arguments.

I would recommend to read trough the Mender Update Module specification linked above and we also have a couple of reference modules described here,

3

I think you’ve misunderstood: State scripts and update modules are two different things. State scripts are single scripts that are specific to each artifact. Update modules are generic installer modules that must be installed on the device first, and then they can be used to install artifacts whose type matches them.

It looks like your general approach with checking the $STATE variable is good, but you need to put all of those checks into one script, and install it on the device under /usr/share/mender/modules/v3 first. Basically stick to the instructions in the Update Modules section, and ignore everything state script related.

4

Thank you for your support.

However in guid you sugest me command is

./mender-artifact write module-image -t $DEVICE_TYPE -o ~/Downloads/web-file-1.mender -T web-file -n web-file-1.1 -f my-file-1 -f my-file-2 -f my-file-3 ...

while my mender-artifact accepts only rootfs-image

root@debian9:/home/marcom/mender-artifact# ./mender-artifact write help
NAME:
   mender-artifact write - Writes artifact file.

USAGE:
   mender-artifact write [global options] command [command options] [arguments...]

VERSION:
   2.3.1

COMMANDS:
     rootfs-image  Writes Mender artifact containing rootfs image

GLOBAL OPTIONS:
   --help, -h  show help
5

You need latest version of the mender-artifact tool (3.0.0b1).

Will update the guide to state this as well

6

Unfortunately version 3 is not accepted by Mender 1.7

Artifact couldn't be uploaded. unsupported version: 3; supported versions are: 1, 2 [Request ID: ade3d46e]

7

That is expected and Update Modules are only supported in 2.0.0b1.

8

Ok! Thank you!
Then, how can I create an artifact to update some files on devices using 1.7 version?

9

That was not possible in earlier releases (including 1.7) and you could only do full image (rootfs) updates.

To create artifacts that only update some files on the device you will need to use the 2.0.0b1 release, which is currently in beta but will soon be converted to an official release.

10

I’m trying to create an artifact with version 1.7 and mender-artifact 2.
I have mender-converter image sdimg file on which I have modified some files using “mount” procedure".
How can I create an artifact starting from it?

11

How can I create an artifact starting from it?

There are certainly ways of doing this, but the recommended workflow id to actually make the changes to the image before you run mender-convert, this way you will always get a Mender Artifact.

But if you want to create one based on the sdimg you can extract the raw root filesystem image and then do this,

https://docs.mender.io/1.7/artifacts/modifying-a-mender-artifact#create-an-artifact-from-a-raw-root-file-system

12

Looking at documentation I think another way shuld be

DEVICE_TYPE="raspberrypi3"
ARTIFACT_NAME="2019-04-09-CHECKPOINT"
MENDER_DISK_IMAGE="./output/2019-04-09-CHECKPOINT.sdimg"

./mender-convert mender-disk-image-to-artifact
            --mender-disk-image $MENDER_DISK_IMAGE
            --device-type $DEVICE_TYPE
            --artifact-name $ARTIFACT_NAME

Is it correct?

13

Yeah forgot about that one. Did it work? :slight_smile:

14

Hi Mirzak,
it works like a charm.
However it is not the correct solution.
After Raspberry installation, customer performs some configurations and upgrading i such may will loose all those settings…
I hope 2.0 version will be released soon, because files upgrade instead of full image upgrade is a must.
Is there an ETA for 2.0 release?

15

I’d say around end of April / beginning of May, but this is just an
estimate.

16

Are customers able to configure the whole system or parts of it? Because if you know what configurations you want to keep before hand you can make sure that they are stored/linked in the persistent data partition and hence keep state across updates.