Hi @Aysha ,
I think you are randomly mixing up things, and therefore they do not work. By custom image, we understand a full linux root filesystem including your modifications. So, the command style mender-artifact write rootfs-image only applies if you have this. The filename docker-container.tar sounds very different. So, it does not apply here!
The docker-artifact-gen script is suitable for packing docker containers into a Mender artifact. Please use it, and please, read the tutorial. It expects the name (and optionally tag) of the container, not an archive. The corresponding section explains the required arguments in detail.
Greetz,
Josef