It turns out the extra reboot has nothing to do with updated raspberry pi firmware.
After I cloned the OS image from the golden device, I used pishrink to shrink the image (I usually don’t do this but my hard drive is running out of space) before running mender-convert. Without passing any argument, pishrink will modify /etc/rc.local to expand rootfs in first boot, and a reboot is triggered in the function (PiShrink/pishrink.sh at 9d9c0dd32cd988f9c3bd9953ca245d33a52fc9e3 · Drewsif/PiShrink · GitHub). Therefore, the Mender artifact created from this modified golden image will reboot an extra time during ArtifactReboot and cause Mender update failure.
So if someone need to shrink their OS image with pishrink before running mender-convert, use option -s to avoid calling the rootfs expansion function.