According to the source, we have to flash vbmeta. But I flashed vbmeta, vbmeta_system, vbmeta_vendor just in case, as follows.(for unlocked device.)
./fastboot reboot fastboot
./fastboot --disable-verity --disable-verification
flash vbmeta vbmeta-sign.img
./fastboot --disable-verity --disable-verification
flash vbmeta_system vbmeta_system.img
./fastboot --disable-verity --disable-verification
flash vbmeta_vendor vbmeta_vendor.img
./fastboot erase system
./fastboot delete-logical-partition product
./fastboot --disable-verity --disable-verification flash system (your gsi.img)
./fastboot -w
(When I flashed system, I also disabled verity and verification, but it may be unneccesary)
When I replaced GSI rom, I seemed to forget to wiping userdata.
So, if you wipe userdata, android 11 GSI rom may be flashable.
(Android 11 GSI is freshly made and under developing, so some trouble may be happened like MTK devices.)
NOTE: If you want to root M40, hovatek already succeeded to get root. Refer this

(Do not simply patch boot.img and flash. Device will boot-loop.)
The Teclast M40 runs on Android 10 Q and Unisoc T618 Processor. Rooting this model is pretty much like rooting the Teclast P80X except there are some important
forum.hovatek.com