So I thought, wth, let's open the back, maybe there're loose connections, like the battery or something. But now the adhesive is tampered with, a small dent in the case (due to the very strong adhesive) and the back does not pop completely back. One corner is a bit loose, so there's no fooling them that I have opened it.
I updated to 4.4.4 and new issue, the display is gone, just backlight. I managed to fix this eventually. Now I'm back to 4.4.2, and the display is still shifting. During bootup, Google logo is stable, but the next animation is shifting. Bootloader is stable, but sometimes it moves up a grid after I press a button. So I assume that the display moves when the system processes something.
So my conclusion now is that this is probably hardware/GPU issue. Or is it? My warranty is as good as void already since I can't communicate with Japanese, and since I opened the back, it really is void already. I have 2 immediate plans of attack: 1) flash a custom rom, if software issue; 2) buy a triwing screwdriver to check the back, if loose hardware.