Why Nexus 4? Why you picked that? Where I live Nexus 5 and X are of almost same price (N5 being slightly higher), and in UK the price is almost same if I am not wrong, unless you are in a contract. N4 is quite outdated, old.
Moto X definitely. Chose it over Nexus 5. The reasons have been said already, a lot of times, still repeating, Active display, AMOLED, Ergonomics and Size, Call quality, Speaker phone quality, and of course the performance. While with Nexus you will get the fastest Android out there at the moment because of better hardware, and a bigger display with more resolution, but it's IPS, so it's dull, call quality can never match X's, everyone knows how the speaker is on the N5, no active display, if that matters to you, and last but not the least an extremely fragile device.
If there were a phone at X's size, with S5's display, Z2/1020's camera and G3's hardware then I would probably let myself switch off from browsing mobile related websites for six months.