Nexus 5 owner
Some months ago I was wondering almost the same thing, between the Nexus 5 and Motorola X. Finally I decided to buy the Nexus 5 (primarily because its price) but the point here is that now I have 3 months with it, and I can tell you I'm very happy with it.
It depends on your preferences, I like the stock android experience and the huge developer support for the Nexus rather than a phone full of features, sometimes unnecessary features. And with XDA and some time you can have a Nexus full of features too.