Go ahead and do it. You can always revert to stock by flashing the stock kernel during the time of updates. You may install a custom ROM as well. Moving from one ROM to another or stock barely takes time and you wouldn't lose any personal data, you can backup everything necessary in a few mins.
The nexus 4 is a Dev friendly device, so feel free to enjoy the nexus by flashing your fav kernel or rom's. Everything will be simple enough once you give it a try.
Sent from my Nexus 4 using xda app-developers app
Hit thanks if you believe the post deserves it.