I came from a S4. I had a AT&T S4. I was lucky enough to have received mine with AMDL firmware so I could still use Loki to flash firmware. However it was always a big pain when I had to odin/flash back to stock, I had to remove SIM card and not connect to wifi to make sure it didn't even attempt to download the OTA Update that broke loki and made custom firmwares a thing of the past with its locked bootloader.
Also the S4 had so many bugs with any custom firmware. It was normal to get reboot hands where it got stuck on the S4 logo on boot and you would have to turn the phone on and off sometimes 5 times for it to boot. So annoying.
On top of that, the Camera sucked. Not sure if it was just the firmware ,or the camera itself. My S4's Camera (13MP) took worse photos than my iPhone 4S (8MP). In turn, my Nexus 5 now takes WAY BETTER photos than both phones even though it's only 8MP. The focusing, contrast and brightness and shutter speed adaptation is perfect (after 4.4.1 update which included a camera firmware update). HDR function actually works as well. The S4, in both Samsung based ROMS and AOSP roms would always take blurry junky HDR photos and I could never leave it on. Nexus 5 i have no problem taken HDR shots even from a moving car.
Overall I find the Nexus 5 to be a MUCH MUCH better phone than the Galaxy S4. For AOSP / Custom Rom support its hands down better, and camera. It's also slightly faster. (Talking 300mhz..not much). The only downfalls are:
1. Non removable Battery. I loved this on the S4. I had 3 batteries, one would be on a charger, and i'd carry a spare with me and just swap it out when needed so I never had to charge the phone. I could start out each morning with a fresh 100% charge without worrying about plugging in my phone at night.Battery life is the same as the S4. Just I liked that feature.
2. No SD card. Not a huge problem. I had a 32gb S4 and a 64gb microSD card. I thought I couldnt live without it but its fine. I just backup my phone to my PC once every few days just in case something happens. (I liked microSD so if the phone died, I still could get my photos). I got this, the Meenova USB OTG card reader. http://www.meenova.com/st/p/m3r.html
Works great! I use it for TWRP backups / nanoids. I also use it to store seasons of TV shows, then copy them over to my phone half a season or so at a time as needed so I dont have to fill up my phone. Very handy.
3. Hardware buttons. I liked the S4's hardware buttons. I've gotten used to not having them, just a preference thing really.
4. Screen. The screen is the same resolution. You will get the same detail on it. However, you're comparing AMOLED to a regular LCD. The AMOLED on the S4 has more "vibrant" colors. It also had a slightly better viewing angle, and higher brightness and contrast in comparison to the S4 screen. Nothing major, however you will notice it if you're a detail freak.
That about sums it up.
All in all, I'm very happy with my switch from my Galaxy S4 to the Nexus 5 and I would do it again given the choice. Never buying another Samsung phone ever. Sticking with Nexus phones from now on.