I had identical problems with two 5X's, intermittent & slow wifi, on 2.4G (I don't have a 5G router yet.) My previous Nexus 4's and all other devices worked fine.
BUT, when I upgraded my wireless AP's firmware (it was fairly old) and changed from 40Mhz to 20Mhz band (there is a lot of 2.4G interference in my area) the problems went away completely and it's been solid. I've heard similar stories in other threads about 5X wifi being fixed by router updates.
So try changing to 20Mhz band if you're on 40 now (on 2.4Ghz). Also upgrade your router firmware if it's older.
I'm not making excuses for the phone not working well in this scenario, just saying it made a tremendous difference for me. I wish I had any idea what 'fix' might have been in the firmware updates, but it was like 2+ years worth I skipped over, and it's not documented well. My AP is a Ubiquiti UAP, if anyone has similar.
So I would say to anyone having issues, it's very unlikely your actual phone is bad, but there may be some incompatibility especially with older wifi routers/AP.