After much research and reading many threads I found that the culprit was google play services. Specifically it was Google (I believe on Now) always trying to figure out location.
It always does a quick wifi check to try and figure out what's what. However due to a bug that has been in the coding since launch of KitKat (hurry up and fix it google) it fails and will just continue to try and try and try.
Solutions I found were to turn off location access on phone or edit Google Play Services Permission, specifically disable stay awake permission.
Turning off location, google now access to location, or both resulted in temporary fixes and I believe google still tried to figure it's location via wifi despite being told not to.
Long story short...if having just wifi location doesn't work.. and having location off doesn't work...
Then why not switch mode to high accuracy in which GPS is left on!
Google uses GPS for location, it works in one try, everyone is happy!
Long story short, if you have KitKat and are experiencing high battery drain with you phone constantly awake when off.
Switch your location settings to High Accuracy. Phone is smart enough to not use GPS when not needed.
It worked for me and it might work for you too!
(Hopefully this fixes it for good)