I've been running ART for several weeks on 4.4.4, and there is absolutely not discernable difference in performance or battery life on ART vs. Dalvik on my phone. I've gone back and forth a few times since the 4.4.4 update (though I've been on ART for a bit because I've been too busy to switch.)
I did find after running it for a few days that the Google Now launcher consumed a bit more battery than Nova Prime, though not much, I think because of the "always listening" "OK Google" support in the launcher (which our phones don't really need with touchless controls.) Obviously, you're seeing something different, I guess.