Default Auto brightness glitch

It goes all over the place and its annoying. Is there a fix beside disabling it?

Sent from my Nexus 5 using XDA Premium 4 mobile app