Yep, it was turned on. Turned it off, will check if it will help.Maybe you have ambient display on (Is on by default)
Forgive my ignorance, but... is there any way to turn off the screen without having to press the power button every time? Something like double tapping the notification bar or holding down the home button?
You can enable Double Tap notification bar to turn off screen. It's in Settings->System->Status Bar->Tap to sleep. (Idk if it's Plus only)