I asked about it and investigated but got no satisfactory responses. Finally I came across this
I uninstalled the camera updates and the CPU usage immediately dropped to normal with only 2 cores being used most of the time.
So, if your phone has been running slow lately, it might be because of the latest camera update. Uninstalling the update might be the answer for you too.