Then I decided it might be my rom breaking after a year due to apps? So reinstalled the same rom and had the same problem.
Lastly I went and updated from a 4.2.2 rom to a 4.4.2 rom and the problem has went from it dropping audio to it just not showing it playing and it plays audio randomly later.
With my 4.2.2 rom my temporary fix was to unpair/repair the audio device.
I have already tested this with multiple bluetooth devices (Philips as140 bt clock, Backbeat go 2's, and the oontz angle)
Tested two audio players (Google music, Rocket player)