What actually was happening was the phone was stuck outputting only through the top speaker meant for phone conversations. Normally it would use the bottom speakers and much louder. Only way I've found to fix this was with a reboot.
Is there a fix or way to stop this from happening? Rather not reboot every time this happens.