The board has the port attached and is really cheap to replace (~US$10 or so), and quite replaceable by most people if you follow Keith's guide. When the board starts acting up, it often shows symptoms you're describing. And as it's a cheap/easy fix, that's what I'd be doing. Try a clean install of stock first (even if you've come from stock recently) to absolutely rule out a rom-side problem if you want to. I'd be surprised if it's still not the same though.
As to specifically what component on the board is causing the issue, no way to say for certain, and it's not really relevant here; the fix is still replacing the entire board given the modular design of phones these days & the low cost of doing so.
I saw Keith post to another thread in Q&A just now, I'm sure he'll also post to this thread & give you a bit more info (he's the resident expert on these things here).