This is a known issue with the Verizon Galaxy Nexus (toro). It happens with pretty much any Verizon phone that gets a major (unofficial) update- data tends to break or be unreliable until there is a radio update from Verizon. All the more reason to switch off them and onto a carrier that uses GSM like the rest of the world and US (aside from Sprint).
As for others reporting not having the issue, to put it bluntly, they're likely wrong. If they're being served by a single tower, don't travel much, or are on wifi constantly, they probably just don't notice it. The vast majority of people are reporting this as an issue from what I've read, however.
I haven't heard anything from the toro maintainer- I don't know if he is still actively maintaining the branch. The only absolute way to solve it is to flash back to a 4.2.2-based ROM, unfortunately.