I can confirm it is something to do with the location services. I turned everything off apart from gps, reset with faster fix, re downloaded the agps data with gps test. Now it works perfect even after turning on the Google
location services

Sent from my Galaxy Nexus using xda app-developers app