My Xperia U seems to have the same issue...
The devices that came with the NovaThor U8500 chipset are experiencing this GPS problem. It is a hardware problem.
When I use the A-GPS function (using 3g data during the satellite search to calculate the distance of the device until the antennas making a triangulation) i can get signal much faster.
Unfortunately the GPS receiver of these devices is disappointing.