This was a problem with my nexus one. Some have mentioned that the US variant, the Xperia Z1s had that issue. It has to do with where the microphone is located. I know the obvious answer is to get blutooth headsets or use the headsets in the box for calls, but sometimes you are doing something and you answer your phone quickly and toss it between your neck and hour head to say something as you quickly finish up a task with your hands..etc
Also, can the headset be used to answer calls and speak to a contact through?