First of all thanks to:
wmi543 and stockwell for helping me get the stock kernel built.
stockwell also for helping me figure out where the problem could be fixed.
genokolar and stockwell for the working recovery.
And everyone else if I forgot to mention someone.

I modified the kernel sound driver (msm7x30.c) so that when the driver is requested to set the headset mic on/off or route to it, the speakerphone mic is used instead. I don't think there's a different mic for the speakerphone physically on the phone, but the device is called "speaker_mono_tx" and it probably uses different settings than when using the phone in non-speakerphone mode. So I figured it's best to use the speakerphone mic device instead of the normal mic settings (When headphones/line-out used).

Obviously DO THIS AT YOUR OWN RISK. It's always a good idea to take a complete backup with perfect recovery (http://forum.xda-developers.com/show....php?t=1067490) before changing anything.

Edit: This mod is now included in the franco.kernel (http://forum.xda-developers.com/show....php?t=1166560). You can just flash that kernel. Below are the old instructions.

Installation (in linux)
Note the attached boot.img contains the ramdisk of the Finnish operator Saunalahti. It seems to contain some wifi modules too, so it's better to create a new boot.img from your own boot.img (instructions below)

- Boot the phone in bootloader mode. (Hold down vol up and vol down and power on).
- In ubuntu the phone partitions are mounted automatically, you might have to mount them manually in some other distros.
- Find the phone's partition which contains the "image" directory. Go to the image directory.
- There should be a file called "boot.img". Take a backup of the original and replace it with the attached boot.img.
- Unmount all the phone's partitions
- Remove battery to exit bootloader mode.
- Put battery back and boot
- Now the internal mic should work with headphones without mic or for example when connected to a car stereo.

Updating your own boot.img
- Get abootimg source from http://gitorious.org/ac100/abootimg. (I couldn't find a compiled binary for ubuntu, except an old version 0.3).
- I had to install package libblkid-dev to build it. It probably also needs other packages too, but I had them already installed.
- Build it:
Code:
make
- After make finishes you should have the built binary "abootimg".
- Extract the attached kernel.zip (Contains zImage, the built kernel)
- The next command updates the boot.img, so copy your original to boot.img.orig for example.
- Update the boot.img file:
Code:
[path to the abootimg you built]abootimg -u [boot.img from your phone] -k [zImage from kernel.zip]
- Now you can use this updated boot.img on your phone following the instructions above.

Details
As Piron said I think the root of the problem is in the construction of 3.5mm socket so that it doesn't headphones correctly. However I tried to force the detection so that it always detects headphones instead of headset. But that didn't work. The libaudio etc probably have some bugs with the headphone mode (it's never been tested since the phone never detects headphones .

This mod forces the use of the int. mic instead of the headset mic. Obviously with this modified kernel you can't use a headset's mic if you want to.

The modified msm7x30.c is in the attached zip.


It's been quite an adventure but it's rewarding to finally get it to work

I already tested it in my car with background noise etc and it worked perfectly (I heard the other person via the car speakers and he heard me). Also there was no echo which is surprising. The phone's mic is probably directional or something. It's a good mic

Post here if it works for you or if you have problems with the mod.
Attached Files
File Type: zip ksatta_u8800_kernel_headphones_fix.zip - [Click for QR Code] (5.00 MB, 154 views)
File Type: zip kernel.zip - [Click for QR Code] (3.30 MB, 45 views)