Oh, goodie... a thread on the problem that's been plaguing me for months.
I wrote an FM Radio app that speaks to the TI and Broadcom combo chip FM portions via HCI. Works well on Bluez based ROMs like CyanogenMod, but no go on MANY stock ROMs with the TI or BC proprietary BT stacks.
I searched for a LONG time, combed through code and tried everything on this thread and then some. On at least one phone and ROM combo I was able to execute the HCI Get Version command (4 1), but not proprietary commands (0x3f).
I've now resorted to taking over the HCI UART and sending commands that way. It works nice but requires root and prevents normal BT.
Despite these bad stacks, there MUST be some way to do this with root at least. I'd have to presume the kernel code can send any HCI commands.
I've even thought of a serial port shim, but BT is not strictly userspace; it's in the kernel too, to support such stuff as BT sockets. I haven't looked too closely at the kernel yet, but there is surely good stuff there, possibly wrapped up in the Android security model.
I ran an hcidump tonight and it "hit me". Perhaps TI and Broadcom don't want us to have easy HCI access so that tools like hcidump can't be used to reverse engineer their precious trade secrets. They certainly don't publish their FM APIs nor I think their BT APIs.