[Dev] ICS Camera Driver Development

[zivan56]

Due to the change in ICS that requires moving the camera driver from libcamera to libhardware (like GPS and other devices for Gingerbread), a new driver has to be written.

The problem with this is that many data structures have changed. I couldn't get a single one to compile with the ICS SDK. There are many fundamental changes in the way memory is shared for preview/pictures/video, so that will need to be addressed. I have addressed picture only functionality in the skeleton driver, which will show how to transfer a picture back to the HAL.


ICS Camera Driver Overview

The driver seems to be very generic to most Qualcomm SoCs, so either approach should work for many other devices.

New Driver

See CameraHal_Module.cpp from the ICS source code to see what functions needs to be implemented. It needs to be placed in /system/lib/hw/camera.<target_name_in_build.prop>.so and have the struct "camera_module_t HAL_MODULE_INFO_SYM" inside the code, which the HAL looks for before loading. Once it is loaded, you must implement a few dozen functions to get it to work.


Old Driver

The old drivers primary interface with Android is through libcamera. This loads liboemcamera, which sends commands to the camera kernel driver and the Snapdragon DSP. Liboemcamera is a propriatary library with no documentation apart from the source for QualcommCameraHardware.cpp (which compiles to libcamera.so). The actual Linux kernel module is just a simple I2C bridge to control the camera, and provides no real interface.

Current code/skeleton driver available:
http://forum.xda-developers.com/showpost.php?p=20281617&postcount=17

Please do not comment in this thread unless you are able to contribute to the actual development process.

 

[KrewsialNL]

Please keep this DEV only!

Thank you.

 

[blarfiejandro]

If you're having a problem with the autolocks not releasing when needed, just do the locking manually. It's definitely tedious, but doable. I would guess, however, it's going to be easier in the long run to just implement a proper CameraHal module. If structures that you're relying on have changed... it's almost certainly better to rewrite from scratch.

 

[zivan56]

If you're having a problem with the autolocks not releasing when needed, just do the locking manually. It's definitely tedious, but doable. I would guess, however, it's going to be easier in the long run to just implement a proper CameraHal module. If structures that you're relying on have changed... it's almost certainly better to rewrite from scratch.

Unfortunately, I cannot recompile it with the current ICS source, as the structures the module uses have changed significantly. I could get the 2.3 source and compile against that, but then I might as well just re-write it as you have suggested.
Re-writing it based off of QualcommCameraHardware.cpp will involve the exact same hurdles. The previous driver (libcamera.so, which is compiled from QualcommCameraHardware.cpp) supplied only some functionality to Android, and passed numerous others to liboemcamera.so (completely closed source) via the same dlsym's. So there still wouldn't be a proper driver, just one less layer wrapper.

Ultimately, the only solution looks to be just to use the very core initialization methods from libcamera and find the proper ioctl calls to /dev/msm_camera/control0
I have tried to do that earlier, but liboemcamera.so has callbacks to libcamera.so (which may be passed back again to libcameraservice in Android), so its going to take some reverse engineering to figure out out.

I have been working on a skeleton camera HAL driver, but all callbacks to Android cause libcameraservice to crash (like autofocus finished event). Perhaps its an issue of the libcameraservice.so being distributed with the roms here, I will try it on an emulator.

 

[blarfiejandro]

Unfortunately, I cannot recompile it with the current ICS source, as the structures the module uses have changed significantly. I could get the 2.3 source and compile against that, but then I might as well just re-write it as you have suggested.
Re-writing it based off of QualcommCameraHardware.cpp will involve the exact same hurdles. The previous driver (libcamera.so, which is compiled from QualcommCameraHardware.cpp) supplied only some functionality to Android, and passed numerous others to liboemcamera.so (completely closed source) via the same dlsym's. So there still wouldn't be a proper driver, just one less layer wrapper.

Ultimately, the only solution looks to be just to use the very core initialization methods from libcamera and find the proper ioctl calls to /dev/msm_camera/control0
I have tried to do that earlier, but liboemcamera.so has callbacks to libcamera.so (which may be passed back again to libcameraservice in Android), so its going to take some reverse engineering to figure out out.

I have been working on a skeleton camera HAL driver, but all callbacks to Android cause libcameraservice to crash (like autofocus finished event). Perhaps its an issue of the libcameraservice.so being distributed with the roms here, I will try it on an emulator.

With the Optimus V, at least, there's a lot of logic in liboemcamera such that any IOCTLs you call on the device will more or less get ignored unless they're what liboemcamera is expecting. I wouldn't spend a lot of time trying to preserve the 2.3 shim.

 

[zivan56]

With the Optimus V, at least, there's a lot of logic in liboemcamera such that any IOCTLs you call on the device will more or less get ignored unless they're what liboemcamera is expecting. I wouldn't spend a lot of time trying to preserve the 2.3 shim.

So that pretty much only leaves direct calls to liboemcamera basically. I don't see any easy way duplicate liboemcameras functionality, as it does lots of non-camera related hardware setup having to do with the DSP.
It might be easier to backport the old camera library instead to ICS if liboemcamera cannot be used.

 

[Visentinel]

Is it an option to simply port the gingerbread camera related AOSP code into use for ICS that way there is no trouble with the camera.

Sent from my HTC HD2 using Tapatalk

 

[zivan56]

Its an option. The camera API presented to apps themselves doesn't appear to have changed, so it shouldn't affect the functionality of them. Hopefully only libcameraservice needs to be changed.
I will be giving calling liboemcamera a go again before trying that.

 

[zivan56]

Just a small update (no progress on the HD2 itself)

I have a partially complete skeleton HAL driver written dealing with taking images (not panorama or video). I figured out how to deal with the various callbacks, and now have most major events handled (auto focus, take picture, transfer picture to Android)...it even returns a fake image to test memory operations. Will post it later if anybody is interested.

 

[zivan56]

update: managed to get the camera HAL to negotiate the settings with the camera app and HD2 (i.e picture size, saturation, etc) and init the hardware. Now I am struggling with the preview stuff.
Seeing this error:

E/mm-camera 8x vfe( 69): vfe_util_updaterollofftbl: sensor doesn't support rolloff correction by VFE

When it tries to register memory for the camera to put images into. I will have to look through the kernel source and see what its tripping up on.

 

[zivan56]

Next issue is I need /dev/pmem_smipool . Only some kernels provide this, so I will have to look for one that supports it.

 

[kylew1212]

Great work! I know this is developer only thread but I want to help. I do not know how to code however I can test.

Sent from my NexusHD2 using xda premium

 

[zivan56]

I'm afraid unless you know how the pmem stuff works, you can't really.
The main problem now is that the driver (based off QualcommCameraHardware.cpp) and the kernel don't agree on where the shared memory is located (one of MSM_PMEM_OUTPUT1, MSM_PMEM_OUTPUT2 or MSM_PMEM_RAW_MAINIMG, MSM_PMEM_PREVIEW).
Now this depends on the 720p option being enabled/disabled in the kernel (which strangely enough, doesn't actually give 720p support for the HD2).
This is where it errors out:
E/mm-camera 8x-vfe snapshot( 69): vfe_snapshot_raw_axi_init: failed
(message comes from liboemcamera.so)

Kernel reports this:
msm_frame_axi_cfg 1293: pmem region lookup error

So it never transfers the picture/video/preview frames to the shared memory region

Likewise, the QSD8250 CPU has some very unique settings that other MSM devices do not. For example, its preview mode is just video capture mode. Whereas others have a dedicated preview mode.

So far what works:
- Init sensor, set effects, zoom, contrast, etc (and adjust on the fly)
- Take JPEG picture process and transfer to camera app (but only returns solid green due to the kernel not copying the picture into the proper pmem)
- Set video mode

 

[kylew1212]

MSM_PMEM_OUTPUT1, MSM_PMEM_OUTPUT2 or MSM_PMEM_RAW_MAINIMG, MSM_PMEM_PREVIEW

Would it be possible to physically assign a memory address for these variables within the kernal? I understand theirs probably not much I can do, but I'm more than happy to if something comes up I can do. (I can't wait to take programming in school soon)

Sent from my NexusHD2 using xda premium

 

[Stickman89]

If your finding yourself getting stuck at any point the best person I can recommend from past knowledge would be XDA member DZO (Martin) who had done alot of sensational work on porting android to nand onto a number of QVGA WinMo devices i.e. Kaiser, vogue, polaris... rewriting many of the drivers himself from scratch.

I hope he is still active here on XDA, his work will be greatly remembered, even more so for myself as I used to own a HTC Kaiser in the past...

If your interested here's a bit about him from an XDA Interview: http://www.xda-developers.com/android/developer-interview-dzo/

XDA Member Profile: http://forum.xda-developers.com/member.php?u=917443

(Looks like he's still active too xD)

Best Regards,

ST1Cl<^^aN

 

[zivan56]

Would it be possible to physically assign a memory address for these variables within the kernal? I understand theirs probably not much I can do, but I'm more than happy to if something comes up I can do. (I can't wait to take programming in school soon)

Sent from my NexusHD2 using xda premium

Unfortunately, it doesn't look possible. The kernel driver (msm_camera.c) needs a special type of memory that the DSP can write to. The userspace driver tells the kernel to set up the memory and passes the details back to the kernel driver which tells the hardware to write to this special shared memory. I can copy the data from this buffer to the camera HAL, that works fine...but there is no data in the memory obviously.

If your finding yourself getting stuck at any point the best person I can recommend from past knowledge would be XDA member DZO (Martin) who had done alot of sensational work on porting android to nand onto a number of QVGA WinMo devices i.e. Kaiser, vogue, polaris... rewriting many of the drivers himself from scratch.

I hope he is still active here on XDA, his work will be greatly remembered, even more so for myself as I used to own a HTC Kaiser in the past...

If your interested here's a bit about him from an XDA Interview: http://www.xda-developers.com/android/developer-interview-dzo/

XDA Member Profile: http://forum.xda-developers.com/member.php?u=917443

(Looks like he's still active too xD)

Best Regards,

ST1Cl<^^aN

Thanks for the contact. As long as they are Qualcomm MSM/QSD devices, it should be really similar.

 

[zivan56]

Since there were a number of requests for the current code, I am releasing the code for two things:

1. Skeleton.zip - A skeleton HAL which shows you how to do simple image transfers back to the HAL/setting negotiation. This should help with developing drivers for any device or if someone wants to start the HD2 driver from scratch. It will also complete the process of picture transfer back to the camera HAL (with a fixed picture of course)


2. libcamera3.zip - Progress so far on the HD2 driver. It will init the sensor, change/negotiate settings with the HAL.
It also implements notify/data callbacks which have changed significantly with ICS. There are scripts in there to help with getting the driver to the HD2 and chmod the devices so you dont have permission issues.
I used tytung's latest ICS build and the HD2 AOSP dev environment (http://forum.xda-developers.com/showthread.php?t=1361859) see post #3 how to set it up. You may need to pull binaries from the ICS build to the prebuilt folders in the SDK in order to link the driver properly.

The scripts assume the git repository above is in ~/ICS/ and the Android SDK (with adb) is in ~/android-sdk-linux/


Documentation is nonexistant, but it should be easy to figure out. I cannot help anyone with what QualcommCameraHardware.cpp does, I did not change it much apart from implementing the new callbacks and the ICS HAL wrapper.

I wont have much time to work on it, so hopefully someone else can get the ball rolling on getting the shared memory stuff to work properly. Likewise, set up a git repo somewhere for others to contribute.

 

[BoneMo]

thx

thanks for effort and work
how does the development continue ?

 

[zivan56]

I am sure someone will pick it up. Its in the interest of owners of the Nexus One and a bunch of other Qualcomm Snapdragon based phone owners who want the camera working in ICS. If someone fixes the driver I was working on, it should support at least a couple more devices with very small changes in the code (or perhaps 1 driver that works on all of them by detecting the phone).

 

[vua777]

from ankuch ICS sd build: http://forum.xda-developers.com/showpost.php?p=20305195&postcount=677

Yes. I got first image from camera

Edit: Sorry not a dev post