[GUIDE] Understanding the Android Source Code

Note - I own an HTC Explorer(Pico), so many things that are written *might* be exclusive to it, or others with some changes.

Describing the Android Source Code Folders -​

I'll take CM11 as an example to explain this.

Now the folders, in alphabetical order-

1. abi - This folder contains a sub folder called cpp which actually contains many C++ files linked to many places.

2. android Remember this?
Yes, it's that android.git folder.
Have a look at this also.

3. art - Yeah, it is the folder that deals with the compilation of the latest android ART runtime. If you're looking into source directories of some other android versions, you won't fins it obviously.

4. bionic - Bionic is mainly a port of the BSD C library to our Linux kernel with the following additions/changes:
- No support for locales.
- No support for wide chars (i.e. multi-byte characters).
- its own smallish implementation of pthreads based on Linux futexes.
- Support for x86, ARM and ARM thumb CPU instruction sets and kernel interfaces.

5. bootable - Boot and startup related code. Some of it is legacy, the fastboot protocol info could be interesting since it is implemented by boot loaders in a number of devices such as the Nexus ones.

6. build - The main entry point of the build system resides here - envsetup.sh, if you follow the instruction in source.android.com you will see that the first step before you do anything to build Android is to use the command source build/envsetup.sh
The script will check few things to make sure all the needed application available in the local machine. It also setup the devices that can be built, which is extracted from the directory device.

7. cts - the compatability tests. The test suite to ensure that a build complies with the Android specification.

8. dalvik - This is the folder responsible for the compilation of the Dalvik runtime for the Android devices.
*Have a look at the difference between the two(art and dalvik) folders and you'll have an idea of how things work in that case*

art -
dalvik -

9. Development - This directory contains application that are not part of the deployed app in the OS. There are some useful application such as widget builder, etc

10. Device - It contains the device specific configurations for many devices.
Note - Many people ask me what, the folders like 'common' and 'generic are for, so here -

> common - This directory contains gps information and also a script that allows you to extract proprietary binary files from your phone to be part of the build process.(You can try to have a look at your device's device tree and then the cm.mk file, where you could find relations of these files. In my case, it shows like this:

> generic - This directory contains the generic device configuration that is called ‘goldfish’. This is the device classification used when building the emulator.

> Google - This directory contains the Android Accessories Kit code. It contains a demokit Android app that allows you to control the ADK board. The ADK firmware can be check out here http://code.google.com/p/microbridge/. There is a good article about this here.

> sample - This directory contains a full example of writing your own Android platform shared library, without changing the Android framework. It also shows how to write JNI code for incorporating native code into the library, and a client application that uses the library. This example is ONLY for people working with the open source platform to create a system image that will be delivered on a device which will include a custom library as shown here. It can not be used to create a third party shared library, which is not currently supported in Android.

11. docs - I contains an important sub-folder called source.android.com. Contains tutorials, references, and miscellaneous information relating to the Android Open Source Project (AOSP). The current iteration of this site is fully static HTML (notably lacking in javascript and doxygen content), and is and/or was maintained by skyler (illustrious intern under Dan Morrill and assistant to the almighty JBQ).

12. external - This directory contains source code for all external open source projects such as SQLite, Freetype, webkit and webview.

13. frameworks - Ah, one of the most important directories. it contains the sources for the framework. Here you will find the implementation of key services such as the System Server with the Package- and Activity managers. A lot of the mapping between the java application APIs and the native libraries is also done here.
A special note on this one - I'd recommend new users to not to play with any file/folder inside the frameworks folder, maybe your ROM doesn't boots then.

14. hardware - Hardware related source code such as the Android hardware abstraction layer specification and implementation. This folder also contains the reference Radio Interface Layer(RIL - To communicate with the modem side) implementation.

15. Kernel - It's not a default folder in the source code, but it's a part of device configuration set-up. It contains the kernel source of your device.

16. libcore - I'll explain this one with the important folders inside this, since every folder performs a different function.

• dalvik - DalvikVM runtime for Android
• dom - Java test classes for DOM
• expectations - Contains information about the test cases
• include - Some C/C++ include files that used for Array and String handling
• json - JSON based Java implementation
• luni - Contains test source code for loading .jar and .dex files
• support - Contains support class file for testing Dalvik
• xml - XML pull and push implementation

17. libnativehelper - I have no idea on this one. If someone knows, share your knowledge. :good:

18. ndk - Contains build scripts and helper files for building the NDK

19. out(Everyone's favorite directory ) - The build output will be placed here after you run make. The folder structure is out/target/product/. In the default build for the emulator the output will be placed in out/target/product/generic. This is where you will find the images used by the emulator to start (or to be downloaded and flashed to a device if you are building for a hardware target).

20. packages - Standard Android application that are available as part of the AOSP - Camera, SMS, Dialer, Launcher, etc

21. pdk - I believe that 'pdk' is the Platform Development Kit, it's basically an SDK/set of tools that Google sends to OEMs to evaluate their framework ahead of each major Android upgrade since Android 4.1.
(Thanks to @yowanvista )

22. prebuilt - Contains files that are distributed in binary form for convenience. Examples include the cross compilations toolchains for different development machines.

23. sdk - This directory contains lots of apps that are not part of operating system. There are quite useful apps that developers can leverage on and can be enhanced further as part of the operating system.

24. system - Source code files for the core Android system. That is the minimal Linux system that is started before the Dalvik VM and any java based services are enabled. This includes the source code for the init process and the default init.rc script that provide the dynamic configuration of the platform.

25. tools - Some external important tools that help in compiling. Not sure though. :/

26. vendor - This directory contains vendors specific libraries. Most of the proprietary binary libraries from non-open source projects are stored here when building AOSP.

One thing - Beyond the above you also have the hidden .repo directory that contains the source for the repo utility. It also holds the manifest specifying what git repositories you want to track for this Android source project. If you have your own additions you could automatically track them by adding a local manifest here. For modifications of the platform framework there are some instructions available in the device/sample folder of the source code tree. That will show you how to add APIs to Android without having to modify the core framework.

Now, many people ask me the difference between the 'Breakfast', 'Lunch' and 'Brunch' command. So, here's what these commands are specific for:​

Breakfast

You may not ever use this command , but in order to explain brunch, we have to explain breakfast first. Breakfast is a function used to configure your build. It keeps track of a list of officially-supported devices to build for, and allows you to choose one.
If you do not put a device, the script will output a list of available devices to build for, and you can then choose yours. Breakfast then goes on to configure your build environment with the correct variables to create your device-specific rom.

Brunch

Defined simply, brunch is equivalent to


This means that it sets up your build environment to be configured for your device, and then commences the build process. mka bacon is just CyanogenMods’s way of saying build the code for your device. It’s generally only used for officially supported devices (ones that you use can choose through the breakfast menu).

Lunch

This ones also pretty simple to explain. It’s used EXACTLY like breakfast, the only difference being the choices you have to build with it. Using lunch, you can choose non-official or non-standard builds for your device. This includes special debug versions and also allows you to build CyanogenMod for use on the Android Emulator. To build after running lunch, simply issue the command mka.

For other commands and help, I recommend you to read this article.

Few common Terms​

Hey, do you ever find few terms on xda, related to development that go over your mind? Well, I recommend you to have a look at this post by Recognized Contributor @TheByteSmasher and give him the Thanks he deserves.

Understanding Android Makefile (Android.mk)​

Well, I bet if go through many folders of the Source Code, you might find the file named Android.mk! Here are few important things regarding that -
These Android.mk files defines how to build that source code. There are well defined specific rules for Android.mk files. Let me summarize them.

Name: We need to define a name for our build (LOCAL_MODULE := )
Local Variables: All builds may have some local variables so to start a new build it is good to clear all local variables (include $(CLEAR_VARS))
Files: We need to write all files we want it to be build (LOCAL_SRC_FILES := main.c)
Tags: Define tags for build. (LOCAL_MODULE_TAGS := eng development)
Libraries: If build needs to be linked to other libraries, we need to define them (LOCAL_SHARED_LIBRARIES := cutils)
Template file: We can define whether our build is executable,library or something else by including template file (include $(BUILD_EXECUTABLE))
BUILD_EXECUTABLE, CLEAR_VARS, etc. variables which are the absolute address of the template files are defined in build/core/config.mk.

Now an example -
The following Android.mk builds a simple APK. The codes beginning with a has(#) are the comments to tell you what the lines actually means.


Last line in Android.mk file builds the APK file. Different source code types must be built differently so we can also use $(BUILD_EXECUTABLE), $(HOST_JAVA_LIBRARY), $(HOST_PREBUILT) etc. variables according to our source code. (Definitions like my-dir, all-subdir-java-files are in build/core/definitions.mk). We can add LOCAL_MODULE_TAGS variable to Android.mk file to determine that module to be installed in that source code built. Here are the some defined tags and their meanings.

Makefile tricks. ​

>Build helper functions
A whole bunch of build helper functions are defined in the file build/core/definitions.mk
Try grep define build/core/definitions.mk for an exhaustive list.
Here are some possibly interesting functions:

1. print-vars - shall all Makefile variables, for debugging
2. emit-line - output a line during building, to a file
3. dump-words-to-file - output a list of words to a file
4. copy-one-file - copy a file from one place to another

>Add a file directly to the output area

You can copy a file directly to the output area, without building anything, using the add-prebuilt-files function.
The following line, extracted from prebuilt/android-arm/gdbserver/Android.mk copies a list of files to the EXECUTABLES directory in the output area:

Understanding the Device Tree​

Note - This is by no means complete, and there will be omissions as have explained all this top of my head and copied pasted certain bits that I have here on my own device tree.

Android.mk - this will tell the build system to include and to build sources specifically for your device. I have explained a bit of it above as well.

AndroidBoard.mk - this is for the kernel, the build system uses that to drop the kernel image in place.

AndroidProducts.mk - specifies the appropriate device's make file, to use for building. i.e. device/htc/pico/pico.mk(In my case), This is device-specific as well.

device_codename.mk - Every device has a codename, and there is a file named as the codename of device. It specifies the properties and extras to copy over into the final output, in this case, it could be for example, pico.mk

BoardConfig.mk - This is the meat of it all, this is where compiler conditional flags are set, partition layouts, boot addresses, ramdisk size, and so on.

Android.mk
^This is how the build will use that to build recovery, sensors, lights and camera (of course there will be more), its saying 'Yo Builder, go into each of the directories specified, and build the respective sources'

AndroidBoard.mk:
^ This one is pretty simple and tells the build system to go to the vendor tree of the device and include AndroidBoardVendor.mk for building.

AndroidProducts.mk
^Just specifying the makefiles of the device.

BoardConfig.mk
Now in this one, I'm taking a BoardConfig.mk file that I found over the internet with a suitable explanation of it. Credits to the post over stackoverflow for the valuable efforts.

^That is an excerpt, notice how we specify kernel's base address, this is how the boot.img gets generated after compilation is done and yet again, gets dropped into out/target/product/lg/gt540/boot.img. Also, more importantly, we're telling the build system to use the target platform for cross-compiling the sources (*TARGET_BOARD_PLATFORM*/*TARGET_CPU_ABI*) There will be more information in there such as conditional flags to pass to the compiler, for an example. we specified the directive HAVE_FM_RADIO to tell it, when it comes to handling the source for the FM radio system, to conditionally compile parts of the source. Again, this is hardware specific and mileage will vary, also this applies to the address for boot. In a nutshell, this is saying 'Yo Builder, read the damn variables and remember them and apply them when cross-compiling those source files!'

Now that the internals of each of those Android build make-files are shown.

Now, onto the vendor/ part of it, in AOSP, simply, once again, correlation and corresponds with the device/ tree, as in continuing with this example, vendor/lg/gt540/ which gets picked up by the lunch. There's more make files in there but the general consensus is there's a directory called proprietary which contains the proprietary libs (due to close-source etc) that gets copied over. The copying over of the libraries gets specified in the file device-vendor-blobs.mk, in this case, gt540-vendor-blobs.mk.

Adding your Own Apps in the Source Code​

The following part of the guide will tell you how to make your own apps to compile with the ROM and be a part of it.

1. Add the App's source code to packages/apps/name_of_app
2. Regardless of how you put the source in packages/apps/, assuming that the source for the app has an Android.mk Makefile, you can get it to automatically build and install the resulting file in your $OUT directory (and thus your .zip) by simply determining the name of the project, which is typically defined in Android.mk with this:
3. Add the project to device.mk (or whatever .mk) in the device folder.
Have a look at an example here

Compiling specific parts of a ROM​

To compile an app-

Wait, there's some more things you must know for compiling a *specific app* ! Thanks to @thewisenerd head over to this post.

To compile other parts you also need to enter the folder name, here are some examples from /frameworks/base [Credits-XDA University Article]


More examples:


Configuring a new Product for the AOSP.
Again, I'd recommend you to have a look at this great document.

A detail information about BoardConfig.mk
Well, I believe that BoardConfig.mk is one of the most important files for the device tree. Now, here are few parameters of the file explained-

• TARGET_ARCH: set to arm for almost all current Android devices.
• BOARD_KERNEL_CMDLINE: not all devices pass boot parameters however if your device does this must be filled out properly in order to boot successfully.
• BOARD_KERNEL_PAGESIZE: the pagesize of the stock boot.img and must be set properly in order to boot. Typical values for this are 2048 and 4096 and this information can be extracted from the stock kernel.
• BOARD_BOOTIMAGE_PARTITION_SIZE: the number of bytes allocated to the kernel image partition.
• BOARD_RECOVERYIMAGE_PARTITION_SIZE: the number of bytes allocated to the recovery image partition.
• BOARD_SYSTEMIMAGE_PARTITION_SIZE: the number of bytes allocated to the Android system filesystem partition.
• BOARD_USERDATAIMAGE_PARTITION_SIZE: the number of bytes allocated to the Android data filesystem partition.
  ^The above information can be gathered by multiplying the size from /proc/partitions by the block size, typically 1024.
• BOARD_HAS_NO_SELECT_BUTTON: (optional), use this if your device needs to use its Power button to confirm selections in recovery.
• BOARD_FORCE_RAMDISK_ADDRESS / BOARD_MKBOOTIMG_ARGS: (optional), use these to force a specific address for the ramdisk. This is usually needed on larger partitions in order for the ramdisk to be loaded properly where it's expected to exist. This value can be obtained from the stock kernel. The former is deprecated as of Android 4.2.x and the latter will now be used in 4.2.x and beyond.

Few *more* external links ​

Few great docs that I found over the web-
One
Two
Three

Credits​

>Firstly, My parents.
> @Red Devil, @galaxyfreak, @speed_bot, @Nihar.G and my Team for helping me with all my doubts and even answering the most supid questions that I asked them
> @corruptionfreeindia for the idea.
> Helpful and informative posts over-
CyanogenMod Wiki
elinux.org
StackOverflow

Got a query? Ask in the thread. Quote or mention me for quicker replies.

Thanks to @eagleeyetom for featuring it on the portal!

Understanding the Kernel Source Code​

Credits for this- Linux.org
I'll basically explain the 'Linux Kernel Source' and since Android is based on the Linux, it's almost same for both.



Now the folders in order-

1. arch - This folder contains a Kconfig which sets up some settings for compiling the source code that belongs in this folder. Each supported processor architecture is in the corresponding folder. So, the source code for Alpha processors belong in the alpha folder. Keep in mind that as time goes on, some new processors will be supported, or some may be dropped.

2. block - This folder holds code for block-device drivers. Block devices are devices that accept and send data in blocks. Data blocks are chunks of data instead of a continual stream.

3. crypto - This folder contains the source code for many encryption algorithms. For example, “sha1_generic.c” is the file that contains the code for the sha1 encryption algorithm.

4. Documentation - This folder contains plain-text documents that provide information on the kernel and many of the files. If a developer needs information, they may be able to find the needed information in here.
See, this folder is just so awesome-


5. drivers - This directory contains the code for the drivers. A driver is software that controls a piece of hardware. For example, for a computer to understand the keyboard and make it usable, a keyboard driver is needed. Many folders exist in this folder. Each folder is named after each piece or type of hardware. For example, the bluetooth folder holds the code for bluetooth drivers. Other obvious drivers are scsi, usb, and firewire. Some drivers may be more difficult to find. For instance, joystick drivers are not in a joystick folder. Instead, they are under ./drivers/input/joystick. Keyboard and mouse drivers are also located in the input folder. The Macintosh folder contains code for hardware made by Apple. The xen folder contains code for the Xen hypervisor. A hypervisor is software or hardware that allows users to run multiple operating systems on a single computer.

6. firmware - The firmware folder contains code that allows the computer to read and understand signals from devices. For illustration, a webcam manages its own hardware, but the computer must understand the signals that the webcam is sending the computer. The Linux system will then use the vicam firmware to understand the webcam. Otherwise, without firmware, the Linux system does not know how to process the information that the webcam is sending. Also, the firmware helps the Linux system to send messages to the device. The Linux system could then tell the webcam to refocus or turnoff.

7. fs - 'fs' stands for File System. All of the code needed to understand and use filesystems is here. Inside this folder, each filesystem's code is in its own folder. For instance, the ext4 filesystem's code is in the ext4 folder. Within the fs folder, developers will see some files not in folders. These files handle filesystems overall. For example, mount.h would contain code for mounting filesystems. A filesystem is a structured way to store and manage files and directories on a storage device. Each filesystem has its own advantages and disadvantages. These are due to the programming of the filesystem. For illustration, the NTFS filesystem supports transparent compression (when enabled, files are automatically compressed without the user noticing). Most filesystems lack this feature, but they could only possess this ability if it is programmed into the files in the fs folder.

8. include - The include folder contains miscellaneous header files that the kernel uses. The name for the folder comes from the C command "include" that is used to import a header into C code upon compilation.

9. init - The init folder has code that deals with the startup of the kernel (INITiation). The main.c file is the core of the kernel. This is the main source code file the connects all of the other files.

10. ipc - IPC stands for Inter-Process Communication. This folder has the code that handles the communication layer between the kernel and processes. The kernel controls the hardware and programs can only ask the kernel to perform a task. Assume a user has a program that opens the DVD tray. The program does not open the tray directly. Instead, the program informs the kernel that the tray should be opened. Then, the kernel opens the tray by sending a signal to the hardware. This code also manages the kill signals. For illustration, when a system administrator opens a process manager to close a program that has locked-up, the signal to close the program is called a kill signal. The kernel receives the signal and then the kernel (depending on which type of kill signal) will ask the program to stop or the kernel will simply take the process out of the memory and CPU. Pipes used in the command-line are also used by the IPC. The pipes tell the kernel to place the output data on a physical page on in memory. The program or command receiving the data is given a pointer to the page on memory.

11. kernel - The code in this folder controls the kernel itself. For instance, if a debugger needed to trace an issue, the kernel would use code that originated from source files in this folder to inform the debugger of all of the actions that the kernel performs. There is also code here for keeping track of time. In the kernel folder is a directory titled "power". Some code in this folder provide the abilities for the computer to restart, power-off, and suspend.

12. lib - the library folder has the code for the kernel's library which is a set of files that that the kernel will need to reference.

13. mm - The Memory Management folder contains the code for managing the memory. Memory is not randomly placed on the RAM. Instead, the kernel places the data on the RAM carefully. The kernel does not overwrite any memory that is being used or that holds important data.

14. net - The network folder contains the code for network protocols. This includes code for IPv6 and Appletalk as well as protocols for Ethernet, wifi, bluetooth, etc. Also, the code for handling network bridges and DNS name resolution is in the net directory.

15. samples - This folder contains programming examples and modules that are being started. Assume a new module with a helpful feature is wanted, but no programmer has announced that they would work on the project. Well, these modules go here. This gives new kernel programmers a chance to help by going through this folder and picking a module they would like to help develop.

16. scripts - This folder has the scripts needed for compiling the kernel. It is best to not change anything in this folder. Otherwise, you may not be able to configure or make a kernel.

17. security - This folder has the code for the security of the kernel. It is important to protect the kernel from computer viruses and hackers. Otherwise, the Linux system(Android in this case ) can be damaged.

18. sound - This directory has sound driver code for sound/audio cards.

19. tools - This directory contains tools that interact with the kernel.

20. usr - The code in this folder creates vmlinuz and similar files after the kernel is compiled.

21. virt - This folder contains code for virtualization which allows users to run multiple operating systems at once. This is different from Xen (mentioned previously). With virtualization, the guest operating system is acting like any other application within the Linux operating system (host system). With a hypervisor like Xen, the two operating systems are managing the hardware together and the same time. In virtualization, the guest OS runs on top of the Linux kernel while in a hypervisor, there is no guest OS and all of the operating systems do not depend on each other.

PS- If I went wrong at any place, please do tell, I'm tend to make mistakes.

Reserved 2

cleverior.ipul said:

great !!

but could you tell me about propietary files and vendor_blobs.mk ? and how to make it for device that using GB (for example) ?

Click to expand...

Click to collapse

Sorry for the wait but I'll answer this after I reach home.

Sent from my HTC Explorer A310e using XDA Premium 4 mobile app

cleverior.ipul said:

great !!

but could you tell me about propietary files and vendor_blobs.mk ? and how to make it for device that using GB (for example) ?

Click to expand...

Click to collapse

Okay, I'll tell this with an example.
Have a look at this vendor tree. This for Galaxy Nexus(Maguro).
Now, the important makefile, vendor-blobs.mk
In this case, it is:

So, in the source code, the vendor tree is kept in /source/vendor/manufacturer/codename(vendor/samsung/maguro in this case) and the makefile parameters tell it to copy the proprietary files(that are the files in the vendor tree, for instance, open up the vendor tree's link I gave above) to certain direstories in the ROM's zip, in this case, for example, this line:
this tells the builder to copy the proprietary file gps.omap4.so to lib/hw folder in the system.
I hope you'll figure out the remaining and this cleared your doubts.
PS- If I went wrong at some place, correct me.

smileydr0id said:

Great guide mate! Maybe you can teach us how to track build error's ?

Click to expand...

Click to collapse

That are mostly easy to track, few can be easily be fixed by seeing the error's line and jumping to that file, and if you are unable to fix, then Google or this thread will help you.