Remove All Ads from XDA
Thread Deleted Email Thread
Prev Previous Post   Next Post Next

Why this thread?

Hello xda-university !

As a newbie ( still remain a big newbie ) , I always had a tough time managing my ROMs and kernel projects.
I blindly followed what guides said .. and then realized how much time and energy I was wasting on downloading unnecessary $#!t from the internet as I have a very limited bandwidth and a particularly slow speed. ( around 80 kbps download speed )

While I am not a pro at this stuff, I would like to share a few tips and tricks about ROM building which will save some of your time , energy, space as well as a lot of the money while returning back to our great community out here.

After all efficiency is what matters in the end right ?

Also , understanding how the process works is more important that the process itself .

For what its worth , even a build bot can compile a ROM.. thats no biggie.

Learning how it works matters more !

Tired of waiting for that repo sync to end ?!!!

Prerequisites / Requirements

Some rough knowledge on compiling a ROM or experience of compiling a ROM.

Check out this thread ! : COMPILE From Source For Any Phone On Linux Distros [BUILD ENVIRONMENT TO COMPILE] by @mithun46
That was just to scare you . Though this will be more useful if you do have that knowledge .

Ok now moving on to the guide :

Google's Repo Tool

I ll just start off with a basic introduction of

Repository :
Repository is basically like a store room for code. It can be either selfmade i.e. you have yourself contributed it or claim to have written that code .. ( circled in yellow )

Or it can also be forked from someone else ... that is .. it is a version of someone else's code that you have modified a bit and pushed it to the remote server/ website. ( circled in black )

Look at the icons on the left of the name of the repository to know .

There are a variety of websites that offer such free hosting for code for eg : I will be using in this tutorial) and among others..

Repo tool :
You maybe very familiar with the words repo init , repo sync , etc while downloading source.

Where did that repo command come from ?

Remember this ? :

curl > ~/bin/repo
Yep thats where it came from .

So in order to use the repo tool you need to run this command at least once.

What this does is it download the repo tool from Google's source and makes it executable for your own use.
How does the repo tool work ?

The repo tool is used for downloading a bunch of remote repositories that are on the internet thru one single command :

repo sync -j4
Details , details and more details
To explain this , I will take an example of a ROM, say AOKP.

For building AOKP you need to download its source code.

AOKP ROM developers have maintained the ROM's source code in a large number of Repositories that are located at

What they have done is they have organized their source code into a number of folders for and uploaded it to separate repositories online.

What this does is , it enables all the ROM developers to work on a specific part of the source on their own computers and then push the source online to remote repository online so that he can share the code with all the developers of the same ROM. ( just a basic idea. it involves a lot more including pull requests and code reviews but we will stick with this for now )

Social coding at its best.

So during the command :

 repo sync -j4
What is done is the repo tool fetches all the remote repositories that the AOKP ROM devs have deemed necessary for building the ROM for a device through 4 threads.

If you have a slow internet connection then it is advised to use -j1 or -j2.
If you have a fast internet connection then you can use a -j16 or a -j32 whatever suits you the best.

Where does the repo tool take all the reference when downloading ??

The repo tool takes all the reference for the code to be downloaded from a Default.xml or a Manifest.xml.

The manifest.xml is a file which is written in xml markup and directs the repo tool to the repositories and their respective branches that need to be downloaded .

This is basically a huge list of all the repositories that need to be downloaded by the repo tool.
Where can this list of yours be found ?

Once you have initialized your repo sync which will be somewhat similar to this :

[email protected]:~$ mkdir -p kang
[email protected]:~$ cd kang
[email protected]:~/kang$ repo init -u -b jb-mr1
remote: Counting objects: 101, done
remote: Finding sources: 100% (101/101)
Receiving objects: 100% (2526/2526), 1.69 MiB | 67 KiB/s, done.
remote: Total 2526 (delta 1442), reused 2526 (delta 1442)
Resolving deltas: 100% (1442/1442), done.
 * [new branch]      maint      -> origin/maint
 * [new branch]      master     -> origin/master
 * [new branch]      stable     -> origin/stable
 * [new tag]         v1.0       -> v1.0
 * [new tag]         v1.0.1     -> v1.0.1
 * [new tag]         v1.0.2     -> v1.0.2
 * [new tag]         v1.0.3     -> v1.0.3
 * [new tag]         v1.0.4     -> v1.0.4
 * [new tag]         v1.0.5     -> v1.0.5
 * [new tag]         v1.0.6     -> v1.0.6
 * [new tag]         v1.0.7     -> v1.0.7
 * [new tag]         v1.0.8     -> v1.0.8
 * [new tag]         v1.0.9     -> v1.0.9
 * [new tag]         v1.1       -> v1.1
 * [new tag]         v1.10.0    -> v1.10.0
 * [new tag]         v1.10.1    -> v1.10.1
 * [new tag]         v1.10.2    -> v1.10.2
 * [new tag]         v1.10.3    -> v1.10.3
 * [new tag]         v1.10.4    -> v1.10.4
 * [new tag]         v1.11.0    -> v1.11.0
 * [new tag]         v1.11.1    -> v1.11.1
 * [new tag]         v1.12.0    -> v1.12.0
 * [new tag]         v1.12.1    -> v1.12.1
 * [new tag]         v1.12.2    -> v1.12.2
 * [new tag]         v1.2       -> v1.2
 * [new tag]         v1.3       -> v1.3
 * [new tag]         v1.3.1     -> v1.3.1
 * [new tag]         v1.3.2     -> v1.3.2
 * [new tag]         v1.4       -> v1.4
 * [new tag]         v1.4.1     -> v1.4.1
 * [new tag]         v1.4.2     -> v1.4.2
 * [new tag]         v1.4.3     -> v1.4.3
 * [new tag]         v1.4.4     -> v1.4.4
 * [new tag]         v1.5       -> v1.5
 * [new tag]         v1.5.1     -> v1.5.1
 * [new tag]         v1.6       -> v1.6
 * [new tag]         v1.6.1     -> v1.6.1
 * [new tag]         v1.6.10    -> v1.6.10
 * [new tag]         v1.6.10.1  -> v1.6.10.1
 * [new tag]         v1.6.10.2  -> v1.6.10.2
 * [new tag]         v1.6.2     -> v1.6.2
 * [new tag]         v1.6.3     -> v1.6.3
 * [new tag]         v1.6.4     -> v1.6.4
 * [new tag]         v1.6.5     -> v1.6.5
 * [new tag]         v1.6.6     -> v1.6.6
 * [new tag]         v1.6.7     -> v1.6.7
 * [new tag]         v1.6.7.1   -> v1.6.7.1
 * [new tag]         v1.6.7.2   -> v1.6.7.2
 * [new tag]         v1.6.7.3   -> v1.6.7.3
 * [new tag]         v1.6.7.4   -> v1.6.7.4
 * [new tag]         v1.6.7.5   -> v1.6.7.5
 * [new tag]         v1.6.8     -> v1.6.8
 * [new tag]         v1.6.8.1   -> v1.6.8.1
 * [new tag]         v1.6.8.10  -> v1.6.8.10
 * [new tag]         v1.6.8.11  -> v1.6.8.11
 * [new tag]         v1.6.8.2   -> v1.6.8.2
 * [new tag]         v1.6.8.3   -> v1.6.8.3
 * [new tag]         v1.6.8.4   -> v1.6.8.4
 * [new tag]         v1.6.8.5   -> v1.6.8.5
 * [new tag]         v1.6.8.6   -> v1.6.8.6
 * [new tag]         v1.6.8.7   -> v1.6.8.7
 * [new tag]         v1.6.8.8   -> v1.6.8.8
 * [new tag]         v1.6.8.9   -> v1.6.8.9
 * [new tag]         v1.6.9     -> v1.6.9
 * [new tag]         v1.6.9.1   -> v1.6.9.1
 * [new tag]         v1.6.9.2   -> v1.6.9.2
 * [new tag]         v1.6.9.3   -> v1.6.9.3
 * [new tag]         v1.6.9.4   -> v1.6.9.4
 * [new tag]         v1.6.9.5   -> v1.6.9.5
 * [new tag]         v1.6.9.6   -> v1.6.9.6
 * [new tag]         v1.6.9.7   -> v1.6.9.7
 * [new tag]         v1.6.9.8   -> v1.6.9.8
 * [new tag]         v1.7       -> v1.7
 * [new tag]         v1.7.1     -> v1.7.1
 * [new tag]         v1.7.2     -> v1.7.2
 * [new tag]         v1.7.3     -> v1.7.3
 * [new tag]         v1.7.3.1   -> v1.7.3.1
 * [new tag]         v1.7.4     -> v1.7.4
 * [new tag]         v1.7.4.1   -> v1.7.4.1
 * [new tag]         v1.7.4.2   -> v1.7.4.2
 * [new tag]         v1.7.4.3   -> v1.7.4.3
 * [new tag]         v1.7.5     -> v1.7.5
 * [new tag]         v1.7.6     -> v1.7.6
 * [new tag]         v1.7.6.1   -> v1.7.6.1
 * [new tag]         v1.7.7     -> v1.7.7
 * [new tag]         v1.7.7.1   -> v1.7.7.1
 * [new tag]         v1.7.7.2   -> v1.7.7.2
 * [new tag]         v1.7.7.3   -> v1.7.7.3
 * [new tag]         v1.7.7.4   -> v1.7.7.4
 * [new tag]         v1.7.7.5   -> v1.7.7.5
 * [new tag]         v1.7.7.6   -> v1.7.7.6
 * [new tag]         v1.7.8     -> v1.7.8
 * [new tag]         v1.7.8.1   -> v1.7.8.1
 * [new tag]         v1.7.8.2   -> v1.7.8.2
 * [new tag]         v1.8.0     -> v1.8.0
 * [new tag]         v1.8.1     -> v1.8.1
 * [new tag]         v1.8.2     -> v1.8.2
 * [new tag]         v1.9.0     -> v1.9.0
 * [new tag]         v1.9.1     -> v1.9.1
 * [new tag]         v1.9.2     -> v1.9.2
 * [new tag]         v1.9.3     -> v1.9.3
 * [new tag]         v1.9.4     -> v1.9.4
 * [new tag]         v1.9.5     -> v1.9.5
 * [new tag]         v1.9.6     -> v1.9.6
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:--  0:00:02 --:--:--     0
curl: (22) The requested URL returned error: 404 Not Found
Server does not provide clone.bundle; ignoring.
remote: Counting objects: 1722, done.
remote: Compressing objects: 100% (1012/1012), done.
remote: Total 1722 (delta 839), reused 1552 (delta 709)
Receiving objects: 100% (1722/1722), 249.52 KiB | 50 KiB/s, done.
Resolving deltas: 100% (839/839), done.
 * [new branch]      ics        -> origin/ics
 * [new branch]      jb         -> origin/jb
 * [new branch]      jb-mr1     -> origin/jb-mr1
 * [new tag]         Build-2    -> Build-2
 * [new tag]         Build-3    -> Build-3
 * [new tag]         Build-4    -> Build-4
 * [new tag]         Build-5    -> Build-5
 * [new tag]         Milestone-1 -> Milestone-1
 * [new tag]         jb-mr1_build-1 -> jb-mr1_build-1
 * [new tag]         jb-mr1_build-4 -> jb-mr1_build-4
 * [new tag]         jb-mr1_build-6 -> jb-mr1_build-6
 * [new tag]         jb-mr1_milestone-1 -> jb-mr1_milestone-1

Your identity is: akshatshenoy <[email protected]>
If you want to change this, please re-run 'repo init' with --config-name

repo has been initialized in /home/akshat/kang
So I have to go to : the /kang folder in my home directory and open the .repo file. Which will look somewhat like the following images :

Important tip ! : While browsing through files in a directory , to view hidden files press control + H and to view the exact directory of the folder which you are browsing through press control + L ..

Opening this file with a text editor will tell you the list of the repos that need to be downloaded by the repo tool :

For AOKP JB-MR1 it will look somewhat like this

Now just a flashback to the original repo init command we had applied earlier
[email protected]:~/kang$ repo init -u -b jb-mr1
When the repo init command is carried out , the repo tool grabs the default.xml / manifest.xml from AOKP's github which is mentioned in the init command :

So open up a web-browser and navigate to that page and you will realize that it is the same manifest.xml that is present in the .repo of the source directory where you have performed a repo init at.

Now when we repo sync ,
the repo tool downloads each repo as a specific project as shown in the picture : (highlighted)

So, basically, there are 415 projects to be synced in order to build AOKP that has been defined in the manifest.xml

Once those 415 projects have been downloaded , they will be stored in the .repo and a copy of it will be synced as a work-tree .

This work-tree comprises of the files and folders that appear automagically once the repo sync is done with all 415 projects synced !

It is that part of the source code which you actually work on and make your device specific changes to !
Understanding the manifest :

1) The remote tag :
syntax :
<remote  name=" any name here "
           fetch="url of the remote website/hosting/server " 
            review="url of the code review of the repo :optional: " />
This tag just specifies a " remote " location for the repo tool to get your repositories from . This allows you to then associate a particular repository with that remote.

examples of remotes :


<remote  name="github"
           fetch="" />

<remote name="bitbucket" fetch=""/>

  <remote  name="aokp"
           review="" />
This allows you to use multiple code hosting sites , private or public and thereby enabling a trouble-free maintenance of your team's source code.

Now like the remote is specified , you can base you're manifest at a particular remote and define it as your default remote.

The advantages of this will be seen in (2) and (3) during the adding of the project tags to the manifest.xml

The syntax of the default remote is as follows :

<remote  name=" any name here "
           fetch="url of the remote website/hosting/server " 
            review="url of the code review of the repo :optional: " />

<default revision="any/path/here"
           remote="remote hosting name defined earlier"
           sync-j="x" />
where x is the default number of threads to be used when it is not defined in the repo sync command.

best example of a default revision that is widely used in almost every ROM default.xml is

  <remote  name="aosp"
           review="" />

   <default revision="refs/tags/android-4.2.2_r1.2"
           sync-j="4" />
2) The base projects
As explained earlier the projects refer to the repositories that have to be fetched by the repo tool.

The 2) tag in the picture above is used to download the repository from your default remote's server.

The basic syntax of it is :

 <project path=" path/in/work/tree" name="/repo/name/in/your/default" />
Examples :

<project path="abi/cpp" name="platform/abi/cpp" /> 
 <project path="bootable/diskinstaller" name="platform/bootable/diskinstaller" />
This downloads those particular repos from the actual android source .

It is advisable to use this particular tag from the remote you will download the most number of repositories from as it cuts down on the effort of editing the manifest.xml

Many ROMs use the AOSP as the base as it is easier for them to keep their source code upto date without wasting their bandwidth on uploading the same repositories to their own gits .

3) The other projects
This is for the non default remote projects

Their general syntax is :
<project path=" directory in work tree" name="name of the repository in remote" remote="name of remote defined earlier" revision="branch of the repository" />
NOTE : Default revision is master. So if you dont define the revision it is considered as the master branch!

Examples for this are :
<project path="bootable/recovery" name="CyanogenMod/android_bootable_recovery" remote="cm" revision="cm-10.1" />
For this of course the earlier remote "cm" should be mentioned.
Quick tip !
Indirectly , the total url path to the repository should be = the fetched URL of the remote + the name of the repo in the project line.

In this case
Code: && CyanogenMod/android_bootable_recovery 

in total :
For the sole purpose of making the life of a developer easier , CyanogenMod / AOKP and almost all other ROMs named their own repositories as

 android_path_to_directory_in_work_tree || path_to_directory_in_work_tree
Such as :

If you notice the name of the repository : vendor_aokp corresponds to the aokp folder in the source folder in my source.

Thus the name of the repository gives you the path to the same files in your repository.

So much win !
Maintaining your own projects using manifest.xml
Now that you have understood how the manifest.xml works .. you can even maintain your own projects with the help of a manifest.xml ! it is not necessary it has to be used only for ROMs !

Thats the best bit !

So the next time you have to pull a large number of repositories and maintain them you can create your own default manifest in your own github and add the repos as mentioned above.

Example I use sometimes :

 <?xml version="1.0" encoding="UTF-8"?>

<remote  name="red-devil"
         fetch="" /> 

<project path="Cherrypicking/frameworks/base" name="AeonROM/android_frameworks_base" remote="red-devil" revision="master" />
<project path="Cherrypicking/frameworks/native" name="AeonROM/android_frameworks_native" remote="red-devil" revision="master" />
<project path="Cherrypicking/frameworks/av" name="AeonROM/android_frameworks_av" remote="red-devil" revision="master" />


Understanding local manifests :

What is the local_manifest.xml ?
A local manifest is like a specific copy of the manifest of a particular ROM that is maintained by YOU the ROM compiler in your own computer.

This local manifest allows you to override some of the projects mentioned in the manifest and hence allows you to delete / add / modify the source code according to your preferences.

Best part about this is that every time you sync your source , even the onces mentioned in the local manifest get updated by themselves.

Hence, you dont need to keep track of all your device trees and kernel sources manually, let the repo tool do the work for you.

Let us see how this wonderful tool works :

  1. Go to the folder where you have inited the repo and open the .repo ( press ctrl + H if it is NOT visible ).
  2. Create a folder named local_manifests
  3. Download this local_manifest.txt and rename it to local_manifest.xml and copy it in the / .repo / local_manifests / directory

NOTE : In some older ROMs where an older version of the repo tool is used the local manifests has to be placed like

As opposed to a folder named local_manifests as illustrated in the above pictures
Functions of the local manifest :

The local manifest looks somewhat similar to the main manifest.xml ... however in this you can even remove projects or add them

> To remove projects the basic syntax is :

<remove-project name="name of the project" />
What to remove :

Consider my case

I am building AOKP JB-MR1 for the HTC Explorer.

As a result of that I do not need the various device trees that are officially supported by AOKP.

Such all these :


So why do I waste my data and time downloading those ?

I think I should modify the default.xml / manifest.xml .

NO !

Beware ! If you try to edit the default.xml / manifest.xml the git will tell you to commit these changes as they do NOT match with the online / and the synced version of the default.xml that you have originally initializeed with.

So this means you will either have to commit those changes , push it to github and update the default.xml
This is not possible unless you are a device maintainer for a ROM and even if you are such drastic changes mean it can affect the other users of the same default.xml . So chances are you can never bring about a change in the default.xml.

So dont even think about editing the default.xml


You will have to revert back to the original default.xml that you was present in your .repo and create a local_manifest.xml and remove projects / add them .

How do I do that ?

Basic syntax for removing them is :
<remove-project name="name_of_project" />
As shown in the image since I dont need those projects the syntax will be somewhat like :

 <remove-project name="AOKP/device_samsung_d2att" />
  <remove-project name="AOKP/device_samsung_d2tmo" />
For removing the 1st two projects as shown in the picture above

In a similar way , I went on removing projects that were not required... only device trees and saved the local_manifests.xml

Remember when I earlier synced there were 415 projects ?

Look at it now .. slimmed down around 50 projects !

That means a much quicker repo sync for me !
> How to add projects to get it synced :

To add projects just use the code like I explained in the manifests.xml

Basic syntax is :
<remote  name=" any name here "
           fetch="url of the remote website/hosting/server " 
            review="url of the code review of the repo :optional: " />

<project path=" directory in work tree" name="name of the repository in remote" remote="name of remote defined earlier" revision="branch 
of the repository" />
EG :

I wanted to add my own device tree of AOKP and the kernel source for my device in the local manifests cause I am too lazy to manually download them

 <project path="device/htc/pico" name="android_device_htc_pico_aokp" remote="red-devil" revision="master"/>
  <project path="kernel/htc/pico" name="android_kernel_htc_pico" remote="red-devil" revision="cm-10.1" />

So ultimately this is the sample local_manifest.xml I used :

<?xml version="1.0" encoding="UTF-8"?>
  <remote  name="red-devil"
           fetch="" />
  <project path="device/htc/pico" name="android_device_htc_pico_aokp" remote="red-devil" revision="master"/>
  <project path="kernel/htc/pico" name="android_kernel_htc_pico" remote="red-devil" revision="cm-10.1" />
  <remove-project name="AOKP/device_acer_a510" />
  <remove-project name="AOKP/device_asus_tilapia" />
  <remove-project name="AOKP/device_htc_evita" />
  <remove-project name="AOKP/device_htc_s4-common" />
  <remove-project name="AOKP/device_lge_iprj-common" />
  <remove-project name="AOKP/device_lge_mako" />
  <remove-project name="AOKP/device_lge_p930" />
  <remove-project name="AOKP/device_lge_su640" />
  <remove-project name="AOKP/device_lge_vs920" />
  <remove-project name="AOKP/device_motorola_maserati" />
  <remove-project name="AOKP/device_motorola_omap4-common" />
  <remove-project name="AOKP/device_motorola_solana" />
  <remove-project name="AOKP/device_motorola_spyder" />
  <remove-project name="AOKP/device_motorola_targa" />
  <remove-project name="AOKP/device_motorola_umts_spyder" />
  <remove-project name="AOKP/device_samsung_aries-common" />
  <remove-project name="AOKP/device_samsung_d2-common" />
  <remove-project name="AOKP/device_samsung_celox-common" />
  <remove-project name="AOKP/device_samsung_d2att" />
  <remove-project name="AOKP/device_samsung_d2tmo" />
  <remove-project name="AOKP/device_samsung_d2usc" />
  <remove-project name="AOKP/device_samsung_d2vzw" />
  <remove-project name="AOKP/device_samsung_galaxys2-common" />
  <remove-project name="AOKP/device_samsung_hercules" />
  <remove-project name="AOKP/device_samsung_i605" />
  <remove-project name="AOKP/device_samsung_i9100" />
  <remove-project name="AOKP/device_samsung_i9100g" />
  <remove-project name="AOKP/device_samsung_i9300" />
  <remove-project name="AOKP/device_samsung_l900" />
  <remove-project name="AOKP/device_samsung_maguro" />
  <remove-project name="AOKP/device_samsung_manta" />
  <remove-project name="AOKP/device_samsung_msm8660-common" />
  <remove-project name="AOKP/device_samsung_msm8960-common" />
  <remove-project name="CyanogenMod/android_device_samsung_omap4-common" />
  <remove-project name="AOKP/device_samsung_smdk4412-common" />
  <remove-project name="CyanogenMod/android_device_samsung_smdk4412-qcom-common" />
  <remove-project name="AOKP/device_samsung_t0lte" />
  <remove-project name="AOKP/device_samsung_t0lteatt" />
  <remove-project name="AOKP/device_samsung_t0ltetmo" />
  <remove-project name="AOKP/device_samsung_toro" />
  <remove-project name="AOKP/device_samsung_toroplus" />
  <remove-project name="AOKP/device_samsung_tuna" />
  <remove-project name="AOKP/device_samsung_vibrantmtd" />
  <remove-project name="AOKP/device_sony_hayabusa" />
  <remove-project name="AOKP/device_sony_mint" />
  <remove-project name="AOKP/device_sony_odin" />
  <remove-project name="AOKP/device_sony_tsubasa" />
  <remove-project name="AOKP/device_sony_yuga" />
  <remove-project name="AOKP/device_sony_blue-common" />
  <remove-project name="AOKP/device_sony_fusion3-common" />
  <remove-project name="CyanogenMod/android_device_sony_qcom-common" />
  <remove-project name="AOKP/device_sony_common" />
  <remove-project name="AOKP/packages_apps_Torch" />

Result of this : the number of projects being downloaded reduced from 415 to 364 !

So , I hope you guys understood at least a bit about manifests and how the repo sync process actually works..

Read on for some tips !
The Following 178 Users Say Thank You to Red Devil For This Useful Post: [ View ] Gift Red Devil Ad-Free
Subscribe to Thread

compiling, guide, repo tool, rom, sync

Guest Quick Reply (no urls or BBcode)
Previous Thread Next Thread
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes