FORUMS
Remove All Ads from XDA

[GUIDE][2018]All you need to know to build Android from scratch!

205 posts
Thanks Meter: 574
 
Post Reply Email Thread
Introduction

So, I will start off by telling a little bit about myself. I got interested in Android building in November 2017. I found most of the guides "inadequate" for pure newbies like me.
This is my second guide and the first one for No-Knowledge newbies so please do correct me . I will also add some humor so you don't get bored
The guide will be divided into a few parts-
Using linux and enjoying it(No offense to GNU/Linux enthusiasts. please don't start a debate, its just my personal opinion)
Basic git for complete newbies
Compiling your first ROM(Lineage)
Compiling your first non-Lineage ROM
Choosing a Programming lanugage
Basics to resolving conflicts
Compiling AOSP-based roms for CAF devices(For which I couldn't find a guide)

It will probably take a while for you to read this but its must to read everything carefully. Lets start off right away

Using Linux and Enjoying it!

There are more than adequate guides for using Linux. I will just help you choose the distro and recommend some guides
Personally, I used Linux wayyyy before I started building android(~2013) I didn't do much at first so I got most of my Linux experience in 2016-2018
Some distros I recommend-

Ubuntu 16.04 or 18.04 (Probably the best for newbies imo) - https://www.ubuntu.com/download/desktop


Arch Linux(Could be problematic to install for newbies so I recommend you install Antergos first - https://Antergos.com/try-it/ )


Linux Mint - https://linuxmint.com/download.php (Newbie friendly)


Debian GNU/Linux (Easier than Arch imo, slightly(very slightly) harder than Ubuntu) - https://www.debian.org/


You absolutely MUST live boot them and feel them. I highly recommend you to install them on a small partition(~40GB) to get the best experience. Please don't use linux on a server at first!

I personally started with Ubuntu 12.04 for a month or so, 14.04 as well for a few months. I became a daily user when 16.04 came out.
Currently, I am using Antergos. However, for building I liked Arch as it had everything "bleeding edge"(Search it if you don't know what this means xD)

Here are some absolutely noob friendly guides-
https://www.youtube.com/watch?v=opBKvGi77cU
For Ubuntu (Long guide but indepth!)

https://www.youtube.com/watch?v=G0AFuhVSvEk
For Linux Mint (Again, Long but indepth!)

Use them for atleast a week or two before you even jump into android building! A month of daily usage is recommended so you know how to tackle basic problems. Youtube will always be your friend and so will be google. You can't hate Linux once you know how to use it! The only reason why I use Windows is because of gaming and applications such as Photoshop and Illustrator(I know there are alternatives, again just my personal opinion!)
See you after a week! No? Seriously use linux for atleast a week!

Using Linux will also hone your google skills(Not even kidding)




Basic git for complete newbies


Personally, I myself don't do advanced stuff with git myself and here are the subcommands you will be using the most with git -
clone
cherry-pick
revert
merge
commit
push

I will give a brief explanation of each and link some guides! Why brief? Because I expect you to have a thinking like a linux user now . Also, you can switch to a server now!

Here is a playlist by github -
https://www.youtube.com/watch?v=noZn...YDuKZSKyJrgwB-

If you are not satisfied(I wasn't) you can search youtube for a specifc command as well! (i.e cherry-pick)


clone -
Simply clones the repository including its commit history!
Usage-
Code:
git clone <repo> -b <branch> <directory>
If you don't specify a branch, it will clone from the default branch.
If you don't specify a directory, it will make a folder with the name of the repo
Example -
Code:
git clone https://github.com/ChimeraKernelProject/chimera_land-current -b lineage-16.0 chimera
(Shameless self advertising xD)


cherry-pick-

Generates a diff from a commit and applies it to the local repo
Usage-
Code:
git cherry-pick sha
You do need to fetch the repository from which you are picking a commit
Using-
Code:
git fetch <repo> <branch>
If the branch isn't specified, it fetches the default branch instead
Cherry-picking multiple commits-
Code:
git cherry-pick sha1^..sha2
This picks all the commits from sha1 to sha2(i.e the commits between sha1 and sha2, including sha1 and sha2)
You may or may not get a conflict for which I will have a seperate section.
Example-
Code:
git fetch https://github.com/ChimeraKernelProject/chimera_land-current lineage-16.0
git cherry-pick 9e8a821ba64f5b498843b025d1804e3818dda480^..8e80e52915492328e80378f7ecff0cb44fdc1344
(Try figuring out what I did :P)
What is SHA you ask? You don't have to know what it actually is so just consider it to be an identifier for a commit.

revert

Generates a diff of the commit and reverses it(pretty much the opposite of cherry-pick)
The usage is same as cherry-pick.

merge

Kind of like cherry-picking multiple commits but it compares the history of the local and the remote and then generates a common diff for the commits that are not in local
Also, you do need to fetch before merging.
Usage-
Code:
git merge <remote>
Example-
Code:
git fetch https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git v3.18.125
git merge FETCH_HEAD
(The example here is of upstreaming the kernel :P )

commit and push

Generates a commit for staged files
Usage-
Code:
git commit -m "some message" -m "sub-message"
some optional parameters-
--signoff
--author="name <author email>
If you forgot to add a file to the commit, you can -
Code:
git commit --ammend
Again, you can use the optional parameter.
how to stage files you ask?
you simply use-
Code:
git add <filepath>
If you did your linux lessons correctly, you might know that, you can use "." and ".." in filepath(redo them if you don't know xD)
finally, to "push" your changes to a remote repo-
Code:
git push -u <repolink>
It will ask for authentication. You use your github(or whatever you prefer) account for that.
After using -u once, you won't have to pass the parameter again.
Lets end this with an example-
Code:
git add .
git commit -m "add 69 hour battery life" -m "best" --signoff --author="someone <[email protected]>"
git push -u https://github.com/ChimeraKernelProject/chimera_land-current
(Also please don't use a stupid commit message like the one I showed xD. This one was just for the lulz)


This one was big. oof. Took me a while to write and probably will take a while for you to read too :P .
You might ask why did I tell you to search xyz in this one so much! Simple reason- I want you to get used to searching your problems!
Mastering git takes alot of time! I am not any good myself and I search alot of stuff on google myself! (Sometimes even problems related to merging and checking out!)




Compiling your first ROM(LineageOS)


I bet you have been getting restless by now! The previous guides were to build the excitement to build your first ROM!
Why Lineage? Simple. Its one of the easiest ROM to build. Pretty well documented and most importantly, alot of video guides to check if you are confused at any point!
It is highly recommended to use a server at this point as you will be pulling alot of data and you will require alot of processing power.
You can build locally if you have good internet.
Here are the minimum requirements(imo)-

i3 6100 or equivalent
8 GB RAM
300GB HDD(500 GB+ is highly recommended)

Lets start off by the dependencies. I am assuming you are on Ubuntu or Linux Mint. These are the dependencies you need to install -
Code:
openjdk-8-jdk
git-core gnupg flex bison gperf build-essential zip curl zlib1g-dev gcc-multilib g++-multilib libc6-dev-i386 lib32ncurses5-dev x11proto-core-dev libx11-dev lib32z-dev libgl1-mesa-dev libxml2-utils xsltproc unzip
You know how to install them don't you now :P


Lets install Repo now-
Code:
mkdir ~/bin
edit your .bashrc in the home directory with your preffered file editor(I use nano) and add this line to the bottom (Hint - hold pgdwn to jump to the bottom quickly)-
Code:
PATH=~/bin:$PATH
Then execute these commands-
Code:
curl https://storage.googleapis.com/git-repo-downloads/repo > ~/bin/repo
Code:
chmod a+x ~/bin/repo
Lets sync Lineage now shall we

First add your github account to the git config using-
Code:
git config --global user.name "Your Name"
git config --global user.email "[email protected]"
Here, you should use your github email! (See https://help.github.com/articles/abo...ail-addresses/ if you don't want to use personal email)

Make a dir in which you will do the rom stuff and cd to it

Before we initialize and sync LineageOS, you have to decide which branch you are choosing as well
You can check your device forum and see what Android versions does the device have. PS- You might wanna check the LineageOS thread. The developer must have provided a link to their kernel source(If they didn't, report the thread to mods kek). Click and navigate through their github. you will find a repo named- android_device_(vendor)_(devicecodename) . you probably know the vendor and the codename from the forum don't you?For example its- android_device_xiaomi_land for my device(Yea right I have an old as hecc device pls no laugh ). Check the branch of the ROM. It will probably something like lineage-16.0 or 15.1 etc. Also if you have an official device, you can check its branch at- https://github.com/LineageOS .

Welp now you know the branch, lets start!
Code:
repo init -u git://github.com/LineageOS/android.git -b <branch>
repo sync
To sync a little bit faster, here's a better command I got from StatiXOS git -
Code:
repo sync -c -f --force-sync --no-tag --no-clone-bundle -j$(nproc --all) --optimized-fetch --prune
Also, if your sync stopped midway because of your ISP(BSNL :'( ), you can just start the sync again.

If you are on another distro such as arch, you might have to do some extra steps- https://wiki.archlinux.org/index.php/android#Building

Now that you have synced, You may choose one of the 2 paths-

If you have official LineageOS, choose me!

Setup the environmental Variables with-
Code:
source bu*/e*
Clone your vendor blobs from TheMuppets repo - http://github.com/TheMuppets
or, from your preferred developer.
The usual repo name is proprietary_vendor_(vendorname). For example, in my case its proprietary_vendor_xiaomi . It should have a folder with your device codename

If not, try changing the branch!
You should be cloning them to vendor/(vendorname)

start bulding right away with-
Code:
brunch lineage_<codename>-<buildtype>
here, codename is your device codename and buildtype is one of these-
user, userdebug, eng .
I recommend userdebug at first. user builds are pretty much as limited as your stock rom. eng builds are actually the one you should use for debugging the rom. These are not secure for daily usage.
This will start the build and you probably won't get an error as its official. If you do, you can just search it up on google. Once the build is done, you will get filepath in the command output. Generally, the build is in
<workingdir>/out/target/product/<codename>/xyz-UNOFFICIAL.zip .


If you have unofficial Lineage, choose me!

I am assuming you are in the working directory

Welp, lets start off by searching for the required trees.
Here's what you need-
Device Tree
Kernel Source
Vendor Blobs

Usually I find them by checking the post of the unofficial Lineage thread and click the kernel source link(Again, if they didn't post it, you should report the thread ). Go to the dev's profile and check his repositories. Here are the usual naming schemes for each one of them-
Device tree - android_device_(vendorname)_(devicecodename)
Kernel Source - android_kernel_(vendorname)_(devicecodename) or! android_kernel_(vendorname)_(soccodename)
Vendor Blobs - proprietary_vendor_(vendorname) or, proprietary_vendor_(vendorname)_(devicecodename)

Device tree should be obvious. It should be cloned to - device/(vendorname)/(devicecodename)

As for kernel tree, soccodename menas the codename of the SoC you have in the device. The thread should have it already as the kernel is licensed in GPL v2(GNU Public License V2, for more info check - https://www.gnu.org/licenses/old-lic...pl-2.0.en.html ). To know where you need to clone the kernel tree, check the "BoardConfig.mk" of your Device tree and check the "TARGET_KERNEL_SOURCE" line. It tells where you need to clone the kernel

vendor name should be obvious as well now. If your vendor name doesn't have the codename, it should be cloned to vendor/<vendorname>
if it has the devicecodename, it may or may not be the device tree and still be like this-

If it doesn't look like this, it should be cloned to vendor/<vendorname>/<devicecodename>

Here's the repo list of an ideal dev-


Setup environmental variables-
Code:
source bu*/e*
Start the build! -
Code:
brunch lineage_<codename>-<buildtype>
here, codename is your device codename and buildtype is one of these-
user, userdebug, eng .
I recommend userdebug at first. user builds are pretty much as limited as your stock rom. eng builds are actually the one you should use for debugging the rom. These are not secure for daily usage.

Now since you are using an unofficial tree, it is likely that you might hit an error. One of the most common error is when brunch is unable to find your target device. a simple fix for that is to navigate to your device tree and make a file called "vendorsetup.sh"
It should have this line in it-
Code:
add_lunch_combo lineage_<codename>-userdebug
where(you guessed it), codename is your device codename. Run the environmental variable setup again and you should be ready to build.
You might hit errors. Google is always your friend. If you are absolutely sure that you can't find it on google, check the help group section in the thread!
Once the ROM is ready, you will find the filepath in the command ouput. Generally its in out/target/product/<codename>/xyz-UNOFFICIAL.zip


Uploading files to google drive

There's a really nice guide availabe here- http://olivermarshall.net/how-to-upl...e-command-line


Congratulations! You just made your first rom. If you find this hard, this is just the beginning! There's a very rough road ahead.




Compiling your first Non-Lineage ROM

PS. - The ROM has to be "Lineage-based".

thee aren't eft f'r aosp bas'd roms!

I will be using XenonHD as an example as it is pretty easy to build and it was pretty much one of the first roms i ever built.
Building a Lineage-based rom is as easy as building Lineage!
You go to the rom's git(I usually google search or check it in one of the rom's thread for any device) and check the repo manifest. The usual naming scheme for a repo is-
android_manifest
platform_manifest
manifest
android
etc.
In xenonHD's case, its platform_manifest(and all other repo names will start with platform prefix)

A proper ROM source should have the syncing instructions and building instructions. Lets check the XenonHD's example

make a working dir for the rom in which you will do your rom stuff.
Initialize and sync it. The initializing and syncing part is always the same across the ROMs. (You know which branch to use now. Don't you?)

Again, clone all the required trees!

Now now now, don't get excited yet! Before you start brunching, there is some stuff you need to change.
Usually you can refer to one of the official trees in the ROM organization. For example, https://github.com/TeamHorizon/andro...e_xiaomi_kenzo has proper commits. You should be looking at the "Initial XenonHD" commit. You can obviously ignore commits like https://github.com/TeamHorizon/andro...bf3ffc63080d1d as they are device specific. The basics are always the same-

Rename lineage.mk to (romname).mk . refer to the device tree if you are unsure about the romname
edit the (romname).mk and change all the lineage instances to (romname). refer to the commit if unsure.
rename lineage.dependencies to (romname).dependencies .
Renaming the kernel defconfig isn't really necessary so you can ignore that.

Setup the environmental variables(Same across all the roms)-
Code:
source bu*/e*
start the build using the supported command given by the ROM devs.
Code:
brunch codename-<buildtype>
in XenonHD's case. However its different across most of the roms so never forget to read manifests!
You are more likely to hit errors with these spinoffs. Google is your friend(I can't remember how many times I have repeated this) and so is LineageOS repo! You have alot of references if the error is not device specific. Learn to fix derps! Check the help groups sections and ask if its absolutely necessary! Again, the command output will have the filepath of the final ROM zip!
You aren't ready yet. You need to gain alot more experience in compiling ROMs and here are some ROMs that you should be able to build at this point- Resurrection Remix, DotOS etc. Most of the lineage based roms support brunch. (Be careful tho, as a few AOSP based roms support it too. You might have a hard time compiling those!)




Choosing a programming language


At this point, it is highly suggested to learn a programming language if you want to escape the Buildbot stage. I am not forcing you, but you will have alot of trouble in doing advanced stuff.
There are several beginner friendly languages-

bash (the easiest)


Python (Recommended)


Kotlin (Android App development, quite easy)

Harder, recomended to learn after the easy langages-

Java (Gotta learn java if you want to make your own custom ROM at some point :3 )


C/C++ (Other than System UI, most of the other stuff for Android is written in C or C++)

Anything else you like! Once you get the basics, its not hard to learn another language :P .

The reason why I recommend Python is because it was the first language I learnt! Its pretty easy(Though, the OOP, Object Oriented Programming part is slightly advanced). It has wide applications and THE preferred language for Machine Learning and AI stuff, Period(OMG BUZZWORDS).

Kotlin is also fairely easy. I haven't programmed much in kotlin though I know basic syntaxes. If you learn it first, you won't have much trouble with learning Java. Probably. You can learn Java at first as well but you might have trouble! C is a must learn if you want to modify the Kernel and C++ is a must if you want to modify other code like HALs!

Python takes a day or two to learn! Not even kidding. I recommend this nice playlist by CSDojo - https://www.youtube.com/watch?v=Z1Yd...D-hhDwSEDxogDg
If you are interested in programming Machine learning, I recommend this playlist- https://www.youtube.com/playlist?lis...6rFYQU_jL2ryal

I learnt Kotlin basics from this Video - https://www.youtube.com/watch?v=H_oGi8uuDpA but there are more advanced videos as well.

I can't link Java tutorials as I haven't learnt it and nor do I plan to anytime soon. You can of course check youtube :P .

I learnt C from books and The C programming language by Dennis Ritchie and Brian W Keringhan is a must read! I quite liked "Programming in ANSI C" by E. Balagurusamy as well. You can of course, learn it on youtube.

Getting into programming takes a while and you might not like it at first but if you do, you will always enjoy it!




Basics To Conflict Resolution

It is assumed that you have a mind of a programmer and thus the guide will have alot less spoon feeding!

Lets get back to git and discuss about Conflict resolution. From time to time, you might want to cherry-pick something or perform a merge and you might get a "conflict".
Here's a basic idea!
The code between <<< HEAD and ===== is your previous code and the code between ====== and >>>> (commit SHA) is the new code from the commit. You get this if a part of the file in the commit is not the same as your local file.
There are 3 possibilities -
The code between <<<< HEAD and ===== is not required, thus you can delete that (Mostly)
The code between ===== and >>>>> is not required, thus the new change is not required (Rarely)
Both the old code and the new code is required (Pretty common!)

How do you decide between the three? You actually check the commit you are picking. In case of a merge, you can just view the history of a file and check the changes on github. Mostly, that should be enough for a decision. In case of a conflict while upstreaming my kernel, I usually refer to android-linux-stable project - http://github.com/android-linux-stable . Other times, I actually use my brain and figure out the option myself!




Compiling AOSP-based roms for CAF devices


Welp, I took alot of your time! You surely learnt plenty. Its the final part of this guide Took me a while to write all this ;_; .
I am assuming that you are fairly experienced now and you have atleast basic programming knowledge and plenty experience with git.

For starters, I recommend you to try AOSIP! It is highly compatible with Lineage trees and it shouldn't be much trouble to modify your device tree for it.
No spoon feeding at this point :P . Lets revise the basic idea! -
Find the rom source
Find the appropriate branch
Check the manifest for syncing and building instructions
Refer to another device tree(Possibly a tree of a similar device) and modify your device tree! (The hard part as its different from lineage based roms)
Environment setup
Start the build

Most of the AOSP-based roms use these commands-
Code:
lunch <codename>-<buildtype>
Code:
mka bacon
The lunch part is always there in Maximum AOSP based ROMs and the building command maybe different(i.e
Code:
time mka kronic
in AOSIP)

Another part I want to mention is about "HALs" . Usually most of the ROMs (even AOSP-based) ship with CAF HALs and support project pathmaps. However, some ROMs don't. Probably because they are not intended for CAF devices or the sources aren't properly complete yet! To build these ROM succesfully, I clone the CAF HALs from the lineage repo. These are- display-caf, media-caf, audio-caf, bt-caf and wlan-caf.
The first 3 HALs are usually platform specific so do check the branch on LineageOS! (for example, its lineage-16.0-caf-8996 branch for 821 series family SoCs for pie). The next step is to remove project pathmpas from both the device tree and the HALs. You can refer to the changes done by PedroKalil -
https://github.com/KAOSP/platform_ha.../aosp-8.1-8996
https://github.com/KAOSP/platform_ha.../aosp-8.1-8996
https://github.com/KAOSP/platform_ha.../aosp-8.1-8937

Don't be lazy. Don't clone these HALs as these are outdated! Refer to them and do the appropriate changes on the latest Lineage HALs. the libbfqio changes may not be required as alot of AOSP-based roms have started to ship with it.

As for the Changes required in Device tree, you can refer to -
https://github.com/rupansh/pie_devic...7adeb7b08ab1d0 (The custom Audio policy is probably already enabled so no need to do that)
and https://github.com/rupansh/pie_devic...f19f3fd111c250 (Depends on the path you cloned your HAL to)

You should now be able to compile any AOSP-based ROM! Experience is the key. Some other AOSP-based roms I recommend - NitrogenOS, AEX, AOSCP, Pixel Experience(Fairly easy), Pixel Dust(One of the hard), AOSPA( teach me when you fix the cneserver error lulz) .

Thats all you need to know to build AOSP-based ROMs for CAF devices .




If you wish to get into Android and Linux Kernel Development, Learning C is a must! I recommend this guide by nathanchance if you are ready! - https://forum.xda-developers.com/and...ernel-t3627297

Groups for help

https://t.me/AndroidBuildersHelp
https://t.me/LinuxKernelNewbies (Linux Kernel only)
Your device groups
Let me know if you want your group here


Credits
@tanish2k09 and @Swapnil Soni ( For guiding me even though I had 0 knowledge, co-operating with my idiotic requests)
ABH (Helping me realise that you can fix most of the errors by just searching properly)
@nathanchance (for his awesome kernel guides)
@KalilDev (For helping me with building AOSP based roms)
@ZeroInfinity (For machine learning tutorials kek)
@riteshsaxena (Bess sources for reference xD, Helping me with pie bringup which caused rapid progress for me)
You, the reader

I hope you learnt plenty of stuff from the guide. It took me a while to type this you see :3 .


The Following 16 Users Say Thank You to rupanshji For This Useful Post: [ View ] Gift rupanshji Ad-Free
10th November 2018, 07:18 PM |#3  
Senior Member
Flag Belo Horizonte
Thanks Meter: 267
 
More
Quote:
Originally Posted by rupanshji

@KalilDev (For helping me with building AOSP based roms)

The Following User Says Thank You to KalilDev For This Useful Post: [ View ] Gift KalilDev Ad-Free
14th May 2019, 06:22 AM |#4  
Junior Member
Flag Nashik
Thanks Meter: 4
 
More
This is one of the best or the most helpful guide I've ever read! I'll Recommend this for anyone who wants to join the ROM building club.
The Following User Says Thank You to SauravKhare For This Useful Post: [ View ] Gift SauravKhare Ad-Free
17th June 2019, 05:42 PM |#6  
Junior Member
Thanks Meter: 8
 
More
Thank you for this, OP
7th November 2019, 07:11 PM |#7  
Junior Member
Thanks Meter: 0
 
More
[GUIDE][2018]All you need to know to build Android from scratch!
any guide like this for mtk devices?
Post Reply Subscribe to Thread

Tags
android, aosp, scract

Guest Quick Reply (no urls or BBcode)
Message:
Previous Thread Next Thread
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes