FORUMS

[GUIDE] How to build a Project Treble GSI ROM from source? [31/08]

3,319 posts
Thanks Meter: 7,023
 
Post Reply Email Thread
Hello guys, in this guide I'll try to simplify building Treble GSI process.
As you read this guide now I'll assume you already have a previous knowledge about How to build android from source, so I won't cover some points with too many basic details.
So, let's start:


What you’ll need
  • A treble enabled device, basically all devices that come with Android 8.1 out of box support it.
  • A relatively recent 64-bit computer (Linux, OS X, or Windows) with a reasonable amount of RAM and about 100 GB of free storage (more if you enable ccache or build for multiple devices). The less RAM you have, the longer the build will take (aim for 8 GB or more). Using SSDs results in considerably faster build times than traditional hard drives.
  • A USB cable compatible with your device
  • A decent internet connection & reliable electricity
  • Some familiarity with basic Android operation and terminology. It would help if you’ve installed custom ROMs on other devices and are familiar with recovery. It may also be useful to know some basic command line concepts such as cd for “change directory”, the concept of directory hierarchies, that in Linux they are separated by /. etc.


Summary
1. Install SDK
2. Install build packages
3. Install the repo command
4. Configure git
5. Turn on caching to speed up build
6. Build using phhusson's script
7. Build using dakkar's script
8. Build using the manual way

1. Install SDK
If you haven’t previously installed adb and fastboot, you can download them from Google. Extract it using:
Code:
unzip platform-tools-latest-linux.zip -d ~
Now we have to add adb and fastboot to our path. Open ~/.profile and add the following:
Code:
# add Android SDK platform tools to path
if [ -d "$HOME/platform-tools" ] ; then
    PATH="$HOME/platform-tools:$PATH"
fi
Then, run this to update your environment.
Code:
source ~/.profile
2. Install build packages
Several packages are needed to build Android. You can install these using your distribution’s package manager.
You’ll need:
Code:
bc bison build-essential curl flex g++-multilib gcc-multilib git gnupg gperf imagemagick lib32ncurses5-dev
lib32readline-gplv2-dev lib32z1-dev libesd0-dev liblz4-tool libncurses5-dev libsdl1.2-dev libwxgtk3.0-dev
libxml2 libxml2-utils lzop pngcrush schedtool squashfs-tools xsltproc zip zlib1g-dev openjdk-8-jdk
3. Install the repo command
Enter the following to download the repo binary and make it executable (runnable):
Code:
mkdir -p ~/bin
curl https://storage.googleapis.com/git-repo-downloads/repo > ~/bin/repo
chmod a+x ~/bin/repo
Put the ~/bin directory in your path of execution

In recent versions of Ubuntu, ~/bin should already be in your PATH. You can check this by opening ~/.profile with a text editor and verifying the following code exists (add it if it is missing):
Code:
# set PATH so it includes user's private bin if it exists
if [ -d "$HOME/bin" ] ; then
    PATH="$HOME/bin:$PATH"
fi
Then, use this to update your environment.
Code:
source ~/.profile
4. Configure git
You’ll need to set up git identity in order to sync the source, run these commands:
Code:
git config --global user.name "your username"
git config --global user.email [email protected]
5. Turn on caching to speed up build
You can speed up subsequent builds by running:
Code:
export USE_CCACHE=1
export CCACHE_COMPRESS=1
And adding that line to your ~/.bashrc file. Then, specify the maximum amount of disk space you want the cache to use by typing this from the top of your Android tree:
Code:
prebuilts/misc/linux-x86/ccache/ccache -M 50G
Where 50G corresponds to 50GB of cache. This needs to be run once. Anywhere from 25GB-100GB will result in very noticeably increased build speeds (for instance, a typical 1hr build time can be reduced to 20min). If you’re only building for one device, 25GB-50GB is fine. If you plan to build for several devices that do not share the same kernel source, aim for 75GB-100GB. This space will be permanently occupied on your drive, so take this into consideration. See more information about ccache on Google’s Android build environment initialization page.

6. Build using phhusson's script
We can't deny that @phhusson has made amazing works and countless contributions to Project Treble ROMs development apart from his experimentations is a build script which make build a GSI super simple job.
By default, it supports building LineageOS - RR - Carbon.
1- Open your terminal and run:
Code:
git clone https://github.com/phhusson/treble_experimentations
2- To clone and build enter the following command and replace "romname" with lineage|rr|carbon
Code:
mkdir romname; cd romname
bash ../treble_experimentations/build-rom.sh android-8.1 romname
3- The script will automatically initialize the repository, sync the source, apply patches and start building.

7. Build using dakkar's script
dakkar's script is another treble building script, originally made by @Dakkar and improved by contributors on treble experimentations repo. It's customizable, easy to understand and can build almost all ROMs with simple edits.

1- Open your terminal and run:
Code:
git clone https://github.com/phhusson/treble_experimentations
2- To start the process:
Code:
bash ../treble_experimentations/build-dakkar.sh romname variant
Quote:

Variants are dash-joined combinations of (in order):
* processor type
* "arm" for ARM 32 bit
* "arm64" for ARM 64 bit
* A or A/B partition layout ("aonly" or "ab")
* GApps selection
* "vanilla" to not include GApps
* "gapps" to include opengapps
* "go" to include gapps go
* SU selection ("su" or "nosu")
for example:
* arm-aonly-vanilla-nosu
* arm64-ab-gapps-su

Note: check patches when you use these auto scripts, if some patch is broken you'll have build errors

8. Build using the manual way
In simple steps:
1. Repo init the rom you want to build GSI for.
Code:
mkdir ~/rom &&  cd ~/rom
repo init -u https://github.com/LineageOS/android.git -b lineage-15.1
2. Add phh repos to your local_manifest
Code:
git clone https://github.com/phhusson/treble_manifest .repo/local_manifests -b android-8.1
After git clone you need to remove replace.xml if you're building any rom expect aosp.
Quote:
Originally Posted by phhusson

The replace.xml is meant only for AOSP, if you build a custom ROM, remove it, but don't forget to cherry-pick the patches.

3. Sync the source
Code:
repo sync -c -j4 --force-sync --no-tags --no-clone-bundle
4. Modify the source to fix issues in other devices using one of these methods:
- Apply phh patches:
Code:
git clone https://github.com/phhusson/treble_patches -b android-8.1
Then apply each path in its project
Code:
patch -p1 < patch
- Or cherry-pick changes by phhusson from here, remember to choose the latest branch
platform_build - platform_external_selinux - platform_frameworks_av - platform_frameworks_base - platform_frameworks_native - platform_frameworks_opt_telephony - platform_system_bt - platform_system_core - platform_system_libvintf - platform_system_vold
Code:
git fetch repo branch && git cherry-pick commit
5. Go to the phh device repo and edit the .mk for your ROM (example lineage.mk)
6. Lunch the build varaint you want (ex. treble_arm64_avN-userdebug) and start the build
Code:
. build/envsetup.sh
lunch treble_arm64_avN-userdebug
WITHOUT_CHECK_API=true make -j8 systemimage
7. If you want to compress the system image after build finishes, go to out/target/product/phh_*/ folder and run
Code:
xz -c system.img > system.img.xz
That's all

Credits:
- @phhusson for all his contributions, without his efforts this can't be possible.
- @Dakkar for his build script
- @fAIyaZ for helping me build my first GSI
- @sooti for his simplified instruction on phh-treble telegram.
- @lineageos guys for their wiki
- And me for writing this guide
The Following 75 Users Say Thank You to yshalsager For This Useful Post: [ View ]
9th June 2018, 03:41 PM |#2  
yshalsager's Avatar
OP Recognized Developer
Flag Cairo
Thanks Meter: 7,023
 
Donate to Me
More
Community Edition
I made a community edition of this guide, available on @phhusson treble experimentations wiki
Feel free to edit it and improve the current one
How to build a GSI?
The Following 12 Users Say Thank You to yshalsager For This Useful Post: [ View ]
9th June 2018, 03:58 PM |#3  
Senior Member
Flag Delhi
Thanks Meter: 9
 
More
Weew thanks. Will try to 😉
12th June 2018, 12:41 PM |#4  
Member
Thanks Meter: 31
 
More
I'm getting an error when trying to repo sync manually the sources to download all of the phhusson's code. The thing is that even modifying the "replace.xml" file (the one "causing" the error) to adapt it to the rom's manifest, the error persists.

Repo is telling me the following: fatal: remove-project element specifies non-existent project: platform/build If I change the "platform/build" reference to the correct path, it's still giving me the error but with the new route, even while the folders and files clearly exist.

Anyone got a clue on why is this happening? I mean, the way to solve the problem is pretty straightforward, I just need to download manually the file from the Github repo and then perform all the needed chances, but it's always easier and cleaner to do it via the repo command.
The Following 2 Users Say Thank You to Sinek_ For This Useful Post: [ View ] Gift Sinek_ Ad-Free
12th June 2018, 02:35 PM |#5  
phhusson's Avatar
Recognized Developer
Flag Paris
Thanks Meter: 4,685
 
Donate to Me
More
Quote:
Originally Posted by Sinek_

I'm getting an error when trying to repo sync manually the sources to download all of the phhusson's code. The thing is that even modifying the "replace.xml" file (the one "causing" the error) to adapt it to the rom's manifest, the error persists.

Repo is telling me the following: fatal: remove-project element specifies non-existent project: platform/build If I change the "platform/build" reference to the correct path, it's still giving me the error but with the new route, even while the folders and files clearly exist.

Anyone got a clue on why is this happening? I mean, the way to solve the problem is pretty straightforward, I just need to download manually the file from the Github repo and then perform all the needed chances, but it's always easier and cleaner to do it via the repo command.

Ah indeed, the replace.xml is meant only for AOSP, if you build a custom ROM, remove it, but don't forget to cherry-pick the patches.

@yshalsager ^^^
The Following 4 Users Say Thank You to phhusson For This Useful Post: [ View ]
13th June 2018, 12:58 PM |#6  
Senior Member
Flag Sydney
Thanks Meter: 391
 
More
I'm sure this thread will see a lot of views in the coming years. Also it gives us noobs (me) a chance to build ROMs that actually build and run on our devices. Thanks for writing this.

Edit: in future if there's a way to add device specific features into here, it would be amazing. (Alert slider etc)
14th June 2018, 12:31 PM |#7  
phhusson's Avatar
Recognized Developer
Flag Paris
Thanks Meter: 4,685
 
Donate to Me
More
Quote:
Originally Posted by Jamie_oppo

Edit: in future if there's a way to add device specific features into here, it would be amazing. (Alert slider etc)

I'm not sure what you mean. Alert slider is already supported in GSIs
Or you're asking for a documentation on how to implement such features for GSIs?
14th June 2018, 10:34 PM |#8  
Senior Member
Flag Sydney
Thanks Meter: 391
 
More
Quote:
Originally Posted by phhusson

I'm not sure what you mean. Alert slider is already supported in GSIs
Or you're asking for a documentation on how to implement such features for GSIs?

Oh I isn't know the alert slider was already implemented in gsi. ( the only one I've flashed is dot os. ) maybe thats why.


Yeah I'm asking if its possible to add documentation for device specific features. Like imagine a new budget Motorola (with something specific like alert slider v3) with treble as a base and a Dev would be able to add the device specific "alert slider V3" from source to the gsi (and that gsi can be the ones that, that specific Motorola will flash).

Its so awesome they all of us involved with treble have an awesome Dev like you.
22nd June 2018, 12:52 PM |#9  
Senior Member
Flag Hessen
Thanks Meter: 19
 
More
Hi
i try to build gsi on windows subsystem for linux with dakkar's script. But it failed with the following error:
Quote:

...
[ 2% 2291/105390] Build hyb /root/tre...phenation-patterns/nn//hyph-nn.pat.txt
12585 unique nodes, 42861 total
[ 2% 2359/105390] Generating TOC: /ro...S/sdk_v8_intermediates/classes.jar.toc
FAILED: /root/treble_experimentations/out/target/common/obj/JAVA_LIBRARIES/sdk_v8_intermediates/classes.jar.toc
/bin/bash -c "(ASAN_OPTIONS=detect_leaks=0 prebuilts/build-tools/linux-x86/bin/ijar /root/treble_experimentations/out/target/common/obj/JAVA_LIBRARIES/sdk_v8_intermediates/classes.jar /root/treble_experimentations/out/target/common/obj/JAVA_LIBRARIES/sdk_v8_intermediates/classes.jar.toc.tmp ) && (if cmp -s /root/treble_experimentations/out/target/common/obj/JAVA_LIBRARIES/sdk_v8_intermediates/classes.jar.toc.tmp /root/treble_experimentations/out/target/common/obj/JAVA_LIBRARIES/sdk_v8_intermediates/classes.jar.toc ; then rm /root/treble_experimentations/out/target/common/obj/JAVA_LIBRARIES/sdk_v8_intermediates/classes.jar.toc.tmp ; else mv /root/treble_experimentations/out/target/common/obj/JAVA_LIBRARIES/sdk_v8_intermediates/classes.jar.toc.tmp /root/treble_experimentations/out/target/common/obj/JAVA_LIBRARIES/sdk_v8_intermediates/classes.jar.toc ; fi )"
ftruncate(fd_out, GetSize()): Invalid argument
/bin/bash: Zeile 1: 2346 Abgebrochen (Speicherabzug geschrieben) ( ASAN_OPTIONS=detect_leaks=0 prebuilts/build-tools/linux-x86/bin/ijar /root/treble_experimentations/out/target/common/obj/JAVA_LIBRARIES/sdk_v8_intermediates/classes.jar /root/treble_experimentations/out/target/common/obj/JAVA_LIBRARIES/sdk_v8_intermediates/classes.jar.toc.tmp )
[ 2% 2362/105390] Build hyb /root/tre...phenation-patterns/hu//hyph-hu.pat.txt
21515 unique nodes, 102669 total
ninja: build stopped: subcommand failed.
11:27:50 ninja failed with: exit status 1
[email protected]:~/treble_experimentations#

I used this command to build lineage with su und gapps:
Quote:

bash ../treble_experimentations/build-dakkar.sh lineage arm64-aonly-gapps-su

26th June 2018, 10:41 AM |#10  
Senior Member
Flag Hessen
Thanks Meter: 19
 
More
I figured out, that the error is a problem with the wsl. To fix this problem a patch is needed: https://forum.xda-developers.com/sho...4&postcount=20.
How can i apply this patch?
The Following User Says Thank You to Saftpresse99 For This Useful Post: [ View ] Gift Saftpresse99 Ad-Free
26th June 2018, 11:28 AM |#11  
yshalsager's Avatar
OP Recognized Developer
Flag Cairo
Thanks Meter: 7,023
 
Donate to Me
More
Quote:
Originally Posted by Saftpresse99

I figured out, that the error is a problem with the wsl. To fix this problem a patch is needed: https://forum.xda-developers.com/sho...4&postcount=20.
How can i apply this patch?

Cherry pick his commit like I explained in the guide
Post Reply Subscribe to Thread

Tags
gsi, phh treble, project treble, treble

Guest Quick Reply (no urls or BBcode)
Message:
Previous Thread Next Thread
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes