FORUMS
Remove All Ads from XDA

[HOW-TO] Building TWRP from source with goodies

822 posts
Thanks Meter: 1,574
 
By NZedPred, Senior Member on 4th January 2020, 02:15 AM
Post Reply Email Thread
I decided to put together a guide on how to build TWRP from source, having successfully built it myself and restored a few of the goodies that disappeared in the 3.3.1-0 release (e.g. install kernel). Also, marlin (Pixel XL) has had a few advancements over sailfish (Pixel), so this is a way to get back on par even if unofficially.

Whilst I'll go into a good level of detail, this will not be a hand-holding guide. You are expected to be able to use the Linux command line, and tools such as Git and repo.

Download sources
This guide is based upon building from the Minimal TWRP sources, which are based upon a subset of Omnirom. At the time of writing, the latest branch is twrp-9.0 (Pie). (If you are reading this some months after I have written this guide, check for later branches.) Follow this link for the repo, and the instructions:
https://github.com/minimal-manifest-.../tree/twrp-9.0

Firstly, create a folder to put this in, e.g. $HOME/twrp-minimal-omni, and change to that directory. Then execute the following for a shallow clone (to save space...):
Code:
repo init --depth=1 -u git://github.com/minimal-manifest-twrp/platform_manifest_twrp_omni.git -b twrp-9.0
Before doing the repo sync, we will need a few extra repositories to be downloaded. Create a subfolder .repo/local_manifests and create an XML file (e.g. custom.xml) with the following content:
Code:
<?xml version="1.0" encoding="UTF-8"?>
<manifest>
  <project name="android_device_google_marlin" path="device/google/marlin" remote="TeamWin" revision="android-9.0" />
  <project path="hardware/qcom/bootctrl" name="platform/hardware/qcom/bootctrl" groups="pdk-qcom" remote="aosp" />
  <project path="hardware/qcom/msm8998" name="platform/hardware/qcom/msm8998" groups="qcom_msm8998,pdk-qcom" remote="aosp" />
</manifest>
  • android_device_google_marlin - contains the configuration and build instructions for marlin and sailfish
  • hardware/qcom/bootctrl - this is part of AOSP, and is required for building TWRP
  • hardware/qcom/msm8998 - extra chipset stuff that is required for marlin/sailfish

You can do a repo sync now to download the sources.

Additionally, we will need some driver binaries that are available from Google. Get the Google (Qcom is not necessary) drivers for the latest Pie release here:
https://developers.google.com/androi...q3a.190801.002

Extract the files into the same folder as you created, and execute e.g as follows, and follow the prompts:
Code:
./extract-google_devices-sailfish.sh
The binaries will be extracted into a vendor/google-devices/ sub-folder. Do not rename the sub-folder - the configuration looks for exactly this path.

You now have all of the source files needed.

Patch to build for Sailfish on Omni
This is no longer required. The patch below has been merged into the official repo.

At the time of writing, the sources only allow building on the lineage branch, and some of the options we want just don't get us a functional build. So I have made appropriate updates to get sailfish on par with marlin.

Copy and paste the following code into a text editor:
Code:
From f42e1e866ba2e6892252d0d48099ce6f682bdfbd Mon Sep 17 00:00:00 2001
From: NZedPred <[email protected]il.com>
Date: Fri, 3 Jan 2020 20:17:51 +1300
Subject: [PATCH] Enable omni_sailfish

Change-Id: I33ad048038d27650dcacaf610e285a6243cb2028
---
 AndroidProducts.mk      |  1 +
 device-common.mk        |  2 +-
 omni_sailfish.mk        | 39 +++++++++++++++++++++++++++++++++++++++
 sailfish/BoardConfig.mk |  6 ++++--
 4 files changed, 45 insertions(+), 3 deletions(-)
 create mode 100644 omni_sailfish.mk

diff --git a/AndroidProducts.mk b/AndroidProducts.mk
index 75bae61..1934c32 100644
--- a/AndroidProducts.mk
+++ b/AndroidProducts.mk
@@ -1,4 +1,5 @@
 PRODUCT_MAKEFILES := 
 	$(LOCAL_DIR)/omni_marlin.mk 
+	$(LOCAL_DIR)/omni_sailfish.mk 
 	$(LOCAL_DIR)/lineage_sailfish.mk 
 	$(LOCAL_DIR)/lineage_marlin.mk
diff --git a/device-common.mk b/device-common.mk
index c98a4a4..c2bd98c 100644
--- a/device-common.mk
+++ b/device-common.mk
@@ -321,7 +321,7 @@ PRODUCT_PROPERTY_OVERRIDES += 
 
 # Write Manufacturer & Model information in created media files.
 # IMPORTANT: ONLY SET THIS PROPERTY TO TRUE FOR PUBLIC DEVICES
-ifneq ($(filter aosp_sailfish% sailfish% omni_marlin% lineage_marlin% marlin% lineage_sailfish%, $(TARGET_PRODUCT)),)
+ifneq ($(filter omni_sailfish% sailfish% omni_marlin% lineage_marlin% marlin% lineage_sailfish%, $(TARGET_PRODUCT)),)
 PRODUCT_PROPERTY_OVERRIDES += 
     media.recorder.show_manufacturer_and_model=true
 else
diff --git a/omni_sailfish.mk b/omni_sailfish.mk
new file mode 100644
index 0000000..70d4369
--- /dev/null
+++ b/omni_sailfish.mk
@@ -0,0 +1,39 @@
+#
+# Copyright 2015 The Android Open Source Project
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+# Sample: This is where we'd set a backup provider if we had one
+# $(call inherit-product, device/sample/products/backup_overlay.mk)
+
+# Inherit from the common Open Source product configuration
+$(call inherit-product, $(SRC_TARGET_DIR)/product/core_64_bit.mk)
+$(call inherit-product, $(SRC_TARGET_DIR)/product/aosp_base_telephony.mk)
+
+PRODUCT_NAME := omni_sailfish
+PRODUCT_DEVICE := sailfish
+PRODUCT_BRAND := Android
+PRODUCT_MODEL := AOSP on msm8996
+PRODUCT_MANUFACTURER := Google
+PRODUCT_RESTRICT_VENDOR_FILES := false
+
+PRODUCT_COPY_FILES += device/google/marlin/fstab.common:$(TARGET_COPY_OUT_VENDOR)/etc/fstab.marlin
+
+$(call inherit-product, device/google/marlin/device-sailfish.mk)
+$(call inherit-product-if-exists, vendor/google_devices/marlin/device-vendor-marlin.mk)
+
+PRODUCT_PACKAGES += 
+    Launcher3QuickStep 
+    WallpaperPicker 
+
diff --git a/sailfish/BoardConfig.mk b/sailfish/BoardConfig.mk
index 1c08e81..b0e39f4 100644
--- a/sailfish/BoardConfig.mk
+++ b/sailfish/BoardConfig.mk
@@ -206,8 +206,10 @@ TW_INPUT_BLACKLIST := "hbtp_vm"
 TW_DEFAULT_BRIGHTNESS := "80"
 TW_INCLUDE_CRYPTO := true
 AB_OTA_UPDATER := true
-TWRP_INCLUDE_LOGCAT := true
-TARGET_USES_LOGD := true
+#TWRP_INCLUDE_LOGCAT := true
+#TARGET_USES_LOGD := true
+TW_INCLUDE_REPACKTOOLS := true
+TW_EXCLUDE_UBSAN := true
 TW_USE_TOOLBOX := true
 TW_EXCLUDE_DEFAULT_USB_INIT := true
 #TARGET_RECOVERY_DEVICE_MODULES += [email protected] [email protected]
-- 
2.20.1
Save it into the device/google/marlin/ sub-folder, and call it e.g. sailfish.patch. Then execute:
Code:
git am sailfish.patch

Do the build

We then more-or-less follow the instructions as per the original minimal manifest page:
Code:
export ALLOW_MISSING_DEPENDENCIES=true
. build/envsetup.sh
lunch omni_sailfish-eng
mka bootimage
The boot image will be created at out/target/product/sailfish/boot.img.

Use fastboot boot out/target/product/sailfish/boot.img to test it in fastboot mode on your device.

Hope this helps someone out there!


XDA:DevDB Information
Building TWRP for Sailfish, Tool/Utility for the Google Pixel

Contributors
NZedPred

Version Information
Status: Stable

Created 2020-01-03
Last Updated 2020-01-16
The Following User Says Thank You to NZedPred For This Useful Post: [ View ] Gift NZedPred Ad-Free
4th January 2020, 02:15 AM |#2  
NZedPred's Avatar
OP Senior Member
Flag Wellington
Thanks Meter: 1,574
 
Donate to Me
More
Downloads
Occasionally I will make available one of my builds. They will be in the folder below.

Please note the following:
  • You are always encouraged to use the official versions of TWRP instead of these
  • I am not liable for anything that goes wrong as a consequence of using them
  • As a general rule, I will not be fixing bugs or adding features to these builds (other than enabling features that already are in the code) - they are built straight from the TWRP sources

Google Pixel (Sailfish) Unofficial TWRP
Google Pixel (Marlin) XL Unofficial TWRP

GPL compliance
The downloads in this folder use pre-compiled Linux kernels. The source for these can be found here:
https://android.googlesource.com/kernel/msm/

Refer to the tags/branches prefixed with "android-msm-marlin-".

Log
  • 20200115 - Added a build of OrangeFox Recovery for Sailfish
  • 20200105 - Added f2fs support (consider the f2fs option to be experimental, and it may not even work)
  • 20200104 - Initial version for sailfish with feature parity with marlin
4th January 2020, 02:15 AM |#3  
NZedPred's Avatar
OP Senior Member
Flag Wellington
Thanks Meter: 1,574
 
Donate to Me
More
Building other recoveries - Orangefox
Orangefox
Quick guide for the impatient
Create a folder that will be your OrangeFox folder, and change into it. All commands below will be relative to this folder.

Code:
# Initialize the repo
repo init --depth=1 -u https://gitlab.com/OrangeFox/Manifest.git -b fox_9.0
# Create local manifests folder
mkdir .repo/local_manifests
# Create the local manifest for sailfish/marlin - use gedit, kate, leafpad, whatever your text editor is
gedit .repo/local_manifests/orangefox-marlin.xml
After the last command, the text editor window will open. Paste the following content into it.
Code:
<?xml version="1.0" encoding="UTF-8"?>
<manifest>
    <remote name="TeamWin" fetch="https://github.com/TeamWin" review="https://gerrit.twrp.me" />
    <project name="external_magisk-prebuilt" path="external/magisk-prebuilt" remote="TeamWin" revision="master" />
    <project name="android_device_google_marlin" path="device/google/marlin" remote="TeamWin" revision="android-9.0" />
    <project path="hardware/qcom/bootctrl" name="platform/hardware/qcom/bootctrl" groups="pdk-qcom" remote="aosp" />
    <project path="hardware/qcom/msm8998" name="platform/hardware/qcom/msm8998" groups="qcom_msm8998,pdk-qcom" remote="aosp" />
</manifest>
Then save and close the editor, and continue with the following commands.
Code:
# Sync the repos
repo sync -j8
Additionally, we will need some driver binaries that are available from Google. Get the Google (Qcom aren't necessary) drivers for the latest Pie release here:
https://developers.google.com/androi...q3a.190801.002

Extract the files into the same folder as you created, and execute as follows, and follow the prompts:
Code:
./extract-google_devices-sailfish.sh
The binaries will be extracted into a vendor/google-devices/ sub-folder. Do not rename the sub-folder - the configuration looks for exactly this path.

You now have all of the source files needed.

At the time of writing, there is a bug that prevents the FB2PNG module building (I think this is FrameBuffer2PNG - screenshots). Edit the file as below
Code:
# Edit sailfish config
gedit device/google/marlin/sailfish/BoardConfig.mk
# Edit marlin config
gedit device/google/marlin/marlin/BoardConfig.mk
Around line 204, change
Code:
TW_INCLUDE_FB2PNG := true
to
Code:
TW_INCLUDE_FB2PNG := false

Now continue to build as normal
Code:
. build/envsetup.sh
export ALLOW_MISSING_DEPENDENCIES=true
# For building sailfish use this line
lunch omni_sailfish-eng
# For building marlin use this line
lunch omni_marlin-eng
mka bootimage
NOTE: If you are building for sailfish, don't run the lunch omni_marlin-eng line. Also, you may get an error "Your device can't be found in device sources..". That's actually OK, it's just because there isn't a folder called device/google/sailfish. If the error bothers you, just create the folder and re-run the lunch command (and ignore the warning about device/google/sailfish/omni.dependencies file not found).

This will create a fastboot bootable image in the $OUT folder called boot.img. You can try booting it with your device in fastboot mode:
Code:
fastboot boot $OUT/boot.img
There are also a number of environment variables that OrangeFox has available. I haven't really tested whether or not they are necessary. Will update this post after I do some investigation.
4th January 2020, 03:21 PM |#4  
Member
Thanks Meter: 6
 
More
Could you make twrp for marlin with support F2FS? I did but unsuccesed.
4th January 2020, 07:53 PM |#5  
NZedPred's Avatar
OP Senior Member
Flag Wellington
Thanks Meter: 1,574
 
Donate to Me
More
Quote:
Originally Posted by chuate92

Could you make twrp for marlin with support F2FS? I did but unsuccesed.

You will need to add a flag to the device/google/marlin/marlin/BoardConfig.mk file:
Code:
TARGET_USERIMAGES_USE_F2FS := true
Additionally, the kernel needs to support f2fs. No idea if the one that comes with the repos supports it, but you could always grab one from a kernel that does and try that.

I'll try doing a build later on, but you will have to test at your own risk.
4th January 2020, 08:06 PM |#6  
Member
Thanks Meter: 6
 
More
Quote:
Originally Posted by NZedPred

You will need to add a flag to the device/google/marlin/marlin/BoardConfig.mk file:

Code:
TARGET_USERIMAGES_USE_F2FS := true
Additionally, the kernel needs to support f2fs. No idea if the one that comes with the repos supports it, but you could always grab one from a kernel that does and try that.

I'll try doing a build later on, but you will have to test at your own risk.

I followed your guide but, I didn't know how to do "Create a subfolder .repo/local_manifests and create an XML file (e.g. custom.xml) with the following content:". So I downloaded manual. Then I do build recoveryimage, I still got error them same I posted here https://forum.xda-developers.com/sho...9#post81384609.
I didn't know why.
4th January 2020, 10:07 PM |#7  
NZedPred's Avatar
OP Senior Member
Flag Wellington
Thanks Meter: 1,574
 
Donate to Me
More
Quote:
Originally Posted by chuate92

I followed your guide but, I didn't know how to do "Create a subfolder .repo/local_manifests and create an XML file (e.g. custom.xml) with the following content:". So I downloaded manual. Then I do build recoveryimage, I still got error them same I posted here https://forum.xda-developers.com/sho...9#post81384609.
I didn't know why.

For the above, using the command line, from the folder you originally created you do this:
Code:
mkdir .repo/local_manifests
gedit .repo/local_manifests/custom.xml
If you don't have gedit, use e.g. kate, or whatever other text editor you have.

Also, you don't build a recovery image - you build a boot image.
Code:
mka bootimage
Anyway, I've updated the second post with a folder for Google Pixel XL as well. I'll upload there later (once AFH sorts itself out - I can't upload anything into the new folder yet).

For now, try using the build here (at your own risk):
https://www.dropbox.com/s/gvkvhklmu2...00105.img?dl=0
8th January 2020, 06:14 PM |#8  
Senior Member
Flag Tel Aviv
Thanks Meter: 149
 
More
tried this unofficial twrp for sailfish. it boots just fine as temporary recovery, when trying to install into ramDisk it acts as the official image, too large to fit the ramDisk
8th January 2020, 07:38 PM |#9  
NZedPred's Avatar
OP Senior Member
Flag Wellington
Thanks Meter: 1,574
 
Donate to Me
More
Quote:
Originally Posted by wizardwiz

tried this unofficial twrp for sailfish. it boots just fine as temporary recovery, when trying to install into ramDisk it acts as the official image, too large to fit the ramDisk

Yes, TWRP is now getting quite big and a lot of features would have to be removed to get it to fit into the ramdisk. This is something I've been spending some time on, but it seems to be a difficult ask at this point. I'll post if I ever get there...
The Following User Says Thank You to NZedPred For This Useful Post: [ View ] Gift NZedPred Ad-Free
9th January 2020, 06:49 PM |#10  
Junior Member
Thanks Meter: 9
 
More
I'm having some trouble with your guide. I am trying to compile not TWRP but orangefox recovery. I have the files and I will link them below. If you could either walk me through building orangefox or building it yourself, that would be amazing. I have a sailfish.

Link: https://gitlab.com/OrangeFox/Recovery
12th January 2020, 02:31 AM |#11  
NZedPred's Avatar
OP Senior Member
Flag Wellington
Thanks Meter: 1,574
 
Donate to Me
More
Quote:
Originally Posted by Spookybear

I'm having some trouble with your guide. I am trying to compile not TWRP but orangefox recovery. I have the files and I will link them below. If you could either walk me through building orangefox or building it yourself, that would be amazing. I have a sailfish.

Link: https://gitlab.com/OrangeFox/Recovery

I've had a look at this, and I don't think it's as straight forward as merely dropping in in the TWRP sailfish device tree and building. It reports success when building, but it hasn't actually built anything...

I'd suggest you go to their Telegram chat for building OrangeFox: https://t.me/OrangeFoxBuilding
The Following User Says Thank You to NZedPred For This Useful Post: [ View ] Gift NZedPred Ad-Free
Post Reply Subscribe to Thread

Guest Quick Reply (no urls or BBcode)
Message:
Previous Thread Next Thread
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes