• Introducing XDA Computing: Discussion zones for Hardware, Software, and more!    Check it out!

[GUIDE] Re-locking the bootloader on the OnePlus 8t with a self-signed build of LOS 18.1

Search This thread

WhitbyGreg

Senior Member
Thanks, WhitbyGreg,

So what you would suggest for periodic recovery tasks like clear cache and backup/restore the userdata partition when the bootloader is locked and signed LineageOS + its usual recovery is installed?
Maybe I can sign TWRP but not flush it and only boot it through the fastboot mode once I need to do a backup/restore?
From my understanding "clear cache" isn't required in modern android, so no issue there. For backup/restore you shoud use tools the work within Android, like seedvault or various backup tools.

You can sign TWRP and use "fastboot boot" if your phone supports it (the 8T and pretty much anything newer than and including the 6 does from OnePlus).

I use NextCloud to host my contacts/schedule, SMS Backup and Restore for SMS/MMS/Calls logs (weekly automated backups to NextCloud), and do manual backups to NextCloud for things like messaging apps etc, and SeedVault (to NextCloud) for everything that supports Android's native backup API.
 
  • Like
Reactions: xHasKx
What is this tutorial?
This tutorial will:
  • Creating an unofficial build of LineageOS 18.1 suitable for using to re-lock the bootloader on a OnePlus 8t
  • Take you through the process of re-locking your bootloader after installing the above

This tutorial will NOT:
  • Remove *all* warning messages during boot (the yellow "Custom OS" message will be present though the orange "Unlocked bootloader" message will not)
  • Allow you to use official builds of LineageOS 18.1 on your device with a re-locked bootloader (more details near the end of the tutorial)
This tutorial will assume you are working on an Ubuntu 18.04 installation, if you are using Windows or another Linux distro, the commands may be different.

Supported devices:
The following devices have been tested and confirmed to work:
  • OnePlus 7 Pro (guacamole)
  • OnePlus 8t (kebab)
  • Pixel 4 (flame)
Other OnePlus devices that support AVBv2 (OnePlus 6t and newer as well as most Pixel devices) and LineageOS 18.1 (see current support list over on the LineageOS download page) should work as well.

For simplicities sake, all further references will only be to the 8t (kebab).

Pre-requisites:
  • a mid level knowledge of terminal commands and features
  • a supported phone
  • a PC with enough CPU/RAM to build LineageOS 18.1 (recommended 8 cores, 24g of RAM)
  • a working USB cable
  • fastboot/adb installed and functional
  • LineageOS 18.1 source code downloaded
  • at least one successful build of LineageOS
  • at least one successful signing of your build with your own keys

Misc. notes:
  • the basics of building/signing of LineageOS is outside the scope of this tutorial, refer to the LineageOS Wiki for details on how to complete these tasks
  • you'll be modifying some code in LineageOS, so if you are not comfortable using basic editing utilities as well as patch, do not proceed any further
  • the path to your LineageOS source code is going to be assumed to be ~/android/lineageos, if it is somewhere else, substitute the correct path in the tutorial
  • the path to your private certificate files is going to be assumed to be ~/android-certs, if it is somewhere else, substitute the correct path in the tutorial


*** WARNING ****
This process may brick your device. Do not proceed unless you are comfortable taking this risk.


*** WARNING ****
This process will delete all data on your phone! Do not proceed unless you have backed up your data!


*** WARNING ****
Make sure you have read through this entire process at least once before attempting, if you are uncomfortable with any steps include in this guide, do not continue.


And now on with the show!

Step 1: Basic setup

You need a few places to store things, so create some working directories:
Code:
mkdir ~/android/kebab
mkdir ~/android/kebab/patches
mkdir ~/android/kebab/pkmd
You also need to add "~/android/lineageos/out/host/linux-x86/bin" to your shell's profile path. Make sure to close and restart your session afterwards otherwise the signing will fail later on with a "file not found" error message (this may no longer be required).

Step 2: Update kebab's BoardConfig.mk

You will need to add a few parameters to the end of ~/android/lineageos/device/oneplus/kebab/BoardConfig.mk, they are:

Code:
BOARD_AVB_ALGORITHM := SHA256_RSA2048
BOARD_AVB_KEY_PATH := /home/<userid>/.android-certs/releasekey.key
Note you cannot use "~" in the path names above to signify your home directory, so give the full absolute path to make sure the files are found.

Step 3: Update sm8250-common's BoardConfigCommon.mk

LineageOS by default disables Android Verified Boot's partition verification, but you can enable it now as all the required parts will be in place.

To enable partition verification do the following:

Code:
cd ~/android/lineageos/device/oneplus/sm8250-common
sed -i 's/^BOARD_AVB_MAKE_VBMETA_IMAGE_ARGS += --flags 2/#BOARD_AVB_MAKE_VBMETA_IMAGE_ARGS += --flags 2/' BoardConfigCommon.mk
sed -i 's/^BOARD_AVB_MAKE_VBMETA_IMAGE_ARGS += --set_hashtree_disabled_flag/#BOARD_AVB_MAKE_VBMETA_IMAGE_ARGS += --set_hashtree_disabled_flag/' BoardConfigCommon.mk
sed -i 's/^BOARD_AVB_VBMETA_SYSTEM_KEY_PATH := external\/avb\/test\/data\/testkey_rsa2048.pem/BOARD_AVB_KEY_PATH := \/home\/<userid>\/.android-certs\/releasekey.key/' BoardConfigCommon.mk

Don't forget to replace your <userid> in the third sed command above with your current logged in user id.

Step 4: Patch the AOSP and Device Makefile

You also need to patch the Makefile included with AOSP as it will otherwise fail during the build.

The required patch can be found here:

Download it and store in ~/android/kebab/patches.

Now apply it with the following command:

Code:
cd ~/android/lineageos/build/core
patch Makefile ~/android/kebab/patches/core-Makefile-fix-18.1.patch

If you would like to know more about this patch, see the additional info at the bottom of this post.

There is also a small addition to the device's common.mk required to enable the OEM unlock option in developers options, do this via the following commands:

Code:
cd ~/android/lineageos/device/oneplus/sm8250-common
sed -i 's/^# OMX/# OEM Unlock reporting\nPRODUCT_DEFAULT_PROPERTY_OVERRIDES += \\\n    ro.oem_unlock_supported=1\n\n# OMX/' common.mk

Step 5: Build LineageOS

You are now ready to build:

Code:
cd ~/android/lineageos
breakfast kebab
source build/envsetup.sh
croot
mka target-files-package otatools

Step 6: Sign the APKs

You are now ready to sign the apks with sign_target_files_apks:

Code:
./build/tools/releasetools/sign_target_files_apks -o -d ~/.android-certs $OUT/obj/PACKAGING/target_files_intermediates/*-target_files-*.zip signed-target_files.zip

Step 7: Build the OTA

Now it is time to complete the OTA package:

Code:
./build/tools/releasetools/ota_from_target_files -k ~/.android-certs/releasekey --block signed-target_files.zip lineage-18.1-[date]-UNOFFICIAL-kebab-signed.zip

Note, replace [date] with today's date in YYYYMMDD format.

Step 8: Create pkmd.bin for your phone

Before you can lock your phone, you have to tell it what your public key is so it knows it can trust your build.

To do this you need to create a pkmd.bin file:

Code:
~/android/lineageos/external/avb/avbtool extract_public_key --key ~/.android-certs/releasekey.key --output ~/android/kebab/pkmd/pkmd.bin

Step 9: Flashing your LineageOS build

It's time to flash your build to your phone. The following steps assume you have already unlocked your phone and have flashed an official version of LineageOS to it. You don't need to have flashed LineageOS yet, you could use TWRP through "fastboot boot" if you prefer. Or, if you want to use the recovery that was just created, it is located in ~/android/lineageos/out/target/product/kebab and is called recovery.img.

  • Reboot your phone in to recovery mode
  • In LineageOS Recovery return to the main menu and select "Apply update"
  • From your PC, run:
Code:
adb sideload ~/android/lineageos/lineage-18.1-[date]-UNOFFICIAL-kebab-signed.zip

When the sideload is complete, reboot in to LineageOS. Make sure everything looks good with your build.

You may also need to format your data partition at this time depending on what you had installed on your phone previously.

Step 10: Flashing your signing key

Now it's time to add your signing key to the Android Verified Boot process. To do so, do the following:

  • Reboot your phone in to fastboot mode
  • From your PC, run:
Code:
fastboot flash avb_custom_key ~/android/kebab/pkmd/pkmd.bin
fastboot reboot bootloader
fastboot oem lock
  • On your phone, confirm you want to re-lock and it will reboot

Your phone will then factory reset and then reboot in to LineageOS.

Which of course means you have to go through the first time setup wizard, so do so now.

Step 11: Disable OEM unlock

Congratulations! Your boot loader is now locked, but you can still unlock it again using fastboot, so it's time to disable that as well.

  • Unlock you phone and go to Settings->About phone
  • Scroll to the bottom and find "Build number"
  • Tap on it you enable the developer options
  • Go to Settings->System->Advanced->Developer options
  • Disable the "OEM unlocking" slider
  • Reboot

Step 12: Profit!


Other things
  • The above will build a standard USERDEBUG version of LineageOS, however this will still allow LineageOS Recovery to sideload non-signed files as well as give you root shell access through ADB. Step 3/4 above protects your system/vendor/boot/dtbo/etc. partitions, but none of the others. Likewise USERDEBUG builds will allow for rolling back to a previous builds/versions of LineageOS. To increase security and disallow both of these scenarios you may want to build a USER version of LineageOS to install. However this brings in other issues, such as flashing newer firmware from OnePlus so make sure you understand the implications of both choices. For more details on build types, see https://source.android.com/setup/develop/new-device#build-variants.
  • In the above example the releasekey from your LineageOS install has been used to sign AVB, but AVB supports other key strengths up to SHA512_RSA8192. You could create a key just for signing AVB that used different options than the default keys generated to sign LineageOS.
  • If you want to remove you signing key from your phone, you can do it by running "fastboot erase avb_custom_key".
  • The changes you made to the AOSP Makefile may conflict with future updates that you pull from LineageOS through repo sync, if you have to reset the file to get repo sync to complete successfully, you'll have to reapply the changes afterwards.

So why can't I do this with official LineageOS builds?

NEW: You can! See this thread for more details.

For Android Verified Boot (AVB) to work, it must have the hash values for each of the system/vendor/boot/dtbo/etc. partitions stored in vbmeta. Official LineageOS builds for kebab do include the vendor.img in them along with everything else that is needed, however that is not true for all phones.

There are two "issues" that stop someone from using the official kebab builds:
  • LineageOS does not provide a pkmd.bin file to flash to your phone to include the public key in your AVB process (NEW: this thread shows you how to extract the key).
  • AVB is enabled in the official LineageOS builds but does not validate the hash trees during boot which limits the protection offered.
Ok, what messages do I see during the boot process then?

During a boot you will of course see the standard OnePlus power up screen, followed by the yellow "custom os" message and then the standard LineageOS boot animation.

For more details on AVB boot messages, see https://source.android.com/security/verifiedboot/boot-flow

So what does that patch to the Makefile do?

AOSP's default Makefile makes an assumption that when AVB is enabled, that all the img files will be available well before vbmeta.img is created. This is simply NOT true and AOSP seems to know this as well from the following comment in the Makefile:

Code:
# Not using INSTALLED_VBMETA_SYSTEMIMAGE_TARGET as it won't be set yet.
ifdef BOARD_AVB_VBMETA_SYSTEM
$(eval $(call check-and-set-avb-args,vbmeta_system))
endif

ifdef BOARD_AVB_VBMETA_VENDOR
$(eval $(call check-and-set-avb-args,vbmeta_vendor))
endif

These two calls eventual evaluate to returning the path to the partitions based upon the INSTALLED_*IMAGE_TARGET variable, which isn't created until later in the build process.

Because of this, the command to build vbmeta.img gets corrupted due to the missing make variable being empty and an invalid command line is passed to avbtool near the end of the build.

The corruption happens due to the fact that the following line from the original Makefile:

Code:
--include_descriptors_from_image $(call images-for-partitions,$(1))))))

Gets added to the avbtool call even if "$(call images-for-partitions,$(1))" turns out to be an empty string. Avbtool then throws an error message as it is expecting a parameter after the "--include_descriptors_from_image" flag that is added for the "empty" partition path.

The fix is to call "$(call images-for-partitions,$(1))" earlier, set it to a variable and check to make sure it isn't an empty string before letting the "--include_descriptors_from_image" be added to the avbtool command line to be used later.

This technically generates an incomplete vbmeta.img file during the build process, but since the signing process recreates it from scratch anyway; no harm, no foul.

Thank You's
thanks for the guide. ps regarding the path requirements and the absolutely annoying signapk.jar issues when on the signing step, one can simple add a flag to the command like so
Code:
./build/tools/releasetools/sign_target_files_apks -p out/host/linux-x86/ -o -d ~/.android-certs $OUT/obj/PACKAGING/target_files_intermediates/*-target_files-*.zip signed-target_files.zip
the addition of -p out/host/linux-x86/ takes care of the pathing issues
If anyone runs into the unicode encoding error with common.py, run repopick 305886
which will cherry-pick https://review.lineageos.org/c/LineageOS/android_build/+/305886 and fix that error.

Aside from those 2 things, (y) 🍻
 
  • Like
Reactions: mattya__ and xHasKx
Code:
checkvintf E 09-03 13:02:08 1018132 1018132 check_vintf.cpp:554] files are incompatible: Framework manifest and device compatibility matrix are incompatible: Vndk version 30 is not supported. Supported versions in framework manifest are:: Success

Traceback (most recent call last):
  File "/home/xstefen/android/los/./build/tools/releasetools/ota_from_target_files", line 2300, in <module>
    main(sys.argv[1:])
  File "/home/xstefen/android/los/./build/tools/releasetools/ota_from_target_files", line 2270, in main
    GenerateAbOtaPackage(
  File "/home/xstefen/android/los/./build/tools/releasetools/ota_from_target_files", line 1957, in GenerateAbOtaPackage
    CheckVintfIfTrebleEnabled(target_file, target_info)
  File "/home/xstefen/android/los/./build/tools/releasetools/ota_from_target_files", line 630, in CheckVintfIfTrebleEnabled
    raise RuntimeError("VINTF compatibility check failed")
RuntimeError: VINTF compatibility check failed
 

WhitbyGreg

Senior Member
Code:
checkvintf E 09-03 13:02:08 1018132 1018132 check_vintf.cpp:554] files are incompatible: Framework manifest and device compatibility matrix are incompatible: Vndk version 30 is not supported. Supported versions in framework manifest are:: Success

Traceback (most recent call last):
  File "/home/xstefen/android/los/./build/tools/releasetools/ota_from_target_files", line 2300, in <module>
    main(sys.argv[1:])
  File "/home/xstefen/android/los/./build/tools/releasetools/ota_from_target_files", line 2270, in main
    GenerateAbOtaPackage(
  File "/home/xstefen/android/los/./build/tools/releasetools/ota_from_target_files", line 1957, in GenerateAbOtaPackage
    CheckVintfIfTrebleEnabled(target_file, target_info)
  File "/home/xstefen/android/los/./build/tools/releasetools/ota_from_target_files", line 630, in CheckVintfIfTrebleEnabled
    raise RuntimeError("VINTF compatibility check failed")
RuntimeError: VINTF compatibility check failed
Not sure what that is, have you added anything to your build that might not support the version of LineageOS you are building?
 
Not sure what that is, have you added anything to your build that might not support the version of LineageOS you are building?
Nah, this is bone stock 18.1. Lineage doesnt do signed builds so theres occasionally little hiccups on these steps. I think it has to do with recent FCM changes tho, in the full context of it it happens with ota_from_target_packages tripping on FCM VNDK API 30 vs null. Ill post my fix

EDIT: Full stdout if interested
 
Last edited:

xHasKx

Member
May 27, 2018
18
1
@WhitbyGreg, thanks for your instruction to build a signed LineageOS build, and for your Reddit post with a lot of clarifications!
In addition, @xstefen, thanks for your hint about -p out/host/linux-x86/ flag!

I've succeeded in building a signed OS for the OnePlus 8T.

Now I'm searching for a way to include MindTheGapps and Magisk in my build process.
For the MindTheGapps, I found its sources at https://gitlab.com/MindTheGapps/vendor_gapps.
Its build process is a simple "make gapps_arm64" command, producing a file "MindTheGapps-11.0.0-arm64-20210904_084129.zip", but I'm searching for a way to built it into a LineageOS signed image.
I found that the MindTheGapps-11.0.0-arm64-20210904_084129.zip archive I've built contains such files:

Code:
system/product/app/GoogleCalendarSyncAdapter/GoogleCalendarSyncAdapter.apk
system/product/app/<...>/<...>.apk
...
system/product/etc/permissions/com.google.android.dialer.support.xml
...
system/product/etc/sysconfig/google-hiddenapi-package-whitelist.xml
...
system/product/framework/com.google.android.dialer.support.jar
system/product/lib/libjni_latinimegoogle.so
system/product/lib64/libjni_latinimegoogle.so
system/product/priv-app/AndroidAutoStubPrebuilt/AndroidAutoStubPrebuilt.apk
system/product/priv-app/<...>/<...>.apk
...
system/system_ext/etc/permissions/privapp-permissions-google-se.xml
system/system_ext/priv-app/GoogleFeedback/GoogleFeedback.apk
...

And my signed LineageOS build directory contains a path ./out/target/product/kebab/obj/PACKAGING/target_files_intermediates/lineage_kebab-target_files-eng.hask/ with sub-folders PRODUCT/app, PRODUCT/etc, SYSTEM_EXT/etc, and others very like to the files in the apoiler above.

Now my thoughts are just to copy the MindTheGapps-11.0.0-arm64-20210904_084129.zip archive content into corresponding folders PRODUCT/etc, SYSTEM_EXT/etc, and then run your instruction from the "Step 6: Sign the APKs" stage to get a signed LineageOS archive with MindTheGapps included.

Do you know maybe there is a better way to include MindTheGapps into a LineageOS build?
Will the way like this work for including Magisk into a signed LineageOS build?
 

xHasKx

Member
May 27, 2018
18
1
@WhitbyGreg, thanks for your instruction to build a signed LineageOS build, and for your Reddit post with a lot of clarifications!
In addition, @xstefen, thanks for your hint about -p out/host/linux-x86/ flag!

I've succeeded in building a signed OS for the OnePlus 8T.

Now I'm searching for a way to include MindTheGapps and Magisk in my build process.
For the MindTheGapps, I found its sources at https://gitlab.com/MindTheGapps/vendor_gapps.
Its build process is a simple "make gapps_arm64" command, producing a file "MindTheGapps-11.0.0-arm64-20210904_084129.zip", but I'm searching for a way to built it into a LineageOS signed image.
I found that the MindTheGapps-11.0.0-arm64-20210904_084129.zip archive I've built contains such files:

Code:
system/product/app/GoogleCalendarSyncAdapter/GoogleCalendarSyncAdapter.apk
system/product/app/<...>/<...>.apk
...
system/product/etc/permissions/com.google.android.dialer.support.xml
...
system/product/etc/sysconfig/google-hiddenapi-package-whitelist.xml
...
system/product/framework/com.google.android.dialer.support.jar
system/product/lib/libjni_latinimegoogle.so
system/product/lib64/libjni_latinimegoogle.so
system/product/priv-app/AndroidAutoStubPrebuilt/AndroidAutoStubPrebuilt.apk
system/product/priv-app/<...>/<...>.apk
...
system/system_ext/etc/permissions/privapp-permissions-google-se.xml
system/system_ext/priv-app/GoogleFeedback/GoogleFeedback.apk
...

And my signed LineageOS build directory contains a path ./out/target/product/kebab/obj/PACKAGING/target_files_intermediates/lineage_kebab-target_files-eng.hask/ with sub-folders PRODUCT/app, PRODUCT/etc, SYSTEM_EXT/etc, and others very like to the files in the apoiler above.

Now my thoughts are just to copy the MindTheGapps-11.0.0-arm64-20210904_084129.zip archive content into corresponding folders PRODUCT/etc, SYSTEM_EXT/etc, and then run your instruction from the "Step 6: Sign the APKs" stage to get a signed LineageOS archive with MindTheGapps included.

Do you know maybe there is a better way to include MindTheGapps into a LineageOS build?
Will the way like this work for including Magisk into a signed LineageOS build?

Unfortunately, this is not working - I've tested the setup I've described and built a new LineageOS signed archive.
Then I've extracted payload.bin from it, then split it into files with payload_dumper.py, then mounted the product.img file and didn't find a GoogleCalendarSyncAdapter.apk file or others from MindTheGapps in it.
Looks like it should be included in some packages building list.
@WhitbyGreg, can you please point to some actual docs describing how to include APK files into the build process?
 

WhitbyGreg

Senior Member
Unfortunately, this is not working - I've tested the setup I've described and built a new LineageOS signed archive.
Then I've extracted payload.bin from it, then split it into files with payload_dumper.py, then mounted the product.img file and didn't find a GoogleCalendarSyncAdapter.apk file or others from MindTheGapps in it.
Looks like it should be included in some packages building list.
@WhitbyGreg, can you please point to some actual docs describing how to include APK files into the build process?
That's not how you want to include gap's, you want to build it as part of your lineage build, see this section of the mindthegapps readme: https://gitlab.com/MindTheGapps/vendor_gapps#build-inline-with-android
 
  • Like
Reactions: xHasKx

xHasKx

Member
May 27, 2018
18
1

xHasKx

Member
May 27, 2018
18
1
Thanks, I've managed to make a signed build with integrated MindTheGApps.

Now trying to include Magisk into my build too, for now without success. Tried these old and modern solutions.
Finally, I built the signed LineageOS with MindTheGApps and Magisk included. For Magisk I used the modern solution with vendor/extendrom. The tricky part was that it should be enabled by several X:=true variables.

Now I'm trying to build a user build instead of userdebug with these commands:
Bash:
cd ~/android/lineageos
source build/envsetup.sh
breakfast kebab # it shows "TARGET_BUILD_VARIANT=userdebug"
croot
lunch lineage_kebab-user # it shows "TARGET_BUILD_VARIANT=user"
mka target-files-package otatools

But the build process is stopping after several minutes with unspecified ninja failure.

@WhitbyGreg, can you please look at the build commands and tell me maybe I used it the wrong way?
 

WhitbyGreg

Senior Member
Finally, I built the signed LineageOS with MindTheGApps and Magisk included. For Magisk I used the modern solution with vendor/extendrom. The tricky part was that it should be enabled by several X:=true variables.

Now I'm trying to build a user build instead of userdebug with these commands:
Bash:
cd ~/android/lineageos
source build/envsetup.sh
breakfast kebab # it shows "TARGET_BUILD_VARIANT=userdebug"
croot
lunch lineage_kebab-user # it shows "TARGET_BUILD_VARIANT=user"
mka target-files-package otatools

But the build process is stopping after several minutes with unspecified ninja failure.

@WhitbyGreg, can you please look at the build commands and tell me maybe I used it the wrong way?

This is what I use, where $LOS_DEVICE=kebab:

Bash:
        # Setup the build environment
        source build/envsetup.sh
        croot

        # Setup our env variables
        RELEASE_TYPE=RELEASE
        export RELEASE_TYPE

        TARGET_BUILD_VARIANT=user
        export TARGET_BUILD_VARIANT

        TARGET_PRODUCT=lineage_$LOS_DEVICE
        export TARGET_PRODUCT

        # Clean the build environment.
        make installclean

        # Start the build
        echo "Running breakfast for $LOS_DEVICE..."
        breakfast $LOS_DEVICE user

        # Package the files
        echo "Making target packages for $DEVICE..."
        mka target-files-package otatools

Make sure you've done a full clean between building userdebug and user builds, to make sure nothing is left over.
 
Unfortunately, this is not working - I've tested the setup I've described and built a new LineageOS signed archive.
Then I've extracted payload.bin from it, then split it into files with payload_dumper.py, then mounted the product.img file and didn't find a GoogleCalendarSyncAdapter.apk file or others from MindTheGapps in it.
Looks like it should be included in some packages building list.
@WhitbyGreg, can you please point to some actual docs describing how to include APK files into the build process?
MindTheGapps git repo has instructions for inline building, and historically I've used a modified version of Akhil's signing script


cloning to vendor/gapps and adding the following to device/oneplus/kebab/lineage_kebab.mk:

Code:
# MindTheGapps
$(call inherit-product-if-exists, vendor/gapps/arm64/arm64-vendor.mk)
WITH_GMS := true
gist: https://gist.github.com/xstefen/900cd8b64f3bfaf494243b2d43e3c0b6

Original script: https://raw.githubusercontent.com/akhilnarang/scripts/master/aosip/sign.sh
my hacked and slashed edit: https://gist.github.com/xstefen/285276eb14094d452b0318423716decb
 
Last edited:
  • Like
Reactions: xHasKx

xHasKx

Member
May 27, 2018
18
1
MindTheGapps git repo has instructions for inline building, and historically I've used a modified version of Akhil's signing script


cloning to vendor/gapps and adding the following to device/oneplus/kebab/lineage_kebab.mk:

Code:
# MindTheGapps
$(call inherit-product-if-exists, vendor/gapps/arm64/arm64-vendor.mk)
WITH_GMS := true
gist: https://gist.github.com/xstefen/900cd8b64f3bfaf494243b2d43e3c0b6

Original script: https://raw.githubusercontent.com/akhilnarang/scripts/master/aosip/sign.sh
my hacked and slashed edit: https://gist.github.com/xstefen/285276eb14094d452b0318423716decb
Yes, I configured my build like this, and succeed with building MindTheGApps into my signed build. Thanks, xstefen
 

xHasKx

Member
May 27, 2018
18
1
This is what I use, where $LOS_DEVICE=kebab:

Bash:
        # Setup the build environment
        source build/envsetup.sh
        croot

        # Setup our env variables
        RELEASE_TYPE=RELEASE
        export RELEASE_TYPE

        TARGET_BUILD_VARIANT=user
        export TARGET_BUILD_VARIANT

        TARGET_PRODUCT=lineage_$LOS_DEVICE
        export TARGET_PRODUCT

        # Clean the build environment.
        make installclean

        # Start the build
        echo "Running breakfast for $LOS_DEVICE..."
        breakfast $LOS_DEVICE user

        # Package the files
        echo "Making target packages for $DEVICE..."
        mka target-files-package otatools

Make sure you've done a full clean between building userdebug and user builds, to make sure nothing is left over.
WhitbyGreg, Thanks, I tested that build commands and faced two issues:

1. command make installclean displays environment variables and one of them is
LINEAGE_VERSION=18.1.0- - looks like some environment variable is missing. Actually, this is not an important issue because the next command breakfast $LOS_DEVICE user shows me
LINEAGE_VERSION=18.1.0-kebab, which looks OK. But anyway this can be a sign of some internal issue in my configuration - when I finally flash recovery built with my setup it shows me its version line as Version 18.1 () above the recovery menu items. Looks some string is missing.

2. The main issue is that after the build I do a signing and OTA (zip archive) building steps and the last one is failing with that error:

Code:
$ time python2 ./build/tools/releasetools/ota_from_target_files -p out/host/linux-x86/ -k ~/.android-certs/releasekey --block signed-target_files.zip ./out/target/product/kebab/lineage-18.1-$(
date +%Y%m%d-%H%M%S)-kebab-user-signed.zip              
2021-09-06 18:42:41 - common.py - WARNING : Failed to read SYSTEM/etc/build.prop
2021-09-06 18:42:41 - common.py - WARNING : Failed to read VENDOR/etc/build.prop
2021-09-06 18:42:41 - common.py - WARNING : Failed to read PRODUCT/etc/build.prop
2021-09-06 18:42:41 - common.py - WARNING : Failed to read SYSTEM_EXT/etc/build.prop
Traceback (most recent call last):
  File "./build/tools/releasetools/ota_from_target_files", line 2300, in <module>
    main(sys.argv[1:])
  File "./build/tools/releasetools/ota_from_target_files", line 2273, in main
    source_file=OPTIONS.incremental_source)
  File "./build/tools/releasetools/ota_from_target_files", line 1957, in GenerateAbOtaPackage
    CheckVintfIfTrebleEnabled(target_file, target_info)
  File "./build/tools/releasetools/ota_from_target_files", line 630, in CheckVintfIfTrebleEnabled
    raise RuntimeError("VINTF compatibility check failed")
RuntimeError: VINTF compatibility check failed

real    3m37,945s
user    24m7,671s
sys     0m10,228s

I tried to add a --skip_compatibility_check parameter for ota_from_target_files script but the resulting zip archive is not passing signature check when I install it in the recovery. The recovery I extracted from the same zip archive and flashed with fastboot into recovery_a and recovery_b slots.

This can be related to the Magisk setup I use, but the same VINTF error I have when disabling all addons (MindTheGApps, Magisk) and building the OTA from scratch. The only difference is that the recovery is successfully accepting my ZIP archive built with --skip_compatibility_check parameter.
So, I'm still investigating what can be wrong with my setup.

Can you please share the entire environment you set before running the user build? I mean can you please show the output of the export shell command leaving only related lines?

Thanks in advance!
 

WhitbyGreg

Senior Member
WhitbyGreg, Thanks, I tested that build commands and faced two issues:

1. command make installclean displays environment variables and one of them is
LINEAGE_VERSION=18.1.0- - looks like some environment variable is missing. Actually, this is not an important issue because the next command breakfast $LOS_DEVICE user shows me
LINEAGE_VERSION=18.1.0-kebab, which looks OK. But anyway this can be a sign of some internal issue in my configuration - when I finally flash recovery built with my setup it shows me its version line as Version 18.1 () above the recovery menu items. Looks some string is missing.

2. The main issue is that after the build I do a signing and OTA (zip archive) building steps and the last one is failing with that error:

Code:
$ time python2 ./build/tools/releasetools/ota_from_target_files -p out/host/linux-x86/ -k ~/.android-certs/releasekey --block signed-target_files.zip ./out/target/product/kebab/lineage-18.1-$(
date +%Y%m%d-%H%M%S)-kebab-user-signed.zip             
2021-09-06 18:42:41 - common.py - WARNING : Failed to read SYSTEM/etc/build.prop
2021-09-06 18:42:41 - common.py - WARNING : Failed to read VENDOR/etc/build.prop
2021-09-06 18:42:41 - common.py - WARNING : Failed to read PRODUCT/etc/build.prop
2021-09-06 18:42:41 - common.py - WARNING : Failed to read SYSTEM_EXT/etc/build.prop
Traceback (most recent call last):
  File "./build/tools/releasetools/ota_from_target_files", line 2300, in <module>
    main(sys.argv[1:])
  File "./build/tools/releasetools/ota_from_target_files", line 2273, in main
    source_file=OPTIONS.incremental_source)
  File "./build/tools/releasetools/ota_from_target_files", line 1957, in GenerateAbOtaPackage
    CheckVintfIfTrebleEnabled(target_file, target_info)
  File "./build/tools/releasetools/ota_from_target_files", line 630, in CheckVintfIfTrebleEnabled
    raise RuntimeError("VINTF compatibility check failed")
RuntimeError: VINTF compatibility check failed

real    3m37,945s
user    24m7,671s
sys     0m10,228s

I tried to add a --skip_compatibility_check parameter for ota_from_target_files script but the resulting zip archive is not passing signature check when I install it in the recovery. The recovery I extracted from the same zip archive and flashed with fastboot into recovery_a and recovery_b slots.

This can be related to the Magisk setup I use, but the same VINTF error I have when disabling all addons (MindTheGApps, Magisk) and building the OTA from scratch. The only difference is that the recovery is successfully accepting my ZIP archive built with --skip_compatibility_check parameter.
So, I'm still investigating what can be wrong with my setup.

Can you please share the entire environment you set before running the user build? I mean can you please show the output of the export shell command leaving only related lines?

Thanks in advance!
There are probably two additional changes you want to make:
  1. in device/oneplus/sm8250-common/common.mk add "PRODUCT_DEFAULT_PROPERTY_OVERRIDES += ro.oem_unlock_supported=1" to enable the unlock/lock option in dev options.
  2. in kernel/oneplus/sm8250/arch/arm64/configs/vendor/kona-perf_defconfig change "CONFIG_DEBUG_FS=y" to "CONFIG_DEBUG_FS=n".
The second one is probably the source of your VINTF compatibility check failure.
 
  • Like
Reactions: xHasKx

xHasKx

Member
May 27, 2018
18
1
There are probably two additional changes you want to make:
  1. in device/oneplus/sm8250-common/common.mk add "PRODUCT_DEFAULT_PROPERTY_OVERRIDES += ro.oem_unlock_supported=1" to enable the unlock/lock option in dev options.
  2. in kernel/oneplus/sm8250/arch/arm64/configs/vendor/kona-perf_defconfig change "CONFIG_DEBUG_FS=y" to "CONFIG_DEBUG_FS=n".
The second one is probably the source of your VINTF compatibility check failure.
Thanks, WhitbyGreg, adding the CONFIG_DEBUG_FS=n option has fixed the error on the last stage (OTA zip archive building).

I have another critical issue.
When bootloader is locked (fastboot oem lock) - the finderprint sensor is not working.
Looks like a software bug: when I'm trying to register a fingerprint - the white circle is appearing, reacting for my finger press with a white backlight but it's not recording by the system - nothing is changed on the screen except backlight appearing. Even after I cancel finger adding process and exit to the homescreen - there is a fingerprint area left highlighted on the screen, until I pull notifications panel.

When I'm unlocking bootloader - the fingerprint sensor is working perfectly, and again stopping working after bootloader locking.
This is reproducing on both signed user and signed userdebug builds.

I found several reports about the same bug in the internet: this for 8T and this for 7T and 8.
On the last link there is a possible solution - to restore "persist" partition from the original OxygenOS firmware.
I restored my phone with ELD unbrick tool, unlocked bootloader, booted into TWRP and downloaded two partitions - "persist" and "persist_bkp". (Maybe there is a way to extract it from the EDL unbrick tool, to be sure it's not changed by unlock process, don't tested yet)

WhitbyGreg, what do you think about this solution with "persist" partition restoring? Is there a way to include it into a OTA zip archive?
 

xHasKx

Member
May 27, 2018
18
1
Thanks, WhitbyGreg, adding the CONFIG_DEBUG_FS=n option has fixed the error on the last stage (OTA zip archive building).

I have another critical issue.
When bootloader is locked (fastboot oem lock) - the finderprint sensor is not working.
Looks like a software bug: when I'm trying to register a fingerprint - the white circle is appearing, reacting for my finger press with a white backlight but it's not recording by the system - nothing is changed on the screen except backlight appearing. Even after I cancel finger adding process and exit to the homescreen - there is a fingerprint area left highlighted on the screen, until I pull notifications panel.

When I'm unlocking bootloader - the fingerprint sensor is working perfectly, and again stopping working after bootloader locking.
This is reproducing on both signed user and signed userdebug builds.

I found several reports about the same bug in the internet: this for 8T and this for 7T and 8.
On the last link there is a possible solution - to restore "persist" partition from the original OxygenOS firmware.
I restored my phone with ELD unbrick tool, unlocked bootloader, booted into TWRP and downloaded two partitions - "persist" and "persist_bkp". (Maybe there is a way to extract it from the EDL unbrick tool, to be sure it's not changed by unlock process, don't tested yet)

WhitbyGreg, what do you think about this solution with "persist" partition restoring? Is there a way to include it into a OTA zip archive?
In addition, I've tested to flash and lock bootloader with the official LineageOS signed build for OnePlus 8T according to this instruction, and got the same fingerprint bug. That's very bad because something is wrong in the LineageOS itself, but not in my build...
 

WhitbyGreg

Senior Member
In addition, I've tested to flash and lock bootloader with the official LineageOS signed build for OnePlus 8T according to this instruction, and got the same fingerprint bug. That's very bad because something is wrong in the LineageOS itself, but not in my build...
Thanks, WhitbyGreg, adding the CONFIG_DEBUG_FS=n option has fixed the error on the last stage (OTA zip archive building).

I have another critical issue.
When bootloader is locked (fastboot oem lock) - the finderprint sensor is not working.
Looks like a software bug: when I'm trying to register a fingerprint - the white circle is appearing, reacting for my finger press with a white backlight but it's not recording by the system - nothing is changed on the screen except backlight appearing. Even after I cancel finger adding process and exit to the homescreen - there is a fingerprint area left highlighted on the screen, until I pull notifications panel.

When I'm unlocking bootloader - the fingerprint sensor is working perfectly, and again stopping working after bootloader locking.
This is reproducing on both signed user and signed userdebug builds.

I found several reports about the same bug in the internet: this for 8T and this for 7T and 8.
On the last link there is a possible solution - to restore "persist" partition from the original OxygenOS firmware.
I restored my phone with ELD unbrick tool, unlocked bootloader, booted into TWRP and downloaded two partitions - "persist" and "persist_bkp". (Maybe there is a way to extract it from the EDL unbrick tool, to be sure it's not changed by unlock process, don't tested yet)

WhitbyGreg, what do you think about this solution with "persist" partition restoring? Is there a way to include it into a OTA zip archive?
I knew the fingerprint sensor wasn't working, but I had assumed that was the user build, interesting that it is in fact the bootloader lock that is doing it.

It's likely a problem with a SELinux Policy, probably the FP sensor is flipped in to read only mode when relocking but the existing policy expects read/write or something like that. You should be able to track it down in logcat.

it doesn't sound like something that persist would resolve as I would expect it to fail when unlocked as well if that was the problem. The persist partition isn't included in the OOS OTA so it's not something that you could flash easily.

I don't use the finger print sensor as it is not particularly secure (in the sense that you don't need conscious consent to use it), and has legal ambiguity to it.

Edit: Can confirm the finger print sensor on the OnePlus 9 works fine with bootloader locked and a user build.
 
Last edited:

xHasKx

Member
May 27, 2018
18
1
I collected some logcat on locked and unlocked bootloaders and found such a difference at the logcat output beginning:

On locked bootloader:
Code:
09-07 08:02:17.768  1361  1787 V FingerprintService: mDaemon was null, reconnect to fingerprint
09-07 08:02:17.770  1361  1787 V FingerprintService: Fingerprint HAL id: -5476376659343548400
09-07 08:02:17.770  1361  1361 D FingerprintService: initConfiguredStrengthInternal(15)
09-07 08:02:17.808  1361  1787 V FingerprintService: Enumerating user(0)

On unlocked bootloader:
Code:
09-07 08:02:18.264  1391  1391 D FingerprintService: initConfiguredStrengthInternal(15)
09-07 08:02:18.301  1391  1776 V FingerprintService: Enumerating user(0)

At the time when I'm trying to register fingerprint there is a lines on unlocked bootloader:
Code:
09-07 08:03:12.773  1391  1391 V FingerprintService: Acquired: 0 0
09-07 08:03:12.804  1391  1391 V FingerprintService: Cancelling enrollment
09-07 08:03:12.977  1391  1391 V FingerprintService: handleError(client=com.android.settings, error = 5)
09-07 08:03:12.978  1391  1391 V FingerprintService: Done with client: com.android.settings
09-07 08:03:13.038  1391  1391 D FingerprintService: setActiveUser(0)
09-07 08:03:13.039  1391  1391 V FingerprintService: starting client EnrollClientImpl(com.android.settings) targetUserId: 0 currentUserId: 0 cookie: 0/0

And no lines with "FingerprintService:" at all on the locked bootloader near the time I tried to register a fingerprint. Looks like the last line with it was only near the end of the system boot time.

WhitbyGreg, does it mean something to you?
 

Top Liked Posts

  • There are no posts matching your filters.
  • 10
    What is this tutorial?
    This tutorial will:
    • Creating an unofficial build of LineageOS 18.1 suitable for using to re-lock the bootloader on a OnePlus 8t
    • Take you through the process of re-locking your bootloader after installing the above

    This tutorial will NOT:
    • Remove *all* warning messages during boot (the yellow "Custom OS" message will be present though the orange "Unlocked bootloader" message will not)
    • Allow you to use official builds of LineageOS 18.1 on your device with a re-locked bootloader (more details near the end of the tutorial)
    This tutorial will assume you are working on an Ubuntu 18.04 installation, if you are using Windows or another Linux distro, the commands may be different.

    Supported devices:
    The following devices have been tested and confirmed to work:
    • OnePlus 7 Pro (guacamole)
    • OnePlus 8t (kebab)
    • Pixel 4 (flame)
    Other OnePlus devices that support AVBv2 (OnePlus 6t and newer as well as most Pixel devices) and LineageOS 18.1 (see current support list over on the LineageOS download page) should work as well.

    For simplicities sake, all further references will only be to the 8t (kebab).

    Pre-requisites:
    • a mid level knowledge of terminal commands and features
    • a supported phone
    • a PC with enough CPU/RAM to build LineageOS 18.1 (recommended 8 cores, 24g of RAM)
    • a working USB cable
    • fastboot/adb installed and functional
    • LineageOS 18.1 source code downloaded
    • at least one successful build of LineageOS
    • at least one successful signing of your build with your own keys

    Misc. notes:
    • the basics of building/signing of LineageOS is outside the scope of this tutorial, refer to the LineageOS Wiki for details on how to complete these tasks
    • you'll be modifying some code in LineageOS, so if you are not comfortable using basic editing utilities as well as patch, do not proceed any further
    • the path to your LineageOS source code is going to be assumed to be ~/android/lineageos, if it is somewhere else, substitute the correct path in the tutorial
    • the path to your private certificate files is going to be assumed to be ~/android-certs, if it is somewhere else, substitute the correct path in the tutorial


    *** WARNING ****
    This process may brick your device. Do not proceed unless you are comfortable taking this risk.


    *** WARNING ****
    This process will delete all data on your phone! Do not proceed unless you have backed up your data!


    *** WARNING ****
    Make sure you have read through this entire process at least once before attempting, if you are uncomfortable with any steps include in this guide, do not continue.



    And now on with the show!

    Step 1: Basic setup

    You need a few places to store things, so create some working directories:
    Code:
    mkdir ~/android/kebab
    mkdir ~/android/kebab/patches
    mkdir ~/android/kebab/pkmd
    You also need to add "~/android/lineageos/out/host/linux-x86/bin" to your shell's profile path. Make sure to close and restart your session afterwards otherwise the signing will fail later on with a "file not found" error message (this may no longer be required).

    Step 2: Update kebab's BoardConfig.mk

    You will need to add a few parameters to the end of ~/android/lineageos/device/oneplus/kebab/BoardConfig.mk, they are:

    Code:
    BOARD_AVB_ALGORITHM := SHA256_RSA2048
    BOARD_AVB_KEY_PATH := /home/<userid>/.android-certs/releasekey.key
    Note you cannot use "~" in the path names above to signify your home directory, so give the full absolute path to make sure the files are found.

    Step 3: Update sm8250-common's BoardConfigCommon.mk

    LineageOS by default disables Android Verified Boot's partition verification, but you can enable it now as all the required parts will be in place.

    To enable partition verification do the following:

    Code:
    cd ~/android/lineageos/device/oneplus/sm8250-common
    sed -i 's/^BOARD_AVB_MAKE_VBMETA_IMAGE_ARGS += --flags 2/#BOARD_AVB_MAKE_VBMETA_IMAGE_ARGS += --flags 2/' BoardConfigCommon.mk
    sed -i 's/^BOARD_AVB_MAKE_VBMETA_IMAGE_ARGS += --set_hashtree_disabled_flag/#BOARD_AVB_MAKE_VBMETA_IMAGE_ARGS += --set_hashtree_disabled_flag/' BoardConfigCommon.mk
    sed -i 's/^BOARD_AVB_VBMETA_SYSTEM_KEY_PATH := external\/avb\/test\/data\/testkey_rsa2048.pem/BOARD_AVB_KEY_PATH := \/home\/<userid>\/.android-certs\/releasekey.key/' BoardConfigCommon.mk

    Don't forget to replace your <userid> in the third sed command above with your current logged in user id.

    Step 4: Patch the AOSP and Device Makefile

    You also need to patch the Makefile included with AOSP as it will otherwise fail during the build.

    The required patch can be found here:

    Download it and store in ~/android/kebab/patches.

    Now apply it with the following command:

    Code:
    cd ~/android/lineageos/build/core
    patch Makefile ~/android/kebab/patches/core-Makefile-fix-18.1.patch

    If you would like to know more about this patch, see the additional info at the bottom of this post.

    There is also a small addition to the device's common.mk required to enable the OEM unlock option in developers options, do this via the following commands:

    Code:
    cd ~/android/lineageos/device/oneplus/sm8250-common
    sed -i 's/^# OMX/# OEM Unlock reporting\nPRODUCT_DEFAULT_PROPERTY_OVERRIDES += \\\n    ro.oem_unlock_supported=1\n\n# OMX/' common.mk

    Step 5: Build LineageOS

    You are now ready to build:

    Code:
    cd ~/android/lineageos
    breakfast kebab
    source build/envsetup.sh
    croot
    mka target-files-package otatools

    Step 6: Sign the APKs

    You are now ready to sign the apks with sign_target_files_apks:

    Code:
    ./build/tools/releasetools/sign_target_files_apks -o -d ~/.android-certs $OUT/obj/PACKAGING/target_files_intermediates/*-target_files-*.zip signed-target_files.zip

    Step 7: Build the OTA

    Now it is time to complete the OTA package:

    Code:
    ./build/tools/releasetools/ota_from_target_files -k ~/.android-certs/releasekey --block signed-target_files.zip lineage-18.1-[date]-UNOFFICIAL-kebab-signed.zip

    Note, replace [date] with today's date in YYYYMMDD format.

    Step 8: Create pkmd.bin for your phone

    Before you can lock your phone, you have to tell it what your public key is so it knows it can trust your build.

    To do this you need to create a pkmd.bin file:

    Code:
    ~/android/lineageos/external/avb/avbtool extract_public_key --key ~/.android-certs/releasekey.key --output ~/android/kebab/pkmd/pkmd.bin

    Step 9: Flashing your LineageOS build

    It's time to flash your build to your phone. The following steps assume you have already unlocked your phone and have flashed an official version of LineageOS to it. You don't need to have flashed LineageOS yet, you could use TWRP through "fastboot boot" if you prefer. Or, if you want to use the recovery that was just created, it is located in ~/android/lineageos/out/target/product/kebab and is called recovery.img.

    • Reboot your phone in to recovery mode
    • In LineageOS Recovery return to the main menu and select "Apply update"
    • From your PC, run:
    Code:
    adb sideload ~/android/lineageos/lineage-18.1-[date]-UNOFFICIAL-kebab-signed.zip

    When the sideload is complete, reboot in to LineageOS. Make sure everything looks good with your build.

    You may also need to format your data partition at this time depending on what you had installed on your phone previously.

    Step 10: Flashing your signing key

    Now it's time to add your signing key to the Android Verified Boot process. To do so, do the following:

    • Reboot your phone in to fastboot mode
    • From your PC, run:
    Code:
    fastboot flash avb_custom_key ~/android/kebab/pkmd/pkmd.bin
    fastboot reboot bootloader
    fastboot oem lock
    • On your phone, confirm you want to re-lock and it will reboot

    Your phone will then factory reset and then reboot in to LineageOS.

    Which of course means you have to go through the first time setup wizard, so do so now.

    Step 11: Disable OEM unlock

    Congratulations! Your boot loader is now locked, but you can still unlock it again using fastboot, so it's time to disable that as well.

    • Unlock you phone and go to Settings->About phone
    • Scroll to the bottom and find "Build number"
    • Tap on it you enable the developer options
    • Go to Settings->System->Advanced->Developer options
    • Disable the "OEM unlocking" slider
    • Reboot

    Step 12: Profit!


    Other things

    • The above will build a standard USERDEBUG version of LineageOS, however this will still allow LineageOS Recovery to sideload non-signed files as well as give you root shell access through ADB. Step 3/4 above protects your system/vendor/boot/dtbo/etc. partitions, but none of the others. Likewise USERDEBUG builds will allow for rolling back to a previous builds/versions of LineageOS. To increase security and disallow both of these scenarios you may want to build a USER version of LineageOS to install. However this brings in other issues, such as flashing newer firmware from OnePlus so make sure you understand the implications of both choices. For more details on build types, see https://source.android.com/setup/develop/new-device#build-variants.
    • In the above example the releasekey from your LineageOS install has been used to sign AVB, but AVB supports other key strengths up to SHA512_RSA8192. You could create a key just for signing AVB that used different options than the default keys generated to sign LineageOS.
    • If you want to remove you signing key from your phone, you can do it by running "fastboot erase avb_custom_key".
    • The changes you made to the AOSP Makefile may conflict with future updates that you pull from LineageOS through repo sync, if you have to reset the file to get repo sync to complete successfully, you'll have to reapply the changes afterwards.

    So why can't I do this with official LineageOS builds?

    NEW: You can! See this thread for more details.

    For Android Verified Boot (AVB) to work, it must have the hash values for each of the system/vendor/boot/dtbo/etc. partitions stored in vbmeta. Official LineageOS builds for kebab do include the vendor.img in them along with everything else that is needed, however that is not true for all phones.

    There are two "issues" that stop someone from using the official kebab builds:
    • LineageOS does not provide a pkmd.bin file to flash to your phone to include the public key in your AVB process (NEW: this thread shows you how to extract the key).
    • AVB is enabled in the official LineageOS builds but does not validate the hash trees during boot which limits the protection offered.
    Ok, what messages do I see during the boot process then?

    During a boot you will of course see the standard OnePlus power up screen, followed by the yellow "custom os" message and then the standard LineageOS boot animation.

    For more details on AVB boot messages, see https://source.android.com/security/verifiedboot/boot-flow

    So what does that patch to the Makefile do?

    AOSP's default Makefile makes an assumption that when AVB is enabled, that all the img files will be available well before vbmeta.img is created. This is simply NOT true and AOSP seems to know this as well from the following comment in the Makefile:

    Code:
    # Not using INSTALLED_VBMETA_SYSTEMIMAGE_TARGET as it won't be set yet.
    ifdef BOARD_AVB_VBMETA_SYSTEM
    $(eval $(call check-and-set-avb-args,vbmeta_system))
    endif
    
    ifdef BOARD_AVB_VBMETA_VENDOR
    $(eval $(call check-and-set-avb-args,vbmeta_vendor))
    endif

    These two calls eventual evaluate to returning the path to the partitions based upon the INSTALLED_*IMAGE_TARGET variable, which isn't created until later in the build process.

    Because of this, the command to build vbmeta.img gets corrupted due to the missing make variable being empty and an invalid command line is passed to avbtool near the end of the build.

    The corruption happens due to the fact that the following line from the original Makefile:

    Code:
    --include_descriptors_from_image $(call images-for-partitions,$(1))))))

    Gets added to the avbtool call even if "$(call images-for-partitions,$(1))" turns out to be an empty string. Avbtool then throws an error message as it is expecting a parameter after the "--include_descriptors_from_image" flag that is added for the "empty" partition path.

    The fix is to call "$(call images-for-partitions,$(1))" earlier, set it to a variable and check to make sure it isn't an empty string before letting the "--include_descriptors_from_image" be added to the avbtool command line to be used later.

    This technically generates an incomplete vbmeta.img file during the build process, but since the signing process recreates it from scratch anyway; no harm, no foul.

    Thank You's
    2
    Also, if you disable OEM unlock, that's pretty secure, but if something goes wrong with your phone, i.e., you can't boot in recovery and system, the only way to restore the phone is flashing with MSM tool. I had it once on my Oneplus 5. I had a paranoid TWRP, with Cancel button removed. So, all of a sudden, TWRP refused to recognize my password. So, you can't press cancel to access system to do a factory reset. All right, I thought, I am just going to boot into system and enable OEM unlock. Well, no can do: the phone refused to boot into system and instead booted into TWRP with the password prompt. Bootloader locked, remote flashing is not allowed. The only option - MSM tool. I set up Windows on my virtual machine, but forget it, MSM tool can't connect to the phone no matter what I tried. So, I had to find on old laptop with Windows and then it worked... .
    Correct, once OEM unlocking is disabled, just like with OxygenOS, if something goes horribly wrong, you're using MSM to get back to stock and re-installing.

    I don't use TWRP, and build both LineageOS and Lineage Recovery in user mode. Building recvoery in user mode (and only with my certificates) also means you can't "rollback" to an older version of Lineage as it won't flash older zips.

    As I said in my reddit post, most people don't really want to do this. Too much risk, too little reward. But if you're wiling to take the risk, you do get as close to a "stock" security footprint as you can without using the OEM's OS.
    2
    What is this tutorial?
    This tutorial will:
    • Creating an unofficial build of LineageOS 18.1 suitable for using to re-lock the bootloader on a OnePlus 8t
    • Take you through the process of re-locking your bootloader after installing the above

    This tutorial will NOT:
    • Remove *all* warning messages during boot (the yellow "Custom OS" message will be present though the orange "Unlocked bootloader" message will not)
    • Allow you to use official builds of LineageOS 18.1 on your device with a re-locked bootloader (more details near the end of the tutorial)
    This tutorial will assume you are working on an Ubuntu 18.04 installation, if you are using Windows or another Linux distro, the commands may be different.

    Supported devices:
    The following devices have been tested and confirmed to work:
    • OnePlus 7 Pro (guacamole)
    • OnePlus 8t (kebab)
    • Pixel 4 (flame)
    Other OnePlus devices that support AVBv2 (OnePlus 6t and newer as well as most Pixel devices) and LineageOS 18.1 (see current support list over on the LineageOS download page) should work as well.

    For simplicities sake, all further references will only be to the 8t (kebab).

    Pre-requisites:
    • a mid level knowledge of terminal commands and features
    • a supported phone
    • a PC with enough CPU/RAM to build LineageOS 18.1 (recommended 8 cores, 24g of RAM)
    • a working USB cable
    • fastboot/adb installed and functional
    • LineageOS 18.1 source code downloaded
    • at least one successful build of LineageOS
    • at least one successful signing of your build with your own keys

    Misc. notes:
    • the basics of building/signing of LineageOS is outside the scope of this tutorial, refer to the LineageOS Wiki for details on how to complete these tasks
    • you'll be modifying some code in LineageOS, so if you are not comfortable using basic editing utilities as well as patch, do not proceed any further
    • the path to your LineageOS source code is going to be assumed to be ~/android/lineageos, if it is somewhere else, substitute the correct path in the tutorial
    • the path to your private certificate files is going to be assumed to be ~/android-certs, if it is somewhere else, substitute the correct path in the tutorial


    *** WARNING ****
    This process may brick your device. Do not proceed unless you are comfortable taking this risk.


    *** WARNING ****
    This process will delete all data on your phone! Do not proceed unless you have backed up your data!


    *** WARNING ****
    Make sure you have read through this entire process at least once before attempting, if you are uncomfortable with any steps include in this guide, do not continue.


    And now on with the show!

    Step 1: Basic setup

    You need a few places to store things, so create some working directories:
    Code:
    mkdir ~/android/kebab
    mkdir ~/android/kebab/patches
    mkdir ~/android/kebab/pkmd
    You also need to add "~/android/lineageos/out/host/linux-x86/bin" to your shell's profile path. Make sure to close and restart your session afterwards otherwise the signing will fail later on with a "file not found" error message (this may no longer be required).

    Step 2: Update kebab's BoardConfig.mk

    You will need to add a few parameters to the end of ~/android/lineageos/device/oneplus/kebab/BoardConfig.mk, they are:

    Code:
    BOARD_AVB_ALGORITHM := SHA256_RSA2048
    BOARD_AVB_KEY_PATH := /home/<userid>/.android-certs/releasekey.key
    Note you cannot use "~" in the path names above to signify your home directory, so give the full absolute path to make sure the files are found.

    Step 3: Update sm8250-common's BoardConfigCommon.mk

    LineageOS by default disables Android Verified Boot's partition verification, but you can enable it now as all the required parts will be in place.

    To enable partition verification do the following:

    Code:
    cd ~/android/lineageos/device/oneplus/sm8250-common
    sed -i 's/^BOARD_AVB_MAKE_VBMETA_IMAGE_ARGS += --flags 2/#BOARD_AVB_MAKE_VBMETA_IMAGE_ARGS += --flags 2/' BoardConfigCommon.mk
    sed -i 's/^BOARD_AVB_MAKE_VBMETA_IMAGE_ARGS += --set_hashtree_disabled_flag/#BOARD_AVB_MAKE_VBMETA_IMAGE_ARGS += --set_hashtree_disabled_flag/' BoardConfigCommon.mk
    sed -i 's/^BOARD_AVB_VBMETA_SYSTEM_KEY_PATH := external\/avb\/test\/data\/testkey_rsa2048.pem/BOARD_AVB_KEY_PATH := \/home\/<userid>\/.android-certs\/releasekey.key/' BoardConfigCommon.mk

    Don't forget to replace your <userid> in the third sed command above with your current logged in user id.

    Step 4: Patch the AOSP and Device Makefile

    You also need to patch the Makefile included with AOSP as it will otherwise fail during the build.

    The required patch can be found here:

    Download it and store in ~/android/kebab/patches.

    Now apply it with the following command:

    Code:
    cd ~/android/lineageos/build/core
    patch Makefile ~/android/kebab/patches/core-Makefile-fix-18.1.patch

    If you would like to know more about this patch, see the additional info at the bottom of this post.

    There is also a small addition to the device's common.mk required to enable the OEM unlock option in developers options, do this via the following commands:

    Code:
    cd ~/android/lineageos/device/oneplus/sm8250-common
    sed -i 's/^# OMX/# OEM Unlock reporting\nPRODUCT_DEFAULT_PROPERTY_OVERRIDES += \\\n    ro.oem_unlock_supported=1\n\n# OMX/' common.mk

    Step 5: Build LineageOS

    You are now ready to build:

    Code:
    cd ~/android/lineageos
    breakfast kebab
    source build/envsetup.sh
    croot
    mka target-files-package otatools

    Step 6: Sign the APKs

    You are now ready to sign the apks with sign_target_files_apks:

    Code:
    ./build/tools/releasetools/sign_target_files_apks -o -d ~/.android-certs $OUT/obj/PACKAGING/target_files_intermediates/*-target_files-*.zip signed-target_files.zip

    Step 7: Build the OTA

    Now it is time to complete the OTA package:

    Code:
    ./build/tools/releasetools/ota_from_target_files -k ~/.android-certs/releasekey --block signed-target_files.zip lineage-18.1-[date]-UNOFFICIAL-kebab-signed.zip

    Note, replace [date] with today's date in YYYYMMDD format.

    Step 8: Create pkmd.bin for your phone

    Before you can lock your phone, you have to tell it what your public key is so it knows it can trust your build.

    To do this you need to create a pkmd.bin file:

    Code:
    ~/android/lineageos/external/avb/avbtool extract_public_key --key ~/.android-certs/releasekey.key --output ~/android/kebab/pkmd/pkmd.bin

    Step 9: Flashing your LineageOS build

    It's time to flash your build to your phone. The following steps assume you have already unlocked your phone and have flashed an official version of LineageOS to it. You don't need to have flashed LineageOS yet, you could use TWRP through "fastboot boot" if you prefer. Or, if you want to use the recovery that was just created, it is located in ~/android/lineageos/out/target/product/kebab and is called recovery.img.

    • Reboot your phone in to recovery mode
    • In LineageOS Recovery return to the main menu and select "Apply update"
    • From your PC, run:
    Code:
    adb sideload ~/android/lineageos/lineage-18.1-[date]-UNOFFICIAL-kebab-signed.zip

    When the sideload is complete, reboot in to LineageOS. Make sure everything looks good with your build.

    You may also need to format your data partition at this time depending on what you had installed on your phone previously.

    Step 10: Flashing your signing key

    Now it's time to add your signing key to the Android Verified Boot process. To do so, do the following:

    • Reboot your phone in to fastboot mode
    • From your PC, run:
    Code:
    fastboot flash avb_custom_key ~/android/kebab/pkmd/pkmd.bin
    fastboot reboot bootloader
    fastboot oem lock
    • On your phone, confirm you want to re-lock and it will reboot

    Your phone will then factory reset and then reboot in to LineageOS.

    Which of course means you have to go through the first time setup wizard, so do so now.

    Step 11: Disable OEM unlock

    Congratulations! Your boot loader is now locked, but you can still unlock it again using fastboot, so it's time to disable that as well.

    • Unlock you phone and go to Settings->About phone
    • Scroll to the bottom and find "Build number"
    • Tap on it you enable the developer options
    • Go to Settings->System->Advanced->Developer options
    • Disable the "OEM unlocking" slider
    • Reboot

    Step 12: Profit!


    Other things
    • The above will build a standard USERDEBUG version of LineageOS, however this will still allow LineageOS Recovery to sideload non-signed files as well as give you root shell access through ADB. Step 3/4 above protects your system/vendor/boot/dtbo/etc. partitions, but none of the others. Likewise USERDEBUG builds will allow for rolling back to a previous builds/versions of LineageOS. To increase security and disallow both of these scenarios you may want to build a USER version of LineageOS to install. However this brings in other issues, such as flashing newer firmware from OnePlus so make sure you understand the implications of both choices. For more details on build types, see https://source.android.com/setup/develop/new-device#build-variants.
    • In the above example the releasekey from your LineageOS install has been used to sign AVB, but AVB supports other key strengths up to SHA512_RSA8192. You could create a key just for signing AVB that used different options than the default keys generated to sign LineageOS.
    • If you want to remove you signing key from your phone, you can do it by running "fastboot erase avb_custom_key".
    • The changes you made to the AOSP Makefile may conflict with future updates that you pull from LineageOS through repo sync, if you have to reset the file to get repo sync to complete successfully, you'll have to reapply the changes afterwards.

    So why can't I do this with official LineageOS builds?

    NEW: You can! See this thread for more details.

    For Android Verified Boot (AVB) to work, it must have the hash values for each of the system/vendor/boot/dtbo/etc. partitions stored in vbmeta. Official LineageOS builds for kebab do include the vendor.img in them along with everything else that is needed, however that is not true for all phones.

    There are two "issues" that stop someone from using the official kebab builds:
    • LineageOS does not provide a pkmd.bin file to flash to your phone to include the public key in your AVB process (NEW: this thread shows you how to extract the key).
    • AVB is enabled in the official LineageOS builds but does not validate the hash trees during boot which limits the protection offered.
    Ok, what messages do I see during the boot process then?

    During a boot you will of course see the standard OnePlus power up screen, followed by the yellow "custom os" message and then the standard LineageOS boot animation.

    For more details on AVB boot messages, see https://source.android.com/security/verifiedboot/boot-flow

    So what does that patch to the Makefile do?

    AOSP's default Makefile makes an assumption that when AVB is enabled, that all the img files will be available well before vbmeta.img is created. This is simply NOT true and AOSP seems to know this as well from the following comment in the Makefile:

    Code:
    # Not using INSTALLED_VBMETA_SYSTEMIMAGE_TARGET as it won't be set yet.
    ifdef BOARD_AVB_VBMETA_SYSTEM
    $(eval $(call check-and-set-avb-args,vbmeta_system))
    endif
    
    ifdef BOARD_AVB_VBMETA_VENDOR
    $(eval $(call check-and-set-avb-args,vbmeta_vendor))
    endif

    These two calls eventual evaluate to returning the path to the partitions based upon the INSTALLED_*IMAGE_TARGET variable, which isn't created until later in the build process.

    Because of this, the command to build vbmeta.img gets corrupted due to the missing make variable being empty and an invalid command line is passed to avbtool near the end of the build.

    The corruption happens due to the fact that the following line from the original Makefile:

    Code:
    --include_descriptors_from_image $(call images-for-partitions,$(1))))))

    Gets added to the avbtool call even if "$(call images-for-partitions,$(1))" turns out to be an empty string. Avbtool then throws an error message as it is expecting a parameter after the "--include_descriptors_from_image" flag that is added for the "empty" partition path.

    The fix is to call "$(call images-for-partitions,$(1))" earlier, set it to a variable and check to make sure it isn't an empty string before letting the "--include_descriptors_from_image" be added to the avbtool command line to be used later.

    This technically generates an incomplete vbmeta.img file during the build process, but since the signing process recreates it from scratch anyway; no harm, no foul.

    Thank You's
    thanks for the guide. ps regarding the path requirements and the absolutely annoying signapk.jar issues when on the signing step, one can simple add a flag to the command like so
    Code:
    ./build/tools/releasetools/sign_target_files_apks -p out/host/linux-x86/ -o -d ~/.android-certs $OUT/obj/PACKAGING/target_files_intermediates/*-target_files-*.zip signed-target_files.zip
    the addition of -p out/host/linux-x86/ takes care of the pathing issues
    If anyone runs into the unicode encoding error with common.py, run repopick 305886
    which will cherry-pick https://review.lineageos.org/c/LineageOS/android_build/+/305886 and fix that error.

    Aside from those 2 things, (y) 🍻
    2
    Hello,

    Just to share with you.

    I successfully built a LineageOS 4 MicroG with OEM unlock support, the build is user build.

    I merged your guide with the needed steps from the build.sh script in the docker-lineage-cicd src/ directory.

    Now, my OnePlus 7 Pro, has lineageos 4 microg installed with OEM unlock support in developer options menu, same as CalyxOS on Pixel phones. 😁

    Thank You again, I really appreciate your shared information.🙏

    Wish you a blessed life.
    1
    Hello,

    I did extract the proprietary blobs from payload-based.

    Do you mean I should compile LinageOS successfully first using:
    source build/envsetup.sh
    breakfast guacamole
    croot
    brunch guacamole

    before i follow the steps listed here in this guide??

    Thank You
    Check the extraction script for errors or switch to the muppets, sometimes the extraction script isn't up to date.

    In general, yes, make sure you have a version of LineageOS that compiles successfully, that way you know you have a valid base to start from.

    Pre-requisites:
    • at least one successful build of LineageOS
    • at least one successful signing of your build with your own keys